Pavement detection data calibration method and related equipment

Abstract

The invention provides a road surface detection data calibration method. The method comprises the following steps: acquiring a target image containing a calibration plate from road surface sensing equipment; acquiring an inertial navigation positioning coordinate and an attitude angle corresponding to the target image acquisition moment from inertial navigation equipment; determining a perspective transformation matrix from a pixel coordinate system to a physical coordinate system according to the four angular points of the calibration plate; determining a conversion matrix from the physical coordinate system to a second inertial navigation coordinate system based on the coordinate of the center point of the calibration plate on the geodetic coordinate system and the inertial navigation positioning coordinate at the corresponding target image acquisition moment; taking a perspective transformation matrix from a pixel coordinate system to a physical coordinate system, a transformation matrix from the physical coordinate system to a second inertial navigation coordinate system and an attitude angle at a target image acquisition moment as calibration parameters; and for any pixel point in a to-be-calibrated image acquired by the pavement sensing equipment, determining the coordinate of the pixel point in the second inertial navigation coordinate system based on the calibration parameter. The invention further provides a related device for calibrating the pavement detection data.

Description

Technical Field

[0001] The present disclosure relates to the technical field of highway pavement management and maintenance, and in particular to a pavement detection data calibration method and related equipment. Background Art

[0002] With the development of computer vision, artificial intelligence, photoelectric sensors and other technologies, there are more and more types of road technical condition detection equipment. These detection equipment have different technical routes, data dimensions and organizational structures, which leads to data fusion only being able to go deep into the indicator evaluation model level, and unable to achieve refined fusion analysis at the basic data level, thus limiting the further development of the highway maintenance scientific decision-making technology system. Therefore, how to achieve accurate alignment and integrated fusion of multi-source, multi-dimensional, and heterogeneous data in a unified spatial coordinate system has become one of the key issues in highway maintenance scientific decision-making technology. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a road surface detection data calibration method, which can convert the coordinates of each pixel point in the road surface detection data collected by the road surface sensing device into the coordinates of the second inertial navigation coordinate system. In this way, after the multi-source, multi-dimensional, and heterogeneous data collected by different road surface sensing devices are uniformly converted into the coordinates of the second inertial navigation coordinate system, the spatial geometric mapping of multi-source heterogeneous data can be realized, and the precise alignment and integrated fusion of multi-source, multi-dimensional, and heterogeneous data can be achieved.

[0004] The road detection data calibration method described in the embodiment of the present disclosure may specifically include the following steps: obtaining a target image including a calibration plate from data collected by a road perception device; obtaining the inertial navigation positioning coordinates and attitude angle corresponding to the target image collection time from the data collected by the inertial navigation device; determining the perspective transformation matrix from the pixel coordinate system to the physical coordinate system according to the relationship between the pixel coordinates of the four corner points of the calibration plate on the target image and the physical coordinates of the four corner points of the calibration plate in the physical coordinate system; determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the geodetic coordinate system and the inertial navigation positioning coordinates corresponding to the target image collection time; using the perspective transformation matrix from the pixel coordinate system to the physical coordinate system, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system, and the attitude angle at the target image collection time as calibration parameters; for any pixel point in the image to be calibrated collected by the road perception device, determining the coordinates of the pixel point in the second inertial navigation coordinate system based on the calibration parameters.

[0005] In an embodiment of the present disclosure, the target image includes: a target front image taken by an area array camera; a target road surface image taken by a line scan camera; and a target ground penetrating radar image obtained by a ground penetrating radar, etc., or a combination thereof.

[0006] In an embodiment of the present disclosure, determining the perspective transformation matrix from the pixel coordinate system to the physical coordinate system includes: reading the pixel coordinates of the four corner points of the calibration plate on the target image from the target image; determining the coordinates of the four corner points in the physical coordinate system by the size of the calibration plate; and constructing a group of linear equations based on the correspondence between the pixel coordinates of the four corner points and the physical coordinates, and calculating the perspective transformation matrix from the pixel coordinate system to the physical coordinate system by solving the group of linear equations.

[0007] In an embodiment of the present disclosure, the method for determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system includes: acquiring the coordinates of the center point of the calibration plate in the geodetic coordinate system; converting the coordinates of the center point of the calibration plate in the geodetic coordinate system into the first inertial navigation coordinate system to obtain the coordinates of the center point of the calibration plate in the first inertial navigation coordinate system; and determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the first inertial navigation coordinate system.

[0008] In an embodiment of the present disclosure, converting the coordinates of the center point of the calibration plate in the geodetic coordinate system to the first inertial navigation coordinate system includes: determining the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system by the following expression:

[0009]

[0010] Among them, R F1 and T F1 They are determined by the following expressions:

[0011]

[0012]

[0013] Among them, N is the radius of the ellipsoid, e is the first eccentricity of the earth, a is the major semi-axis of the reference ellipsoid, and b is the minor semi-axis of the reference ellipsoid. is the inertial navigation positioning coordinate recorded at the time of target image acquisition.

[0014] In an embodiment of the present disclosure, determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the first inertial navigation coordinate system includes: determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system Calculate the coordinate origin of the first inertial navigation coordinate system from the center point O of the calibration plate The translation vector as well as The transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system is obtained according to the following expression: F2 :

[0015]

[0016] In an embodiment of the present disclosure, determining the coordinates of the pixel point in the second inertial navigation coordinate system includes: based on the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system F1 Convert the pixel coordinates P(u,v) of the pixel point to physical coordinates P II ; and the attitude angle based on the image acquisition time to be calibrated γ P and θ P , the attitude angle of the target image at the time of acquisition γ F and θ F , and the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 Determine the coordinates P of the pixel point in the second inertial navigation coordinate system IV .

[0017] In the embodiment of the present disclosure, the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system is F1 Convert the pixel coordinates P(u,v) of the pixel point to physical coordinates P II Including: Determine the physical coordinate P based on the following expression II In the physical coordinate system X II , Y II Coordinate value in direction and

[0018]

[0019] Determine the physical coordinates of the corresponding point of any pixel point P(u,v) in the image to be calibrated in the physical coordinate system as follows:

[0020] In an embodiment of the present disclosure, the coordinate P of the pixel point in the second inertial navigation coordinate system is determined based on the following expression: IV :P IV =R F3 ·T F2 ·P II ; Among them, R F3 Determined by the following expression:

[0021]

[0022] In an embodiment of the present disclosure, the method further comprises: respectively measuring the Y coordinates of the center point of the multi-point laser system to the center point of the inertial navigation device in the second inertial navigation coordinate system; IV The first distance s1 in the direction of Z IV A second distance s2 in the direction; based on the first distance and the second distance, obtaining the coordinates (0, s1, s2) of the center point of the multi-point laser system in the second inertial navigation coordinate system; and by determining the positional relationship between each laser point and the center point of the inertial navigation device, respectively obtaining the coordinates of the laser cross-section data collected by each laser point in the second inertial navigation coordinate system.

[0023] Corresponding to the above method, an embodiment of the present disclosure further provides a road surface detection data calibration device, including:

[0024] A target image acquisition module is used to acquire a target image including a calibration plate from data collected by a road surface sensing device;

[0025] An inertial navigation data acquisition module is used to obtain the inertial navigation positioning coordinates and attitude angle corresponding to the target image acquisition time from the data collected by the inertial navigation device;

[0026] A perspective transformation matrix determination module, used to determine the perspective transformation matrix from the pixel coordinate system to the physical coordinate system according to the relationship between the pixel coordinates of the four corner points of the calibration plate on the target image and the physical coordinates of the four corner points of the calibration plate on the physical coordinate system;

[0027] A conversion matrix determination module, used to determine the conversion matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the geodetic coordinate system and the inertial navigation positioning coordinates corresponding to the target image acquisition time;

[0028] a calibration parameter determination module, configured to use the perspective transformation matrix from the pixel coordinate system to the physical coordinate system, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system, and the attitude angle at the time of target image acquisition as calibration parameters; and

[0029] The data calibration module is used to determine the coordinates of any pixel point in the to-be-calibrated image collected by the road surface sensing device in the second inertial navigation coordinate system based on the calibration parameters.

[0030] In addition, an embodiment of the present disclosure further provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned road surface detection data calibration method when executing the program.

[0031] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the above-mentioned road surface detection data calibration method.

[0032] An embodiment of the present disclosure further provides a computer program product, including computer program instructions, which, when executed on a computer, enable the computer to execute the above-mentioned road surface detection data calibration method.

[0033] It can be seen that the above-mentioned road detection data calibration method and its related equipment can convert the coordinates of each pixel point in the road detection data collected by the road sensing device into the coordinates of the second inertial navigation coordinate system. In this way, after the multi-source, multi-dimensional, and heterogeneous data collected by different road sensing devices are uniformly converted into the coordinates of the second inertial navigation coordinate system, the spatial geometric mapping of multi-source heterogeneous data can be realized, and the precise alignment and integrated fusion of multi-source, multi-dimensional, and heterogeneous data can be achieved.

[0034] The embodiments of the present disclosure can comprehensively utilize satellite space positioning technology and sensor positioning technology, build a detection equipment carrier coordinate system based on inertial navigation equipment, and use a joint calibration method for each detection module of a universal calibration tool to integrate the coordinates of data collected by various road surface sensing devices such as area array cameras, linear array cameras, multi-point laser systems or ground penetrating radars into the detection equipment carrier coordinate system based on inertial navigation equipment, thereby realizing multi-source heterogeneous data space geometric mapping, providing support for refined fusion analysis at the basic data level, and promoting the further development of the scientific decision-making technology system for highway maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 The implementation process of the road surface detection data calibration method described in the embodiment of the present disclosure is shown.

[0037] Figure 2 An example of the shape of the calibration plate described in the embodiment of the present disclosure is shown.

[0038] Figure 3 The implementation process of the method for determining the perspective transformation matrix from the pixel coordinate system to the physical coordinate system described in the embodiment of the present disclosure is shown.

[0039] Figure 4The implementation process of the method for determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system described in an embodiment of the present disclosure is shown.

[0040] Figure 5 The implementation process of the method for converting laser cross-section data into a second inertial navigation coordinate system described in an embodiment of the present disclosure is shown.

[0041] Figure 6 The internal structure of the road surface detection data calibration device described in the embodiment of the present disclosure is shown.

[0042] Figure 7 A more specific schematic diagram of the hardware structure of an electronic device described in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0044] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0045] As mentioned earlier, with the increasing number of types of road technical condition detection equipment, how to achieve accurate alignment and integrated fusion of multi-source, multi-dimensional, and heterogeneous data in a unified spatial coordinate system has become one of the key issues in scientific decision-making technology for highway maintenance.

[0046] In order to solve the above problems, the embodiments of the present invention propose a road surface detection data calibration method, which can comprehensively utilize satellite space positioning technology and sensor positioning technology, take inertial navigation equipment as a benchmark, construct a detection equipment carrier coordinate system, and use a joint calibration method of each detection module based on a universal calibration tool to integrate the coordinates of data collected by various road surface sensing devices such as area array cameras, line array cameras, multi-point laser systems or ground penetrating radars into the detection equipment carrier coordinate system, thereby realizing multi-source heterogeneous data space geometric mapping, providing support for refined fusion analysis at the basic data level, and promoting the further development of the scientific decision-making technology system for highway maintenance.

[0047] It should be noted that the data that needs to be calibrated in the road surface detection data calibration method described in the embodiment of the present disclosure may include: the front image taken by the area array camera; the road surface image taken by the line scan camera; the laser cross-section data (including: rutting, flatness, etc.) obtained by the multi-point laser system; and the ground penetrating radar image obtained by the ground penetrating radar, etc., one of the data collected by various road surface sensing devices or any combination thereof. In addition, the data used as the reference for data calibration in the road surface detection data calibration method described in the embodiment of the present disclosure may further include: the spatial positioning coordinates and attitude angles obtained by the inertial navigation equipment (such as the inertial navigation real-time dynamic measurement equipment, that is, the inertial navigation RTK equipment). Among them, the above-mentioned attitude angles may include: heading angle That is, the angle between the projection line of the longitudinal axis of the carrier on the local horizontal plane and the local geographic north direction, where the clockwise direction of north-east is positive; the roll angle γ, that is, the angle between the vertical axis of the carrier and the plumb plane where the longitudinal axis is located, where it is positive when the carrier is tilted to the right; and the pitch angle θ, that is, the angle between the longitudinal axis of the carrier and its horizontal projection line, where it is positive when the carrier is raising its head.

[0048] In the embodiment of the present disclosure, in order to calibrate multi-source data, it is necessary to install one or any combination of various road surface sensing devices such as an area array camera, a line scan camera, a multi-point laser system, a ground penetrating radar, and an inertial navigation device on the same vehicle in advance. Wherein, when a multi-point laser system is installed, the center point of the inertial navigation device and the center point of the multi-point laser system are usually located on a straight line, and the straight line is parallel to the front direction of the vehicle.

[0049] In addition, in the embodiments of the present disclosure, in order to realize the spatial geometric mapping of multi-source heterogeneous data, it is also necessary to pre-establish the earth coordinate system, the physical coordinate system, the first inertial navigation coordinate system and the second inertial navigation coordinate system. In addition, for the front image taken by the array camera, the road image taken by the line scan camera and the ground penetrating radar image obtained by the ground penetrating radar, their own pixel coordinate system is used to represent the position of each pixel point. The goal of the embodiments of the present disclosure is to use the pre-established earth coordinate system, the physical coordinate system and the first inertial navigation coordinate system to map the pixel coordinates of each pixel point in the front image taken by the array camera, the road image taken by the line scan camera and the ground penetrating radar image obtained by the ground penetrating radar to the second inertial navigation coordinate system. For the laser section data obtained by the multi-point laser system, the laser section data collected by each laser point can also be mapped to the second inertial navigation coordinate system, so as to realize the precise alignment and integrated fusion of multi-source, multi-dimensional and heterogeneous data.

[0050] Specifically, the geodetic coordinate system can be established by the following method: take the center of mass of the earth as the origin, take the direction pointing to the agreement earth pole defined by BIH1984.0 as the Z axis, take the direction pointing to the intersection of the zero meridian plane of BIH1984.0 and the agreement earth pole equator as the X axis, and take the direction determined by the right-hand rule as the Y axis to establish the geodetic coordinate system. In the embodiment of the present disclosure, the geodetic coordinate system is also referred to as coordinate system I. The coordinates of a point in the geodetic coordinate system can be expressed as (B I ,L I ,H I In addition, for the convenience of description and to distinguish the axes in other coordinate systems, in the embodiments of the present disclosure, the X, Y, and Z axes of the above-mentioned geodetic coordinate system can be represented by X, Y, and Z axes, respectively. I , Y I and Z I express.

[0051] The above physical coordinate system can be established by the following method: on a sufficiently long straight lane, place the calibration plate parallel to the center of the lane, take the center point O of the calibration plate as the origin, take the center cross of the calibration plate as the X axis and Y axis respectively, and take the direction perpendicular to the plane where the calibration plate is located as the Z axis to establish the physical coordinate system. In the embodiment of the present disclosure, the above physical coordinate system can also be referred to as coordinate system II or coordinate system OXYZ. In order to facilitate the description and distinguish the axes in other coordinate systems, in the embodiment of the present disclosure, the X, Y, and Z axes of the above physical coordinate system can be respectively represented by X II , Y II and Z II The structure of the calibration plate will be described in detail later, so it is omitted here.

[0052] The first inertial navigation coordinate system can be established by the following method: take the center point I of the inertial navigation device as the origin, take the direction pointing to the east as the X axis, take the direction pointing to the north as the Y axis, and take the direction pointing to the sky as the Z axis to establish the first inertial navigation coordinate system. In the embodiment of the present disclosure, the first inertial navigation coordinate system can also be referred to as coordinate system III. For the convenience of description and to distinguish the axes in other coordinate systems, in the embodiment of the present disclosure, the X, Y, and Z axes of the first inertial navigation coordinate system can be respectively represented by X III , Y III and Z III express.

[0053] The second inertial navigation coordinate system can be established by the following method: take the center point I of the inertial navigation device as the origin. When the vehicle is in a horizontal state, the direction parallel to the horizontal plane and pointing to the right side of the vehicle is the X-axis, the direction parallel to the horizontal plane and pointing to the front of the vehicle is the Y-axis, and the direction perpendicular to the horizontal plane and pointing to the top of the vehicle is the Z-axis to establish the second inertial navigation coordinate system. In the embodiment of the present disclosure, the second inertial navigation coordinate system can also be referred to as coordinate system IV. For the convenience of description and to distinguish the axes in other coordinate systems, in the embodiment of the present disclosure, the X, Y, and Z axes of the second inertial navigation coordinate system can be respectively represented by X, Y, and Z. IV , Y IV and Z IV In the embodiment of the present disclosure, the second inertial navigation coordinate system can be regarded as the detection equipment carrier coordinate system.

[0054] After the above-mentioned coordinate system is established, the pavement detection data calibration method provided in the embodiment of the present disclosure can utilize the pre-established geodetic coordinate system, physical coordinate system and first inertial navigation coordinate system to map the pixel coordinates of each pixel point in the front image taken by the area array camera, the pavement image taken by the line scan camera or the ground penetrating radar image obtained by the ground penetrating radar to the second inertial navigation coordinate system, thereby achieving precise alignment of multi-source, multi-dimensional and heterogeneous data, providing a basis for realizing refined fusion analysis at the basic data level, and further ensuring the further development of the scientific decision-making technology system for highway maintenance.

[0055] Specifically, Figure 1 The implementation process of the road surface detection data calibration method described in some embodiments of the present disclosure is shown. Figure 1 As shown, the road surface detection data calibration method described in the embodiment of the present disclosure may include the following steps.

[0056] In step 110, a target image including a calibration plate is obtained from data collected by a road surface sensing device.

[0057] In the embodiments of the present disclosure, the road surface sensing device may include: one or more of an area array camera, a line array camera, or a ground penetrating radar. Correspondingly, the target image may include: a front image taken by an area array camera; a road surface image taken by a line scan camera; and a ground penetrating radar image obtained by a ground penetrating radar, or a combination thereof.

[0058] In the embodiments of the present disclosure, the calibration plate is a calibration tool. In order to achieve data calibration more accurately, the calibration plate can generally have a grid shape, which should include a center point and cross lines passing through the center point of the calibration plate and perpendicular to each other. For example, in some embodiments of the present disclosure, the calibration plate shape can be as follows: Figure 2 shown. Figure 2 The calibration plate shown includes four plates arranged alternately in white and black, wherein the white plate may be engineering plastic and the black plate may be metal. In addition, the intersection point of the above four plates is the center point O of the calibration plate, and the intersection line of the above four plates is the above mutually perpendicular cross lines, such as Figure 2 Further, in some embodiments of the present disclosure, the black plate may include a plurality of strip-shaped areas, and the height (or thickness) thereof increases sequentially from the center of the calibration plate to the edge of the calibration plate.

[0059] In step 120, the inertial navigation positioning coordinates I corresponding to the target image acquisition time are obtained from the data collected by the inertial navigation device. I , attitude angle γ F and θ F .

[0060] In the embodiment of the present disclosure, in the above step 120, it is necessary to make the front bumper of the vehicle and the X coordinate system of the physical coordinate system II The axis is parallel to the calibration plate and passes over the calibration plate to ensure that the scanning area of ​​the road sensing device covers the calibration plate. At the same time, it is also necessary to ensure that the road sensing device records the time of each image acquisition moment and the inertial navigation positioning coordinates corresponding to each image acquisition moment during the data acquisition process. I And attitude angle γ F and θ F .

[0061] In step 130, the perspective transformation matrix from the pixel coordinate system to the physical coordinate system is determined according to the relationship between the pixel coordinates of the four corner points of the calibration plate on the target image and their physical coordinates on the physical coordinate system.

[0062] In the embodiment of the present disclosure, the determination of the perspective transformation matrix from the pixel coordinate system to the physical coordinate system in step 130 can be performed by Figure 3 Specifically, Figure 3 As shown, the method for determining the perspective transformation matrix from the pixel coordinate system to the physical coordinate system may include:

[0063] In step 310, pixel coordinates of the four corner points of the calibration plate on the target image are read from the target image.

[0064] It can be understood that the pixel coordinates of the above four corner points on the target image can be directly read from the above target image.

[0065] In step 320, the coordinates of the four corner points in the physical coordinate system are determined by calibrating the size of the plate.

[0066] In an embodiment of the present disclosure, the coordinates of the above four corner points in the physical coordinate system can be directly determined based on the size of the calibration plate and the units of each axis in the physical coordinate system. For example, assuming that the calibration plate is 1 meter long and 1 meter wide, the size of each black or white grid in the calibration plate is 0.5 meters * 0.5 meters, and the unit of each axis in the physical coordinate system is 1 meter, it can be deduced that the physical coordinates of the four corner points of the above calibration plate in the physical coordinate system are (-0.5, 0.5, 0), (0.5, 0.5, 0), (-0.5, -0.5, 0) and (-0.5, 0.5, 0).

[0067] In step 330, a linear equation group is constructed according to the correspondence between the pixel coordinates and the physical coordinates of the four corner points, and the perspective transformation matrix from the pixel coordinate system to the physical coordinate system is calculated by solving the linear equation group.

[0068] For example, based on the correspondence between the pixel coordinates and the physical coordinates of the four corner points, the coordinate pairs of the four corner points can be obtained, where each coordinate pair can be expressed as (u, v) and Further, assuming that the perspective transformation matrix from the pixel coordinate system to the physical coordinate system has the structure of the following expression (1), where each parameter is the coefficient to be solved, according to the coordinate pairs of the above four corner points, four linear equations associated with the above coefficients to be solved can be obtained, that is, eight equations. The equation group corresponding to each coordinate pair can be shown in the following expression (2). By solving the above four linear equations, the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system can be solved. F1 The specific values ​​of each coefficient in .

[0069]

[0070] Further, it can be understood that after determining the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system F1 After that, through the perspective transformation matrix C F1The corresponding point P in the physical coordinate system of any pixel point P(u,v) in the target image or other images collected by the road perception device can be calculated by the following expression: II Respectively in X II , Y II Coordinate value in direction and

[0071]

[0072] In addition, since point P II In the physical coordinate system Z II The direction coordinate value can be represented by 0, and finally the corresponding point P in the physical coordinate system of any pixel point P(u,v) in the above target image or other images can be obtained. II The physical coordinates are

[0073] In step 140, a transformation matrix from the physical coordinate system to the second inertial navigation coordinate system is determined based on the coordinates of the center point O of the calibration plate in the geodetic coordinate system.

[0074] In the embodiment of the present disclosure, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system in step 140 can be determined by Figure 4 Specifically, Figure 4 As shown, the method for determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system may include:

[0075] In step 410, the coordinates of the center point O of the calibration plate in the geodetic coordinate system are obtained.

[0076] In the embodiment of the present disclosure, the coordinates of the center point O of the calibration plate in the geodetic coordinate system are It can be measured by handheld RTK equipment.

[0077] In step 420, the coordinates of the center point O of the calibration plate in the geodetic coordinate system are Convert to the first inertial navigation coordinate system and obtain the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system

[0078] In the embodiment of the present disclosure, the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system can be obtained by the following expression (3):

[0079]

[0080] Among them, R F1 and T F1 They can be determined by the following expressions:

[0081]

[0082] Where N is the radius of the mortise and tenon circle. As an alternative, it can also be determined by the following calculation formula in practical applications: e is the first eccentricity of the earth; a is the major semi-axis of the reference ellipsoid; b is the minor semi-axis of the reference ellipsoid; is the inertial navigation positioning coordinate recorded at the above target image acquisition time, that is, the inertial navigation positioning coordinate corresponding to the target image acquisition time.

[0083] In step 430, based on the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system Determine the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 .

[0084] In the embodiment of the present disclosure, first, based on the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system, Calculate the coordinate origin of the first inertial navigation coordinate system from the center point O of the calibration plate (That is, I III The translation vector of (0,0,0) as well as Then, the transformation matrix T from the above physical coordinate system to the second inertial navigation coordinate system is obtained according to the following expression (4): F2 .

[0085]

[0086] In step 150, the perspective transformation matrix from the pixel coordinate system to the physical coordinate system, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system, and the attitude angle at the time of target image acquisition are used as calibration parameters.

[0087] In step 160, for any pixel point in the image to be calibrated collected by the road surface sensing device, the coordinates of the pixel point in the second inertial navigation coordinate system are determined based on the calibration parameters.

[0088] Specifically, the above step 160 may include:

[0089] First, the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system F1 Convert the pixel coordinates P(u,v) of the pixel point to physical coordinates P II Specifically, the physical coordinate P can be obtained using the following expression: II In the physical coordinate system X II , Y II Coordinate value in direction and Since point P II In the physical coordinate system Z II The direction coordinate value can be represented by 0, and finally the corresponding point P in the physical coordinate system of any pixel point P(u,v) in the above-mentioned image to be calibrated can be obtained. II The physical coordinates are

[0090]

[0091] Secondly, based on the attitude angle of the image to be calibrated at the time of acquisition γ P and θ P , the attitude angle of the target image at the time of acquisition γ F and θ F , the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 Determine the coordinates P of the pixel point in the second inertial navigation coordinate system IV .

[0092] P IV =R F3 ·T F2 ·P II

[0093] Among them, R F3 It can be determined by the following expression:

[0094]

[0095] in, γ P and θ P is the attitude angle of the image to be calibrated at the time of image acquisition.

[0096] It can be seen that the coordinates of each pixel point in the road detection data collected by the road sensing device can be converted to the coordinates of the second inertial navigation coordinate system through the above method. In this way, after the multi-source, multi-dimensional, and heterogeneous data collected by different road sensing devices are uniformly converted to the coordinates of the second inertial navigation coordinate system, the spatial geometric mapping of multi-source heterogeneous data can be realized, and the precise alignment of multi-source, multi-dimensional, and heterogeneous data can be realized, thereby providing a basis for the integrated fusion of multi-source data. Specifically, after the coordinate system conversion of each image in the detection data collected by the road sensing device, the position of the target to be monitored (for example, the road detection element of interest) in the image in the global geographic coordinate system (that is, the second inertial navigation coordinate system) can be determined based on the converted coordinates. Since the second inertial navigation coordinate system is associated with the physical world, the point-to-point correspondence between the image data and the physical world can be realized, which can help maintenance workers quickly determine the location of road diseases and take corresponding measures. Further, the spatial coordinate alignment of historical detection data can be realized to conduct research such as tracking and comparing the historical status of the detection target.

[0097] Specifically, for the front image captured by the area array camera, the perspective transformation matrix C from the pixel coordinate system corresponding to the front image to the physical coordinate system can be first determined by the road surface detection data calibration method. F1 , the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 And the attitude angle of the target image acquisition time γ F and θ F Then, the calibration parameters are used to convert any pixel point in any front image taken by the area array camera into coordinates in the second inertial navigation coordinate system, thereby realizing the calibration of the front image.

[0098] For a road surface image captured by a line scan camera with a fixed preset length, the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system corresponding to the road surface image can be first determined by the above road surface detection data calibration method. F1 , the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 And the attitude angle at the time of road image acquisition γ F and θ F Then, the calibration parameters are used to convert any pixel point in any road image taken by the line scan camera into coordinates in the second inertial navigation coordinate system, thereby realizing the calibration of the road image.

[0099] For a ground penetrating radar image collected by a ground penetrating radar with a fixed preset length, the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system corresponding to the ground penetrating radar image can be first determined by the above road surface detection data calibration method. F1 , the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 And the attitude angle at the time of road image acquisition γ F and θ F The parameters are used as the calibration parameters of the ground penetrating radar image. Then, the calibration parameters are used to convert any pixel point in any ground penetrating radar image collected by the ground penetrating radar into coordinates in the second inertial navigation coordinate system, thereby realizing the calibration of the ground penetrating radar image.

[0100] In this way, the front image taken by the area array camera, the road surface image taken by the line scan camera with a fixed preset length, and the ground penetrating radar image collected by the ground penetrating radar with a fixed preset length can all be uniformly converted to the second inertial navigation coordinate system, so that the precise alignment of the above-mentioned multi-source, multi-dimensional, and heterogeneous data can be achieved, thereby providing a basis for the integrated fusion of multi-source data.

[0101] In order to further integrate the laser cross-section data (including rutting, flatness, etc.) obtained by the multi-point laser system, the embodiment of the present disclosure can further include the following on the basis of the above method: Figure 5 The method of converting laser cross-section data into the second inertial navigation coordinate system is shown.

[0102] In step 510, the Y coordinates of the center point of the multi-point laser system to the center point of the inertial navigation device in the second inertial navigation coordinate system (i.e., coordinate system IV) are measured respectively. IV The first distance s1 in the direction of Z IV The second distance s2 in the direction.

[0103] In step 520, the coordinates (0, s1, s2) of the center point of the multi-point laser system in the second inertial navigation coordinate system are acquired based on the first distance and the second distance.

[0104] In step 530, the coordinates of the laser cross-section data collected by each laser point in the second inertial navigation coordinate system are respectively obtained by determining the positional relationship between each laser point and the center point of the inertial navigation device.

[0105] For example, the position offset between each laser point and the center point of the inertial navigation device can be determined first, and the coordinates of the laser cross-section data collected by each laser point in the second inertial navigation coordinate system can be determined based on the position offset and the coordinates (0, s1, s2) of the center point of the multi-point laser system in the second inertial navigation coordinate system.

[0106] In some embodiments of the present disclosure, the Y coordinates of the Nth laser point and the center point of the inertial navigation device in the second inertial navigation coordinate system (i.e., coordinate system IV) can also be measured respectively. IV The first distance S1n in the direction of Z IV The second distance S2n in the direction can thereby obtain the coordinates (0, S1n, S2n) of the laser cross-section data collected by each laser point in the second inertial navigation coordinate system.

[0107] In this way, the laser cross-section data obtained by the multi-point laser system is also converted to the second inertial navigation coordinate system, so as to achieve precise alignment with the data collected by other road sensing devices and complete integrated fusion.

[0108] The road surface detection data calibration method described in the embodiment of the present disclosure will be further described in detail below with reference to specific examples.

[0109] In this example, the area array camera, line scan camera, multi-point laser system, ground penetrating radar and inertial navigation equipment are first installed on the same vehicle. The center point of the inertial navigation equipment and the center point of the multi-point laser system are located on a straight line, and the straight line is parallel to the front direction of the vehicle. Then, on a sufficiently long straight lane, the calibration plate is placed parallel to the center of the lane, and the coordinate system I coordinate O of the center point O of the calibration plate is measured by a handheld RTK device. I (40.0698,116.2047,27.05).

[0110] Next, let the front bumper of the vehicle be aligned with the X coordinate system of the physical coordinate system. II The axis is parallel and passes over the calibration plate to ensure that the scanning area of ​​the above-mentioned detection equipment covers the calibration plate and that each detection equipment records the time of collection and the inertial navigation positioning coordinates I at the time of collection during data collection. I and attitude angle γ F and θ F .

[0111] For the front image captured by the area array camera, find a target front image that completely captures the calibration plate, and determine the pixel coordinates of the four corner points of the calibration plate from the target front image (824,554), (993,554), (800,652), and (1010,653).

[0112] Assuming that the calibration plate is 1 meter (m) long and 1 meter (m) wide, and the grid size in the calibration plate is 0.5m*0.5m, the physical coordinates of the four corner points of the calibration plate in the physical coordinate system can be inferred from the size of the calibration plate as (-0.5, 0.5), (0.5, 0.5), (-0.5, -0.5), and (-0.5, 0.5).

[0113] Next, according to the correspondence between the pixel coordinates and the physical coordinates of the above four corner points, the perspective transformation matrix C from the pixel coordinate system to the physical coordinate system can be solved: F1 .

[0114]

[0115] Further, the coordinate system I coordinate O of the center point O of the calibration plate I (40.0698,116.2047,27.05) is converted to the first inertial navigation coordinate system, and we can get O III (0,11.103645,-1.600009). The specific calculation process is as follows:

[0116]

[0117] Where N is the radius of the ellipsoid, e is the first eccentricity of the earth, a = 6378137.0 is the major semi-axis of the reference ellipsoid, b = 6356752.31414 is the minor semi-axis of the reference ellipsoid, I I (40.0697,116.2047,28.65) are the inertial navigation positioning coordinates recorded at the time of the above-mentioned front image acquisition.

[0118] Furthermore, we calculate O in coordinate system III III To I III The translation vector of the physical coordinate system is obtained to obtain the transformation matrix T from the second inertial navigation coordinate system F2 .

[0119]

[0120] Then, record the above C F1 , T F2 And the attitude angle when the above-mentioned target front image is collected γ F and θ F As the front image calibration parameter.

[0121] Finally, for any pixel point P(950,580) in the subsequent front image, the coordinates of the pixel point in the second inertial navigation coordinate system are determined based on the above calibration parameters.

[0122] Specifically, first, determine the physical coordinates of the above pixel points in the physical coordinate system:

[0123]

[0124] And set point P II In the physical coordinate system Z II The direction coordinate value is 0.

[0125] Then, based on the attitude angle at the time of subsequent front image acquisition γ P and θ P , the attitude angle of the target at the time of image acquisition γ F and θ F , and the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 Determine the coordinates P of the pixel point in the second inertial navigation coordinate system IV .

[0126] P IV =R F3 ·T F2 ·P II

[0127] Among them, R F3 It can be determined as:

[0128]

[0129] Finally, the coordinates P of the pixel point in the second inertial navigation coordinate system can be determined IV (2.07061478,-10.74013763,1.41670673).

[0130] The calibration method for the road surface image and radar image data is similar to that for the above-mentioned front image data, and will not be repeated here.

[0131] For the laser cross-section data obtained by the multi-point laser system, first, measure the Y coordinate of the center point of the multi-point laser system to the center point of the inertial navigation device in the second inertial navigation coordinate system. IV The direction distance is 3 meters, Z IV If the direction distance is 1 meter, the coordinates of the center point of the multi-point laser system in the second inertial navigation coordinate system can be obtained as (0, 3, -1). Furthermore, the coordinates of each laser point acquisition data in the second inertial navigation coordinate system can be obtained through the positional relationship between each laser point and the center point.

[0132] It can be seen that through the above-mentioned road surface detection data calibration method, whether it is the front image taken by the area array camera, the road surface image taken by the line scan camera, the ground penetrating radar image collected by the ground penetrating radar, or the laser cross-section data obtained by the multi-point laser system, they can all be uniformly converted to the second inertial navigation coordinate system, thereby realizing the precise alignment and integrated fusion of the above-mentioned multi-source, multi-dimensional and heterogeneous data.

[0133] Based on the above-mentioned road surface detection data calibration method, an embodiment of the present disclosure also provides a road surface detection data calibration device. Figure 6The internal structure of the road surface detection data calibration device according to the embodiment of the present disclosure is shown. Figure 6 As shown, the above-mentioned road surface detection data calibration device may include the following modules.

[0134] The target image acquisition module 610 is used to acquire a target image including a calibration plate from data collected by the road surface sensing device;

[0135] The inertial navigation data acquisition module 620 is used to obtain the inertial navigation positioning coordinates and attitude angle corresponding to the target image acquisition time from the data collected by the inertial navigation device;

[0136] A perspective transformation matrix determination module 630 is used to determine a perspective transformation matrix from a pixel coordinate system to the physical coordinate system according to a relationship between pixel coordinates of the four corner points of the calibration plate on the target image and physical coordinates of the four corner points of the calibration plate on the physical coordinate system;

[0137] A conversion matrix determination module 640 is used to determine the conversion matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the geodetic coordinate system and the inertial navigation positioning coordinates corresponding to the target image acquisition time;

[0138] a calibration parameter determination module 650, configured to use the perspective transformation matrix from the pixel coordinate system to the physical coordinate system, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system, and the attitude angle at the time of acquisition of the target image as calibration parameters; and

[0139] The data calibration module 660 is used to determine the coordinates of any pixel point in the to-be-calibrated image collected by the road surface sensing device in the second inertial navigation coordinate system based on the calibration parameters.

[0140] It should be noted that the specific implementation methods of the various modules included in the road surface detection data calibration device described in the embodiments of the present disclosure can refer to the aforementioned embodiments, and will not be repeated here. Furthermore, the road surface detection data calibration device described in the above embodiments has the same beneficial effects as the corresponding method embodiments, and will not be repeated here.

[0141] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present disclosure also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the road surface detection data calibration method described in any of the above embodiments is implemented.

[0142] Figure 7A more specific hardware structure diagram of an electronic device provided in this embodiment is shown, and the device may include: a processor 2010, a memory 2020, an input / output interface 2030, a communication interface 2040, and a bus 2050. The processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040 are connected to each other in communication within the device through the bus 2050.

[0143] The processor 2010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0144] The memory 2020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 2020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 2020 and are called and executed by the processor 2010.

[0145] The input / output interface 2030 is used to connect input / output devices to realize information input and output. The input / output devices can be configured in the device as components, or can be externally connected to the device to provide corresponding functions. The input devices can include microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0146] The communication interface 2040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0147] The bus 2050 includes a path that transmits information between the various components of the device (eg, the processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040).

[0148] It should be noted that, although the above device only shows the processor 2010, the memory 2020, the input / output interface 2030, the communication interface 2040, and the bus 2050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0149] The electronic device of the above embodiment is used to implement the corresponding road surface detection data calibration method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0150] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the road surface detection data calibration method described in any of the above embodiments.

[0151] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0152] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the road surface detection data calibration method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0153] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0154] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0155] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0156] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A road surface detection data calibration method, comprising: Obtain a target image including a calibration plate from data collected by a road surface sensing device; Acquire the inertial navigation positioning coordinates and attitude angle corresponding to the target image acquisition time from the data collected by the inertial navigation device; Determine a perspective transformation matrix from a pixel coordinate system to the physical coordinate system according to a relationship between pixel coordinates of the four corner points of the calibration plate on the target image and physical coordinates of the four corner points of the calibration plate on the physical coordinate system; Determine a transformation matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the geodetic coordinate system and the inertial navigation positioning coordinates corresponding to the target image acquisition moment; The perspective transformation matrix from the pixel coordinate system to the physical coordinate system, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system, and the attitude angle at the time of target image acquisition are used as calibration parameters; For any pixel point in the image to be calibrated collected by the road surface perception device, the coordinates of the pixel point in the second inertial navigation coordinate system are determined based on the calibration parameters.

2. The method according to claim 1, wherein: The target image includes: a target front image taken by an area array camera; a target road surface image taken by a line scan camera; and a target ground penetrating radar image obtained by a ground penetrating radar, or a combination thereof.

3. The method according to claim 1, wherein: Determining the perspective transformation matrix from the pixel coordinate system to the physical coordinate system includes: Reading pixel coordinates of four corner points of the calibration plate on the target image from the target image; Determining the coordinates of the four corner points in a physical coordinate system based on the size of the calibration plate; and According to the correspondence between the pixel coordinates of the four corner points and the physical coordinates, a linear equation group is constructed, and the perspective transformation matrix from the pixel coordinate system to the physical coordinate system is calculated by solving the linear equation group.

4. The method according to claim 1, wherein: The method for determining a transformation matrix from a physical coordinate system to a second inertial navigation coordinate system comprises: Obtaining the coordinates of the center point of the calibration plate in the geodetic coordinate system; Converting the coordinates of the center point of the calibration plate in the geodetic coordinate system to the first inertial navigation coordinate system to obtain the coordinates of the center point of the calibration plate in the first inertial navigation coordinate system; and A transformation matrix from the physical coordinate system to the second inertial navigation coordinate system is determined based on the coordinates of the center point of the calibration plate in the first inertial navigation coordinate system.

5. The method according to claim 4, wherein: Converting the coordinates of the center point of the calibration plate in the geodetic coordinate system to the first inertial navigation coordinate system includes: determining the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system by the following expression: Among them, R F1 and T F1 They are determined by the following expressions: Among them, N is the radius of the ellipsoid, e is the first eccentricity of the earth, a is the major semi-axis of the reference ellipsoid, and b is the minor semi-axis of the reference ellipsoid. is the inertial navigation positioning coordinate recorded at the time of target image acquisition.

6. The method according to claim 4, wherein: Determining the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the first inertial navigation coordinate system includes: Based on the coordinates of the center point O of the calibration plate in the first inertial navigation coordinate system Calculate the coordinate origin of the first inertial navigation coordinate system from the center point O of the calibration plate The translation vector as well as The transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system is obtained according to the following expression: F2 :

7. The method according to claim 1, wherein: Determining the coordinates of the pixel point in the second inertial navigation coordinate system includes: Perspective transformation matrix C from pixel coordinate system to physical coordinate system F1 Convert the pixel coordinates P(u,v) of the pixel point to physical coordinates P II ;as well as Based on the attitude angle at the time of image acquisition to be calibrated γ P and θ P , the attitude angle of the target image at the time of acquisition γ F and θ F , and the transformation matrix T from the physical coordinate system to the second inertial navigation coordinate system F2 Determine the coordinates P of the pixel point in the second inertial navigation coordinate system IV .

8. The method according to claim 7, wherein: Perspective transformation matrix C from pixel coordinate system to physical coordinate system F1 Convert the pixel coordinates P(u,v) of the pixel point to physical coordinates P II include: The physical coordinate P is determined based on the following expression II In the physical coordinate system X II , Y II Coordinate value in direction and Determine the physical coordinates of the corresponding point of any pixel point P(u,v) in the image to be calibrated in the physical coordinate system as follows:

9. The method according to claim 7, wherein: The coordinates P of the pixel point in the second inertial navigation coordinate system are determined based on the following expression: IV :P IV =R F3 ·T F2 ·P II ; Among them, R F3 Determined by the following expression:

10. The method according to claim 1, further comprising: Measure the Y coordinates of the center point of the multi-point laser system to the center point of the inertial navigation device in the second inertial navigation coordinate system. IV The first distance s1 in the direction of Z IV The second distance s2 in the direction; Acquire the coordinates (0, s1, s2) of the center point of the multi-point laser system in the second inertial navigation coordinate system based on the first distance and the second distance; as well as The coordinates of the laser cross-section data collected by each laser point in the second inertial navigation coordinate system are respectively obtained by determining the positional relationship between each laser point and the center point of the inertial navigation device.

11. A road surface detection data calibration device, comprising: A target image acquisition module is used to acquire a target image including a calibration plate from data collected by a road surface sensing device; An inertial navigation data acquisition module is used to obtain the inertial navigation positioning coordinates and attitude angle corresponding to the target image acquisition time from the data collected by the inertial navigation device; A perspective transformation matrix determination module, used to determine the perspective transformation matrix from the pixel coordinate system to the physical coordinate system according to the relationship between the pixel coordinates of the four corner points of the calibration plate on the target image and the physical coordinates of the four corner points of the calibration plate on the physical coordinate system; A conversion matrix determination module, used to determine the conversion matrix from the physical coordinate system to the second inertial navigation coordinate system based on the coordinates of the center point of the calibration plate in the geodetic coordinate system and the inertial navigation positioning coordinates corresponding to the target image acquisition time; A calibration parameter determination module, used to use the perspective transformation matrix from the pixel coordinate system to the physical coordinate system, the transformation matrix from the physical coordinate system to the second inertial navigation coordinate system, and the attitude angle at the time of target image acquisition as calibration parameters; as well as The data calibration module is used to determine the coordinates of any pixel point in the to-be-calibrated image collected by the road surface sensing device in the second inertial navigation coordinate system based on the calibration parameters.

12. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the road surface detection data calibration method as described in any one of claims 1 to 10 is implemented.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the road surface detection data calibration method according to any one of claims 1 to 10.

14. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the road surface detection data calibration method according to any one of claims 1 to 10.

Images