Road surface roughness detection method and device based on image matching combined with point cloud data

Abstract

The present invention discloses a road surface flatness detection method and device based on image matching combined with point cloud data, which obtains three images of the road surface to be detected at the same time through a three-eye camera; performs pixel-by-pixel parallax comparison and matching in the road surface area in the three images; obtains point cloud data of the road surface to be detected at the same time through a laser radar when the number of mismatched pixels is greater than a first preset threshold; calculates the distance difference of the same horizontal line of the mismatched pixels corresponding to the point cloud data, and when the distance difference is greater than or equal to a second preset threshold, performs a road hazard warning based on the mean of the variance of the distance difference between points on each horizontal line. When used to assist manual driving, it can remind the driver of the road conditions ahead so that the driver can make adjustments in time to prevent the vehicle from being bumpy or in danger due to the driver's negligence in observation; when used in an automatic driving system, it can warn the decision module to facilitate the system to autonomously adjust the vehicle speed and ensure driving comfort and safety.

Description

Technical Field

[0001] The present invention relates to the field of computers and artificial intelligence, and in particular to a method and device for detecting road surface flatness based on image matching combined with point cloud data. Background Art

[0002] The transportation system is composed of people, vehicles, roads and the environment. Among them, the road is an important factor. Its road surface flatness largely constrains the upper limit of driving speed. Road surface flatness and driving speed jointly determine driving comfort and safety, which in turn affects the operating efficiency of the entire transportation system.

[0003] The smoothness of the road surface is so important that its evaluation value is one of the important indicators for road construction acceptance. However, due to the long-term use of roads, they will be exposed to the sun or rain, and the smoothness of the road surface may decrease over time, such as cracks, subsidence, partial breakage, etc., affecting driving safety.

[0004] In manual driving, the driver usually observes the road surface flatness problem autonomously, makes subjective judgments and responds, and adjusts the speed, direction, etc. as needed. In intelligent driving systems, most current algorithms can only detect: 1) the flatness of the road surface that has been driven on, such as judging by the sensor changes such as speed and vibration recorded by the instrument during driving; 2) large-scale road surface problems on the road surface that has not been driven on, such as obstacles such as pedestrians and railings, or large-scale damage such as ditch collapse (judged by images, ranging information, etc.). There are omissions in the detection of small-scale defects on horizontal roads that also threaten driving safety.

[0005] The existing algorithms in intelligent driving systems have a large-scale detection scale for road problems ahead of the vehicle, such as large-scale collapse or pedestrian-level road obstacles. They are unable to accurately identify minor road defects and cannot better ensure driving comfort and safety. Summary of the invention

[0006] In view of the above problems, the present invention proposes a road surface flatness detection method and device based on image matching combined with point cloud data, aiming to solve the current problem of being unable to accurately identify tiny road surface defects and being unable to better ensure driving comfort and safety.

[0007] In a first aspect, an embodiment of the present invention provides a road surface flatness detection method based on image matching combined with point cloud data, comprising:

[0008] Acquire three images of the road surface to be inspected at the same time through a three-eye camera; the three lenses of the three-eye camera have consistent internal parameters and are located at the same height;

[0009] The road surface area is detected on the image obtained by the central camera based on a preset scene segmentation algorithm;

[0010] The central camera image and the left and right cameras are used for two verifications respectively, and the disparity comparison and matching of the three images is performed pixel by pixel in the road area;

[0011] When the number of mismatched pixels is greater than a first preset threshold, acquiring point cloud data of the road surface to be detected at the same time through a laser radar; the mismatched pixels are non-flat pixels;

[0012] The distance difference of the same horizontal line of the uneven pixels is calculated. When the distance difference is greater than or equal to a second preset threshold, the mean of the variance of the distance difference between the points on each horizontal line is fed back to the driver or the intelligent driving program for road hazard warning.

[0013] In one embodiment, two verifications are performed using the center camera image and the left and right cameras respectively, and the disparity comparison and matching of the three images is performed pixel by pixel in the road surface area, including:

[0014] A coordinate system is constructed with the driving direction as the X-axis, the road plane as the XY plane, and the vertical road direction as the Z-axis; the ground point vertically corresponding to the central camera is the origin O;

[0015] Suppose the coordinates of a point on the horizontal road in the center camera image are (X m , Y m ), then the imaging coordinates of the left and right cameras are expressed as (-w / h*Y m +X m , Y m ) and (w / h*Y m +X m , Y m ), where w represents the distance between the left and right cameras and the center camera; h represents the distance between the three cameras and the ground;

[0016] Calculate the coordinate position of each pixel in the left and right images according to the coordinates of the central camera, and compare the pixel value difference of the corresponding coordinate positions.

[0017] In one embodiment, the coordinate position of each pixel point in the left and right images is calculated according to the coordinates of the central camera, and the pixel value difference of the corresponding coordinate positions is compared, including:

[0018] Assume that the pixel value of the center camera image point (x, y) is I 中 (x,y), then calculate the mean difference between its pixel values ​​and the corresponding pixels on the left and right:

[0019]

[0020] (4) In the formula, I 左 (-w / h*y+x,y) represents the imaging coordinates of the image point (x,y) on the left camera; I 右 (w / h*y+x,y) represents the imaging coordinates of the image point (x,y) on the right camera.

[0021] In one embodiment, calculating the distance difference of the same horizontal line of the non-flat pixels includes:

[0022] The non-planar pixels are divided into a number of connected domains, and for each connected domain, the distance difference of the same horizontal line where the non-planar pixels exist is calculated using the laser point cloud data.

[0023] In one embodiment, the method further includes: when the distance difference is less than a second preset threshold, determining that the road surface to be detected is a slope, and performing a slope reminder.

[0024] In one embodiment, the mean of the variance of the distance difference between points on each horizontal line is fed back to the driver or the intelligent driving program for road hazard warning, including:

[0025] The road undulations are divided into several levels according to the mean of the variance of the distance differences between points on each horizontal line; the corresponding levels are fed back to the driver or the intelligent driving program for road hazard warning.

[0026] In a second aspect, an embodiment of the present invention provides a road surface flatness detection device based on image matching combined with point cloud data, comprising:

[0027] A first acquisition module is used to acquire three images of the road surface to be detected at the same time through a three-eye camera; the three lenses of the three-eye camera have consistent internal parameters and are located at the same height;

[0028] A segmentation module, used for detecting a road surface area on an image obtained by a central camera based on a preset scene segmentation algorithm;

[0029] A matching module, for performing two verifications using the center camera image and the left and right cameras respectively, and performing pixel-by-pixel parallax comparison and matching of the three images in the road surface area;

[0030] A second acquisition module is used to acquire point cloud data of the road surface to be detected at the same time through a laser radar when the number of mismatched pixels is greater than a first preset threshold; the mismatched pixels are non-flat pixels;

[0031] The detection feedback module is used to calculate the distance difference of the same horizontal line of the uneven pixels. When the distance difference is greater than or equal to a second preset threshold, the mean of the variance of the distance difference between the points on each horizontal line is fed back to the driver or the intelligent driving program for road hazard warning.

[0032] In one embodiment, the matching module includes:

[0033] A construction submodule is used to construct a coordinate system with the driving direction as the X-axis, the road plane as the XY plane, and the vertical road direction as the Z-axis; the ground point vertically corresponding to the central camera is the origin O;

[0034] The matching submodule is used to set the coordinates of a point on the horizontal road surface in the center camera image as (X m ,Y m ), then the imaging coordinates of the left and right cameras are expressed as (-w / h*Y m +X m ,Y m ) and (w / h*Y m +X m ,Y m ), where w represents the distance between the left and right cameras and the center camera; h represents the distance between the three cameras and the ground; the coordinate position of each pixel point in the left and right images is calculated according to the coordinates of the center camera, and the pixel value difference of the corresponding coordinate position is compared.

[0035] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0036] The embodiment of the present invention provides a road surface roughness detection method based on image matching combined with point cloud data. The method performs road surface roughness detection based on three-eye camera image matching combined with lidar point cloud data and gives corresponding warnings. When used to assist manual driving, the method can remind the driver of the road conditions ahead so that the driver can make adjustments in time to prevent vehicle bumps or danger caused by the driver's negligence of observation. When used in an automatic driving system, the decision-making module can be warned to facilitate the system to autonomously adjust the vehicle speed and ensure driving comfort and safety.

[0037] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0040] Figure 1A flow chart of a road surface roughness detection method based on image matching combined with point cloud data provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the road surface flatness detection principle provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the flatness detection principle of a three-eye camera provided in an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the installation position of the device provided in the embodiment of the present invention;

[0044] Figure 5 A block diagram of a road surface flatness detection device based on image matching combined with point cloud data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] Reference Figure 1 As shown, the road surface flatness detection method based on image matching combined with point cloud data provided by the embodiment of the present invention includes:

[0047] S100, acquiring three images of the road surface to be inspected at the same time through a three-eye camera; the three lenses of the three-eye camera have consistent internal parameters and are located at the same height;

[0048] S200, detecting a road surface area on an image obtained by a central camera based on a preset scene segmentation algorithm;

[0049] S300, performing two verifications using the center camera image and the left and right cameras respectively, and performing disparity comparison and matching of the three images pixel by pixel in the road surface area;

[0050] S400, when the number of mismatched pixels is greater than a first preset threshold, acquiring point cloud data of the road surface to be detected at the same time through a laser radar; the mismatched pixels are non-flat pixels;

[0051] S500, calculating the distance difference of the uneven pixels on the same horizontal line, and when the distance difference is greater than or equal to a second preset threshold, feeding back the mean of the variance of the distance difference between points on each horizontal line to the driver or the intelligent driving program for road hazard warning.

[0052] In this embodiment, the above steps are not in a strict execution order. For example, in actual applications, the acquisition processes in S100 and S400 are performed simultaneously; wherein the three-eye camera and the lidar work in collaboration to simultaneously acquire image information and point cloud data of the road environment.

[0053] This method can be applied in intelligent driving systems to detect the flatness of the road ahead of the vehicle. The equipment involved includes, for example, a vehicle-mounted three-eye camera, a laser radar, and a data processing module (algorithm chip, etc.). Figure 2 As shown in the figure, the three-eye camera is used to detect the uneven part of the road surface, and the laser radar is used to estimate the unevenness of the road surface. During operation, the three lenses of the three-eye camera (with consistent internal parameters) are located at the same height. The scene segmentation algorithm is first used to detect the road surface area on the central camera image, and then the possible uneven area is calculated by comparing and matching the disparity of the three images pixel by pixel in the road surface area. If the number of mismatched pixels is greater than the first preset threshold, the pixel fluctuation degree of the uneven part (the connected domain composed of uneven pixels) is further detected by the point cloud information of the laser radar, that is, the distance difference of the same horizontal line of the uneven pixels is calculated. When the distance difference is greater than or equal to the second preset threshold, the mean of the variance of the distance difference of the points on each horizontal line is fed back to the driver or the intelligent driving program for road danger warning. It is possible to predict the flatness of the road surface that the vehicle is about to drive on, and to warn the driver or the automatic driving system, which helps to improve the comfort and safety of driving.

[0054] The above-mentioned preset scene segmentation algorithm, for example, can be a model obtained by training on a training set (such as Cityscapes) based on a fully convolutional neural network; it can also be other segmentation models, as long as the algorithm can accurately distinguish between road and non-road areas, and the embodiments of the present disclosure are not limited to this. The above-mentioned first preset threshold is, for example, an empirical value obtained by manual judgment during the training phase, which can maximize the possibility of distinguishing between flat and uneven roads; similarly, the specific value of the first preset threshold is not limited by the embodiments of the present disclosure; it can be adjusted accordingly according to the specific training algorithm and training effect.

[0055] Combine the following Figure 2-4 Each of the above steps is described in detail.

[0056] 1) The three-eye camera captures three images at the same time and transmits them to the data processing center.

[0057] 2) The data processing center first uses a fully convolutional neural network to perform semantic segmentation based on the images taken by the central camera to determine the road area.

[0058] 3) After obtaining the road area, the data center uses a matching algorithm to verify the matching of pixels in the road area. It uses the center camera image and the left and right cameras for two verifications respectively, and classifies pixels with a mismatch number greater than a first preset threshold as non-horizontal pixels. The threshold here is an empirical value observed during the algorithm debugging process.

[0059] 4) The data processing center counts the number of uneven pixels. If the number is small (for example, the number of pixels in each connected uneven area accounts for less than 1% of the number of pixels in the road area of ​​the entire map), it is considered that the road ahead is level and has good flatness, and no warning is needed.

[0060] 5) If the data center believes that an early warning is necessary, the point cloud data of the laser radar is further introduced to estimate the severity. First, the non-flat pixels are divided into several connected domains. For each connected domain, the laser point cloud data is used to calculate the distance difference of the same horizontal line where the non-flat pixels exist. If there is basically no difference (for example, the distance variance of the points on the horizontal line is less than 5mm), for example, when the distance difference is less than the second preset threshold, the road surface is judged to be a ramp (the road surface is not level but not bumpy), and a ramp reminder is issued; if the distance fluctuations are large, that is, the distance difference is greater than or equal to the second preset threshold, the fluctuations are divided into three levels: weak, moderate, and severe according to the mean of the distance difference variance of the points on each horizontal line, and feedback is given to the driver or the intelligent driving program for road hazard warning.

[0061] Furthermore, in the above step 3), based on the pixel matching principle of the three-eye camera:

[0062] like Figure 3 As shown in the figure, assuming that the focal length of the three homogeneous lenses F1, F2, and F3 of the three-eye camera is f, a coordinate system is established with the driving direction as the x-axis, the road plane as the xy plane, and the vertical road direction as the z-axis (the ground point perpendicular to the center camera is the origin O). If the road ahead is horizontal and flat, the z coordinate of the road surface point A is 0, otherwise it is similar to the undulating point B: the z-axis coordinate is non-0. The center camera F2 is located on the y-axis. Assuming that the distance between the left and right cameras and the center camera is w, and the distance between the three cameras and the ground is h, the three lenses can be abstracted as three points (0,-w,h), (0,0,h), and (0,w,h), and the center points of their imaging focal planes are (-f,-w,h), (-f,0,h), and -f,-w,h).

[0063] Assume that the horizontal road surface is level point A(x A ,y A ,0) The image point coordinates on the three imaging planes are A1(x1,y1,z1), A2(x2,y2,z2) and A3(x3,y3,z3), then:

[0064]

[0065]

[0066]

[0067] Where x1=x2=x3=-f, the three-dimensional space coordinates of the image points A1A2A3 are: A1(-f,-wf(w+y A ) / x A ,h+hf / x A )、A2(-f,-fy A / x A ,h+hf / x A )、A3(-f,wf(y A -w) / x A ,h+hf / x A ).

[0068] The three images taken take the center point of the imaging plane as the origin, and three image points of A can be obtained. The two-dimensional coordinates are:

[0069]

[0070]

[0071] If the road pixel is flat, set the image coordinates of the image point in the center camera image as (X m ,Y m ), then the imaging coordinates of the left and right cameras can be expressed as (-w / h*Y m +X m ,Y m ) and (w / h*Y m +X m ,Y m ).

[0072] In the pixel matching algorithm, assuming that the street ahead is flat, the above formula is used to calculate the coordinate position of each pixel point in the left and right images according to the coordinates of the center camera, and the pixel value difference of the corresponding coordinate position is compared (the absolute value difference of the corresponding pixels in the center left and center right images is compared and the average is taken). That is, assuming that the pixel value of the center camera image point (x, y) is I 中 (x,y), then calculate the mean difference between its pixel values ​​and the corresponding pixels on the left and right:

[0073]

[0074] If the difference is small, it is considered to be a corresponding image point. If the difference is large, it is considered that the pixel is not located on the plane road surface, and the pixel point is marked as mismatched. After calculating the mismatched pixels of the entire image, the connected domain detection is performed on the mismatched pixel area, and the smaller connected domains (the number of pixels is less than 1% of the road surface pixels) are screened out, and the lidar ranging information is read for further judgment.

[0075] Furthermore, in the above step 5), the danger judgment based on the laser radar point cloud data is:

[0076] If the pixel matching algorithm believes that a certain number of pixels are not on the horizontal road surface, then according to the correspondence between the central camera image and the radar data (calculated based on the hardware position distribution and internal parameters), further read the information of the laser radar point cloud data in the segmented road area. Obtain from the point cloud data whether there is a large distance difference between the points on the corresponding horizontal lines in the connected domain of the mismatched pixels. If there is a large-scale pixel mismatch in front, and there is almost no difference in the point cloud distance on each horizontal line (small distance value variance), it is judged that there is a ramp ahead; if the distance on the horizontal line corresponding to the mismatched pixels fluctuates greatly, it is considered that there is an uneven defect in the road surface, and the unevenness is classified into three severity levels according to the size of the distance variance mean, and then a corresponding warning is given.

[0077] The road surface flatness detection method based on image matching combined with point cloud data provided in an embodiment of the present invention performs road surface flatness detection based on three-eye camera image matching combined with laser radar point cloud data, which can predict the integrity of the road surface that the vehicle is about to drive on and give corresponding warnings; when used to assist manual driving, it can remind the driver of the road conditions ahead so that the driver can make timely adjustments to prevent vehicle bumps or danger caused by the driver's negligence of observation; when used in an automatic driving system, it can give a warning to the decision-making module, so that the system can autonomously adjust the vehicle speed to ensure driving comfort and safety.

[0078] Based on the same inventive concept, an embodiment of the present invention also provides a road surface flatness detection device based on image matching combined with point cloud data. Since the principle of the problem solved by the device is similar to the aforementioned road surface flatness detection method based on image matching combined with point cloud data, the implementation of the device can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0079] The embodiment of the present invention provides a road surface flatness detection device based on image matching combined with point cloud data, such as Figure 5 As shown, including:

[0080] A first acquisition module 51 is used to acquire three images of the road surface to be inspected at the same time through a three-eye camera; the three lenses of the three-eye camera have consistent internal parameters and are located at the same height;

[0081] A segmentation module 52, configured to detect a road surface area on an image obtained by a central camera based on a preset scene segmentation algorithm;

[0082] A matching module 53, for performing two verifications using the center camera image and the left and right cameras respectively, and performing disparity comparison and matching of the three images pixel by pixel in the road surface area;

[0083] A second acquisition module 54 is used to acquire point cloud data of the road surface to be detected at the same time through a laser radar when the number of mismatched pixels is greater than a first preset threshold; the mismatched pixels are non-flat pixels;

[0084] The detection feedback module 55 is used to calculate the distance difference of the same horizontal line of the uneven pixels. When the distance difference is greater than or equal to a second preset threshold, the mean of the variance of the distance difference between the points on each horizontal line is fed back to the driver or the intelligent driving program for road hazard warning.

[0085] In one embodiment, the matching module 53 includes:

[0086] A construction submodule 531 is used to construct a coordinate system with the driving direction as the X-axis, the road plane as the XY plane, and the vertical road direction as the Z-axis; the ground point vertically corresponding to the central camera is the origin O;

[0087] The matching submodule 532 is used to set the coordinates of a point on the horizontal road surface in the center camera image as (X m ,Y m ), then the imaging coordinates of the left and right cameras are expressed as (-w / h*Y m +X m ,Y m ) and (w / h*Y m +X m ,Y m ), where w represents the distance between the left and right cameras and the center camera; h represents the distance between the three cameras and the ground; the coordinate position of each pixel point in the left and right images is calculated according to the coordinates of the center camera, and the pixel value difference of the corresponding coordinate position is compared.

[0088] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0089] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A road surface roughness detection method based on image matching combined with point cloud data, characterized in that: include: Acquire three images of the road surface to be inspected at the same time through a three-eye camera; the three lenses of the three-eye camera include a left camera, a center camera and a right camera, the three lenses have the same internal parameters and are located at the same height; The road surface area is detected on the image obtained by the central camera based on a preset scene segmentation algorithm; The central camera image and the left and right cameras are used for two verifications respectively, and the disparity comparison and matching of the three images is performed pixel by pixel in the road area; When there is a large difference between the pixel value of the image point of the central camera and the pixel value of the corresponding image point of the left and right cameras, it is considered that the image point is not located on the plane road surface, and the image point is marked as mismatched; when the number of mismatched pixels is greater than the first preset threshold, the point cloud data of the road surface to be detected at the same time is obtained through the laser radar; the mismatched pixels are non-flat pixels; wherein the three-eye camera and the laser radar work together to simultaneously obtain image information and point cloud data of the road environment; The uneven pixels are divided into several connected domains. For each connected domain, the distance difference of the same horizontal line where the uneven pixels exist is calculated using the laser point cloud data. When the distance difference is greater than or equal to a second preset threshold, the mean of the variance of the distance difference between the points on each horizontal line is fed back to the driver or the intelligent driving program for road hazard warning.

2. The road surface roughness detection method based on image matching combined with point cloud data according to claim 1, characterized in that: Two verifications are performed using the center camera image and the left and right cameras respectively, and the disparity comparison and matching of the three images is performed pixel by pixel in the road area, including: A coordinate system is constructed with the driving direction as the X-axis, the road plane as the XY plane, and the vertical road direction as the Z-axis; the ground point vertically corresponding to the central camera is the origin O; Suppose the coordinates of a point on the horizontal road in the center camera image are (X m ,Y m ), then the imaging coordinates of the left and right cameras are expressed as (-w / h*Y m +X m ,Y m ) and (w / h*Y m +X m ,Y m ), where w represents the distance between the left and right cameras and the center camera; h represents the distance between the three cameras and the ground; Calculate the coordinate position of each pixel in the left and right images according to the coordinates of the central camera, and compare the pixel value difference of the corresponding coordinate positions.

3. The road surface roughness detection method based on image matching combined with point cloud data as claimed in claim 2, characterized in that: Calculate the coordinate position of each pixel in the left and right images according to the center camera coordinates, and compare the pixel value difference of the corresponding coordinate positions, including: Assume that the pixel value of the center camera image point (x, y) is I 中 (x,y), then calculate the mean difference between its pixel values ​​and the corresponding pixels on the left and right: (4) In the formula, I 左 (-w / h*y+x,y) represents the imaging coordinates of the image point (x,y) on the left camera; I 右 (w / h*y+x,y) represents the imaging coordinates of the image point (x,y) on the right camera.

4. The road surface roughness detection method based on image matching combined with point cloud data as claimed in claim 1, characterized in that: Also includes: When the distance difference is less than a second preset threshold, it is determined that the road surface to be detected is a slope, and a slope reminder is performed.

5. The road surface roughness detection method based on image matching combined with point cloud data as claimed in claim 1, characterized in that: According to the mean of the variance of the distance difference between points on each horizontal line, feedback is given to the driver or the intelligent driving program to issue a road hazard warning, including: The road undulations are divided into several levels according to the mean of the variance of the distance differences between points on each horizontal line; the corresponding levels are fed back to the driver or the intelligent driving program for road hazard warning.

6. A road surface roughness detection device based on image matching combined with point cloud data, characterized in that: include: A first acquisition module is used to acquire three images of the road surface to be detected at the same time through a three-eye camera; the three lenses of the three-eye camera include a left camera, a center camera and a right camera, and the three lenses have consistent internal parameters and are located at the same height; A segmentation module, used for detecting a road surface area on an image obtained by a central camera based on a preset scene segmentation algorithm; A matching module, for performing two verifications using the center camera image and the left and right cameras respectively, and performing pixel-by-pixel parallax comparison and matching of the three images in the road surface area; The second acquisition module is used to, when there is a large difference between the pixel value of the image point of the central camera and the pixel value of the corresponding image point of the left and right cameras, consider that the image point is not located on the flat road surface, and mark the image point as mismatched; when the number of mismatched pixels is greater than the first preset threshold, obtain the point cloud data of the road surface to be detected at the same time through the laser radar; the mismatched pixels are non-flat pixels; wherein the three-eye camera and the laser radar work together to simultaneously obtain image information and point cloud data of the road environment; The detection feedback module is used to divide the uneven pixels into several connected domains. For each connected domain, the laser point cloud data is used to calculate the distance difference of the same horizontal line where the uneven pixels exist. When the distance difference is greater than or equal to a second preset threshold, the mean of the variance of the distance difference between the points on each horizontal line is fed back to the driver or the intelligent driving program for road hazard warning.

7. The road surface flatness detection device based on image matching and point cloud data according to claim 6, characterized in that: The matching module comprises: A construction submodule is used to construct a coordinate system with the driving direction as the X-axis, the road plane as the XY plane, and the vertical road direction as the Z-axis; the ground point vertically corresponding to the central camera is the origin O; The matching submodule is used to set the coordinates of a point on the horizontal road surface in the center camera image as (X m ,Y m ), then the imaging coordinates of the left and right cameras are expressed as (-w / h*Y m +X m ,Y m ) and (w / h*Y m +X m ,Y m ), where w represents the distance between the left and right cameras and the center camera; h represents the distance between the three cameras and the ground; the coordinate position of each pixel point in the left and right images is calculated according to the coordinates of the center camera, and the pixel value difference of the corresponding coordinate position is compared.

Images