A vehicle-mounted road pothole identification method and system, and a vehicle

Abstract

The present application relates to a kind of vehicle-mounted road pothole identification method, system and car, method is in at:Camera real-time data acquisition front road;Judge whether there is the vehicle that masks front road;If there is no masking vehicle, it is based on the road image information in front, judge whether there is pothole in front road;If there is masking vehicle, it is based on the jitter amplitude of masking vehicle, judge whether there is pothole in front road.The present application is based on visual recognition technology, based on real-time road condition pothole judgment, with good real-time, simultaneously, road condition is divided into two kinds of scenes, and the identification mode is distinguished for two kinds of scenes, improve the accuracy of identification, and then make the present method and system can be under all road conditions Road pothole risk analysis and early warning, improve the comfort and safety on pothole section when automatic driving.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically to image recognition technology. Background Technology

[0002] With the development of intelligent driving technology, more and more vehicles are using IACC (Integrated Adaptive Cruise Control) on the road. However, when these vehicles encounter potholes on the road, due to issues such as low recognition rates, they currently lack effective deceleration or avoidance strategies, leading to serious safety accidents or a poor driving experience. This is especially true on highways, where drivers using autonomous driving functions may have their attention elsewhere. Without effective human intervention and the intelligent driving system's visual perception module failing to effectively recognize potholes, vehicles traveling at high speeds over them can easily cause tire blowouts, loss of control, and other accidents, posing significant safety hazards. Currently, when encountering potholes, drivers are primarily alerted by in-vehicle navigation or high-precision maps that collect road condition information to avoid risks.

[0003] Traffic information updates for in-car navigation systems or high-precision maps require a certain period, resulting in low data real-time performance. Furthermore, some data comes from uploads by car owners, lacking reliable digital support.

[0004] Existing technology proposes a method and system for vehicle-mounted road pothole warning based on big data. The method includes the following steps: during vehicle operation, road image information of the road ahead is collected; when potholes are detected on the road ahead, the collected images of the potholes and their location information are uploaded to a cloud platform; the presence of potholes is determined based on vehicle driving data; when potholes are detected based on vehicle driving data, the vehicle driving data and location information are uploaded to the cloud platform; the cloud platform marks the corresponding area on the map based on the location information; when a vehicle drives into the corresponding area, the cloud platform sends pothole information for that area to the vehicle; when the vehicle is driving in that area, it responds accordingly based on the pothole markings on the map. During vehicle operation, pothole sections can be effectively warned, reducing safety issues in such scenarios, and the cloud platform makes the data more real-time.

[0005] Existing technologies are based on big data, and this solution relies on the big data recorded by the car through potholes to provide warnings. However, it lacks real-time performance and reliability, and cannot avoid newly occurring potholes in a timely manner. Summary of the Invention

[0006] One objective of this invention is to provide a vehicle-mounted road pothole recognition method to solve the problem that existing technologies cannot avoid newly occurring potholes in a timely manner; a second objective is to provide a vehicle-mounted road pothole recognition system; and a third objective is to provide a vehicle.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for identifying potholes on vehicle-mounted roads.

[0009] The camera collects data on the road ahead in real time;

[0010] Determine if there are any obstructing vehicles on the road ahead;

[0011] If there are no obstructing vehicles, determine whether there are potholes on the road ahead based on the road image information;

[0012] If there is a vehicle obscuring the road, the presence of potholes on the road ahead can be determined based on the vibration amplitude of the obscuring vehicle.

[0013] Based on the above technical means, this method is based on image recognition technology and divides the scene into vehicles with and without occlusion, and responds to the two scenes, which improves the accuracy and reliability of recognition. At the same time, image recognition technology is based on the current road conditions and has good real-time performance, which helps vehicles avoid potholes in advance.

[0014] Furthermore, if there are no obstructing vehicles, the method for determining whether there are potholes on the road ahead based on the road image information is as follows:

[0015] Acquire an image of the road ahead and process the image into grayscale;

[0016] The contours of areas where grayscale values ​​change abruptly are obtained using edge detection algorithms;

[0017] Obtain the average depth value of the grayscale value abruptly changed. If the average depth value is greater than or equal to a first set value, the grayscale value abruptly changed is determined to be a pit or depression. If the average depth value is less than the first set value, the grayscale value abruptly changed is determined not to be a pit or depression.

[0018] Based on the aforementioned technical means, the accuracy of recognition is improved in scenarios where there is no vehicle obstruction.

[0019] Furthermore, the method for determining whether there are potholes on the road ahead based on the vibration amplitude of the obstructing vehicle if there is an obstructing vehicle is as follows:

[0020] Acquire video from the camera in front of you, read each frame of the video and display it;

[0021] If the occluded vehicle in each frame of the image is selected, and the following formula is satisfied, then it is considered that there are potholes in the road ahead; otherwise, there are no potholes.

[0022] H n+1 -H n ≥H 设定 ,

[0023] Among them, H n =L n1 -L n2 ;

[0024] L n1 This indicates the distance between the leftmost and rightmost points of the obscured vehicle.

[0025] L n2 This indicates the distance between the topmost and bottommost points of the obscured vehicle.

[0026] n represents the number of frames;

[0027] H 设定 This indicates the second set value.

[0028] Based on the aforementioned technical means, the accuracy of vehicle recognition has been improved in scenarios where vehicles are obscured.

[0029] Furthermore, if there are potholes ahead, the location of the potholes is transmitted to the autonomous driving controller, which then plans a route to avoid them.

[0030] Furthermore, the planned route to avoid potholes is either A: changing to the right lane to avoid potholes; or B: changing to the left lane to avoid potholes; or C: directly passing potholes by slowing down.

[0031] Furthermore, the autonomous driving controller first determines whether route A can be used. If route A cannot be used, it then determines whether route B can be used. If route B cannot be used, it then determines whether route C can be used.

[0032] Based on the aforementioned technical means, a priority for avoidance methods is set, which helps the autonomous driving controller select the best method to avoid the impact of potholes.

[0033] Furthermore, before occluding the vehicle in each frame of the image, each frame of the image is processed using the following method:

[0034] Each frame of the image is converted to grayscale and binarized, and the edges of the binarized image are eroded or enlarged.

[0035] The aforementioned technical methods can obtain clearer images and improve the accuracy of recognition.

[0036] A vehicle-mounted road pothole recognition system based on the above method includes:

[0037] The judgment module is configured to receive information about the road ahead captured by the camera and determine whether there are any vehicles obstructing the road ahead.

[0038] The processing module is configured to determine whether there are potholes on the road ahead based on the image information of the road ahead if there are no obstructing vehicles on the road ahead; and to determine whether there are potholes on the road ahead based on the shaking amplitude of the obstructing vehicles if there are obstructing vehicles on the road ahead.

[0039] Furthermore, when the processing module detects potholes on the road ahead, it sends the location of the potholes to the autonomous driving controller, which then plans a route to avoid them.

[0040] A car equipped with the aforementioned system.

[0041] The beneficial effects of this invention are:

[0042] This invention is based on visual recognition technology and judges potholes based on real-time road conditions, which has good real-time performance. At the same time, it divides road conditions into two scenarios and distinguishes the recognition methods for the two scenarios, which improves the recognition accuracy. Thus, this method and system can analyze and warn of road pothole risks under all road conditions, better avoid or reduce the impact of potholes, and improve the comfort and safety of autonomous driving on pothole-prone road sections. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method described in this invention;

[0044] Figure 2 This is a schematic diagram of scenario A;

[0045] Figure 3 This is a schematic diagram of scenario B;

[0046] Figure 4 This is a schematic diagram of the system structure of the method described in this invention.

[0047] Among them, 1-camera; 2-judgment module; 3-processing module; 4-autonomous driving controller; 5-actuator. Detailed Implementation

[0048] The following description, with reference to the accompanying drawings and preferred embodiments, illustrates the implementation of the technical solution of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0049] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0050] This embodiment proposes a method for identifying potholes on vehicle-mounted roads, such as... Figure 1 As shown, the specific method is as follows:

[0051] While the vehicle is in motion, it collects real-time data about the road ahead using cameras, at which point the vehicle is in autonomous driving mode.

[0052] By using data collected from the road ahead via cameras, it can be determined whether there are any vehicles obstructing the road ahead.

[0053] If there are no obstructing vehicles, the system determines whether there are potholes on the road ahead based on the road image information.

[0054] Its scene is as follows Figure 2 As shown, the specific method is as follows:

[0055] Acquire an image of the road ahead and process the image into grayscale;

[0056] Edge detection algorithms are applied to images. An image edge is a set of pixels between two adjacent regions in the image; it represents the end of one region and the beginning of another. Alternatively, an image edge can be understood as the set of pixels in an image where their grayscale values ​​undergo a spatial abrupt change. Gradient direction and magnitude are two properties of image edges. Along the direction perpendicular to the edge, the change in pixel value is relatively gradual; while along the direction parallel to the edge, the change in pixel value is relatively large. Therefore, based on this characteristic, methods such as calculating the first or second derivative are typically used to describe and detect image edges.

[0057] The basic idea of ​​image segmentation methods based on edge detection is to first detect the edge pixels in the image, and then connect these edge pixel sets together to form the boundary of the desired target region. Edges in an image can be detected and determined by taking the derivative of grayscale values; however, taking the derivative can be achieved by calculating differential operators. In the field of digital image processing, differential operations are usually approximated by difference operations. This allows for the acquisition of the contours of regions where grayscale values ​​change abruptly.

[0058] Calculate the average depth value of the grayscale abrupt change in the outline of the grayscale abrupt change. If the average depth value is greater than or equal to a first set value, the grayscale abrupt change is determined to be a pit or depression. If the average depth value is less than the first set value, the grayscale abrupt change is determined not to be a pit or depression.

[0059] The first set value can be obtained by setting multiple image samples with pits and bumps, determining the contours of the parts where gray values ​​change abruptly using an edge detection method, calculating the depth value of the contours, and then determining the first set value based on the depth values ​​of all contours.

[0060] If there is a vehicle obscuring the road, the presence of potholes on the road ahead can be determined based on the vibration amplitude of the obscuring vehicle.

[0061] like Figure 3 As shown, the specific method is as follows:

[0062] VideoCapture is used to load local videos, read each frame in a loop, and display them.

[0063] To improve computer processing speed, images are generally converted to grayscale before processing.

[0064] Because color images are 3-channel (RGB) 24-bit images, while grayscale images are single-channel 8-bit images, processing grayscale images is much faster than processing color images. `frontMat` represents the previous frame, and `afterMat` represents the next frame.

[0065] The `threshold` function binarizes an image, operating on the binarized data, primarily targeting the highlighted areas. Algorithms are used to erode the image edges, removing "burrs" from the object's edges. Alternatively, it operates on the binarized data, again focusing on highlighted areas, using algorithms to enlarge the image edges. This essentially fills in the edges or internal pits of the object.

[0066] The principle of selecting the occluded vehicle in each frame of the image is to find the leftmost and rightmost points of the white square and calculate the size difference between them (rectangle width), and find the topmost and bottommost points of the white square and calculate the size difference between them (rectangle height H). Using these dimensions, a rectangle can be drawn that encompasses the white square. The coordinates of the top-left corner of the rectangle are determined by the values ​​of the topmost and leftmost points of the white square.

[0067] If the following formula is met, it indicates that the vibration amplitude of the vehicle obscured in front is large, and it is assumed that there are potholes in the road ahead; otherwise, it indicates that the vibration amplitude is large.

[0068] The surface is relatively flat and there are no potholes.

[0069] Hn+1 -H n ≥H 设定 ,

[0070] Among them, H n =L n1 -L n2 ;

[0071] L n1 This indicates the distance between the leftmost and rightmost points of the obscured vehicle.

[0072] L n2 This indicates the distance between the topmost and bottommost points of the obscured vehicle.

[0073] n represents the number of frames;

[0074] H 设定 This indicates the second set value.

[0075] If a pothole is detected ahead, its location is transmitted to the autopilot controller, which then plans a route to avoid it. The planned routes are: A) changing to the right lane; B) changing to the left lane; or C) directly traversing the pothole by slowing down. The priorities of these methods are as follows (in descending order):

[0076] Route A, Route B, Route C.

[0077] That is, the autonomous driving controller first determines whether route A can be used. If route A cannot be used, it then determines whether route B can be used. If route B cannot be used, it then determines whether route C can be used.

[0078] This embodiment also proposes a vehicle-mounted road pothole recognition system, based on the above method, specifically:

[0079] Includes judgment module 2 and processing module 3.

[0080] The judgment module 2 is configured to receive information about the road ahead captured by the camera 1 and determine whether there are vehicles obstructing the road ahead.

[0081] Processing module 3 is configured to determine whether there are potholes on the road ahead based on the image information of the road ahead if there are no obstructing vehicles on the road ahead; and to determine whether there are potholes on the road ahead based on the shaking amplitude of the obstructing vehicles if there are obstructing vehicles on the road ahead.

[0082] When the processing module 3 detects a pothole on the road ahead, it sends the location of the pothole to the autonomous driving controller 4. The autonomous driving controller 4 then plans a route to avoid the pothole, determines the avoidance measures, and begins to automatically change lanes, issuing instructions to the actuator 5.

[0083] This embodiment also proposes a car equipped with the above-mentioned vehicle-mounted road pothole recognition system.

[0084] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A method for identifying potholes on vehicle-mounted roads, characterized in that: The camera collects data on the road ahead in real time; Determine if there are any obstructing vehicles on the road ahead; If there are no obstructing vehicles, determine whether there are potholes on the road ahead based on the road image information; If there is a vehicle obscuring the road, the presence of potholes on the road ahead can be determined based on the vibration amplitude of the obscuring vehicle. The method for determining whether there are potholes on the road ahead based on the vibration amplitude of the obscured vehicle, if there is an obscured vehicle, is as follows: Acquire video from the camera in front of you, read each frame of the video and display it; If the occluded vehicle in each frame of the image is selected, and the following formula is satisfied, then it is considered that there are potholes in the road ahead; otherwise, there are no potholes. H n+1 -H n ≥H 设定 , Among them, H n =L n1 -L n2 ; L n1 This indicates the distance between the leftmost and rightmost points of the obscured vehicle. L n2 This indicates the distance between the topmost and bottommost points of the obscured vehicle. n represents the number of frames; H 设定 This indicates the second set value.

2. The method according to claim 1, characterized in that: If there are no obstructing vehicles, the method for determining whether there are potholes on the road ahead based on the road image information is as follows: Acquire an image of the road ahead and process the image into grayscale; The contours of areas where grayscale values ​​change abruptly are obtained using edge detection algorithms; Obtain the average depth value of the grayscale value abruptly changed. If the average depth value is greater than or equal to a first set value, the grayscale value abruptly changed is determined to be a pit or depression. If the average depth value is less than the first set value, the grayscale value abruptly changed is determined not to be a pit or depression.

3. The method according to claim 2, characterized in that: If there is a pothole ahead, the location of the pothole is transmitted to the autonomous driving controller, which then plans a route to avoid it.

4. The method according to claim 3, characterized in that: The planned routes to avoid potholes are: A) changing to the right lane; B) changing to the left lane; or C) slowing down to pass directly over the pothole.

5. The method according to claim 4, characterized in that: The autonomous driving controller first determines whether route A can be used. If route A cannot be used, it then determines whether route B can be used. If route B cannot be used, it then determines whether route C can be used.

6. The method according to claim 1, characterized in that: Before selecting each frame of an image to mask the vehicle, each frame of the image is processed using the following method: Each frame of the image is converted to grayscale and binarized, and the edges of the binarized image are eroded or enlarged.

7. A vehicle-mounted road pothole recognition system based on the method described in any one of claims 1-6, characterized in that: It includes a judgment module, configured to receive information about the road ahead captured by the camera and determine whether there are vehicles obstructing the road ahead; The processing module is configured to determine whether there are potholes on the road ahead based on the image information of the road ahead if there are no vehicles obscuring the road ahead. If there are obstructing vehicles on the road ahead, the presence of potholes on the road ahead can be determined based on the vibration amplitude of the obstructing vehicles.

8. The system according to claim 7, characterized in that: When the processing module detects potholes on the road ahead, it sends the location of the potholes to the autonomous driving controller, which then plans a route to avoid them.

9. A car, characterized in that: The system is configured as described in claim 7 or 8.

Images