Machine vision-based road full-period condition real-time monitoring system and method

Abstract

The invention relates to the technical field of visual recognition, in particular to a road full-cycle condition real-time monitoring system and method based on machine vision, and the system comprises an image collection unit which is used for collecting road surface images of a road at different time points and driving images of an automobile; the road surface image processing unit is used for determining an accumulated water contour and an accumulated water position and determining a water flow direction; the driving image processing unit is used for analyzing road bumping frequency and abnormal frequency and determining the vertical displacement distance of the vehicle; the pavement analysis unit is used for coupling to obtain a simulated water flow direction; the driving analysis unit is used for determining a road depression position; the defect analysis unit is used for determining whether road collapse and road depression occur or not and judging whether the road is repaired or not; according to the invention, the above units cooperate with each other, so that the road monitoring efficiency is improved, and the accuracy and reliability of the real-time monitoring system for the full-cycle condition of the road based on machine vision are further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition, and in particular to a real-time monitoring system and method for the full life cycle condition of a road based on machine vision. Background Art

[0002] During the long-term use of a road, it will be affected by the combined action of natural factors (such as rain washing, freeze-thaw cycle, sunlight exposure, etc.) and traffic loads (such as vehicle rolling, impact, etc.), resulting in diseases such as cracks, potholes, and bumps on the road surface. If these diseases cannot be discovered and treated in time, they will gradually deteriorate, seriously affecting the service performance and traffic safety of the road. The monitoring system based on machine vision can detect the road with high frequency and high precision, discover early diseases and potential safety hazards in time, provide accurate basis for the preventive maintenance of the road, thereby reducing the risk of accidents and ensuring the safety of people's lives and property.

[0003] Chinese Patent Application Publication No.: CN114511551A discloses a ground damage recognition system based on machine vision. The invention provides a ground damage recognition system based on machine vision, including a data collection mobile terminal and a human-computer interaction terminal. The data collection mobile terminal includes an unmanned aerial vehicle (UAV) aircraft, a data collection module, and a data transmission module. The invention collects data on the ground to be recognized by the data collection module carried by the UAV aircraft, which is more convenient, time-saving and labor-saving compared with the traditional manual detection method. By performing preprocessing such as conversion, enhancement, and edge detection on the collected ground images, the ground images are made clearer, and then it is convenient for subsequent image feature extraction. By judging the damage of the preprocessed image and identifying the damage type according to the judgment result, the recognition of ground damage based on machine vision is realized. Compared with some existing ground damage recognition systems, the recognition accuracy is higher, and the ground damage situation in the area to be recognized can be accurately recognized, thus bringing effective help to the accurate prediction of accidents such as ground collapse.

[0004] The Chinese patent application publication number: CN117523502A discloses an intelligent monitoring system for urban road garbage based on machine vision. This invention provides an intelligent monitoring system for urban road garbage based on machine vision, including: a data acquisition module for acquiring garbage can images; a planar fitting goodness acquisition module for establishing a feature window of pixel points, obtaining the noise performance degree of pixel points, and obtaining the planar fitting goodness of pixel points; a corrected pixel point acquisition module for determining the median point of pixel points, obtaining the edge influence degree of the median point of pixel points, obtaining the correction possibility of pixel points, and marking corrected pixel points; a corrected median gray level acquisition module for determining the non-central points of corrected pixel points, determining the weights of non-central points, and obtaining the corrected median gray level of corrected pixel points; a garbage monitoring module for denoising the garbage can image according to the corrected median gray level of corrected pixel points, obtaining the denoised garbage can image, and realizing the intelligent monitoring of urban road garbage based on the denoised garbage can image. This invention solves the problem of inaccurate monitoring of urban road garbage.

[0005] However, the above method has the following problems: simply relying on the images taken by drones to judge the road surface conditions is not accurate enough, and simply processing the images fails to well monitor the target. Summary of the Invention

[0006] Therefore, the present invention provides a real-time monitoring system and method for the full-cycle condition of a road based on machine vision to overcome the problems in the prior art that simply relying on the images taken by drones to judge the road surface conditions is not accurate enough, and simply processing the images fails to well monitor the target.

[0007] To achieve the above object, on the one hand, the present invention provides a real-time monitoring system for the full-cycle condition of a road based on machine vision, including:

[0008] An image acquisition unit for acquiring road surface images and vehicle driving images at different time points of the road;

[0009] A road surface image processing unit connected to the image acquisition unit for determining the water accumulation contour and position according to the road surface image, and determining the water flow direction in the state of water accumulation flowing;

[0010] A driving image processing unit connected to the image acquisition unit for determining the driving trajectories and driving postures of each vehicle according to the driving image, analyzing the road bump frequency and abnormal frequency, and determining the vehicle vertical displacement distance according to the driving posture;

[0011] A road surface analysis unit connected to the image acquisition unit for constructing a terrain slope model according to the road surface image and coupling with the positions of road drainage holes to obtain a simulated water flow direction;

[0012] A driving analysis unit, which is respectively connected to the image acquisition unit and the driving image processing unit, and is used to compare the road bump frequency with the abnormal frequency, and determine the road depression position according to the comparison result;

[0013] A defect analysis unit, which is respectively connected to the road surface image processing unit, the road surface analysis unit and the driving analysis unit, and is used to compare the water flow direction with the simulated water flow direction to determine whether a road collapse occurs, and compare the road depression position with the water accumulation position to determine whether a road depression occurs, and combine the vehicle vertical displacement distance and the water accumulation contour to construct a road depression estimation model to determine whether to repair the road. The degree of damage to the road caused by the road collapse is greater than that of the road depression.

[0014] Further, the road surface image processing unit includes:

[0015] A water accumulation image sub-unit, which is connected to the image acquisition unit, and is used to determine whether the reflective area in the road surface image is a road reflective area or a water accumulation reflective area, and determine the water accumulation contour and the water accumulation position according to the water accumulation reflective area;

[0016] A flowing water image sub-unit, which is connected to the image acquisition unit, and is used to determine the water flow direction according to the road surface image.

[0017] Further, the driving image processing unit includes:

[0018] A driving frequency sub-unit, which is connected to the image acquisition unit, and is used to count the bump frequencies corresponding to different driving speeds of each vehicle on the driving track, draw a road bump frequency curve graph, and select the bump frequency of the outlier in the road bump frequency curve graph as the abnormal frequency;

[0019] A bump displacement sub-unit, which is connected to the image acquisition unit, and is used to determine the edge length of the fuzzy area in the vertical direction of the corresponding vehicle according to the driving posture of each vehicle, and calculate the vehicle vertical displacement distance.

[0020] Further, the road surface analysis unit extracts the edge information of the road in different road surface images, and generates the terrain slope model by surface fitting based on the parallax of the corresponding points in different road surface images.

[0021] Further, the driving analysis unit calculates the growth rate of the abnormal frequency relative to the road bump frequency at the same speed, and determines whether the road position corresponding to the abnormal frequency is a depression position according to the comparison result between the growth rate and the preset growth rate. Among them,

[0022] If the growth rate is greater than or equal to the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is the sunken position;

[0023] The preset growth rate is positively correlated with the vehicle driving speed.

[0024] Furthermore, the defect analysis unit includes:

[0025] A collapse analysis subunit, which is respectively connected to the road surface image processing unit and the road surface analysis unit, compares the included angle between the water flow direction and the simulated water flow direction with a preset included angle, and determines whether a road collapse has occurred according to the comparison result.

[0026] Furthermore, the defect analysis unit further includes:

[0027] A depression analysis subunit, which is connected to the road surface image processing unit and the driving analysis unit, is used to compare the road depression position and the water accumulation position to determine whether a road depression has occurred. Among them,

[0028] If the road depression position and the water accumulation position overlap each other, it is determined that a road depression has occurred.

[0029] Furthermore, the defect analysis unit further includes:

[0030] A repair analysis subunit, which is respectively connected to the road surface image processing unit, the driving image processing unit and the depression analysis subunit, is used to construct a road depression estimation model by combining the vehicle vertical displacement distance and the water accumulation contour, and calculate the depression volume according to the road depression estimation model.

[0031] Furthermore, the repair analysis subunit compares the depression volume with a preset volume, and determines whether to repair the road according to the comparison result. Among them,

[0032] If the depression volume is less than the preset volume, it is determined not to repair;

[0033] If the depression volume is greater than or equal to the preset volume, it is determined to repair the road;

[0034] The preset volume is positively correlated with the size of the water accumulation contour area.

[0035] On the other hand, the present invention provides a real-time monitoring method for the whole life cycle condition of a road based on machine vision, including:

[0036] Collect road surface images and vehicle driving images at different time points of the road;

[0037] Determine the water accumulation contour and position based on the road surface image, and determine the water flow direction in the state where the water accumulation is flowing;

[0038] Determine the driving trajectories and postures of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, determine the vehicle vertical displacement distance according to the driving posture, construct a terrain slope model based on the road surface image, and couple with the position of the road drainage hole to obtain the simulated water flow direction;

[0039] Compare the road bump frequency with the abnormal frequency, determine the pending road depression position according to the comparison result, and compare the water flow direction with the simulated water flow direction to determine whether a road collapse has occurred;

[0040] Compare the pending road depression position with the water accumulation position to determine whether a road depression has occurred, and construct a road depression estimation model in combination with the vehicle vertical displacement distance and the water accumulation contour to determine whether to repair the road. The degree of damage to the road caused by the road collapse is greater than that of the road depression.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows. By collecting road surface images and driving images of the vehicle, the present invention determines the flatness of the road according to the state of water accumulation on the road. It does not need to be in direct contact with the road surface like traditional flatness detection methods (such as using equipment like a flatness meter), and can obtain road surface and water accumulation information only through an image acquisition device (such as a camera). This greatly reduces the installation and operation complexity of the detection equipment, can perform detection synchronously during the normal driving of the vehicle, does not affect the normal traffic, improves the detection efficiency, can cover a long road mileage in a short time, reduces the detection cost and time cost, can collect road surface images in real time, and timely discovers changes in road flatness. Once the road flatness anomaly caused by water accumulation (such as local depression and water accumulation) occurs, the system can quickly make a judgment and issue an alarm, providing timely road condition information for the road management department. This helps the management department quickly take measures, such as arranging maintenance personnel for inspection and repair, to avoid more serious traffic accidents and road damage caused by road flatness problems. The state of road water accumulation can not only reflect the depression and protrusion of the road surface, but also comprehensively reflect the actual use performance of the road under different weather conditions (especially after rainfall). Compared with simply relying on manual inspection or static detection equipment, this judgment method based on images and water accumulation state can capture more comprehensively the subtle changes in road flatness. The long-term collected road surface images and water accumulation state data can form a huge database. By analyzing and mining these data, the change rules of road flatness can be deeply understood, such as the change trends of road flatness in different sections, different seasons, and different service years. This has important reference value for the formulation and optimization of road design, construction, and maintenance standards, helps to improve the scientificity and standardization of road construction and management, and effectively improves the accuracy and reliability of the real-time monitoring system for the full life cycle condition of the road based on machine vision while comprehensively reflecting the road condition.

[0042] Furthermore, the present invention analyzes the road bump frequency and abnormal frequency by combining the driving trajectory and driving attitude of the vehicle, and then determines the road depression position. By using the driving trajectory and attitude data of the vehicle, it is possible to accurately locate the road depression position. When the vehicle encounters a sunken section during driving, its driving trajectory will deviate and its driving attitude will also change abnormally. By real-time analyzing these data, the specific position of the road depression can be quickly and accurately determined, providing an accurate repair target for road maintenance personnel, improving the pertinence and efficiency of the repair work. By long-term accumulating and analyzing the driving data of a large number of vehicles, the occurrence law and development trend of road depressions in different sections can be grasped. For example, some sections may be more prone to road depression problems due to factors such as geological conditions and traffic flow. Based on these data, the road management department can formulate a more scientific and reasonable road maintenance plan, reasonably arrange maintenance funds and manpower, and conduct preventive maintenance on potential road depression risk areas in advance to avoid further deterioration of the road conditions and reduce the road maintenance cost. The relevant road depression data can be shared, providing valuable references for fields such as urban planning and traffic research. The urban planning department can optimize the road design and construction plan according to the distribution of road depressions, improve the overall quality of urban roads, and further enhance the accuracy and reliability of the real-time monitoring system for the whole life cycle condition of roads based on machine vision.

[0043] Furthermore, the present invention mutually verifies the road depression position analyzed by the vehicle with the water accumulation position on the directly captured road surface image to determine the position where the road has a depression. Through the mutual verification of the two, the limitations of a single method can be effectively reduced. The two different monitoring methods corroborate each other, increasing the diversity of data sources. When one method shows abnormalities or malfunctions, the other method can be used as a supplement to ensure the continuous progress of the monitoring work for the road depression position. The depression position analyzed by the vehicle focuses on reflecting the unevenness of the road from the perspective of the vehicle driving experience, while the water accumulation position on the road surface image intuitively shows the geometric shape changes of the road surface. Combining the two can provide a more comprehensive understanding of the actual situation of the road depression, including the depth, scope of the depression, and the impact of water accumulation on the depression. An accurate judgment of the depression position can make the road maintenance work more targeted, avoiding blind large-scale inspections and repairs. Based on the depression position determined through mutual verification, the maintenance personnel can directly go to the problem area for repair, reducing the waste of manpower, material resources, and time, and lowering the road maintenance cost. By mutually verifying the two methods, the accuracy of the depression position judgment is improved, which means that road problems can be dealt with more promptly, providing better guarantee for driving safety, and further enhancing the accuracy and reliability of the real-time monitoring system for the whole life cycle condition of roads based on machine vision.

[0044] Furthermore, the present invention constructs a road depression estimation model by analyzing the vertical displacement distance of the vehicle when encountering bumps and the water accumulation contour. The volume of the road depression is calculated by the model to determine whether to repair the road. This model comprehensively considers the vertical displacement distance of the vehicle when bumping and the water accumulation contour, and can more comprehensively, objectively and accurately evaluate the degree of road depression. The vertical displacement distance of the vehicle directly reflects the impact degree of the depression on vehicle driving, while the water accumulation contour shows the geometric shape and scope of the depression from the side. By combining the two, the volume of the depression can be accurately calculated, and this can be used as a quantitative index to more scientifically judge whether the road depression has reached the degree that needs to be repaired, avoiding the situations of over-repair or untimely repair. Road maintenance resources (such as human, material and financial resources) are usually limited. After accurately calculating the volume of the road depression by using this model, the depressions can be sorted according to the severity, and the depression areas with larger volume and serious impact on driving safety can be repaired preferentially. In this way, the limited maintenance resources can be more reasonably allocated, the resource utilization efficiency can be improved, and the resources can be prevented from being wasted on some minor depressions that do not need to be repaired immediately. The volume data of the road depression calculated by this model can provide valuable reference for road planning and design departments. By analyzing the depression volume of different road sections, the areas and reasons where road depressions are likely to occur during road use can be understood, such as the influence of geological conditions, traffic flow and other factors. When planning and designing new roads, corresponding measures can be taken to optimize the road structure and design parameters, improve the durability and stability of the road, and reduce the occurrence of future road diseases such as road depressions, further enhancing the accuracy and reliability of the real-time monitoring system for the whole life cycle condition of the road based on machine vision. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is the structural block diagram of the real-time monitoring system for the whole life cycle condition of the road based on machine vision of the present invention;

[0046] Figure 2 is the structural block diagram of the road surface image processing unit in the embodiment of the present invention;

[0047] Figure 3 is the determination diagram for determining the position of the road depression in the embodiment of the present invention;

[0048] Figure 4 is the flow chart of the real-time monitoring method for the whole life cycle condition of the road based on machine vision of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] In order to make the purpose and advantages of the present invention clearer and more understandable, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principle of the present invention and do not limit the protection scope of the present invention.

[0051] It should be noted that in the description of the present invention, the terms indicating directions or positional relationships such as "upper", "lower", "left", "right", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention.

[0052] In addition, it should also be noted that in the description of the present invention, unless otherwise clearly specified and defined, the terms "installation", "connection", "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0053] It can be understood that road waterlogging can be caused by rainy weather or road cleaning and sprinkler trucks.

[0054] Please refer to Figure 1 as shown, which is a structural block diagram of the real-time monitoring system for the full-cycle condition of the road based on machine vision of the present invention. The embodiment of the present invention provides a real-time monitoring system for the full-cycle condition of the road based on machine vision, including:

[0055] An image acquisition unit for acquiring road surface images and vehicle driving images at different time points of the road;

[0056] A road surface image processing unit connected to the image acquisition unit for determining the water accumulation contour and position according to the road surface image, and determining the water flow direction in the state of water accumulation flowing;

[0057] A driving image processing unit connected to the image acquisition unit for determining the driving trajectories and driving postures of each vehicle according to the driving images, analyzing the road bump frequency and abnormal frequency, and determining the vehicle vertical displacement distance according to the driving posture;

[0058] A road surface analysis unit connected to the image acquisition unit for constructing a terrain slope model according to the road surface image and coupling with the position of the road drainage holes to obtain a simulated water flow direction;

[0059] A driving analysis unit, which is respectively connected to an image acquisition unit and a driving image processing unit, and is used to compare the road bump frequency with the abnormal frequency, and determine the road depression position according to the comparison result;

[0060] A defect analysis unit, which is respectively connected to a road surface image processing unit, a road surface analysis unit and a driving analysis unit, and is used to compare the water flow direction with the simulated water flow direction to determine whether a road collapse occurs, and compare the road depression position with the water accumulation position to determine whether a road depression occurs, and combine the vehicle vertical displacement distance and the water accumulation contour to construct a road depression estimation model to determine whether to repair the road. The degree of damage to the road caused by road collapse is greater than that of road depression.

[0061] Specifically, the present invention collects road surface images and driving images of the vehicle, and judges the flatness of the road according to the state of water accumulation on the road. It does not need to be in direct contact with the road surface like traditional flatness detection methods (such as using equipment like a flatness meter), and can obtain road surface and water accumulation information only through an image acquisition device (such as a camera). This greatly reduces the installation and operation complexity of the detection equipment, can perform detection synchronously during the normal driving process of the vehicle, does not affect the normal traffic, improves the detection efficiency, can cover a long road mileage in a short time, reduces the detection cost and time cost, can collect road surface images in real time, and timely discovers changes in the flatness of the road. Once the flatness of the road is abnormal due to water accumulation (such as local depression and water accumulation), the system can quickly make a judgment and issue an alarm, providing timely road condition information for the road management department. This helps the management department quickly take measures, such as arranging maintenance personnel for inspection and repair, to avoid more serious traffic accidents and road damage caused by road flatness problems. The state of road water accumulation can not only reflect the depression and protrusion of the road surface, but also comprehensively reflect the actual use performance of the road under different weather conditions (especially after rainfall). Compared with simply relying on manual inspection or static detection equipment, this judgment method based on images and water accumulation state can capture more subtle changes in road flatness more comprehensively. The long-term collected road surface images and water accumulation state data can form a huge database. By analyzing and mining these data, the change law of road flatness can be deeply understood, such as the change trend of road flatness in different road sections, different seasons, and different service years. This has important reference value for the formulation and optimization of road design, construction and maintenance standards, helps to improve the scientificity and standardization of road construction and management, and effectively improves the accuracy and reliability of the real-time monitoring system for the full life cycle condition of the road based on machine vision while comprehensively reflecting the road condition.

[0062] Please refer to Figure 2 shown, which is a structural block diagram of the road surface image processing unit in the embodiment of the present invention. The road surface image processing unit includes:

[0063] A ponding image subunit, connected to the image acquisition unit, is used to determine whether the reflective area in the road surface image is a road reflection area or a ponding reflection area, and to determine the ponding contour and ponding position according to the ponding reflection area;

[0064] It can be understood that the reflective area in the road surface image can be determined as a road reflection area or a ponding reflection area through image feature analysis. For example, the texture of the reflective area of an asphalt road surface will show fine granularity or irregular patterns. The texture of the ponding reflection area is often smoother because the ponding surface is relatively flat, and there will also be fine ripple textures generated by the water surface fluctuations.

[0065] A flowing water image subunit, connected to the image acquisition unit, is used to determine the water flow direction according to the road surface image.

[0066] It can be understood that for consecutive video image frames, the inter-frame difference method can be used to detect the movement of the water flow. By calculating the difference between two adjacent frames of images, the changed areas are highlighted. These changed areas are the result of the water flow movement. Analyze these changed areas, track the movement trajectory of the water flow, and thus determine the water flow direction.

[0067] Specifically, the driving image processing unit includes:

[0068] A driving frequency subunit, connected to the image acquisition unit, is used to count the bump frequencies corresponding to different driving speeds of each vehicle on the driving trajectory, draw a road bump frequency curve graph, and select the bump frequency of the outlier in the road bump frequency curve graph as the abnormal frequency;

[0069] It can be understood that the bump frequencies corresponding to different driving speeds of each vehicle on the driving trajectory are statistically analyzed, the collected data is checked, and obvious error or abnormal data points are removed, such as data with negative speed values, acceleration values exceeding the reasonable range, etc. According to the data of the acceleration sensor, a bump judgment criterion is defined. For example, when the change in acceleration exceeds a certain threshold (such as 0.5g, where g is the acceleration due to gravity), it is considered that the vehicle has had a bump. For the data points in each driving speed interval, the number of bumps that the vehicle has had within a certain time window (such as 1 minute) is statistically analyzed, and then this number is divided by the length of the time window to obtain the bump frequency of this speed interval (unit: times / minute). The abscissa represents the driving speed of the vehicle, and the ordinate represents the corresponding bump frequency. The average bump frequency of each driving speed interval is plotted on the coordinate graph, and then these points are connected with a curve to form a road bump frequency curve graph. Different curve fitting methods can be selected as needed to make the curve more smooth and accurately reflect the relationship between the bump frequency and the driving speed. According to the selected outlier detection method, the data points in the road bump frequency curve graph are analyzed to find out those outliers that deviate from the normal data distribution. The bump frequencies corresponding to these outliers are the abnormal frequencies.

[0070] The bump displacement sub-unit, which is connected to the image acquisition unit, is used to determine the edge length of the fuzzy area in the vertical direction of the corresponding vehicle according to the driving attitude of each vehicle and calculate the vertical displacement distance of the vehicle.

[0071] It can be understood that the collected images are preprocessed, including grayscale conversion, noise reduction (such as using Gaussian filtering to remove image noise), and contrast enhancement. Based on the driving attitude data of the vehicle and image features, object detection algorithms or image segmentation algorithms are used to identify the fuzzy areas in the vertical direction of the vehicle. For the identified fuzzy areas, edge detection algorithms are used to extract their edges. These algorithms can detect the pixel points with large gray-scale changes in the image, thereby determining the boundaries of the fuzzy areas. According to the extracted edge information, image measurement techniques are used to calculate the edge length of the fuzzy area. During the calculation process, the resolution and scale of the image need to be considered to convert the pixel length into the actual physical length.

[0072] Specifically, the present invention analyzes the road bump frequency and abnormal frequency by combining the driving trajectory and driving attitude of the vehicle, and then determines the position of the road depression. By using the driving trajectory and attitude data of the vehicle, it is possible to accurately locate the position of the road depression. When the vehicle is driving and encounters a sunken section, its driving trajectory will deviate and its driving attitude will also change abnormally. By real-time analyzing these data, the specific position of the road depression can be quickly and accurately determined, providing an accurate repair target for road maintenance personnel, improving the pertinence and efficiency of the repair work. By long-term accumulating and analyzing the driving data of a large number of vehicles, the occurrence rules and development trends of road depressions in different sections can be grasped. For example, certain sections may be more prone to road depression problems due to factors such as geological conditions and traffic flow. Based on these data, the road management department can formulate a more scientific and reasonable road maintenance plan, reasonably arrange maintenance funds and manpower, and conduct preventive maintenance on potential road depression risk areas in advance to avoid further deterioration of the road conditions and reduce the road maintenance cost. The relevant road depression data can be shared, providing valuable references for fields such as urban planning and traffic research. The urban planning department can optimize road design and construction plans according to the distribution of road depressions, improve the overall quality of urban roads, and further enhance the accuracy and reliability of the real-time monitoring system for the full life cycle condition of roads based on machine vision.

[0073] Specifically, the road surface analysis unit extracts the edge information of the road in different road surface images, and generates a terrain slope model by surface fitting based on the parallax of corresponding points in different road surface images.

[0074] Please refer to Figure 3 as shown, which is the determination diagram for determining the position of the road depression in the embodiment of the present invention. The driving analysis unit calculates the growth rate of the abnormal frequency relative to the road bump frequency at the same speed, and determines whether the road position corresponding to the abnormal frequency is the depression position according to the comparison result between the growth rate and the preset growth rate. Among them,

[0075] If the growth rate is greater than or equal to the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is the depression position;

[0076] If the growth rate is less than the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is not the depression position;

[0077] In a specific embodiment, the preset growth rate is set to 20%. If the growth rate is 34% which is greater than the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is the depression position;

[0078] If the growth rate is 14% which is less than the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is not the depression position.

[0079] The preset growth rate is positively correlated with the driving speed of the vehicle.

[0080] It can be understood that the faster the vehicle travels, the greater the bump and the higher the abnormal frequency generated when encountering road potholes. Therefore, the preset growth rate is positively correlated with the vehicle speed.

[0081] Specifically, the defect analysis unit includes:

[0082] A subsidence analysis subunit, which is respectively connected to the road surface image processing unit and the road surface analysis unit, compares the angle between the water flow direction and the simulated water flow direction with a preset angle, and determines whether a road subsidence has occurred according to the comparison result.

[0083] If the angle between the water flow direction and the simulated water flow direction is less than the preset angle, it is determined that no road subsidence has occurred;

[0084] If the angle between the water flow direction and the simulated water flow direction is greater than or equal to the preset angle, it is determined that a road subsidence has occurred.

[0085] It can be understood that according to the image of the water flow, the optical flow method is used to calculate the motion vector of the pixel points in the water flow, so as to obtain the water flow direction.

[0086] In a specific embodiment, the preset angle is set to 45°. If the angle between the water flow direction and the simulated water flow direction is 30°, which is less than the preset angle, it is determined that no road subsidence has occurred;

[0087] If the angle between the water flow direction and the simulated water flow direction is 65°, which is greater than the preset angle, it is determined that a road subsidence has occurred.

[0088] The preset angle is positively correlated with the road slope.

[0089] It can be understood that the greater the road slope, the faster the water flow velocity, the greater the inertia, and the greater the angle between the water flow branch direction and the main road water flow direction at the road subsidence. Therefore, the preset angle is positively correlated with the road slope.

[0090] Specifically, the defect analysis unit further includes:

[0091] A depression analysis subunit, which is connected to the road surface image processing unit and the driving analysis unit, is used to compare the road depression position and the water accumulation position to determine whether a road depression has occurred. Among them,

[0092] If the road depression position and the water accumulation position overlap with each other, it is determined that a road depression has occurred.

[0093] It can be understood that when the overlapping part of the road depression position and the water accumulation position exceeds one-fourth of either of them, it can be determined that a road subsidence has occurred.

[0094] Specifically, the present invention determines the position where the road is sunken by cross-checking the position of the road depression analyzed by the vehicle with the position of the water accumulation on the directly captured road surface image. Through the mutual cross-check of the two, the limitations of a single method can be effectively reduced. The two different monitoring methods corroborate each other, increasing the diversity of data sources. When one method has an anomaly or malfunction, the other method can be used as a supplement to ensure the continuous progress of the monitoring work for the road depression position. The depression position analyzed by the vehicle focuses on reflecting the unevenness of the road from the perspective of the vehicle driving experience, while the position of the water accumulation on the road surface image intuitively shows the geometric shape change of the road surface. Combining the two can provide a more comprehensive understanding of the actual situation of the road depression, including the depth, scope of the depression, and the impact of water accumulation on the sunken area, etc. An accurate judgment of the depression position can make the road maintenance work more targeted, avoiding blind large-scale inspections and repairs. Based on the depression position determined by the mutual cross-check, the maintenance personnel can directly go to the problem area for repair, reducing the waste of manpower, material resources, and time, lowering the road maintenance cost. By cross-checking the two methods, the accuracy of the depression position judgment is improved, which means that road problems can be processed more promptly, providing better protection for driving safety, and further enhancing the accuracy and reliability of the real-time monitoring system for the full-cycle condition of the road based on machine vision.

[0095] Specifically, the defect analysis unit further includes:

[0096] A repair analysis subunit, which is respectively connected to the road surface image processing unit, the driving image processing unit, and the depression analysis subunit, and is used to construct a road depression estimation model by combining the vehicle vertical displacement distance and the water accumulation contour, and calculate the depression volume according to the road depression estimation model.

[0097] It can be understood that preprocessing the captured water accumulation image, including operations such as grayscale conversion, noise reduction, and contrast enhancement, to improve the image quality. Then, an image segmentation algorithm is used to separate the water accumulation area from the background, extract the contour of the water accumulation, analyze the extracted water accumulation contour, calculate its geometric features, such as area, perimeter, major axis length, minor axis length, determine the input variables and output variables of the model. The input variables include the geometric features of the water accumulation contour and the vertical displacement of the vehicle in the water accumulation area, which is the estimated value of the water accumulation depth; the output variable is the road depression volume.

[0098] Specifically, the repair analysis subunit compares the depression volume with a preset volume, and determines whether to repair the road according to the comparison result. Among them,

[0099] If the depression volume is less than the preset volume, it is determined not to repair;

[0100] If the depression volume is greater than or equal to the preset volume, it is determined to repair the road;

[0101] In a specific embodiment, a preset volume is set to 0.005 cubic meters. If the sunken volume is 0.0012 cubic meters, which is less than the preset volume, it is determined that no repair will be carried out.

[0102] If the sunken volume is 0.008 cubic meters, which is greater than the preset volume, it is determined that the road will be repaired.

[0103] The preset volume is positively correlated with the size of the water accumulation contour area.

[0104] It can be understood that when the road depth is certain, the larger the water accumulation contour area, the larger the volume of accumulated water. Therefore, the preset volume is positively correlated with the size of the water accumulation contour area.

[0105] Specifically, the present invention analyzes the vertical displacement distance of the vehicle when encountering bumps and the water accumulation contour to construct a road depression estimation model. The volume of the road depression is calculated through the model to determine whether to repair the road. This model comprehensively considers the vertical displacement distance of the vehicle when bumping and the water accumulation contour, and can more comprehensively, objectively and accurately evaluate the degree of road depression. The vertical displacement distance of the vehicle directly reflects the impact degree of the depression on vehicle driving, and the water accumulation contour shows the geometric shape and range of the depression from the side. By combining the two, the sunken volume can be accurately calculated. As a quantitative index, it can more scientifically judge whether the road depression has reached the level that needs to be repaired, avoiding over-repair or untimely repair. Road maintenance resources (such as manpower, material resources and financial resources) are usually limited. After accurately calculating the road depression volume using this model, the depressions can be sorted according to the severity of the depression, and priority can be given to repairing the depression areas with larger volumes and serious impacts on driving safety. In this way, limited maintenance resources can be more reasonably allocated, improving resource utilization efficiency and avoiding wasting resources on some minor depressions that do not need to be repaired immediately. The road depression volume data calculated by this model can provide valuable reference for road planning and design departments. By analyzing the depression volumes of different road sections, the areas and reasons where road depressions are likely to occur during road use can be understood, such as the influence of geological conditions, traffic flow and other factors. When planning and designing new roads, corresponding measures can be taken to optimize the road structure and design parameters, improve the durability and stability of the road, and reduce the occurrence of future road diseases such as depressions, further improving the accuracy and reliability of the real-time monitoring system for the full life cycle condition of the road based on machine vision.

[0106] Specifically, in the state where the defect analysis unit determines not to carry out repair, the road position corresponding to the water accumulation contour is marked as a key area for periodic scanning, and the interval time of the periodic scanning is negatively correlated with the number of rainfall times.

[0107] Please refer to Figure 4As shown, it is a flowchart of the real-time monitoring method for the full-cycle condition of a road based on machine vision. The present invention provides a clothing cutting method based on machine vision, including:

[0108] Step S1, collect road surface images and vehicle driving images at different time points of the road;

[0109] Step S2, determine the water accumulation contour and location based on the road surface image, and determine the water flow direction when the water accumulation is in a flowing state;

[0110] Step S3, determine the driving trajectories and postures of each vehicle based on the driving images, analyze the road bump frequency and abnormal frequency, determine the vertical displacement distance of the vehicle according to the driving posture, construct a terrain slope model based on the road surface image, and couple with the positions of road drainage holes to obtain the simulated water flow direction;

[0111] Step S4, compare the road bump frequency with the abnormal frequency, determine the pending road depression positions according to the comparison results, and compare the water flow direction with the simulated water flow direction to determine whether a road collapse has occurred;

[0112] Step S5, compare the pending road depression positions with the water accumulation positions to determine whether a road depression has occurred, and construct a road depression estimation model by combining the vehicle vertical displacement distance and the water accumulation contour to determine whether to repair the road. The degree of damage to the road caused by a road collapse is greater than that caused by a road depression.

[0113] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

[0114] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A real-time monitoring system for the full-cycle condition of a road based on machine vision, characterized in that, Including: An image acquisition unit for acquiring road surface images and vehicle driving images at different time points of the road; A road surface image processing unit connected to the image acquisition unit for determining the water accumulation contour and position based on the road surface image, and determining the water flow direction in the state where the water accumulation is flowing; A driving image processing unit connected to the image acquisition unit for determining the driving trajectories and postures of each vehicle based on the driving images, analyzing the road bump frequency and abnormal frequency, and determining the vehicle vertical displacement distance based on the driving posture; A road surface analysis unit connected to the image acquisition unit for constructing a terrain slope model based on the road surface image and coupling with the positions of road drainage holes to obtain a simulated water flow direction; A driving analysis unit connected to the image acquisition unit and the driving image processing unit respectively for comparing the road bump frequency with the abnormal frequency and determining the pending road depression position according to the comparison result; A defect analysis unit connected to the road surface image processing unit, the road surface analysis unit and the driving analysis unit respectively for comparing the water flow direction with the simulated water flow direction to determine whether a road collapse occurs, comparing the pending road depression position with the water accumulation position to determine whether a road depression occurs, and constructing a road depression estimation model by combining the vehicle vertical displacement distance and the water accumulation contour to determine whether to repair the road, wherein the degree of damage to the road caused by the road collapse is greater than that of the road depression.

2. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 1, characterized in that, The road surface image processing unit includes: A water accumulation image sub-unit connected to the image acquisition unit for determining whether the reflective area in the road surface image is a road reflective area or a water accumulation reflective area, and determining the water accumulation contour and position based on the water accumulation reflective area; A flowing water image sub-unit connected to the image acquisition unit for determining the water flow direction based on the road surface image.

3. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 2, wherein The driving image processing unit includes: A driving frequency sub-unit connected to the image acquisition unit for counting the bump frequencies corresponding to different driving speeds of each vehicle on the driving trajectory, plotting a road bump frequency curve graph, and selecting the bump frequency of the outlier in the road bump frequency curve graph as the abnormal frequency; A bump displacement sub-unit connected to the image acquisition unit for determining the edge length of the fuzzy area in the vertical direction of the corresponding vehicle based on the driving postures of each vehicle and calculating the vehicle vertical displacement distance.

4. The real-time monitoring system for the whole life cycle condition of a road based on machine vision according to claim 3, wherein The road surface analysis unit extracts the edge information of the road in different road surface images and generates the terrain slope model by surface fitting based on the parallax of corresponding points in different road surface images.

5. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 4, characterized in that, The driving analysis unit calculates the growth rate of the abnormal frequency relative to the road bump frequency at the same speed, and determines whether the road position corresponding to the abnormal frequency is a depression position according to the comparison result between the growth rate and the preset growth rate, where if the growth rate is greater than or equal to the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is the depression position; The preset growth rate is positively correlated with the vehicle driving speed.

6. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 5, characterized in that The defect analysis unit includes: A collapse analysis sub-unit, which is respectively connected to the road surface image processing unit and the road surface analysis unit, compares the included angle between the water flow direction and the simulated water flow direction with a preset included angle, and determines whether a road collapse has occurred according to the comparison result.

7. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 6, wherein The defect analysis unit further includes: A depression analysis sub-unit, which is connected to the road surface image processing unit and the driving analysis unit, and is used to compare the to-be-determined road depression position and the water accumulation position to determine whether a road depression has occurred. Among them, If the to-be-determined road depression position and the water accumulation position overlap each other, it is determined that a road depression has occurred.

8. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 7, characterized in that The defect analysis unit further includes: A repair analysis sub-unit, which is respectively connected to the road surface image processing unit, the driving image processing unit and the depression analysis sub-unit, and is used to construct a road depression estimation model by combining the vehicle vertical displacement distance and the water accumulation contour, and calculate the depression volume according to the road depression estimation model.

9. The real-time monitoring system for the full-cycle condition of a road based on machine vision according to claim 8, characterized in that, The repair analysis sub-unit compares the depression volume with a preset volume, and determines whether to repair the road according to the comparison result. Among them, If the depression volume is less than the preset volume, it is determined not to repair; If the depression volume is greater than or equal to the preset volume, it is determined to repair the road; The preset volume is positively correlated with the size of the water accumulation contour area.

10. A real-time monitoring method for the full-cycle condition of a road based on machine vision using the real-time monitoring system for the full-cycle condition of a road based on machine vision according to any one of claims 1-9, characterized in that, It includes: Collect road surface images and vehicle driving images at different time points; Determine the water accumulation contour and water accumulation position according to the road surface image, and determine the water flow direction in the state where the water accumulation is flowing; Determine the driving trajectories and driving postures of each vehicle according to the driving image, analyze the road bump frequency and abnormal frequency, determine the vehicle vertical displacement distance according to the driving posture, construct a terrain slope model according to the road surface image, and couple with the road drainage hole position to obtain the simulated water flow direction; Compare the road bump frequency with the abnormal frequency, determine the to-be-determined road depression position according to the comparison result, and compare the water flow direction with the simulated water flow direction to determine whether a road collapse has occurred; Compare the to-be-determined road depression position and the water accumulation position to determine whether a road depression has occurred, and construct a road depression estimation model by combining the vehicle vertical displacement distance and the water accumulation contour to determine whether to repair the road. The degree of damage to the road caused by the road collapse is greater than that of the road depression.

Images