A real-time monitoring system and method for full-cycle road conditions based on machine vision

Abstract

The present invention relates to the field of visual recognition technology, and in particular to a system and method for real-time monitoring of the full-cycle condition of a road based on machine vision. The system comprises: an image acquisition unit for acquiring road surface images and vehicle driving images at different time points; a road surface image processing unit for determining the contour and location of accumulated water, as well as the direction of water flow; a driving image processing unit for analyzing the frequency of road bumps and abnormal frequencies, and determining the vertical displacement distance of the vehicle; a road surface analysis unit for coupling to obtain a simulated water flow direction; a driving analysis unit for determining the location of a road depression; a defect analysis unit for determining whether a road collapse occurs, whether a road depression occurs, and whether to judge whether the road should be repaired. The present invention utilizes the above-mentioned units to cooperate with each other, thereby improving the efficiency of road monitoring and further enhancing the accuracy and reliability of the real-time monitoring system for the full-cycle condition of a road based on machine vision.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and in particular to a system and method for real-time monitoring of road conditions throughout a full cycle based on machine vision. Background Art

[0002] During long-term use, roads are subject to the combined effects of natural factors (such as rain erosion, freeze-thaw cycles, and sunlight exposure) and traffic loads (such as vehicle rolling and impact), resulting in cracks, potholes, bumps, and other road surface defects. If these defects are not discovered and treated in a timely manner, they will gradually develop and worsen, seriously affecting road performance and traffic safety. Machine vision-based monitoring systems can perform high-frequency and high-precision inspections on roads, promptly detecting early-stage defects and potential safety hazards, providing an accurate basis for preventive road maintenance, thereby reducing the risk of accidents and protecting people's lives and property.

[0003] Chinese patent application publication number: CN114511551A discloses a ground damage identification system based on machine vision. The invention provides a ground damage identification system based on machine vision, including a data acquisition mobile terminal and a human-computer interaction terminal. The data acquisition mobile terminal includes an unmanned aerial vehicle (UAV) aircraft, a data acquisition module and a data transmission module. The invention uses an unmanned aerial vehicle (UAV) aircraft equipped with a data acquisition module to collect data on the ground to be identified. Compared with traditional manual detection methods, the invention is more convenient, saves time and effort, and pre-processes the collected ground image through conversion, enhancement and edge detection to make the ground image clearer, thereby facilitating subsequent image feature extraction. Damage judgment is performed on the pre-processed image and the damage type is identified based on the judgment result, thereby realizing the identification of ground damage based on machine vision. Compared with some existing ground damage identification systems, the identification accuracy is higher, and the ground damage situation in the area to be identified can be accurately identified, thereby effectively helping to accurately predict accidents such as ground collapse.

[0004] Chinese patent application publication number CN117523502A discloses a machine vision-based intelligent monitoring system for urban road garbage. The system comprises a data acquisition module for collecting garbage bin images; a plane fit goodness-of-fit acquisition module for establishing a pixel feature window, obtaining the noise representation of the pixels, and obtaining the plane fit goodness of the pixels; a pixel correction acquisition module for determining the pixel median, obtaining the edge influence of the pixel median, obtaining the possibility of pixel correction, and marking the corrected pixels; a corrected median grayscale acquisition module for determining the non-center points of the corrected pixels, determining the weights of the non-center points, and obtaining the corrected median grayscale of the corrected pixels; and a garbage monitoring module for denoising the garbage bin image based on the corrected median grayscale of the corrected pixels to obtain the denoised garbage bin image. This denoised garbage bin image is then used to implement intelligent monitoring of urban road garbage. This invention addresses the problem of inaccurate urban road garbage monitoring.

[0005] However, the above method has the following problems: relying solely on drone images to judge road conditions is not accurate enough, and simply processing the images fails to monitor the target well. Summary of the Invention

[0006] To this end, the present invention provides a real-time monitoring system and method for the full-cycle road condition based on machine vision, which is used to overcome the problems in the prior art of relying solely on drone-captured images to accurately judge road conditions and failing to monitor targets well through image processing alone.

[0007] To achieve the above objectives, the present invention provides a real-time monitoring system for road conditions throughout the entire cycle based on machine vision, comprising:

[0008] An image acquisition unit, which is used to acquire road surface images and vehicle driving images at different time points;

[0009] a road surface image processing unit connected to the image acquisition unit, for determining the outline and location of the accumulated water based on the road surface image, and determining the direction of the water flow when the accumulated water is flowing;

[0010] a driving image processing unit connected to the image acquisition unit, configured to determine the driving trajectory and driving posture of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, and determine the vertical displacement distance of the vehicle based on the driving posture;

[0011] a road surface analysis unit connected to the image acquisition unit, configured to construct a terrain slope model based on the road surface image and obtain a simulated water flow direction by coupling with the position of road drainage holes;

[0012] a driving analysis unit, connected to the image acquisition unit and the driving image processing unit, respectively, for comparing the road bump frequency with the abnormal frequency and determining the road depression position according to the comparison result;

[0013] a defect analysis unit, which is respectively connected to the road surface image processing unit, the road surface analysis unit and the driving analysis unit, and is used to compare the water flow direction with the simulated water flow direction to determine whether a road collapse has occurred, and to compare the road depression position with the water accumulation position to determine whether a road depression has occurred, and to construct a road depression estimation model based on the vehicle vertical displacement distance and the water accumulation contour to determine whether the road should be repaired, wherein the degree of damage to the road caused by the road collapse is greater than that caused by the road depression.

[0014] Furthermore, the road surface image processing unit includes:

[0015] a water accumulation image subunit, connected to the image acquisition unit, for determining whether a reflective area in the road surface image is a road reflective area or a water accumulation reflective area, and determining the water accumulation contour and the water accumulation position according to the water accumulation reflective area;

[0016] The water flow image subunit is connected to the image acquisition unit and is used to determine the water flow direction according to the road surface image.

[0017] Furthermore, the driving image processing unit includes:

[0018] a driving frequency subunit, connected to the image acquisition unit, for counting the bumping frequencies corresponding to different driving speeds of each vehicle on the driving trajectory, drawing a road bumping frequency curve, and selecting the bumping frequencies of outliers in the road bumping frequency curve as the abnormal frequencies;

[0019] The bump displacement subunit is connected to the image acquisition unit and is used to determine the edge length of the fuzzy area in the vertical direction of the corresponding vehicle according to the driving posture of each vehicle and calculate the vertical displacement distance of the vehicle.

[0020] Furthermore, the road surface analysis unit extracts edge information of the road in different road surface images, and generates the terrain slope model by surface fitting based on the disparity of corresponding points in different road surface images.

[0021] Furthermore, the driving analysis unit calculates the growth rate of the abnormal frequency relative to the road bump frequency at the same speed, and determines whether the road position corresponding to the abnormal frequency is a sunken position based on a comparison result of the growth rate with a preset growth rate, wherein:

[0022] If the growth rate is greater than or equal to the preset growth rate, determining that the road position corresponding to the abnormal frequency is the sunken position;

[0023] The preset growth rate is positively correlated with the driving speed of the vehicle.

[0024] Furthermore, the defect analysis unit includes:

[0025] The collapse analysis subunit is connected to the road surface image processing unit and the road surface analysis unit respectively, compares the angle between the water flow direction and the simulated water flow direction with a preset angle, and determines whether a road collapse occurs based on the comparison result.

[0026] Furthermore, the defect analysis unit further includes:

[0027] A depression analysis subunit is connected to the road surface image processing unit and the driving analysis unit, and is used to compare the road depression position with the water accumulation position to determine whether a road depression occurs, wherein:

[0028] If the road depression position and the water accumulation position overlap with each other, it is determined that a road depression has occurred.

[0029] Furthermore, the defect analysis unit further includes:

[0030] The repair analysis subunit is connected to the road surface image processing unit, the driving image processing unit and the depression analysis subunit respectively, and is used to construct a road depression estimation model based on the vehicle vertical displacement distance and the water accumulation contour, and calculate the depression volume according to the road depression estimation model.

[0031] Furthermore, the repair analysis subunit compares the sunken volume with a preset volume and determines whether to repair the road according to the comparison result, wherein:

[0032] If the volume of the depression is smaller than the preset volume, it is determined that no repair is to be performed;

[0033] If the volume of the depression is greater than or equal to the preset volume, it is determined that the road should be repaired;

[0034] The preset volume is positively correlated with the size of the water accumulation contour area.

[0035] In another aspect, the present invention provides a method for real-time monitoring of road conditions throughout a full cycle based on machine vision, comprising:

[0036] Collect road surface images and vehicle driving images at different time points;

[0037] Determining the contour and location of the accumulated water based on the road surface image, and determining the direction of water flow when the accumulated water is flowing;

[0038] Determine the driving trajectory and driving posture of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, determine the vertical displacement distance of the vehicle based on the driving posture, construct a terrain slope model based on the road surface image, and obtain a simulated water flow direction by coupling with the location of road drainage holes;

[0039] comparing the road bump frequency with the abnormal frequency, determining a potential road depression location based on the comparison result, and comparing the water flow direction with the simulated water flow direction to determine whether a road collapse occurs;

[0040] The undetermined road depression position and the water accumulation position are compared to determine whether a road depression occurs, and a road depression estimation model is constructed in combination with the vehicle vertical displacement distance and the water accumulation contour to determine whether the road should be repaired. The degree of damage to the road caused by the road collapse is greater than that caused by the road depression.

[0041] Compared with the prior art, the beneficial effect of the present invention is that the present invention determines the flatness of the road according to the state of water accumulation on the road by collecting road surface images and vehicle driving images. There is no need for direct contact with the road surface as in traditional flatness detection methods (such as using equipment such as a flatness meter). Road surface and water accumulation information can be obtained only through image acquisition equipment (such as a camera), which greatly reduces the installation and operation complexity of the detection equipment. The detection can be carried out synchronously during the normal driving of the vehicle without affecting the normal passage of traffic, thereby improving the detection efficiency, being able to cover a long road mileage in a short time, reducing the detection cost and time cost, being able to collect road surface images in real time, and promptly discovering changes in road flatness. Once the road has flatness abnormalities caused by water accumulation (such as local depressions and water accumulation, etc.), the system can quickly make a judgment and issue an alarm to provide timely road condition information to the road management department, which helps the management department to take measures quickly, such as arranging maintenance personnel to inspect and repair. To avoid more serious traffic accidents and road damage caused by road roughness problems, the state of road water accumulation can not only reflect the depressions and bulges on the road surface, but also comprehensively reflect the actual performance of the road under different weather conditions (especially after rainfall). Compared with relying solely on manual inspection or static detection equipment, this judgment method based on images and water accumulation status can more comprehensively capture subtle changes in road smoothness. The road surface images and water accumulation status data collected over a long period of time can form a huge database. By analyzing and mining these data, we can gain an in-depth understanding of the changing patterns of road smoothness, such as the changing trends of road smoothness in different sections, different seasons, and different years of use. This has important reference value for the formulation and optimization of road design, construction, and maintenance standards, and helps to improve the scientificity and standardization of road construction and management. While comprehensively reflecting the road conditions, it effectively improves the accuracy and reliability of the real-time monitoring system for the full cycle of road conditions based on machine vision.

[0042] Furthermore, the present invention analyzes the frequency of road bumps and abnormal frequencies by combining the driving trajectory and driving posture of the car, and then determines the location of the road depression. The use of the car's driving trajectory and posture data can achieve accurate positioning of the road depression. When the vehicle encounters a sunken road section during driving, its driving trajectory will deviate and its driving posture will also show abnormal changes. Through real-time analysis of these data, the specific location of the road depression can be quickly and accurately determined, providing road maintenance personnel with accurate repair targets, improving the pertinence and efficiency of the repair work, and through long-term accumulation and analysis of a large number of vehicle driving data, the occurrence law and development trend of road depressions in different sections can be mastered. For example, some Road sections may be more prone to road sags due to factors such as geological conditions and traffic volume. Based on this data, road management departments can formulate more scientific and reasonable road maintenance plans, rationally allocate maintenance funds and manpower, and carry out preventive maintenance in potential road sag risk areas in advance to avoid further deterioration of road conditions and reduce road maintenance costs. Relevant road sag data can be shared, providing valuable reference for fields such as urban planning and traffic research. Urban planning departments can optimize road design and construction plans based on the distribution of road sags, improve the overall quality of urban roads, and further enhance the accuracy and reliability of the real-time monitoring system for road conditions throughout the entire cycle based on machine vision.

[0043] Furthermore, the present invention will cross-check the road depression position analyzed by the car with the water accumulation position on the directly photographed road surface image to determine the position where the road depression occurs. By cross-checking the two, the limitations of a single method can be effectively reduced. The two different monitoring methods verify each other, which increases the diversity of data sources. When one method is abnormal or fails, the other method can be used as a supplement to ensure that the monitoring of the road depression position can be carried out continuously. The depression position analyzed by the car focuses on reflecting the unevenness of the road from the perspective of vehicle driving experience, while the water accumulation position on the road surface image intuitively shows the geometric changes of the road surface. Combining the two can better A comprehensive understanding of the actual situation of road sags, including the depth and scope of the sags and the impact of accumulated water on the sags, can make accurate sag location judgment more targeted and avoid blind large-scale inspections and repairs. By cross-checking the confirmed sag locations, maintenance personnel can go directly to the problem area for repairs, reducing the waste of manpower, material resources and time, and reducing road maintenance costs. The accuracy of sag location judgment is improved by cross-checking the two methods, which means that road problems can be handled more promptly, providing better protection for driving safety, and further improving the accuracy and reliability of the real-time monitoring system for road conditions throughout the entire cycle based on machine vision.

[0044] Furthermore, the present invention constructs a road depression estimation model by analyzing the vertical displacement distance and water accumulation contour of the car when encountering bumps, and calculates the volume of the road depression through the model to determine whether the road should be repaired. The model comprehensively considers the vertical displacement distance and water accumulation contour of the car when it encounters bumps, and can more comprehensively, objectively and accurately evaluate the degree of road depression. The vertical displacement distance of the car directly reflects the impact of the depression on vehicle driving, and the water accumulation contour shows the geometric shape and range of the depression from the side. By combining the two, the depression volume can be accurately calculated. Using this as a quantitative indicator, it can be more scientifically judged whether the road depression has reached the degree that requires repair, avoiding excessive repair or untimely repair. Road maintenance resources (such as manpower, material resources, and financial resources) are usually limited. After using the model to accurately calculate the road depression volume, it can be based on the depression The road dents are ranked by severity, and priority is given to repairing the larger dents that have a serious impact on driving safety. This will enable a more reasonable allocation of limited maintenance resources, improve resource utilization efficiency, and avoid wasting resources on minor dents that do not need to be repaired immediately. The road dent volume data calculated by this model can provide valuable reference for road planning and design departments. By analyzing the dent volumes of different road sections, we can understand the areas and causes of road dents during use, such as the influence of geological conditions, traffic flow and other factors. When planning and designing new roads, targeted measures can be taken to optimize road structure and design parameters, improve road durability and stability, and reduce the occurrence of road dents and other diseases in the future, further improving the accuracy and reliability of the real-time monitoring system for road full-cycle conditions based on machine vision. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a block diagram of the real-time monitoring system for road conditions throughout the entire cycle based on machine vision of the present invention;

[0046] Figure 2 This is a structural block diagram of a road surface image processing unit according to an embodiment of the present invention;

[0047] Figure 3 A determination diagram for determining a road depression location according to an embodiment of the present invention;

[0048] Figure 4 The figure is a flow chart of the method for real-time monitoring of the full-cycle road conditions based on machine vision of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0050] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0051] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0052] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0053] It is understandable that water accumulation on roads can be caused by rainy weather or road cleaning sprinkler trucks.

[0054] See also Figure 1 As shown in FIG, which is a structural block diagram of a real-time monitoring system for road conditions throughout a cycle based on machine vision according to the present invention, an embodiment of the present invention provides a real-time monitoring system for road conditions throughout a cycle based on machine vision, comprising:

[0055] An image acquisition unit, which is used to acquire road surface images and vehicle driving images at different time points;

[0056] a road surface image processing unit connected to the image acquisition unit, for determining the outline and location of accumulated water based on the road surface image, and determining the direction of water flow when the accumulated water is flowing;

[0057] A driving image processing unit, connected to the image acquisition unit, is used to determine the driving trajectory and driving posture of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, and determine the vertical displacement distance of the vehicle based on the driving posture;

[0058] A road surface analysis unit is connected to the image acquisition unit and is used to construct a terrain slope model based on the road surface image and obtain a simulated water flow direction by coupling with the location of the road drainage holes;

[0059] a driving analysis unit, connected to the image acquisition unit and the driving image processing unit, respectively, for comparing the road bump frequency with the abnormal frequency and determining the road depression position based on the comparison result;

[0060] The defect analysis unit is respectively connected to the road surface image processing unit, the road surface analysis unit and the driving analysis unit, and is used to compare the water flow direction with the simulated water flow direction to determine whether a road collapse has occurred, and to compare the road depression position with the water accumulation position to determine whether a road depression has occurred, and to construct a road depression estimation model based on the vehicle vertical displacement distance and the water accumulation contour to determine whether the road should be repaired. The degree of damage to the road caused by road collapse is greater than that caused by road depression.

[0061] Specifically, the present invention collects road surface images and vehicle driving images to judge the flatness of the road according to the state of water accumulation on the road. There is no need for direct contact with the road surface as in traditional flatness detection methods (such as using equipment such as a flatness meter). Road surface and water accumulation information can be obtained only through image acquisition equipment (such as a camera), which greatly reduces the installation and operation complexity of the detection equipment. Detection can be carried out simultaneously during normal vehicle driving without affecting the normal passage of traffic, improving detection efficiency, and being able to cover a longer road mileage in a short time, reducing detection costs and time costs. It can collect road surface images in real time and promptly detect changes in road flatness. Once the road has flatness abnormalities caused by water accumulation (such as local depressions and water accumulation, etc.), the system can quickly make a judgment and issue an alarm to provide road management departments with timely road condition information, which helps management departments to take measures quickly, such as arranging maintenance personnel to inspect and repair, to avoid changes due to road flatness. Road surface waterlogging not only reflects the depressions and bulges on the road surface, but also comprehensively reflects the actual performance of the road under different weather conditions (especially after rainfall). Compared with relying solely on manual inspection or static detection equipment, this judgment method based on images and waterlogging status can more comprehensively capture subtle changes in road smoothness. The long-term collection of road surface images and waterlogging status data can form a huge database. By analyzing and mining this data, we can gain an in-depth understanding of the changing patterns of road smoothness, such as the changing trends of road smoothness in different sections, seasons, and years of use. This has important reference value for the formulation and optimization of road design, construction, and maintenance standards, and helps to improve the scientific and standardized nature of road construction and management. While comprehensively reflecting road conditions, it effectively improves the accuracy and reliability of the real-time monitoring system for road conditions throughout the entire cycle based on machine vision.

[0062] See also Figure 2 FIG. 1 is a block diagram of a road surface image processing unit according to an embodiment of the present invention. The road surface image processing unit includes:

[0063] The water accumulation image subunit is connected to the image acquisition unit and is used to determine whether the reflective area in the road surface image is a road reflective area or a water accumulation reflective area, and to determine the outline and location of the water accumulation based on the water accumulation reflective area;

[0064] It is understandable that image feature analysis can be used to determine whether a reflective area in a road surface image is a road reflective area or a water reflective area. For example, the texture of a reflective area on an asphalt road surface will appear finely granular or irregular. In contrast, the texture of a water reflective area is often smoother because the surface of the water is relatively flat, and may also have tiny ripples caused by surface fluctuations.

[0065] The water flow image subunit is connected to the image acquisition unit and is used to determine the direction of water flow based on the road surface image.

[0066] It can be understood that for continuous video image frames, the inter-frame difference method can be used to detect the movement of water flow. By calculating the difference between two adjacent frames of images, the changed areas are highlighted. These changed areas are the result of water flow movement. These changed areas are analyzed and the movement trajectory of the water flow is tracked to determine the direction of the water flow.

[0067] Specifically, the driving image processing unit includes:

[0068] The driving frequency subunit is connected to the image acquisition unit and is used to count the bumping frequencies corresponding to different driving speeds of each car on the driving trajectory, draw a road bumping frequency curve, and select the bumping frequencies of outliers in the road bumping frequency curve as abnormal frequencies;

[0069] It is understood that the bump frequency corresponding to different driving speeds of each vehicle along the driving trajectory is counted. The collected data is examined and data points with obvious errors or anomalies are removed, such as negative speed values ​​or acceleration values ​​outside a reasonable range. A bump determination criterion is defined based on the acceleration sensor data. For example, when the change in acceleration exceeds a certain threshold (e.g., 0.5g, where g is the acceleration due to gravity), the vehicle is considered to have experienced a bump. For each data point within a driving speed interval, the number of vehicle bumps within a certain time window (e.g., 1 minute) is counted. This number is then divided by the length of the time window to obtain the bump frequency (unit: times / minute) for that speed interval. The horizontal axis represents the vehicle's driving speed, and the vertical axis represents the corresponding bump frequency. The average bump frequency for each driving speed interval is plotted on a graph, and then these points are connected by a curve to form a road bump frequency curve. Different curve fitting methods can be selected as needed to make the curve smoother and more accurately reflect the relationship between bump frequency and driving speed. Based on the selected outlier detection method, the data points in the road bump frequency curve are analyzed to identify outliers that deviate from the normal data distribution. The bump frequencies corresponding to these outliers are called abnormal frequencies.

[0070] The bump displacement subunit is connected to the image acquisition unit and is used to determine the edge length of the corresponding fuzzy area in the vertical direction of each car according to the driving posture of each car, and calculate the vertical displacement distance of the vehicle.

[0071] It is understandable that the collected images are preprocessed, including grayscale conversion, noise reduction (such as using Gaussian filtering to remove image noise), and contrast enhancement. Based on the vehicle's driving posture data and image features, a target detection algorithm or an image segmentation algorithm is used to identify the blurred area in the vertical direction of the vehicle. For the identified blurred area, an edge detection algorithm is used to extract its edge. These algorithms can detect pixels with large grayscale changes in the image, thereby determining the boundary of the blurred area. Based on the extracted edge information, image measurement technology is used to calculate the length of the edge of the blurred area. During the calculation process, the resolution and scale of the image need to be considered to convert the pixel length into the actual physical length.

[0072] Specifically, the present invention analyzes the frequency of road bumps and abnormal frequencies by combining the driving trajectory and driving posture of the car, and then determines the location of the road depression. By using the driving trajectory and posture data of the car, it is possible to accurately locate the location of the road depression. When the vehicle encounters a sunken section during driving, its driving trajectory will deviate and its driving posture will also show abnormal changes. Through real-time analysis of these data, the specific location of the road depression can be quickly and accurately determined, providing road maintenance personnel with accurate repair targets, improving the pertinence and efficiency of the repair work, and through long-term accumulation and analysis of a large number of vehicle driving data, the occurrence law and development trend of road depressions in different sections can be mastered. For example, some Road sections may be more prone to road sags due to factors such as geological conditions and traffic volume. Based on this data, road management departments can formulate more scientific and reasonable road maintenance plans, rationally allocate maintenance funds and manpower, and carry out preventive maintenance in potential road sag risk areas in advance to avoid further deterioration of road conditions and reduce road maintenance costs. Relevant road sag data can be shared, providing valuable reference for fields such as urban planning and traffic research. Urban planning departments can optimize road design and construction plans based on the distribution of road sags, improve the overall quality of urban roads, and further enhance the accuracy and reliability of the real-time monitoring system for road conditions throughout the entire cycle based on machine vision.

[0073] Specifically, the road surface analysis unit extracts edge information of the road in different road surface images, and generates a terrain slope model through surface fitting based on the disparity of corresponding points in different road surface images.

[0074] See also Figure 3 As shown in FIG, which is a determination diagram for determining a road depression position according to an embodiment of the present invention, the driving analysis unit calculates the growth rate of the abnormal frequency relative to the road bump frequency at the same speed, and determines whether the road position corresponding to the abnormal frequency is a depression position based on the comparison result of the growth rate with the preset growth rate, wherein:

[0075] If the growth rate is greater than or equal to the preset growth rate, the road position corresponding to the abnormal frequency is determined to be a sunken position;

[0076] If the growth rate is less than the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is not a sunken position;

[0077] In a specific embodiment, the preset growth rate is set to 20%. If the growth rate is 34% and is greater than the preset growth rate, the road position corresponding to the abnormal frequency is determined to be a sunken position.

[0078] If the growth rate of 14% is smaller than the preset growth rate, it is determined that the road position corresponding to the abnormal frequency is not a sunken position.

[0079] The preset growth rate is positively correlated with the vehicle's driving speed.

[0080] It is understandable that the faster the car travels, the greater the bumps it encounters on the road potholes and the greater the frequency of abnormalities, so the preset growth rate is positively correlated with the car's travel speed.

[0081] Specifically, the defect analysis unit includes:

[0082] The collapse analysis subunit is connected to the road surface image processing unit and the road surface analysis unit respectively, compares the angle between the water flow direction and the simulated water flow direction with the preset angle, and determines whether road collapse occurs based on the comparison result.

[0083] If the angle between the water flow direction and the simulated water flow direction is smaller than the preset angle, it is determined that no road collapse has occurred;

[0084] If the angle between the water flow direction and the simulated water flow direction is greater than or equal to the preset angle, it is determined that the road collapse has occurred.

[0085] It is understandable that, based on the image of the water flow, the optical flow method is used to calculate the motion vector of the pixel points in the water flow, thereby obtaining the direction of the water flow.

[0086] In a specific embodiment, the preset angle is set to 45°. If the angle between the water flow direction and the simulated water flow direction is 30°, which is less than the preset angle, it is determined that no road collapse has occurred.

[0087] If the angle between the water flow direction and the simulated water flow direction is 65°, which is greater than the preset angle, it is determined that road collapse has occurred.

[0088] The preset angle is positively correlated with the road slope.

[0089] It can be understood that the greater the road slope, the faster the water flow rate and the greater the inertia, the greater the angle between the water flow branch direction and the water flow direction of the main road at the road collapse point, so the preset angle is positively correlated with the road slope.

[0090] Specifically, the defect analysis unit also includes:

[0091] The depression analysis subunit is connected to the road surface image processing unit and the driving analysis unit, and is used to compare the road depression position and the water accumulation position to determine whether a road depression occurs, wherein:

[0092] If the road depression position and the water accumulation position overlap with each other, it is determined that a road depression has occurred.

[0093] It can be understood that when the overlap between the sunken road position and the water accumulation position exceeds one quarter of either one, it can be determined that a road collapse has occurred.

[0094] Specifically, the present invention will cross-check the road depression position analyzed by the car with the water accumulation position on the directly photographed road surface image to determine the position where the road depression occurs. By cross-checking the two, the limitations of a single method can be effectively reduced. The two different monitoring methods verify each other, which increases the diversity of data sources. When one method is abnormal or fails, the other method can be used as a supplement to ensure that the monitoring of the road depression position can be carried out continuously. The depression position analyzed by the car focuses on reflecting the unevenness of the road from the perspective of vehicle driving experience, while the water accumulation position on the road surface image intuitively shows the geometric changes of the road surface. Combining the two can better A comprehensive understanding of the actual situation of road sags, including the depth and scope of the sags and the impact of accumulated water on the sags, can make accurate sag location judgment more targeted and avoid blind large-scale inspections and repairs. By cross-checking the confirmed sag locations, maintenance personnel can go directly to the problem area for repairs, reducing the waste of manpower, material resources and time, and reducing road maintenance costs. The accuracy of sag location judgment is improved by cross-checking the two methods, which means that road problems can be handled more promptly, providing better protection for driving safety, and further improving the accuracy and reliability of the real-time monitoring system for road conditions throughout the entire cycle based on machine vision.

[0095] Specifically, the defect analysis unit also includes:

[0096] The repair analysis subunit is connected to the road surface image processing unit, the driving image processing unit and the depression analysis subunit respectively, and is used to build a road depression estimation model based on the vehicle vertical displacement distance and the water accumulation contour, and calculate the depression volume according to the road depression estimation model.

[0097] It can be understood that the captured waterlogged images are preprocessed, including grayscale conversion, noise reduction, contrast enhancement and other operations, to improve image quality. Then, an image segmentation algorithm is used to separate the waterlogged area from the background, extract the outline of the waterlogged area, analyze the extracted waterlogged area outline, calculate its geometric features, such as area, perimeter, major axis length, and minor axis length, and determine the input and output variables of the model. The input variables include the geometric features of the waterlogged area outline and the vertical displacement of the vehicle in the waterlogged area, which is the estimated value of the waterlogged depth; the output variable is the road depression volume.

[0098] Specifically, the repair analysis subunit compares the sunken volume with the preset volume and determines whether to repair the road according to the comparison result.

[0099] If the dent volume is smaller than the preset volume, it is determined that no repair will be performed;

[0100] If the volume of the depression is greater than or equal to the preset volume, it is determined that the road should be repaired;

[0101] In a specific embodiment, the preset volume is set to 0.005 cubic meters. If the depressed volume is 0.0012 cubic meters, which is smaller than the preset volume, it is determined that no repair is to be performed.

[0102] If the sunken volume is 0.008 cubic meters, which is greater than the preset volume, the road will be repaired.

[0103] The preset volume is positively correlated with the size of the water accumulation contour area.

[0104] It is understandable that, when the road depth is constant, the larger the water accumulation contour area, the larger the volume of the accumulated water, so the preset volume is positively correlated with the size of the water accumulation contour area.

[0105] Specifically, the present invention constructs a road depression estimation model by analyzing the vertical displacement distance and water accumulation contour of a car when it encounters a bump, and calculates the volume of the road depression through the model to determine whether the road should be repaired. The model comprehensively considers the vertical displacement distance and water accumulation contour of the car when it encounters a bump, and can more comprehensively, objectively and accurately evaluate the degree of road depression. The vertical displacement distance of the car directly reflects the impact of the depression on vehicle driving, and the water accumulation contour shows the geometric shape and range of the depression from the side. By combining the two, the depression volume can be accurately calculated. Using this as a quantitative indicator, it can be more scientifically judged whether the road depression has reached the degree that requires repair, avoiding excessive repair or untimely repair. Road maintenance resources (such as manpower, material resources, and financial resources) are usually limited. After using the model to accurately calculate the road depression volume, it can be based on the depression. The road dents are ranked by severity, and priority is given to repairing the larger dents that have a serious impact on driving safety. This will enable a more reasonable allocation of limited maintenance resources, improve resource utilization efficiency, and avoid wasting resources on minor dents that do not need to be repaired immediately. The road dent volume data calculated by this model can provide valuable reference for road planning and design departments. By analyzing the dent volumes of different road sections, we can understand the areas and causes of road dents during use, such as the influence of geological conditions, traffic flow and other factors. When planning and designing new roads, targeted measures can be taken to optimize road structure and design parameters, improve road durability and stability, and reduce the occurrence of road dents and other diseases in the future, further improving the accuracy and reliability of the real-time monitoring system for road full-cycle conditions based on machine vision.

[0106] Specifically, when the defect analysis unit determines that no repair is to be performed, the road position corresponding to the waterlogging contour is marked as a key area for periodic scanning. The interval time of the periodic scanning is negatively correlated with the number of rainfalls.

[0107] See also Figure 4As shown in FIG, which is a flow chart of a method for real-time monitoring of full-cycle road conditions based on machine vision of the present invention, the present invention provides a method for cutting clothing based on machine vision, comprising:

[0108] Step S1, collecting road surface images and vehicle driving images at different time points;

[0109] Step S2, determining the outline and location of the accumulated water based on the road surface image, and determining the direction of the water flow when the accumulated water is flowing;

[0110] Step S3: Determine the driving trajectory and driving posture of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, determine the vertical displacement distance of the vehicle based on the driving posture, construct a terrain slope model based on the road surface image, and obtain a simulated water flow direction by coupling with the location of the road drainage holes;

[0111] Step S4, comparing the road bump frequency with the abnormal frequency, determining the location of the road depression based on the comparison result, and comparing the water flow direction with the simulated water flow direction to determine whether the road collapse occurs;

[0112] In step S5, the undetermined road depression position and the water accumulation position are compared to determine whether a road depression has occurred, and a road depression estimation model is constructed in combination with the vehicle vertical displacement distance and the water accumulation contour to determine whether the road should be repaired. The degree of damage to the road caused by road collapse is greater than that caused by road depression.

[0113] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0114] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A real-time monitoring system for the entire road cycle based on machine vision, characterized in that: include: An image acquisition unit, which is used to acquire road surface images and vehicle driving images at different time points; a road surface image processing unit connected to the image acquisition unit, for determining the outline and location of the accumulated water based on the road surface image, and determining the direction of the water flow when the accumulated water is flowing; a driving image processing unit connected to the image acquisition unit, configured to determine the driving trajectory and driving posture of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, and determine the vertical displacement distance of the vehicle based on the driving posture; a road surface analysis unit connected to the image acquisition unit, configured to construct a terrain slope model based on the road surface image and obtain a simulated water flow direction by coupling with the position of road drainage holes; a driving analysis unit, connected to the image acquisition unit and the driving image processing unit, respectively, for comparing the road bump frequency with the abnormal frequency and determining a pending road depression position based on the comparison result; a defect analysis unit, which is respectively connected to the road surface image processing unit, the road surface analysis unit and the driving analysis unit, and is used to compare the water flow direction with the simulated water flow direction to determine whether a road collapse has occurred, and to compare the undetermined road depression position with the water accumulation position to determine whether a road depression has occurred, and to construct a road depression estimation model based on the vehicle vertical displacement distance and the water accumulation contour to determine whether the road should be repaired, wherein the degree of damage to the road caused by the road collapse is greater than that caused by the road depression.

2. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 1 is characterized in that: The road surface image processing unit includes: a water accumulation image subunit, connected to the image acquisition unit, for determining whether a reflective area in the road surface image is a road reflective area or a water accumulation reflective area, and determining the water accumulation contour and the water accumulation position according to the water accumulation reflective area; The water flow image subunit is connected to the image acquisition unit and is used to determine the water flow direction according to the road surface image.

3. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 2 is characterized in that: The driving image processing unit includes: a driving frequency subunit, connected to the image acquisition unit, for counting the bumping frequencies corresponding to different driving speeds of each vehicle on the driving trajectory, drawing a road bumping frequency curve, and selecting the bumping frequencies of outliers in the road bumping frequency curve as the abnormal frequencies; The bump displacement subunit is connected to the image acquisition unit and is used to determine the edge length of the fuzzy area in the vertical direction of the corresponding vehicle according to the driving posture of each vehicle and calculate the vertical displacement distance of the vehicle.

4. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 3 is characterized in that: The road surface analysis unit extracts edge information of the road in different road surface images, and generates the terrain slope model through surface fitting based on the parallax of corresponding points in different road surface images.

5. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 4 is characterized in that: The driving analysis unit calculates the growth rate of the abnormal frequency relative to the road bump frequency at the same speed, and determines whether the road position corresponding to the abnormal frequency is a concave position based on the comparison result of the growth rate and a preset growth rate, wherein: If the growth rate is greater than or equal to the preset growth rate, determining that the road position corresponding to the abnormal frequency is the sunken position; The preset growth rate is positively correlated with the driving speed of the vehicle.

6. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 5 is characterized in that: The defect analysis unit includes: The collapse analysis subunit is connected to the road surface image processing unit and the road surface analysis unit respectively, compares the angle between the water flow direction and the simulated water flow direction with a preset angle, and determines whether a road collapse occurs based on the comparison result.

7. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 6 is characterized in that: The defect analysis unit further includes: A depression analysis subunit is connected to the road surface image processing unit and the driving analysis unit, and is used to compare the undetermined road depression position with the water accumulation position to determine whether a road depression occurs, wherein: If the undetermined road depression position and the water accumulation position overlap with each other, it is determined that a road depression occurs.

8. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 7 is characterized in that: The defect analysis unit further includes: The repair analysis subunit is connected to the road surface image processing unit, the driving image processing unit and the depression analysis subunit respectively, and is used to construct a road depression estimation model based on the vehicle vertical displacement distance and the water accumulation contour, and calculate the depression volume according to the road depression estimation model.

9. The real-time monitoring system for road conditions throughout a full cycle based on machine vision according to claim 8, characterized in that: The repair analysis subunit compares the sunken volume with a preset volume and determines whether to repair the road according to the comparison result, wherein: If the volume of the depression is smaller than the preset volume, it is determined that no repair is to be performed; If the volume of the depression is greater than or equal to the preset volume, it is determined that the road should be repaired; The preset volume is positively correlated with the size of the water accumulation contour area.

10. A method for real-time monitoring of road conditions based on machine vision using the real-time monitoring system for road conditions based on machine vision according to any one of claims 1 to 9, characterized in that: include: Collect road surface images and vehicle driving images at different time points; Determining the contour and location of the accumulated water based on the road surface image, and determining the direction of water flow when the accumulated water is flowing; Determine the driving trajectory and driving posture of each vehicle based on the driving image, analyze the road bump frequency and abnormal frequency, determine the vertical displacement distance of the vehicle based on the driving posture, construct a terrain slope model based on the road surface image, and obtain a simulated water flow direction by coupling with the location of road drainage holes; comparing the road bump frequency with the abnormal frequency, determining a potential road depression location based on the comparison result, and comparing the water flow direction with the simulated water flow direction to determine whether a road collapse occurs; The undetermined road depression position and the water accumulation position are compared to determine whether a road depression occurs, and a road depression estimation model is constructed in combination with the vehicle vertical displacement distance and the water accumulation contour to determine whether the road should be repaired. The degree of damage to the road caused by the road collapse is greater than that caused by the road depression.

Images