A method for identifying road surface defects based on vehicle motion

Abstract

The application discloses a kind of road surface defect identification methods based on vehicle motion, it is related to road engineering field, by defining basis and combination road defect, the car equipped with sensor drives through these defects, data is collected to form feature dataset.Utilize machine learning algorithm to train a defect identification model, real-time analysis data in the process of car driving, identify road defect type.Model is deployed in cloud, regularly analyze road image, newly appeared defect is retrained by incremental learning, to improve the precision and adaptability of model.At the same time, the classified and prioritized road defects identified are helpful for targeted repair of the road by maintenance parties.The present application maximizes the use of basic and combined road defects, and can identify various possible road defects, including some newly appeared defects, which is beneficial to improve road safety, enhance driving experience, and help to more effectively use public resources for road repair.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of road engineering, more particularly, to a pavement defect identification method based on vehicle motion. BACKGROUND

[0002] With the acceleration of urbanization, the maintenance and management of urban roads are increasingly important. In particular, the identification and repair of road surface defects are related to the service life of the road, the safety of driving, and the operational efficiency of the city. However, existing road defect identification technologies have some problems and cannot meet the increasingly stringent road management requirements.

[0003] Existing road defect identification technologies mainly rely on image recognition technology, which analyzes real-time images of road surfaces to detect defects on road surfaces. These technologies generally use cameras to capture real-time images of road surfaces, and then use image recognition algorithms to analyze the images to identify defects on road surfaces.

[0004] This image recognition-based road defect identification technology has some problems, including:

[0005] 1) Image recognition technology is highly dependent on environmental conditions, such as lighting conditions and weather conditions, which can affect image quality and thus the accuracy of defect identification;

[0006] 2) Image recognition technology can only identify obvious defects in images, and the recognition effect is often poor for defects that are not easily displayed in images, such as small cracks and surface wear.

[0007] 3) Image recognition technology can only handle existing defects, and has difficulty identifying defects that may not have occurred at all.

[0008] 4) Image recognition involves a large amount of data processing, which can result in high costs.

[0009] In addition, in existing technologies, road defect identification relies on pre-set defect types, which lacks flexibility and may not be able to cope with various complex and changing defects in actual roads. Furthermore, in existing technologies, road defect identification and repair are usually carried out independently, and there is a lack of effective defect classification and repair priority ranking, which makes the efficiency and effectiveness of repair work need to be improved. SUMMARY

[0010] The technical problem to be solved by the present application is to provide a pavement defect identification method based on vehicle motion to solve the problems mentioned in the background.

[0011] To achieve the above-mentioned purpose, the present application adopts the following technical scheme:

[0012] A road defect identification method based on vehicle motion, comprising the following steps:

[0013] S1: defining a set of basic road defects, randomly splitting and combining the basic road defects to form a plurality of combined road defects;

[0014] S2: using a car equipped with various sensors to drive through each basic road defect and combined road defect, recording sensor data and labeling the corresponding defect type to form a feature data set;

[0015] S3: applying one or more machine learning algorithms to train a defect identification model using the feature data set obtained in step S2, the defect identification model being used to receive real-time data from the sensors during the driving process and output the corresponding road defect type;

[0016] S4: deploying the defect identification model to a cloud server, transmitting the sensor data during the driving process to the cloud server in real time through the vehicle communication system, and the cloud server analyzes and outputs the defect type and the corresponding recognition time in real time using the defect identification model;

[0017] Wherein, at regular intervals, on the cloud server, according to the time of identifying the defect, the road image photographed by the camera on the vehicle at the corresponding time is called, if the defect on the corresponding road image is not the basic road defect or combined road defect obtained in S1, then it is included in the data set in S2 for incremental learning to retrain the defect identification model;

[0018] S5: classifying and prioritizing the road defects predicted by the defect identification model at regular intervals.

[0019] In some embodiments, the sensors include but are not limited to acceleration sensors, gyroscopes, speed sensors.

[0020] In some embodiments, the basic road defects include but are not limited to various shapes of depressions, protrusions, cracks.

[0021] In some embodiments, the basis for the classification in S5 includes the severity of the defect, the location of the defect, and the resources required for repair.

[0022] In some embodiments, the basis for the repair priority ranking in S5 includes the severity of the defect and the location of the defect; the more severe the defect, the higher the priority; the closer the defect is to the main road or highway, the higher the repair priority of the defect.

[0023] The advantages of the present application over the prior art are:

[0024] The present application firstly defines a set of basic road defects, including various shapes, sizes of concave, convex, cracks and the like, which provides a basis for identifying diversified road defects. Then, the present application further forms combined road defects by randomly splitting and combining these basic road defects, greatly expanding the identification range of the model, which can cover more actual road defect conditions. In fact, in this way, the present application can not only identify known and common road defects, but also predict and identify possible and even some never-appeared road defects. This way can consider all possible defects to the greatest extent, so that the model has stronger adaptability and foresight.

[0025] The model of the present application not only uses basic and combined road defects in the initial training, but also in the actual operation process, every certain period of time, according to the real-time road images shot by the vehicle camera, the newly appeared, non-basic or combined road defects are included in the data set for incremental learning type model training. This makes the model can be dynamically updated and optimized according to the actual road conditions, and more accurately identify road defects.

[0026] The present application can automatically identify various defects on the road in real time by combining machine learning technology, including those defects that the camera may not identify. Since the data measured by vehicle sensors such as accelerometers and the like is greatly reduced compared to image data, the cost and computing amount of machine learning is smaller than image recognition, which makes the present application more cost-effective and can identify more diverse defects.

[0027] The present application can also classify and prioritize repair according to the severity, location and repair resources of the defects, which can more effectively help road repair work. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flowchart of the method of the present application. DETAILED DESCRIPTION

[0029] The specific embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0030] As Figure 1 shown is a schematic diagram of the method of the present application.

[0031] First, a set of basic road defects need to be defined, such as: circular concave with diameter of 10, 20, 30 cm; cracks with width of 5, 10, 15 cm; convex with height of 1, 2, 3 cm……In addition to this, other shapes of defects can also be considered, which can be determined artificially according to the basic defects commonly seen on the road.

[0032] Then, these basic road defects are randomly split and combined, such as combining a 10cm depression and a 5cm crack, such as splitting a circular defect into a semicircular defect and combining it with a crack, and so on, to form a new combined road defect… Combined road defects can be selected according to specific circumstances, such as if the storage capacity of the database is large enough, the training time allowed by the subsequent model is long enough, then enough combined road defects can be selected, and after obtaining the combined road defects, judge which combined road defects are often appeared combined with historical road pictures, so as to preferentially select the combined road defects often appeared.

[0033] As for defect types, there are actually many kinds to consider, some of which are as follows:

[0034] Depression: a pit on the road surface, which may be caused by the collapse of the underlying structure or the erosion of the road surface material.

[0035] Bulge: a protrusion on the road surface, which may be caused by the expansion of the underlying material or uneven road construction.

[0036] Crack: a crack on the road surface, which may be caused by temperature changes, moisture erosion, or pressure from heavy vehicles.

[0037] Water puddle: water accumulation on the road surface, which may be caused by unreasonable road design or problems with the drainage system.

[0038] Soft soil: loose soil on the road surface, which may be caused by rain or unconsolidated foundation.

[0039] Rockfall: rockfall that may occur on mountain roads, which may be caused by natural erosion or human damage.

[0040] Texture damage: loss of road surface texture, which may be caused by long-term wear and tear or construction quality problems.

[0041] Delamination: one or more layers of material on the road surface are peeled off, which may be caused by aging, wear and tear, or construction quality problems.

[0042] And the basic defects also need to consider the shape and specific size, the following table shows part of them, when constructing the data set, there can be more kinds of basic defects:

[0043] Type Shape Size Depression Circular Small (diameter < 10 cm) Depression Circular Medium (10 cm < diameter < 30 cm) Depression Circular Large (diameter > 30 cm) Depression Oblong Small (length < 30 cm, width < 10 cm) Depression Oblong Large (length > 30 cm, width > 10 cm) Elevation Circular Small (diameter < 10 cm) Elevation Circular Medium (10 cm < diameter < 30 cm) Elevation Circular Large (diameter > 30 cm) Elevation Oblong Small (length < 30 cm, width < 10 cm) Elevation Oblong Large (length > 30 cm, width > 10 cm) Crack Linear Small (length < 30 cm, width < 1 cm) Crack Linear Medium (30 cm < length < 100 cm, width < 1 cm) Crack Linear Large (length > 100 cm, width > 1 cm) Crack Net-shaped Small (area < 100 cm²) Crack Net-shaped Large (area > 100 cm2)

[0044] Of course, the basic defects can be constructed according to some defects that can be often seen in daily life.

[0045] Next, a car equipped with various sensors (such as gyroscopes, accelerometers, etc.) is driven over each of the basic road defects and the combined road defects. The car records the sensor data as it drives over the defects and labels which defect it is currently driving over. In this way, a feature dataset containing various defects is obtained.

[0046] Then, a defect recognition model is trained using this feature dataset using a machine learning algorithm (such as a neural network). This model can receive real-time data from the sensors of a car during its driving and output the corresponding road defect type.

[0047] After the model is trained, it is deployed to a cloud server. In this way, when a car (although this model can be deployed to other vehicles, if the road-related department can often send vehicles for road patrol, the same type of vehicle originally used for training can be used for detection, which can relatively improve the accuracy) is driving, its sensor data is transmitted to the cloud server in real time, and the cloud server analyzes these data in real time using the defect recognition model and outputs the road defect type.

[0048] Finally, during the driving of the car, the cloud will predict the defects of the road in real time. The model can be updated every certain period of time, for example, if a defect is predicted and the corresponding road image is viewed using the on-board camera, it is found that the defect on this road image is not in the basic road defect or the combined road defect, then this new defect and its corresponding sensor data will be included in the dataset and incremental learning model training will be performed, so as to continuously update and optimize the defect recognition model.

[0049] Every certain period of time, for example, every week or every month, according to all the road defects measured and predicted by the defect recognition model, combined with the conditions required for repair, detailed classification and repair priority ranking are performed. The classification and repair priority ranking can be set manually, for example:

[0050] According to the severity of the defect. Serious defects such as deep depressions or wide cracks may pose a greater safety risk to drivers and therefore need to be addressed first. At the same time, serious defects may require more complex or higher cost repair methods. Similarly, classification can be made according to severity, such as defects that seriously affect driving are classified into one category.

[0051] According to the location of the defect. For example, defects on highways or main roads may affect more drivers and therefore may need to be addressed first. While defects in residential areas or alleys, although they also need to be repaired, may have lower priority. Similarly, classification can be made according to the location of the defect, such as defects located in an area can be classified into one category and repaired together.

[0052] According to the resources required for repair. Different defects may require different repair methods and resources. For example, a depression may require a filling material, a crack may require a sealing material, and a protrusion may require a polishing device. The availability of these resources required for repair also needs to be considered. Similarly, classification can be made according to the resources required for repair.

[0053] Through this method, the road maintenance team of the city can more accurately understand the defect situation on the road, and can timely repair, improve the service life of the road and the safety of driving. At the same time, this method can also learn and respond to new road defects in real time, improve the adaptability and robustness of the model.

[0054] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for identifying a road surface defect based on vehicle motion, characterized by, The method comprises the following steps: S1: defining a set of basic road defects, randomly splitting and combining the basic road defects to form a plurality of combined road defects; after obtaining the combined road defects, judging the occurrence frequency of each combined road defect in an actual road in combination with historical road pictures, and preferentially selecting a combined road defect with a higher occurrence frequency as a combined road defect for training; S2: using a car equipped with various sensors to drive over each basic road defect and combined road defect, recording sensor data and labeling the corresponding defect type to form a feature data set; S3: applying one or more machine learning algorithms to train a defect recognition model using the feature data set obtained in step S2, wherein the defect recognition model is used to receive real-time data of the sensors during the driving of the car and output the corresponding road defect type; S4: deploying the defect recognition model to a cloud server, transmitting the sensor data of the vehicle during driving to the cloud server in real time through a vehicle-mounted communication system, and using the defect recognition model to analyze and output the defect type and the corresponding recognition time in real time; wherein, at regular intervals, the cloud server retrieves the road images taken by the camera on the vehicle at the corresponding time according to the time of the recognized defect, and if the defect on the corresponding road image is not a basic road defect or a combined road defect obtained in S1, it is included in the data set in S2 for incremental learning to retrain the defect recognition model; during real-time recognition and incremental learning, the defect recognition model uses the sensor data as input for defect recognition, and the road images taken by the vehicle-mounted camera are only used for visual verification of the defect type after the defect is recognized and to drive the incremental update of the feature data set; S5: classifying and prioritizing the road defects predicted by the defect recognition model for repair at regular predetermined intervals.

2. The road defect recognition method based on vehicle motion according to claim 1, wherein The sensors include but are not limited to acceleration sensors, gyroscopes, and speed sensors.

3. The method of claim 1, wherein The basic road defects include but are not limited to various shapes of depressions, protrusions, and cracks.

4. The method of claim 1, wherein The basis for the classification in S5 includes the severity of the defect, the location of the defect, and the resources required for repair.

5. The method of claim 1, wherein The basis for the repair priority ranking in S5 includes the severity of the defect and the location of the defect; the more severe the defect, the higher the priority; the closer the defect is to the main road or highway, the higher the repair priority of the defect.

Images