A method for detecting the range and size of rutting, subsidence, and bumps of road lines by laser

Abstract

The present invention relates to a method for detecting the range and size of rutting, subsidence, and bumping of road lines by laser, comprising the following steps: S1. Install a high-definition camera, a central industrial control computer, and a line laser emitter device on the inspection vehicle. The high-definition camera acquires images according to the instructions of the central industrial control computer, and a 3D convolutional neural network algorithm deployed on the central industrial control computer fits the coordinate points [u, v] of the laser line in the image; S2. Use a line type discrimination algorithm to perform linear and curvature analysis on the fitted coordinate points [u, v] of the laser line in a single-frame image, and calculate the disease index value. Set a certain threshold value threshold. The range of the laser line where the value exceeds this threshold in a single-frame image is the disease width in this image; S3. Continuously analyze through the 3D convolutional neural network in the time dimension to obtain the disease calculation index value in the continuous frames of the picture, calculate the continuous frame number ftime when the calculated index value exceeding this threshold appears, and calculate and obtain the length of the disease; S4. Save the recognized results locally and transmit them back to the cloud through the mobile network.

Description

Technical Field

[0001] The invention relates to the technical field of road engineering, and in particular to a method for detecting the range and size of road line laser rutting subsidence. Background Art

[0002] With the rapid development of road traffic construction in my country, my country has built a huge network of municipal, expressway, medium and low-level roads. The large-scale road facilities also bring great challenges to pavement disease detection and pavement maintenance.

[0003] In recent years, with the promotion and application of deep learning in the field of computer vision, new road lightweight detection systems have also partially replaced traditional detection methods. However, the recognition effect of deformation-related diseases is not significant enough, and the scope and size of the disease cannot be accurately detected, especially deformation-related diseases such as subsidence, bumps, rutting, and misalignment. Accurately detecting the scope and size of deformation-related diseases such as rutting has practical guiding significance for road maintenance.

[0004] Among the existing technologies, the common methods for detecting road rutting, subsidence and bumps mainly rely on manual detection, multifunctional road surface detection vehicles and 3D laser point clouds. The manual method can only detect road rutting by random sampling, and cannot comprehensively and systematically reflect the road rutting and bump conditions. It is inefficient, highly dangerous and highly subjective. The automatic road rutting detection method uses a multifunctional road surface detection vehicle to measure road rutting, but the vehicle needs to be specially modified. The number of equipment is limited and cannot be used on a large scale and at a high frequency. Moreover, this type of equipment is generally only used for annual road inspections and cannot guide daily maintenance. The 3D laser point cloud method uses laser radar to obtain 3D laser point cloud data of the road surface, and then analyzes and extracts rutting, subsidence and bumps. However, this method is not conducive to promotion and use due to the high cost of laser radar. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a road line laser rutting, subsidence, and bump range and size detection method, which uses lightweight equipment to collect road rutting, subsidence and other deformation diseases, and uses deep learning algorithms to identify the length of road bumps, subsidence, rutting and other deformation diseases. With its low-cost, lightweight sensors and intelligent algorithms, it can quickly identify the range and size of road deformation diseases, and track the development trend of deformation diseases through high-frequency inspections, reduce the labor cost of road maintenance, and promote efficient and intelligent road maintenance management.

[0006] The above-mentioned object of the present invention is achieved through the following technical solutions:

[0007] A method for detecting the range and size of road line laser rutting subsidence, comprising the following steps:

[0008] S1. Install high-definition cameras, central industrial control computers, and line laser emitter devices on the inspection vehicle. Start the relevant devices. During the driving process of the inspection vehicle, the high-definition camera collects images according to the instructions of the central industrial control computer, and the 3D convolutional neural network algorithm deployed in the central industrial control computer fits the coordinate points [u, v] of the laser line in the image.

[0009] S2. Use the line type discrimination algorithm to perform linear and curvature analysis on the laser line fitting coordinate points [u, v] in a single-frame image, and calculate the disease index value. Set a certain threshold threshold. The range of the laser line where the value exceeds this threshold in a single-frame image is the disease width in this image.

[0010] S3. Continuously analyze through the 3D convolutional neural network in the time dimension to obtain the disease calculation index value within the continuous frames of the picture. By calculating the continuous frame times ftime when the calculation index value exceeding this threshold appears, combined with the actual vehicle speed, the length of the disease can be calculated.

[0011] S4. Save the recognized results locally and transmit them back to the cloud through the mobile network.

[0012] In a preferred example of the present invention, it can be further configured as: in step S1, during the driving process of the inspection vehicle, start the system and start the line laser emitter. On a flat road surface, a straight line laser can be observed in the image collected by the high-definition camera.

[0013] In a preferred example of the present invention, it can be further configured as: in step S1, use the 3D convolutional neural network algorithm deployed in the industrial control computer to perform regression fitting on the laser line in the image to obtain the coordinate list [u, v] on the laser line.

[0014] In a preferred example of the present invention, it can be further configured as: in step S2, when using the line type discrimination algorithm to perform linear and curvature analysis on the laser line fitting coordinate points [u, v] in a single-frame image, use the least squares method to perform linear fitting on [u, v], calculate the disease index value, set a certain threshold threshold, and the range of the laser line where the value exceeds this threshold in a single-frame image is the disease width in this image.

[0015] In a preferred example of the present invention, it can be further configured as: the frame rate of the high-definition camera is not less than 60fps.

[0016] In a preferred example of the present invention, it can be further configured as: the inspection vehicle needs to maintain a constant speed during the driving process.

[0017] In a preferred example, the present invention can be further configured such that the central industrial control computer is powered by the inspection vehicle.

[0018] In summary, the present invention includes at least one of the following beneficial technical effects:

[0019] 1. Compared with manual inspection, the method of the present application can quickly and intelligently analyze the scope of road surface deformation diseases. The staff only need to focus on driving, which greatly improves the work efficiency of the staff and reduces safety risks.

[0020] 2. Compared with existing heavy detection equipment, the method and system of the present application mainly rely on lightweight sensors. By using intelligent algorithms, it can cope with external interference, greatly reducing equipment costs and operating costs. With the help of edge computing and artificial intelligence algorithms, the scope of road surface deformation can be calculated in real time, and the inspection efficiency of road surface deformation diseases is significantly improved.

[0021] 3. The 3D convolutional neural network algorithm and judgment logic can solve problems such as external light interference, unclear laser line imaging, laser line interruption caused by excessive road surface deformation, and road surface deformation caused by facilities such as manhole covers. The analysis has a higher fine-grained level and better robustness.

[0022] 4. In terms of maintenance benefits, the method mentioned in the present application quickly calculates the scope of deformation diseases, effectively guiding the maintenance department to reasonably formulate maintenance plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flowchart of the present invention.

[0024] Figure 2 is a continuous multi-frame picture of image acquisition of the present invention.

[0025] Figure 3 is a multi-frame picture of line laser fitted by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application; obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0027] In the description of the present application, it should be noted that the orientation or positional relationship indicated by terms such as "upper", "lower", "inner", "outer", "top / bottom end", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present application. In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance.

[0028] In the description of the present application, it should be noted that unless otherwise clearly specified and limited, terms such as "installed", "equipped with", "sheathed / connected", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or a signal connection; it can be directly connected, or indirectly connected through an intermediate medium, and can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0029] Embodiment 1:

[0030] Referring to Figure 1 , a method for detecting the range and size of rutting, subsidence, and potholes of road lines disclosed by the present invention includes the following steps:

[0031] S1. Install a high-definition camera, a central industrial control computer, and a line laser emitter device on the inspection vehicle, start the relevant devices. During the driving process of the inspection vehicle, the high-definition camera collects images according to the instructions of the central industrial control computer, and the 3D convolutional neural network algorithm deployed on the central industrial control computer fits the coordinate points [u, v] of the laser line in the image.

[0032] S2. Use the line type discrimination algorithm to perform linear and curvature analysis on the laser line fitting coordinate points [u, v] in a single-frame image, and calculate the disease index value. Set a certain threshold threshold. The range of the laser line where the value exceeds the threshold in a single-frame image is the disease width in this image.

[0033] S3. Continuously analyze through the 3D convolutional neural network in the time dimension to obtain the disease calculation index value within the continuous frames of the picture. By calculating the continuous frame times ftime when the calculation index value exceeding the threshold appears, and combining the actual vehicle speed, the length of the disease can be calculated.

[0034] S4. Save the recognized results locally and transmit them back to the cloud through the mobile network.

[0035] The implementation method of the road rut settlement and bump range detection method based on 3D convolutional neural network proposed in this application is as follows: Install devices such as line laser emitters, high-definition cameras, and central industrial control computers on the inspection vehicle. The high-definition camera is connected to the central industrial control computer. After the device is started, the line laser emitter can emit obvious line lasers, and the central industrial control computer sends an image acquisition instruction to the high-definition camera. The central industrial control computer is powered by the inspection vehicle.

[0036] It is required that the frame rate of the on-vehicle camera is not less than 60fps, and the vehicle needs to keep a constant speed during the detection process.

[0037] (1) Start the system and start the line laser emitter. On a flat road surface, a straight line laser can be observed in the image collected by the camera.

[0038] (2) Use the 3D convolutional neural network algorithm deployed in the industrial control computer to perform regression fitting on the laser line in the image to obtain the coordinate list [u, v] on the laser line.

[0039] (3) Use the least squares method to perform linear fitting on [u, v], calculate the disease index value, set a certain threshold threshold, and the laser line range where value exceeds this threshold in a single-frame image is the disease width in this image.

[0040] (4) Continuously analyze through the 3D convolutional neural network in the time dimension to obtain the disease calculation index value in consecutive frames of the picture. By calculating the consecutive frame times ftime when the calculation index value exceeding this threshold appears, combined with the actual vehicle speed, the length of the disease can be calculated.

[0041] The recognition results are saved locally and transmitted back to the cloud through the mobile network.

[0042] The implementation principle of this embodiment is as follows: The present invention discloses a method for detecting the range and size of road line laser ruts, settlements, and bumps. It uses lightweight devices to collect deformation diseases such as road ruts and settlements on the road surface, and uses deep learning algorithms to identify the disease lengths of deformation diseases such as bumps, settlements, and ruts on the road surface. With its cost, lightweight sensors, and intelligent algorithms, it can quickly identify the range and size of road deformation diseases, and track the development trend of deformation diseases through high-frequency inspections, reducing the labor cost of road maintenance and promoting the high-efficiency and intelligence of road maintenance management.

[0043] The embodiments of this specific implementation manner are all preferred embodiments of the present invention, and do not limit the protection scope of the present invention accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A method for detecting the range and size of rutting, subsidence and bump of road lines by laser, characterized in that: Including the following steps: S1. Install a high-definition camera, a central industrial control computer, and a line laser emitter device on the inspection vehicle, start the relevant devices. During the driving process of the inspection vehicle, the high-definition camera collects images according to the instructions of the central industrial control computer, and the 3D convolutional neural network algorithm deployed in the central industrial control computer fits the coordinate points [u, v] of the laser line in the image; S2. Use the line type discrimination algorithm to perform linear and curvature analysis on the laser line fitting coordinate points [u, v] in a single-frame image, and calculate the disease index value. Set a certain threshold threshold. The range of the laser line where the value exceeds this threshold in a single-frame image is the disease width in this image; S3. Continuously analyze through the 3D convolutional neural network in the time dimension to obtain the disease calculation index value within the continuous frames of the picture. By calculating the continuous frame times ftime when the calculated index value exceeding this threshold appears, combined with the actual vehicle speed, the length of the disease can be calculated; S4. Save the recognized results locally and transmit them back to the cloud through the mobile network.

2. The method for detecting the range and size of rutting, settlement and bump of road lines by laser according to claim 1, wherein In step S1, during the driving process of the inspection vehicle, start the system and start the line laser emitter. On a flat road surface, a straight line laser can be observed in the image collected by the high-definition camera.

3. The method for detecting the range and size of rutting, subsidence and heaving of road lines according to claim 1, wherein In step S1, use the 3D convolutional neural network algorithm deployed in the industrial control computer to perform regression fitting on the laser line in the image to obtain the coordinate list [u, v] on the laser line.

4. A method for detecting the range and size of rutting, settlement, and heaving of road lines by laser according to claim 1, characterized in that In step S2, when using the line type discrimination algorithm to perform linear and curvature analysis on the laser line fitting coordinate points [u, v] in a single-frame image, use the least squares method to perform linear fitting on [u, v], and calculate the disease index value. Set a certain threshold threshold. The range of the laser line where the value exceeds this threshold in a single-frame image is the disease width in this image.

5. A method for detecting the range and size of rutting, settlement and bump of road lines by laser, according to claim 1, characterized in that The frame rate of the high-definition camera is not less than 60fps.

6. The method for detecting the range and size of rutting, settlement and bump of road lines according to claim 1, characterized in that, The inspection vehicle needs to maintain a constant speed during the driving process.

7. A method for detecting the range and size of rutting, subsidence and heaving of road lines by laser according to claim 1, characterized in that The central industrial control computer is powered by the inspection vehicle.

Images