Road inspection method and device, computer device and storage medium

Abstract

The application relates to a road inspection method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: forming a structured light grid on a to-be-inspected road through a structured light module, collecting an image of the structured light grid to obtain a structured light grid image, obtaining camera calibration parameters according to the structured light grid image, calibrating a camera acquisition module by using the camera calibration parameters, collecting an image through the calibrated camera acquisition module during the inspection process to obtain a road inspection image, detecting a target object in the road inspection image to obtain road disease detection information, obtaining driving comfort information of the to-be-inspected road, obtaining road structure depth information of the to-be-inspected road, and obtaining road inspection information corresponding to the to-be-inspected road according to the road disease detection information, the driving comfort information and the road structure depth information. The method can improve the road inspection information acquisition efficiency and the road inspection efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road inspection, in particular to a road inspection method and device, computer equipment, a storage medium and a computer program product. BACKGROUND

[0002] With the continuous development of society, the transportation industry is growing rapidly. As an important part of transportation, road safety is particularly important. After the road is put into use, various road diseases will occur over time. In order to ensure the safety of driving, road maintenance is very important, and road inspection is a very important part of road maintenance.

[0003] However, the road inspection method in the prior art has the problems of complicated operation steps and low detection efficiency, which is not conducive to improving the acquisition efficiency of road inspection information. SUMMARY

[0004] Therefore, it is necessary to provide a road inspection method, device, computer equipment, computer readable storage medium and computer program product capable of improving the acquisition efficiency of road inspection information.

[0005] In a first aspect, the present application provides a road inspection method, which comprises:

[0006] forming a structured light grid on the road to be inspected by a structured light module on a vehicle, and collecting images of the structured light grid to obtain a structured light grid image;

[0007] obtaining camera calibration parameters according to the structured light grid image, and calibrating a camera acquisition module on the vehicle using the camera calibration parameters;

[0008] collecting images of the road to be inspected by the calibrated camera acquisition module to obtain road inspection images;

[0009] detecting target objects in the road inspection images to obtain road disease detection information corresponding to the road to be inspected;

[0010] obtaining driving comfort information corresponding to the road to be inspected, and obtaining road structure depth information corresponding to the road to be inspected, and obtaining road inspection information corresponding to the road to be inspected according to the road disease detection information, the driving comfort information and the road structure depth information.

[0011] In one embodiment, the driving comfort information corresponding to the road to be inspected is obtained by:

[0012] Collecting a vehicle vibration signal of the inspection vehicle in a driving process through a vibration data collection module on the inspection vehicle;

[0013] According to the vehicle vibration signal, determining driving comfort information corresponding to the to-be-inspected road.

[0014] In one of the embodiments, the vehicle vibration signal includes vertical acceleration, and the determining of the driving comfort information corresponding to the to-be-inspected road according to the vehicle vibration signal includes:

[0015] Converting the vertical acceleration into a road surface flatness corresponding to the to-be-inspected road;

[0016] According to the road surface flatness corresponding to the to-be-inspected road, determining a driving quality index corresponding to the to-be-inspected road as the driving comfort information.

[0017] In one of the embodiments, the obtaining of the road structure depth information corresponding to the to-be-inspected road includes:

[0018] Performing a laser radar scan on the to-be-inspected road through a laser radar module on the inspection vehicle to obtain laser point cloud data;

[0019] According to the laser point cloud data, determining the road structure depth information corresponding to the to-be-inspected road.

[0020] In one of the embodiments, the obtaining of the camera calibration parameter according to the structured light grid image includes:

[0021] According to the structured light grid image, determining position information and depth information of a target road surface by using a triangulation principle; the target road surface is a road surface on which the structured light grid is located;

[0022] According to the position information and the depth information, generating a coordinate system parameter;

[0023] According to the coordinate system parameter, determining the camera calibration parameter.

[0024] In one of the embodiments, the method further includes:

[0025] Obtaining a global positioning system time through a pre-installed time synchronization protocol;

[0026] Adjusting a local system time according to the global positioning system time, so that the adjusted local system time is synchronized with the global positioning system time.

[0027] In a second aspect, the application further provides a road inspection device, which includes:

[0028] a structured light collection module, configured to form a structured light grid on a road to be inspected and collect an image of the structured light grid to obtain a structured light grid image;

[0029] a camera calibration module, configured to obtain camera calibration parameters according to the structured light grid image, and calibrate a camera collection module on the inspection vehicle by using the camera calibration parameters;

[0030] an image collection module, configured to collect an image of the road to be inspected by using the calibrated camera collection module to obtain a road inspection image;

[0031] a disease detection module, configured to detect a target object in the road inspection image to obtain road disease detection information corresponding to the road to be inspected;

[0032] an inspection information generation module, configured to obtain driving comfort information corresponding to the road to be inspected and road structure depth information corresponding to the road to be inspected, and obtain road inspection information corresponding to the road to be inspected according to the road disease detection information, the driving comfort information and the road structure depth information.

[0033] In a third aspect, the present application further provides a road inspection system, which is arranged on an inspection vehicle and includes a structured light module, a laser radar module, a vibration data collection module, a camera collection module and a control module, wherein:

[0034] The control module is configured to implement the steps of the method according to any one of the aspects when executed by a processor.

[0035] In a fourth aspect, the present application further provides a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0036] a structured light grid is formed on a road to be inspected by using a structured light module on an inspection vehicle, and an image of the structured light grid is collected to obtain a structured light grid image;

[0037] camera calibration parameters are obtained according to the structured light grid image, and a camera collection module on the inspection vehicle is calibrated by using the camera calibration parameters;

[0038] an image of the road to be inspected is collected by using the calibrated camera collection module to obtain a road inspection image;

[0039] a target object in the road inspection image is detected to obtain road disease detection information corresponding to the road to be inspected;

[0040] Obtaining the driving comfort information corresponding to the to-be-inspected road, and obtaining the road structure depth information corresponding to the to-be-inspected road, and obtaining the road inspection information corresponding to the to-be-inspected road according to the road disease detection information, the driving comfort information and the road structure depth information.

[0041] In a fifth aspect, a computer readable storage medium is provided. The computer readable storage medium stores a computer program. The computer program, when executed by a processor, implements the steps of the method in any of the preceding aspects.

[0042] In a sixth aspect, a computer program product is provided. The computer program product includes a computer program. The computer program, when executed by a processor, implements the steps of the method in any of the preceding aspects.

[0043] The road inspection method, device, computer device, storage medium and computer program product, utilize the structured light module on the inspection vehicle to form a structured light grid on the to-be-inspected road, and perform image acquisition on the structured light grid to obtain a structured light grid image, obtain camera calibration parameters based on the structured light grid image, and calibrate the camera acquisition module on the inspection vehicle using the camera calibration parameters; utilize the calibrated camera acquisition module to perform image acquisition on the to-be-inspected road in the process that the inspection vehicle drives on the to-be-inspected road according to the road inspection task, to obtain a road inspection image; perform disease detection based on a target object in the road inspection image to obtain road disease detection information corresponding to the to-be-inspected road; obtain driving comfort information and road structure depth information corresponding to the road inspection image, and obtain road inspection information corresponding to the road inspection task based on the road disease detection information, the driving comfort information and the road structure depth information corresponding to the to-be-inspected road; in this way, the camera acquisition module of the inspection vehicle can be accurately and quickly calibrated and corrected by utilizing the structured light module to form a structured light grid on the to-be-inspected road, avoiding complex and tedious calibration operations on the camera acquisition module of the inspection vehicle before the road inspection task, so that the calibrated camera acquisition module can accurately acquire the road image of the to-be-inspected road, and the detection efficiency of road diseases and the acquisition efficiency of road inspection information can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] For better describing and illustrating those embodiments and examples disclosed herein, one or more drawings can be referred to. Additional details or examples used to describe the drawings should not be considered as limiting the scope of any of the disclosed inventions, the presently described embodiments and / or examples, and the best mode presently understood of these inventions.

[0045] Figure 1An application environment diagram of a road inspection method in an embodiment;

[0046] Figure 2 A flowchart diagram of a road inspection method in an embodiment;

[0047] Figure 3 A flowchart diagram of obtaining driving comfort in an embodiment;

[0048] Figure 4 A flowchart diagram of determining driving comfort in an embodiment;

[0049] Figure 5 A flowchart diagram of obtaining road construction depth in an embodiment;

[0050] Figure 6 A framework diagram of a road disease collection system in an embodiment;

[0051] Figure 7 A flowchart diagram of a road inspection method in an embodiment;

[0052] Figure 8 A structural block diagram of a road inspection device in an embodiment;

[0053] Figure 9 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0055] The road inspection method provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment shown. Among them, the road disease collection system 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. Among them, the road disease collection system 102 forms a structured light grid on the road to be inspected by the structured light module on the inspection vehicle, and collects images of the structured light grid to obtain a structured light grid image; the road disease collection system 102 obtains camera calibration parameters according to the structured light grid image; and uses the camera calibration parameters to calibrate the camera acquisition module on the inspection vehicle; the road disease collection system 102 obtains road inspection images by the calibrated camera acquisition module during the inspection vehicle driving on the road to be inspected according to the road inspection task; the road disease collection system 102 detects the target object in the road inspection image to obtain the road disease detection information corresponding to the road to be inspected; the road disease collection system 102 obtains the driving comfort information corresponding to the road to be inspected, and obtains the road structure depth information corresponding to the road to be inspected, and obtains the road inspection information corresponding to the road inspection task according to the road disease detection information, the driving comfort information and the road structure depth information; the road disease collection system 102 sends the road inspection information to the server 104. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0056] In some embodiments, as shown in Figure 2 , a road inspection method is provided, which is applied to the road inspection system in Figure 1 for example, including the following steps:

[0057] Step 210, forming a structured light grid on the road to be inspected by a structured light module on the inspection vehicle, and collecting images of the structured light grid to obtain a structured light grid image.

[0058] Among them, the inspection vehicle can be a vehicle used for road inspection of the road to be inspected, the road to be inspected can be a road that needs to be inspected, the structured light module can be a system structure composed of a projector and a camera, the structured light grid can be a grid formed by structured light on the road to be inspected, and the structured light grid image can be a grid image collected by the camera.

[0059] As an example, the equipment that can be carried by the inspection vehicle includes but is not limited to camera, light camera, vibration sensor, GPS, laser emitter and radar; the road to be inspected can be the road determined by the specific inspection task to be inspected.

[0060] In an embodiment, the structure light module is started by the inspection operator through the road inspection system or is automatically started by the system based on preset requirements. The structure light module can project structure light to the road surface of the road to be inspected to form a structure light grid. The structure light grid image is obtained by the camera acquisition module.

[0061] In step 220, the camera calibration parameters are obtained based on the structure light grid image, and the camera acquisition module on the inspection vehicle is calibrated based on the camera calibration parameters.

[0062] The camera calibration parameters can be parameters for calibrating the camera, and the calibration can be a process of establishing a corresponding relationship between the camera imaging geometric model and the actual space object.

[0063] As an example, the calibration picture obtained by the camera acquisition module can be a structure light grid picture. The calibration picture is uploaded to the cloud platform, and the cloud platform downloads the calibration picture to the camera acquisition module. The camera calibration parameters are obtained based on the calibration picture.

[0064] As an example, the camera calibration parameters include but are not limited to camera intrinsic parameters, camera extrinsic parameters, and focal length of the camera. The camera extrinsic parameters can be divided into a rotation matrix and a translation vector. The camera extrinsic parameters can represent the conversion relationship between a point in the world coordinate system and a point in the camera coordinate system.

[0065] In an embodiment, the position information and the depth information of the road to be inspected are obtained based on the structure light grid image by a triangulation principle. At least two structure light cameras are used to cover a space with a preset area size, which includes but is not limited to 10m*10m. The coordinate system parameters are obtained. The camera calibration parameters are obtained based on the coordinate system parameters. The corresponding relationship between the camera imaging geometric model of the camera acquisition module and the road surface of the road to be inspected is established based on the camera calibration parameters. The calibration of the camera acquisition module on the inspection vehicle is completed.

[0066] In step 230, the road to be inspected is imaged by the calibrated camera acquisition module to obtain a road inspection image.

[0067] The road inspection image can be a road image of the road to be inspected obtained by the camera acquisition module.

[0068] As an example, the road inspection image can be collected by the camera collection module, and the file format of the road inspection image includes, but is not limited to, bitmap format, Tag Image File Format (TIFF), Joint Photographic Experts Group (JPEG) and Portable Network Graphics (PNG) format.

[0069] In an embodiment, the inspection vehicle drives from the starting point of the road to be inspected according to the road inspection task, and during the process of driving to the end point of the road to be inspected, the camera collection module collects images of the road to be inspected to obtain the road inspection image.

[0070] In step 240, the target object in the road inspection image is detected to obtain the road disease detection information corresponding to the road to be inspected.

[0071] The disease detection can detect the road use degree and the road integrity.

[0072] As an example, the target object in the road inspection image includes, but is not limited to, the road surface of the road to be inspected and the along-line facility of the road to be inspected, and the disease detection can be realized by the control module in the road inspection system. The control module includes, but is not limited to, a vehicle-mounted screen, a transmission module and an edge box. The transmission module includes, but is not limited to, an antenna, an Internet of Things card and a router.

[0073] In an embodiment, the edge box can perform picture inference on the target object in the road inspection image by using a deep learning vision algorithm to realize the disease detection of the target object and obtain the road disease detection information corresponding to the road to be inspected.

[0074] In step 250, the driving comfort information corresponding to the road to be inspected and the road construction depth information corresponding to the road to be inspected are obtained, and the road inspection information corresponding to the road to be inspected is obtained according to the road disease detection information, the driving comfort information and the road construction depth information.

[0075] The driving comfort information can be information for representing the driving comfort of the road, and the road construction depth information can be information for representing the road construction depth.

[0076] As an example, the driving comfort information includes, but is not limited to, the driving quality index.

[0077] In an embodiment, after the edge box obtains the road inspection information corresponding to the road inspection task, the edge box can further upload the road inspection information to a cloud platform through the transmission module, and the transmission mode includes but is not limited to wireless communication; the cloud platform supports exporting disease reports and visualizing real-time display of road data, and the cloud platform is further used for detecting the running state of the system, and sending a response instruction for remote or local maintenance when the cloud platform detects that the running state of the system is abnormal.

[0078] In the road inspection method, a structured light module on the inspection vehicle is used to form a structured light grid on the to-be-inspected road, an image of the structured light grid is acquired by image acquisition on the structured light grid, camera calibration parameters are obtained based on the structured light grid image, and the camera acquisition module on the inspection vehicle is calibrated using the camera calibration parameters; the calibrated camera acquisition module is used to acquire images of the to-be-inspected road during driving of the inspection vehicle on the to-be-inspected road according to the road inspection task, to obtain road inspection images; target objects in the road inspection images are detected to obtain road disease detection information corresponding to the to-be-inspected road; driving comfort information and road structure depth information corresponding to the road inspection images are obtained, and road inspection information corresponding to the road inspection task is obtained based on the road disease detection information, the driving comfort information, and the road structure depth information corresponding to the to-be-inspected road. In this way, the camera acquisition module of the inspection vehicle can be accurately and quickly calibrated and corrected by forming a structured light grid on the to-be-inspected road using a structured light module, avoiding complex and tedious calibration operations on the camera acquisition module of the inspection vehicle before the road inspection task, so that the calibrated camera acquisition module can accurately acquire road images of the to-be-inspected road, thereby effectively improving the detection efficiency of road diseases and effectively improving the acquisition efficiency of road inspection information.

[0079] In some embodiments, as shown in Figure 3 The driving comfort information corresponding to the to-be-inspected road includes at least one of the following:

[0080] Step 310: acquiring a vehicle vibration signal of the inspection vehicle during driving through a vibration data acquisition module on the inspection vehicle.

[0081] The vibration data acquisition module can be a vibration sensor for acquiring vibration data generated by the inspection vehicle during the inspection.

[0082] As an example, the vibration data acquisition module includes but is not limited to a vibration sensor, and the vehicle vibration signal includes but is not limited to vertical acceleration.

[0083] In one embodiment, the vibration sensor measures the vertical acceleration of the inspection vehicle during the driving of the to-be-inspected road.

[0084] At step 320, the driving comfort information corresponding to the to-be-inspected road is determined according to the vehicle vibration signal.

[0085] The driving comfort information can be information used to represent the driving comfort of the road.

[0086] In one embodiment, the vertical acceleration is converted into the road flatness based on a preset conversion rule, and the riding quality index of the to-be-inspected road is obtained based on the road flatness, and the riding quality index is taken as the driving comfort information corresponding to the to-be-inspected road.

[0087] Through the above embodiment, the vibration data of the inspection vehicle during the inspection process is collected by the vibration data collection module, and the driving comfort information of the to-be-inspected road is obtained, so that the driving comfort information of the to-be-inspected road can be obtained in time.

[0088] In some embodiments, as shown in Figure 4 The vehicle vibration signal includes the vertical acceleration, and the driving comfort information corresponding to the to-be-inspected road is determined according to the vehicle vibration signal, including:

[0089] At step 410, the vertical acceleration is converted into the road flatness corresponding to the to-be-inspected road.

[0090] As an example, the road flatness (International Roughness Index, IRI) can be the vertical deviation of the road surface relative to the ideal plane.

[0091] As an example, the conversion of the vertical acceleration into the road flatness can be realized by a preset software; for example, after laser ranging, the vertical acceleration of the vehicle during the driving of the to-be-inspected road is measured by an acceleration sensor, the vertical displacement of the detector is obtained by twice integration of the vertical acceleration, the laser ranging result is corrected based on the vertical displacement, the influence of the vehicle vibration on the measurement result is eliminated, the accurate road profile is obtained, and the road flatness is obtained.

[0092] In one embodiment, the vertical acceleration is converted into the road flatness corresponding to the to-be-inspected road based on a preset conversion rule.

[0093] At step 420, the riding quality index (RQI) corresponding to the to-be-inspected road is determined according to the road flatness corresponding to the to-be-inspected road, and is taken as the driving comfort information.

[0094] As an example, the driving quality index can be used to evaluate the relationship between the road surface flatness and the driving comfort.

[0095] As an example, the greater the road surface flatness, the smaller the driving quality index; for example, the relationship between the road surface flatness and the driving quality index can be indicated by the following formula:

[0096] Wherein, a0, a1 can be a preset fixed value.

[0097] In one embodiment, based on the above road surface flatness, the driving quality index corresponding to the to-be-inspected road is obtained, and the driving quality index is used as the driving comfort information corresponding to the to-be-inspected road.

[0098] Through the above embodiment, based on the preset conversion rule, the vehicle vibration signal is converted into the road surface flatness corresponding to the to-be-inspected road, and the driving comfort information corresponding to the to-be-inspected road is obtained based on the road surface flatness. The driving comfort information corresponding to the to-be-inspected road can be obtained based on the preset calculation method.

[0099] In some embodiments, as shown in Figure 5 The above obtaining the road structure depth information corresponding to the to-be-inspected road comprises:

[0100] Step 510, performing laser radar scanning on the to-be-inspected road by the laser radar module on the inspection vehicle to obtain laser point cloud data;

[0101] Wherein, the laser radar module can be a radar system that detects target position, speed and other characteristic quantities by emitting laser.

[0102] As an example, the laser radar module includes but is not limited to a laser emitter and a receiving radar.

[0103] In one embodiment, the laser emitter emits laser to the to-be-inspected road, and the radar receives the returned laser to scan the road and obtain the laser point cloud data.

[0104] Step 520, determining the road structure depth information corresponding to the to-be-inspected road according to the laser point cloud data.

[0105] Wherein, the laser point cloud data can be data composed of scanning point coordinates of laser.

[0106] As an example, the laser point cloud data can be laser scanning point data emitted by the laser emitter and received by the radar.

[0107] In one embodiment, based on the above laser point cloud data, the construction depth information of the road is calculated, and the above laser point cloud data can also be used for road modeling and auxiliary road setting digital asset construction.

[0108] Through the above embodiment, the laser radar module of the above system is used to scan the to-be-inspected road to obtain the construction depth information corresponding to the to-be-inspected road, so that the construction depth information of the to-be-inspected road can be obtained.

[0109] In some embodiments, the camera calibration parameters are obtained according to the above structured light grid image, including:

[0110] According to the above structured light grid image, the position information and the depth information of the target road surface are determined by using the triangulation principle; the target road surface is the road surface on which the structured light grid is located; the coordinate system parameters are generated according to the position information and the depth information; and the camera calibration parameters are determined according to the coordinate system parameters.

[0111] The triangulation principle can be a method of measuring the target distance by the angle of the target point and the known end point of the fixed reference line.

[0112] As an example, the triangulation principle applies a triangulation method, but is not limited to the direction method and the full combination angle measurement method.

[0113] In one embodiment, the structured light grid image collected by the camera acquisition module is obtained, the position information and the depth information of the to-be-inspected road are calculated and obtained based on the structured light grid image by using the triangulation principle, the space with a preset area size is covered by at least two structured light cameras, the preset area includes but is not limited to 10m*10m, the coordinate system parameters corresponding to the to-be-inspected road are obtained, the camera calibration parameters are obtained based on the coordinate system parameters, and the camera calibration parameters can be used for calibrating the camera acquisition module on the inspection vehicle.

[0114] Through the above embodiment, the camera acquisition module on the inspection vehicle is calibrated based on the structured light grid image, so that the accurate calibration of the camera before inspection can be realized, and the accuracy of collecting the image of the to-be-inspected road is improved.

[0115] In some embodiments, the method further includes: obtaining a global positioning system time by using a pre-installed time synchronization protocol; and adjusting a local system time according to the global positioning system time, so that the adjusted local system time is synchronized with the global positioning system time.

[0116] The time synchronization protocol can be a protocol for synchronizing computer time, the global positioning system can be a positioning system based on artificial satellite radio navigation, and the local system time can be the current time representing the time zone set by the local system.

[0117] As an example, the time synchronization protocol includes but is not limited to the Network Time Protocol (NTP), the global positioning system includes but is not limited to the GPS positioning system, and the local system time includes but is not limited to the system time in the above-mentioned road inspection system. The above-mentioned road inspection system also includes a positioning module, and the above-mentioned positioning module includes but is not limited to a GPS receiver and an inertial navigation.

[0118] In one embodiment, the above-mentioned road inspection system runs a pre-installed time synchronization protocol, obtains a global positioning system time in a global positioning system, and adjusts a clock source of the edge box based on the global positioning system time, so that the adjusted local system time is synchronized with the global positioning system time; for example, using the Network Time Protocol to synchronize the system time of the edge box and the GPS positioning system time, and the synchronization frequency can be once per second.

[0119] Through the above-mentioned embodiment, the time of the system and the time of the global positioning system are synchronized based on the pre-installed time synchronization protocol, which can improve the timeliness in the inspection process and ensure the time accuracy of the inspection result.

[0120] In order to facilitate the understanding of those skilled in the art, Figure 6 An example of a framework diagram of a road disease collection system is provided; as shown in Figure 6 The road disease collection system includes a vehicle-mounted control system, a central control system, a network transmission module, a camera collection module, a vibration data collection module, a high-precision positioning module, a laser radar module, and a 3D structured light module.

[0121] The vehicle-mounted control system can include a vehicle-mounted screen and a control system client. The central control system includes an edge box and central control system software. The network transmission module includes a 5G antenna, a 5G Internet of Things card, and a 5G router. The camera collection module includes front and rear view cameras. The vibration data collection module includes two vibration sensors. The high-precision positioning module includes a GPS receiver and an inertial navigation. The laser radar module includes two rear short-range laser radar devices. The 3D structured light module includes two front structured light depth camera devices. The remaining network lines, cables, etc.

[0122] In order to facilitate the understanding of those skilled in the art, Figure 7 An example of a framework diagram of a road disease collection system is provided; as shown in Figure 7As shown, the inspection operator starts the structured light module through the road disease collection system to start 3D structured light calibration, projects the structured light to the road surface of the road to be inspected, forms a structured light grid, collects images of the structured light grid through the camera collection module, uploads the structured light grid image to the cloud platform, and obtains camera calibration parameters based on the structured light grid image. The camera calibration parameters are used to calibrate the camera collection module, correct the positioning information of the road disease collection system, and after calibration, the camera collection module, the high-precision positioning module, the vibration sensor and the laser radar module perform real-time data collection to obtain camera data, GPS data, vibration data and laser point cloud data and store them. The vehicle-mounted control system can preview the data in real time, and the data will be uploaded to the platform in real time. When the inspection task starts, the inspection vehicle arrives at the starting point of the specified inspection section, starts the inspection task through the vehicle-mounted control system, and the road disease collection system performs picture reasoning through a deep learning vision algorithm to identify road surface and facility diseases along the line. The disease data is uploaded to the platform in real time. The vertical acceleration during driving is measured by the vibration sensor, which is converted into flatness through a preset conversion rule, and then the driving quality index is obtained to obtain the driving comfort of the road section. The laser radar module scans the driving section to obtain laser point cloud data, and based on the laser point cloud data, the depth information is obtained. The laser point cloud data can also assist in building road facility digital assets and road modeling. When the inspection vehicle completes the road section inspection, the vehicle-mounted control system ends the inspection task. At this time, the edge box stops picture reasoning and other data collection, and completes the uploading of the remaining disease data, flatness data and laser point cloud data. The vehicle-mounted control system is closed, and the platform end completes data reception. The platform end also supports data visualization display, export of disease report and other operations.

[0123] It should be understood that although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0124] Based on the same inventive concept, the embodiments of the present application also provide a road inspection device for implementing the road inspection method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more road inspection device embodiments provided below can refer to the limitations of the road inspection method described above, which will not be repeated here.

[0125] In one embodiment, as shown in Figure 8 A road inspection device is provided, comprising: a structured light acquisition module 810, a camera calibration module 820, an image acquisition module 830, a disease detection module 840, and an inspection information generation module 850, wherein:

[0126] The structured light acquisition module 810 is configured to form a structured light grid on the road to be inspected, and to acquire an image of the structured light grid.

[0127] The camera calibration module 820 is configured to obtain camera calibration parameters according to the structured light grid image, and to calibrate the camera acquisition module on the inspection vehicle using the camera calibration parameters.

[0128] The image acquisition module 830 is configured to acquire an image of the road to be inspected by the calibrated camera acquisition module, and to obtain a road inspection image.

[0129] The disease detection module 840 is configured to detect a target object in the road inspection image, and to obtain road disease detection information corresponding to the road to be inspected.

[0130] The inspection information generation module 850 is configured to obtain driving comfort information corresponding to the road to be inspected, and to obtain road structure depth information corresponding to the road to be inspected, and to obtain road inspection information corresponding to the road to be inspected according to the road disease detection information, the driving comfort information, and the road structure depth information.

[0131] In one embodiment, the inspection information generation module 850 is specifically configured to acquire a vehicle vibration signal of the inspection vehicle during driving by a vibration data acquisition module on the inspection vehicle during driving of the inspection vehicle on the road to be inspected according to the road inspection task, and to determine the driving comfort information corresponding to the road to be inspected according to the vehicle vibration signal.

[0132] In one embodiment, the inspection information generation module 850 is specifically configured to convert the vertical acceleration into a road surface flatness corresponding to the road to be inspected, and to determine a driving quality index corresponding to the road to be inspected as the driving comfort information according to the road surface flatness corresponding to the road to be inspected.

[0133] In one embodiment, the inspection information generation module 850 is specifically used to perform a laser radar scan on the road to be inspected by using a laser radar module on the inspection vehicle while the inspection vehicle is driving on the road to be inspected according to the road inspection task, to obtain laser point cloud data, and to determine the road structure depth information corresponding to the road to be inspected based on the laser point cloud data.

[0134] In one embodiment, the camera calibration module 820 is specifically used to determine the position information and depth information of the target road surface based on the above-mentioned structured light grid image using the principle of triangulation. The above-mentioned target road surface is the road surface where the above-mentioned structured light grid is located. According to the above-mentioned position information and depth information, coordinate system parameters are generated, and according to the above-mentioned coordinate system parameters, the above-mentioned camera calibration parameters are determined.

[0135] In one embodiment, the above-mentioned device also includes a time synchronization module, which is used to obtain the global positioning system time through a pre-installed time synchronization protocol; adjust the local system time according to the above-mentioned global positioning system time, so that the adjusted local system time is synchronized with the above-mentioned global positioning system time.

[0136] Each module in the aforementioned road inspection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a computer device's memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a road inspection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0138] Those skilled in the art can understand that Figure 9 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment, a road inspection system is provided, the system is arranged on an inspection vehicle, and the system comprises a structured light module, a laser radar module, a vibration data acquisition module, a camera acquisition module, and a control module. When the control module is executed by a processor, the steps in the method embodiments are implemented.

[0140] In one embodiment, a computer device is also provided, which comprises a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method embodiments.

[0141] In one embodiment, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps in the method embodiments are implemented.

[0142] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0144] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0145] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A road inspection method, characterized in that: The method comprises: A structured light grid is formed on the road to be inspected by a structured light module on the inspection vehicle, and an image of the structured light grid is acquired by a camera acquisition module on the inspection vehicle to obtain a structured light grid image; Determine the position and depth information of a target road surface based on the structured light grid image using the triangulation principle; the target road surface is the road surface where the structured light grid is located; generating coordinate system parameters according to the position information and depth information; Determining camera calibration parameters based on the coordinate system parameters; and calibrating the camera acquisition module on the inspection vehicle using the camera calibration parameters; Capturing images of the road to be inspected by the calibrated camera acquisition module to obtain a road inspection image; Performing disease detection on the target object in the road inspection image to obtain road disease detection information corresponding to the road to be inspected; Acquire driving comfort information corresponding to the road to be inspected, and acquire road structure depth information corresponding to the road to be inspected; and obtain road inspection information corresponding to the road to be inspected based on the road defect detection information, the driving comfort information, and the road structure depth information.

2. The method according to claim 1, characterized in that The obtaining of the driving comfort information corresponding to the road to be inspected includes: The vibration data acquisition module on the inspection vehicle is used to collect the vehicle vibration signal during the driving process of the inspection vehicle; Driving comfort information corresponding to the road to be inspected is determined according to the vehicle vibration signal.

3. The method according to claim 2, characterized in that The vehicle vibration signal includes a vertical acceleration, and determining the driving comfort information corresponding to the road to be inspected based on the vehicle vibration signal includes: Converting the vertical acceleration into the road surface smoothness corresponding to the road to be inspected; According to the road surface smoothness corresponding to the road to be inspected, a driving quality index corresponding to the road to be inspected is determined as the driving comfort information.

4. The method according to claim 1, wherein The obtaining of the road structure depth information corresponding to the road to be inspected includes: Performing a laser radar scan on the road to be inspected by using a laser radar module on the inspection vehicle to obtain laser point cloud data; Determine the road structure depth information corresponding to the road to be inspected based on the laser point cloud data.

5. The method according to claim 1, wherein The method further comprises: Obtain GPS time through pre-installed time synchronization protocol; According to the global positioning system time, the local system time is adjusted so that the adjusted local system time is synchronized with the global positioning system time.

6. A road inspection device, characterized in that: The device comprises: A structured light acquisition module is used to form a structured light grid on the road to be inspected, and to acquire an image of the structured light grid through a camera acquisition module on an inspection vehicle to obtain a structured light grid image; a camera calibration module configured to determine, based on the structured light grid image, the position and depth information of a target road surface using the principle of triangulation; the target road surface being the road surface on which the structured light grid is located; generate coordinate system parameters based on the position and depth information; determine camera calibration parameters based on the coordinate system parameters; and calibrate the camera acquisition module on the inspection vehicle using the camera calibration parameters; An image acquisition module is used to acquire images of the road to be inspected by using the calibrated camera acquisition module when the inspection vehicle is driving on the road to be inspected according to the road inspection task, so as to obtain a road inspection image; a disease detection module, configured to perform disease detection on target objects in the road inspection image to obtain road disease detection information corresponding to the road to be inspected; The inspection information generation module is used to obtain the driving comfort information corresponding to the road to be inspected, and obtain the road structure depth information corresponding to the road to be inspected, and obtain the road inspection information corresponding to the road to be inspected based on the road disease detection information, the driving comfort information and the road structure depth information.

7. The device according to claim 6, characterized in that The inspection information generation module is further used to collect vehicle vibration signals of the inspection vehicle during driving through the vibration data collection module on the inspection vehicle, and determine the driving comfort information corresponding to the road to be inspected based on the vehicle vibration signals.

8. A road inspection system, characterized in that: The system is set on the inspection vehicle and includes a structured light module, a laser radar module, a vibration data acquisition module, a camera acquisition module, and a control module, wherein: The control module is configured to execute the steps of the method according to any one of claims 1 to 5.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images