Road inspection method and system based on data fusion and edge calculation

Abstract

The invention discloses a road inspection method based on data fusion and edge calculation, and relates to the field of road inspection, and the method comprises the steps: obtaining a pre-marked weak signal road section according to a preset road inspection route; acquiring real-time point cloud picture information and real-time driving information before the inspection unmanned vehicle enters the weak signal road section, constructing a road simulation model and a driving state model, and updating the road simulation model and the driving state model in real time after the inspection unmanned vehicle enters the weak signal road section; carrying out model fusion on the updated road simulation model and the driving state model to obtain a road driving model; acquiring real-time environment image information after the inspection unmanned vehicle enters the weak signal road section; and determining disease feature types according to the real-time point cloud image information and the real-time environment image information, and obtaining road section disease position information of the weak signal road section based on the disease feature types and a road driving model. The method has the effect of efficiently and accurately detecting the types of the pavement diseases and the accurate positions of the pavement diseases.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of road inspection, and in particular, to a road inspection method and system based on data fusion and edge computing. Background Art

[0002] With the rapid development of society, the transportation network plays a vital role in the process of social development. Among them, the quality of roads greatly affects the transportation efficiency of urban transportation networks. Due to factors such as sunlight, weathering and vehicle friction, roads will be damaged to a certain extent, resulting in road cracks, potholes and other road defects, which will affect the speed of passing vehicles and even cause traffic accidents in severe cases. Therefore, it is extremely important to detect road defects in a timely manner and take corresponding measures.

[0003] At present, most inspections are carried out manually, with inspection vehicles driven manually to inspect the road surface. When road defects are found, the inspectors need to get off the vehicle to take photos of the road defects, and then locate the area where the road defects occur through the navigation system, and upload the captured images and navigation positioning information through a handheld terminal. This method is not only inefficient and prone to omissions, but also very likely to cause safety accidents, affecting the normal operation of traffic. Although some areas have adopted unmanned inspection vehicles for intelligent road inspections, which makes up for the shortcomings of low efficiency and easy omissions of manual inspections, for some areas where the navigation system cannot accurately locate, such as tunnels and roads in remote areas, due to weak signals, even if the road defect area is detected, it is difficult to accurately locate the exact location of the road defect area. Summary of the invention

[0004] The embodiments of the present application provide a road inspection method and system based on data fusion and edge computing, which are used to solve the problem in the prior art that it is difficult to accurately locate the precise position of a road surface defect area even if a road surface defect area is detected for sections with weak signals.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a road inspection method based on data fusion and edge computing is provided, the method comprising:

[0007] According to a preset road inspection plan, the inspection unmanned vehicle is controlled to perform a road inspection task along a preset inspection route, wherein the inspection route includes a pre-marked weak signal section;

[0008] Before the unmanned inspection vehicle enters the weak signal section, the laser radar module is used to continuously collect real-time point cloud information, and the real-time driving information of the unmanned inspection vehicle is obtained at the same time, and the edge computing module is used to preliminarily build a road simulation model and a driving state model based on the real-time point cloud information and the real-time driving information. When the unmanned inspection vehicle enters the weak signal section, the road simulation model and the driving state model are updated in real time;

[0009] The road simulation model and the driving state model updated in real time are fused in real time to obtain a road driving model;

[0010] After entering the weak signal section, the unmanned inspection vehicle is controlled to continuously collect section environment information of the weak signal section using the image acquisition module to obtain real-time environment image information;

[0011] The real-time environmental image information and the real-time point cloud image information are subjected to real-time image recognition using image recognition technology, and the section defect location information of the weak signal section is obtained by combining the image recognition result with the road driving model.

[0012] Optionally, before the unmanned inspection vehicle enters the weak signal section, the laser radar module is used to continuously collect real-time point cloud information, and real-time driving information of the unmanned inspection vehicle is obtained at the same time, and the edge computing module is used to preliminarily construct a road simulation model and a driving state model based on the real-time point cloud information and the real-time driving information. When the unmanned inspection vehicle enters the weak signal section, the road simulation model and the driving state model are updated in real time, including the following steps:

[0013] Before entering the weak signal section, continuously scan the environment of the weak signal section using the laser radar module to obtain real-time point cloud information of the weak signal section;

[0014] Using the edge computing module to preliminarily construct a road simulation model based on the real-time point cloud information obtained before entering the weak signal road section;

[0015] Acquire the real-time driving information of the unmanned road inspection vehicle, and use the satellite positioning module installed on the unmanned road inspection vehicle to acquire the satellite positioning information of the unmanned road inspection vehicle in real time;

[0016] Preliminarily constructing a driving state model based on the real-time driving information and the satellite positioning information;

[0017] After entering the weak signal section, the road simulation model is updated in real time based on the real-time point cloud information acquired after entering the weak signal section;

[0018] The driving state model is updated in real time based on the real-time driving information acquired after entering the weak signal section.

[0019] Optionally, the using the edge computing module to preliminarily construct a road simulation model according to the real-time point cloud information acquired before entering the weak signal road section comprises the following steps:

[0020] Preprocessing the real-time point cloud information;

[0021] Segmenting the real-time point cloud map information to obtain road point cloud map information;

[0022] Fitting the road point cloud information to obtain road feature information of the weak signal section;

[0023] The edge computing module is used to preliminarily construct a road simulation model based on the road feature information.

[0024] Optionally, the real-time updating of the driving state model based on the real-time driving information acquired after entering the weak signal section comprises the following steps:

[0025] Constructing a position prediction model of the road inspection unmanned vehicle based on a recursive neural network;

[0026] Using the real-time driving information and the satellite positioning information before entering the weak signal road section as a training set to train the position prediction model;

[0027] After entering the weak signal section, the driving state model obtained after entering the weak signal section is input into the trained position prediction model to obtain the predicted position of the road inspection unmanned vehicle;

[0028] The driving state model is continuously updated based on the predicted position.

[0029] Optionally, the step of performing real-time image recognition on the real-time environmental image information and the real-time point cloud image information by using image recognition technology, and obtaining the road section defect location information of the weak signal section by combining the image recognition result and the road driving model comprises the following steps:

[0030] Registering the collected real-time environment image information and the real-time point cloud image information;

[0031] Performing image enhancement on the registered real-time environment image information using an image enhancement algorithm;

[0032] Extracting a region of interest containing road section disease characteristics from the real-time environmental image information using an image recognition algorithm, wherein the region of interest is characterized as disease image information;

[0033] Mapping the region of interest from the real-time environmental image information to the real-time point cloud image information at the same time node to obtain disease point cloud image information;

[0034] Performing edge feature comparison between the defect point cloud information and the defect image information, and obtaining the defect feature type of the weak signal section according to the edge feature comparison result;

[0035] The road section defect location information of the weak signal section is obtained by combining the defect feature type and the road driving model.

[0036] Optionally, performing edge feature comparison between the defect point cloud information and the defect image information, and obtaining the defect feature type of the weak signal section according to the edge feature comparison result comprises the following steps:

[0037] Performing denoising on the disease point cloud image information;

[0038] Extracting the edge points of the diseased point cloud image after denoising based on an adaptive multi-feature fusion method;

[0039] Clustering the extracted disease edge points and performing fitting to obtain the point cloud edge features of the disease point cloud information;

[0040] grayscale the disease image information to obtain grayscale disease image information;

[0041] Extracting all edge points of the grayscale disease image information by using an edge enhancement operator, and integrating all the edge points to obtain image edge features of the grayscale disease image information;

[0042] Performing edge matching on the edge features of the point cloud image and the edge features of the image, and determining whether the edge features of the point cloud image and the edge features of the image are the same according to the edge matching result;

[0043] If the edge feature of the point cloud image is the same as the edge feature of the image, determining that the defect point cloud image information of the weak signal section is three-dimensional defect information;

[0044] If the edge feature of the point cloud image is different from the edge feature of the image, it is determined that the defect image information of the weak signal section is plane defect information.

[0045] Optionally, the denoising process for the disease point cloud image information comprises the following steps:

[0046] Calculating the point cloud density of each point in the disease point cloud map information;

[0047] Removing points whose point cloud density is less than a preset point cloud density threshold to obtain the defect point cloud image information after preliminary denoising;

[0048] Performing plane fitting on the defect point cloud image information after preliminary denoising to obtain a defect point cloud plane;

[0049] Calculating the plane distance between each point in the disease point cloud image information and the point cloud plane;

[0050] The points whose plane distance is greater than the preset point cloud distance are eliminated to obtain the denoised point cloud image information of the defect.

[0051] Optionally, the obtaining the road section defect location information of the weak signal road section by combining the defect feature type and the road driving model comprises the following steps:

[0052] If the defect point cloud image information of the weak signal section is three-dimensional defect information, then the center point of the three-dimensional defect information is selected as the three-dimensional defect point;

[0053] Selecting the center point of the inspection unmanned vehicle as the disease inspection point;

[0054] The spatial distance between the three-dimensional defect point and the defect inspection point in the road driving model is calculated using the Euclidean distance formula;

[0055] Acquire the unmanned vehicle position information of the unmanned inspection vehicle on the weak signal section based on the road driving model;

[0056] The spatial distance and the position information of the unmanned vehicle are combined to obtain the section defect position information of the three-dimensional defect information in the weak signal section.

[0057] Optionally, the method further includes:

[0058] If the damage image information of the weak signal road section is plane damage information, determining the position information of the plane damage information in the road driving model according to the position information of the unmanned vehicle;

[0059] Obtaining device parameters of an image acquisition device in the real-time image acquisition module;

[0060] The position information is corrected according to the equipment parameters to obtain the section defect position information of the plane defect information in the weak signal section.

[0061] In the second aspect, the present application provides a road inspection system based on data fusion and edge computing, including:

[0062] a memory configured to store instructions; and

[0063] A processor is configured to call the instructions from the memory and to implement the road inspection method based on data fusion and edge computing according to any one of the first aspects when executing the instructions.

[0064] The present invention specifically adopts the following technical scheme: obtaining a pre-marked weak signal section from a preset inspection route, obtaining real-time point cloud information and real-time driving information before the inspection unmanned vehicle enters the weak signal section, constructing a road simulation model and a driving state model through the real-time point cloud information and real-time driving information, and updating the road simulation model and the driving state model in real time after entering the weak signal section, and fusing the updated road simulation model and the driving state model to obtain a road driving model, and after controlling the unmanned vehicle to enter the weak signal section, performing image acquisition on the weak signal section through an image acquisition module to obtain real-time environmental information. Image recognition technology is used to perform image recognition on the real-time environmental information, and the edge feature extraction of the disease point cloud information and the disease image information in the real-time point cloud information and the real-time environmental information is performed according to the image recognition results, and the edge features of the two are compared, and the types of the disease features of the disease point cloud information and the disease image information are determined according to whether the edge features of the two are the same, and different methods are used to determine the road section disease location information in the weak signal section according to different types of disease features. Through the above technical solution, the present invention can realize efficient and accurate detection of the type and precise location of road surface defects in weak signal sections.

[0065] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart of a road inspection method based on data fusion and edge computing provided in an embodiment of the present application.

[0067] Figure 2 A schematic diagram of a flow chart for obtaining the type of damage feature of a weak signal section based on edge feature comparison results provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0069] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0070] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0071] Figure 1 The following schematically shows a flow chart of a road inspection method based on data fusion and edge computing according to an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a road inspection method based on data fusion and edge computing, which may include the following steps:

[0072] S101. According to a preset road inspection plan, the unmanned inspection vehicle is controlled to perform a road inspection task along a preset inspection route, where the inspection route includes pre-marked weak signal sections.

[0073] In this embodiment, weak signal sections refer to sections where the inspection unmanned vehicle cannot receive satellite signals and perform satellite positioning, such as tunnels, mountain roads, etc. Before performing the inspection task, the weak signal sections will be marked in the inspection route, and the position information of the inspection unmanned vehicle in the weak signal section will be calculated based on the real-time point cloud information of the weak signal section and the real-time driving information of the inspection unmanned vehicle. Thus, the position information of the road section disease collected by the image acquisition module at the current time node is calculated based on the position information of the inspection unmanned vehicle.

[0074] S102. Before the inspection unmanned vehicle enters a weak signal section, the laser radar module is used to continuously collect real-time point cloud map information, and the real-time driving information of the inspection unmanned vehicle is obtained at the same time. The edge computing module is used to preliminarily build a road simulation model and a driving status model based on the real-time point cloud map information and the real-time driving information. When the inspection unmanned vehicle enters a weak signal section, the road simulation model and the driving status model are updated in real time.

[0075] In this embodiment, the edge computing module is arranged inside the inspection unmanned vehicle. The edge computing module has a high computing power support of 100TOPS (the number of operations that the processor can perform per second), and can process the data obtained by other modules in real time and at high speed. Before entering the weak signal section, the laser radar inside the inspection unmanned vehicle remains turned on, and the laser radar is used to continuously scan the surrounding environment to collect real-time point cloud information of the surrounding environment. The laser radar can emit laser pulses. Since the pulse contains a large amount of energy, a long and thin beam is formed. When obstacles appear around the inspection unmanned vehicle, the laser pulse will be reflected back immediately, and the reflected beam will be received by the laser radar receiver. The laser radar generates real-time point cloud information according to the reflection condition of the laser pulse. After denoising and road segmentation of the real-time point cloud information, the edge computing module is used to preliminarily construct a road simulation model. The preliminarily constructed road simulation model is a three-dimensional point cloud model that can reflect the basic road conditions of the road, such as the width of the current weak signal section, road flatness and other information.

[0076] While the laser radar is collecting real-time point cloud information, the inertial measurement module and satellite positioning module of the unmanned inspection vehicle are also collecting the real-time driving information and satellite positioning information of the unmanned inspection vehicle in real time. The real-time driving information includes the angular velocity, acceleration and vehicle deflection direction of the unmanned inspection vehicle collected by the inertial measurement module. The satellite positioning information is the position information of the unmanned inspection vehicle obtained in real time by receiving satellite information through the satellite positioning module. The driving state model is constructed using the real-time driving information of the unmanned inspection vehicle. The driving state model can reflect the angular velocity, acceleration and vehicle deflection direction of the unmanned inspection vehicle during driving. After entering the weak signal section, the preliminary constructed road simulation model and driving state model are updated using the real-time point cloud information and real-time driving information collected when entering the weak signal section.

[0077] S103, fusing the updated road simulation model and the driving state model in real time to obtain a road driving model.

[0078] In this embodiment, whenever the road simulation model and the driving state model are updated, the edge computing module will immediately fuse the two models in real time to generate a comprehensive road driving model. The model fusion process adopts multi-source data fusion technology, which mainly includes two stages: data layer fusion and feature layer fusion. First, in the data layer fusion stage, the point cloud data from the lidar and the motion data of the inertial measurement unit are time synchronized and spatially aligned. Time synchronization ensures that the data collected by different sensors correspond to the same time, while the spatial alignment standard unifies the data in different coordinate systems into the same reference coordinate system. It is specifically implemented by the Kalman filter algorithm, which can effectively process the noise and error in the sensor data and improve the accuracy of the data. In the feature layer fusion stage, key features are extracted from the synchronized and aligned data. For the road simulation model, the main road geometric features are extracted, such as road width, curvature, slope, etc.; for the driving state model, the vehicle motion features are extracted, such as speed, acceleration, heading angle, etc. These features are fused through a preset deep learning model, which has been trained with a large amount of historical data and can automatically learn the complex relationship between different features. The result is a comprehensive road driving model that includes the current position, speed, direction, and predicted trajectory. This model not only reflects the vehicle’s immediate state, but also includes the perception of the surrounding environment and the prediction of future movement.

[0079] S104, after entering the weak signal section, the inspection unmanned vehicle is controlled to continuously collect section environment information using the image acquisition module on the weak signal section to obtain real-time environment image information.

[0080] In this embodiment, after the inspection unmanned vehicle enters the weak signal section, it uses the image acquisition device in the image acquisition module, such as a high-definition camera, to continuously acquire images of the road conditions and surrounding environment of the weak signal section to obtain real-time environmental image information. This step is to capture road surface defects, road spills, and other road section defects that may affect the driving of passing vehicles.

[0081] S105. Perform real-time image recognition on the real-time environmental image information and the real-time point cloud image information using image recognition technology, and obtain the location information of the road section defects on the weak signal section by combining the image recognition result with the road driving model.

[0082] In this embodiment, after entering the weak signal section, the collected real-time environment image information and real-time point cloud map information are first registered. This is because the laser radar module and the image acquisition module have different installation angles when collecting information, resulting in an angle difference in the collected information. Registration can make the information collected by the laser radar module and the image acquisition module consistent in spatial position, which is convenient for further analysis and processing. After the registration step is completed, the real-time environment image information is enhanced using an image enhancement algorithm to make the real-time environment image information clearer, emphasize certain features of interest, expand the differences between the features of different objects in the image, and suppress features of no interest, thereby improving image quality, enriching the amount of information, and strengthening image interpretation and recognition effects, so as to reduce recognition errors and improve the accuracy of image recognition when performing image recognition on the real-time environment image information.

[0083] After completing the image enhancement step, the image recognition technology is used to extract the region of interest in the real-time environmental image information. The region of interest refers to the area containing road section diseases such as road diseases and road spills. The region of interest in the real-time environmental information is named as the disease image information. At the same time, the region of interest is synchronously marked in the real-time point cloud information, and the region of interest synchronized to the real-time point cloud information is named as the disease point cloud information. The edge features of the traffic defects in the disease image information and the real-time point cloud information are extracted respectively, and the type of disease feature of the weak signal section is determined based on whether the edge features of the two are the same. The type of disease feature is the type of road section disease. If the edge features of the point cloud are the same as the edge features of the image, it means that the type of disease feature of the road section disease in the disease point cloud information and the disease image information is three-dimensional disease information. Three-dimensional disease information refers to traffic facilities such as water barriers, crash buckets, and vehicle parts or accidental drops on the road. If the edge features of the point cloud map are different from the edge features of the image, it means that the type of the road section disease in the disease point cloud map information and the disease image information is plane disease information, which refers to road defects such as cracks and potholes on the road surface. After completing all the above steps, different methods are used to determine the specific location of the road section disease in the weak signal section according to the different types of disease features, that is, the road section disease location information. Finally, the inspection unmanned vehicle is controlled to transmit the acquired disease feature type and its corresponding road section disease location information to the terminal system, reminding relevant staff to correct the road in time and eliminate road section diseases to facilitate the travel of subsequent vehicles.

[0084] After obtaining the location information of all road defects on weak signal sections, the types of defects, the time when the defects were found, and the location information of the road defects will be uploaded to the cloud storage through the 4G / 5G network to provide data support for subsequent maintenance management. After the disease data is uploaded, the relevant sections will be marked to prevent repeated uploading and waste of resources. After uploading to the cloud, the maintenance platform will classify the road defects according to the time when the road defects were found, the types of defects, and the severity of the defects. According to the classification results, the maintenance platform will automatically dispatch orders. Road maintenance personnel can receive work orders through mobile programs. After completing the road maintenance task, they will take pictures of the current road conditions and upload the road section information of the current road section through the mobile program. At the same time, the maintenance platform will also update the road section information of the road section in a timely manner to realize the business closed loop. This can effectively improve the efficiency of business processing and supervision, timely maintain roads with road section defects, reduce unnecessary economic losses, and avoid traffic accidents.

[0085] In one embodiment, before the unmanned inspection vehicle enters a weak signal section, the laser radar module is used to continuously collect real-time point cloud information, and the real-time driving information of the unmanned inspection vehicle is obtained at the same time. The edge computing module is used to preliminarily build a road simulation model and a driving state model based on the real-time point cloud information and the real-time driving information. When the unmanned inspection vehicle enters the weak signal section, the road simulation model and the driving state model are updated in real time, including the following steps:

[0086] Before entering a weak signal section, the laser radar module is used to continuously scan the environment of the weak signal section to obtain real-time point cloud information of the weak signal section;

[0087] Use the edge computing module to preliminarily build a road simulation model based on the real-time point cloud information obtained before entering the weak signal section;

[0088] Obtain the real-time driving information of the road inspection unmanned vehicle, and use the satellite positioning module installed on the road inspection unmanned vehicle to obtain the satellite positioning information of the road inspection unmanned vehicle in real time;

[0089] Preliminary construction of driving status model based on real-time driving information and satellite positioning information;

[0090] After entering the weak signal section, the road simulation model is updated in real time based on the real-time point cloud information obtained after entering the weak signal section;

[0091] The driving state model is updated in real time based on the real-time driving information obtained after entering the weak signal section.

[0092] In this embodiment, before entering the weak signal section, the laser radar inside the laser radar module is used to emit laser pulses in all directions, and the real-time point cloud map information of the weak signal section is generated according to the reflection condition of the laser pulse. Since there is still some distance from the weak signal section at this time, the real-time point cloud map information obtained at this time is not comprehensive, so the road simulation model constructed by the real-time point cloud map information obtained at this time is not accurate. Therefore, after entering the weak signal section, the real-time point cloud map of the weak signal section is continuously collected to continuously update the road simulation model. While collecting the real-time point cloud map information, the inertial measurement module and satellite positioning module of the inspection unmanned vehicle are also controlled to collect the real-time driving information and satellite positioning information of the inspection unmanned vehicle in real time. Among them, the inertial measurement module for collecting real-time driving information includes devices such as accelerometers, gyroscopes and magnetometers, which can measure the angular velocity, acceleration, speed and vehicle deflection direction of the unmanned inspection vehicle during driving. For example, when the unmanned inspection vehicle turns, the inertial measurement module can measure the direction of the unmanned inspection vehicle turning and the angular velocity when turning, thereby calculating the driving state of the unmanned inspection vehicle at this time. The satellite positioning module receives satellite signals through the global positioning system or the Beidou satellite navigation system to calculate the latitude and longitude coordinates of the unmanned inspection vehicle, thereby obtaining the precise location information of the unmanned inspection vehicle, that is, the satellite positioning information. Since the satellite positioning module is difficult to play a role after entering the weak signal section, it is necessary to preliminarily construct a driving state model based on the real-time driving information and satellite positioning information before the unmanned inspection vehicle enters the weak signal road. Since the driving state of the unmanned inspection vehicle will also change after entering the weak signal section, it is necessary to update the driving state model in real time according to the real-time driving information collected after entering the weak signal section after entering the weak signal section. The road simulation model and driving state model obtained in this way are highly accurate and timely.

[0093] In one embodiment, using the edge computing module to preliminarily construct a road simulation model based on the real-time point cloud information obtained before entering the weak signal section includes the following steps:

[0094] Preprocess the real-time point cloud information;

[0095] Segment the real-time point cloud information to obtain road point cloud information;

[0096] Fit the road point cloud information to obtain the road feature information of the weak signal section;

[0097] The edge computing module is used to preliminarily build a road simulation model based on road feature information.

[0098] In this embodiment, before entering the weak signal section, once the laser radar module collects the real-time point cloud information, the real-time point cloud information will be analyzed and processed immediately. The real-time point cloud information is preprocessed first. The main function of the preprocessing step is to denoise the real-time point cloud information. In the process of collecting real-time point cloud information, it will be affected by random errors, and noise will inevitably exist, which will affect the accuracy of the road simulation model constructed later. Since the elevation of most noise points is high, the height difference between the noise point and the adjacent point on the scan line will be much higher than other points. Therefore, the noise can be removed by judging the average height difference between the selected point and its adjacent points. When the average height difference is greater than the preset average height difference threshold, it means that the point is a noise point and should be removed. After traversing all points, all noise points are removed to obtain the denoised real-time point cloud information.

[0099] The denoised real-time point cloud information is segmented. Since the real-time point cloud information also contains road point cloud information and non-road point cloud information, such as trees and guardrails around the road, these non-road point cloud information will occupy too many computing units in the process of building the road simulation model, affecting the computing efficiency. Therefore, it is necessary to segment the real-time point cloud information to obtain the road point cloud information. The main principle of road segmentation is to filter out the ground point cloud from the real-time point cloud information. The methods for filtering the ground point cloud include 2.5D grid method, plane fitting method and linear extraction method. Taking the 2.5D grid method as an example, the 3D point cloud data is mapped to the 2D grid. Each grid unit contains the average height or other statistical features of the points in the area. The ground point cloud is filtered out according to the preset average height threshold, so as to obtain the road point cloud information. The image processing technology is used to extract the features of the road point cloud information, extract the road boundary line and fit it, and obtain the road feature information of the weak signal section. The road simulation model is generated by the extracted road feature information. The road simulation model can reflect the width of the current weak signal section, road smoothness and other information.

[0100] In one embodiment, updating the driving state model in real time based on the real-time driving information obtained after entering the weak signal section includes the following steps:

[0101] Construct a location prediction model for road inspection unmanned vehicles based on recurrent neural networks;

[0102] The real-time driving information and satellite positioning information before entering the weak signal section are used as training sets to train the location prediction model;

[0103] After entering the weak signal section, the driving state model obtained after entering the weak signal section is input into the trained position prediction model to obtain the predicted position of the road inspection unmanned vehicle;

[0104] The driving state model is continuously updated based on the predicted position.

[0105] In this embodiment, a position prediction model of a road inspection unmanned vehicle is constructed based on a recursive neural network, for example, a position prediction model is constructed based on a long short-term memory network LSTM, which can predict the position information of the inspection unmanned vehicle according to the real-time driving information of the inspection unmanned vehicle, and the real-time driving information and satellite positioning information before entering the weak signal section are used as a training set to train the position prediction model. The data in the training set is first cleaned to remove noise, abnormal values ​​and missing values ​​in the data to ensure the quality and accuracy of the data. After completing the data cleaning of the training set, the structure of the LSTM model is determined, including the number of nodes in the input layer and the hidden layer, the connection weights and bias items in the network are initialized, the training set is input into the position prediction model for forward propagation, the output of each layer is calculated, and then back propagation is performed to calculate the error between the forward propagation output result and the back propagation output result, and the connection weights and bias items are updated according to the error until the error is reduced, and the forward propagation and back propagation are repeated many times until the preset maximum number of iterations is reached, and the position prediction model training is completed.

[0106] Since the inspection unmanned vehicle cannot perform satellite positioning through satellite signals after entering a weak signal section, a position prediction model is constructed to predict the position. The driving state model obtained after entering the weak signal section is input into the trained position prediction model to obtain the predicted position of the road inspection unmanned vehicle. The driving state model is updated according to the predicted position, so that the driving state model can accurately display the position, speed, acceleration and other information of the road inspection unmanned vehicle at different times.

[0107] In one embodiment, using image recognition technology to perform real-time image recognition on real-time environmental image information and real-time point cloud image information, and combining the image recognition results with the road driving model to obtain the section disease location information of the weak signal section includes the following steps:

[0108] Registering the collected real-time environmental image information and the real-time point cloud image information;

[0109] Use image enhancement algorithm to enhance the registered real-time environment image information;

[0110] An image recognition algorithm is used to extract an area of ​​interest containing road section disease characteristics from real-time environmental image information, and the area of ​​interest is represented as disease image information;

[0111] Map the area of ​​interest from the real-time environmental image information to the real-time point cloud information at the same time node to obtain the disease point cloud information;

[0112] Compare the edge features of the defect point cloud information with the defect image information, and obtain the defect feature type of the weak signal section based on the edge feature comparison result;

[0113] The road defect location information of weak signal sections is obtained by combining the types of defect characteristics and the road driving model.

[0114] In this embodiment, the real-time environmental image information and real-time point cloud information collected after the inspection unmanned vehicle enters the weak signal section are registered. This is because the positions of the laser radar module and the image acquisition module on the top of the inspection unmanned vehicle are slightly different, so the acquisition angles of the real-time point cloud information and the real-time environmental image information are also different. The real-time environmental image information is used as a reference image, and the real-time point cloud information is geometrically corrected by translation, rotation, and affine transformation until the angle and position of the road section defects contained in the real-time point cloud information and the real-time environmental image information are consistent in the image, and the registration is completed.

[0115] The image enhancement algorithm is used to enhance the real-time environment image information after registration. The algorithms that can be used for image enhancement include histogram equalization, gray world algorithm and automatic white balance. Taking the histogram equalization method as an example, the histogram of the original image is corrected to a uniformly distributed histogram through a transformation function. The image contrast is adjusted using the histogram, which can make the image look clearer and facilitate subsequent image recognition. The image recognition technology is used to extract the region of interest containing the road section disease characteristics in the real-time environment image information. The region of interest extracted from the real-time environment image information is the disease image information. The real-time environment image information is framed and mapped to the real-time point cloud information at the same time node, that is, the same region of interest is extracted at the same position in the real-time point cloud information obtained at the same time as the real-time environment image information. The region of interest extracted from the real-time point cloud information is the disease point cloud information. This is because the real-time environmental image information and the real-time point cloud map information have been registered. Theoretically, at the same time node, the road section defects in the real-time point cloud map information can overlap with the road section defect positions in the real-time environmental image information. Therefore, only by performing image recognition on the real-time environmental image information, the positions of the road section defects in the image and point cloud map can be preliminarily marked, which effectively reduces the calculation amount of image recognition and improves image recognition efficiency.

[0116] The edge features of the defect point cloud information and the defect image information are extracted respectively. If the edge features of the two are the same, it means that the defect feature type of the road section defect in the defect point cloud information and the defect image information is three-dimensional defect information. Three-dimensional defect information refers to traffic facilities or accidental drops such as water barriers, crash barrels, and vehicle parts on the road. Since these traffic facilities and accidental drops will affect road traffic and even cause traffic accidents, they are called road section defects. Since this type of road section defect is three-dimensional and has geometric features, it is called three-dimensional defect information. If the edge features of the two are different, it means that the defect feature type of the road section defect in the defect point cloud information and the defect image information is plane defect information. Plane defect information refers to road defects such as cracks and potholes on the road surface. Since these road defects will increase the wear of passing vehicles and increase the gasoline consumption of passing vehicles, thereby increasing transportation costs and causing economic losses, they are also road section defects. Since this type of road section defect is planar, it is named plane defect information. Since the geometric features of three-dimensional damage information can be identified through lidar scanning and edge features can be extracted in the point cloud map, while planar damage information is difficult to identify through lidar scanning, it is difficult to extract edge features of planar damage information in the point cloud map, or the extracted edge features are incomplete. Therefore, the type of damage features of the road section can be determined by comparing the edge features.

[0117] In one embodiment, referring to Figure 2 , the edge feature comparison between the defect point cloud information and the defect image information is performed, and the defect feature type of the weak signal section is obtained according to the edge feature comparison result, including the following steps:

[0118] S201, performing denoising processing on the disease point cloud image information;

[0119] S202, extracting the edge points of the diseased point cloud image after denoising based on an adaptive multi-feature fusion method;

[0120] S203, clustering the extracted disease edge points and performing fitting to obtain the point cloud edge features of the disease point cloud information;

[0121] S204, graying the disease image information to obtain grayscale disease image information;

[0122] S205, extracting all edge points of the grayscale disease image information by using an edge enhancement operator, and integrating all edge points to obtain image edge features of the grayscale disease image information;

[0123] S206, performing edge matching on the edge features of the point cloud image and the edge features of the image, and judging whether the edge features of the point cloud image and the edge features of the image are the same according to the edge matching result;

[0124] S207, if the edge feature of the point cloud image is the same as the edge feature of the image, then determining that the defect point cloud image information of the weak signal section is three-dimensional defect information;

[0125] S208: If the edge features of the point cloud image are different from the edge features of the image, it is determined that the defect image information of the weak signal section is plane defect information.

[0126] In this embodiment, the radius filtering method is used to perform preliminary denoising on the defect point cloud information, and the points with lower density in the defect point cloud information are removed as noise points. It is first assumed that each point in the defect point cloud information contains at least a certain number of neighboring points in the specified radius neighborhood, and the number of neighboring points of each point within the preset search radius is calculated. If the number of neighborhood points is less than the preset threshold number of neighborhood points, the point is regarded as a noise point and removed. After traversing each point in the defect point cloud information, the defect point cloud information after preliminary denoising is obtained, and then the fitting plane method is used to perform local denoising on the defect point cloud information after preliminary denoising, and the least squares method is used to perform plane fitting on the defect point cloud information, and the distance between each point and the plane is calculated. Points exceeding the preset plane distance threshold are extracted to finally obtain the denoised defect point cloud information.

[0127] After denoising, the defect edge points are extracted from the denoised defect point cloud information based on the adaptive multi-feature fusion method, and the neighboring point height difference, smoothness and neighborhood point angle of each point in the defect point cloud information are calculated. The neighboring point height difference, smoothness and neighborhood point angle are feature fused to form a multi-feature fusion strategy to determine whether a point is a defect edge point, and the defect edge points whose neighboring point height difference is less than the preset height difference threshold, whose smoothness is greater than the preset smoothness threshold, and whose neighborhood point angle is less than the preset neighborhood point angle threshold are extracted. All the extracted defect edge points are aggregated using a clustering algorithm, such as the DBSCAN (density-based clustering method with noise) algorithm, which aggregates the defect edge points whose point cloud density is greater than the preset threshold, and then uses the least squares method to curve fit the aggregated defect edge points to obtain the point cloud edge features of the defect point cloud information, which can reflect the geometric shape characteristics of the road section defect.

[0128] The damage image information is grayed out to make the road damage contour in the damage image information clearer. After obtaining the grayscale damage image information, the edge points are extracted using the edge enhancement operator. Commonly used edge enhancement operators include the Sobel operator, the Laplace operator, and the Roberts operator. Taking the Sobel operator as an example, the Sobel operator contains two sets of 3x3 matrices. The horizontal and vertical matrices are plane-convolved with the grayscale reservoir water image information to obtain the horizontal and vertical brightness difference approximations, respectively. The horizontal and vertical brightness difference approximations are squared, added, and then the square root is taken to obtain the image gradient of the grayscale reservoir water image information. Then set two high and low thresholds (for example, the high threshold can be set to 70 and the low threshold can be set to 40), and traverse the entire image gradient matrix. If the image gradient of a point is higher than the high threshold, it is set to 1 in the result. If the image gradient of the point is lower than the low threshold, it is set to 0 in the result. If the image gradient of the point is between the high and low thresholds, the following judgment is required: Check the 8 neighboring points of the point (regard it as the center point) to see if there is a point with an image gradient higher than the high threshold. If so, it means that the center point is connected to the determined edge point, so it is set to 1 in the result, otherwise it is set to 0. All points set to 1 are the edge points of the grayscale disease image information, and all edge points are integrated to obtain the image edge features of the grayscale disease image information.

[0129] The edge features of the point cloud map and the edge features of the image are matched to see if the edge features of the two can overlap. If they can overlap, it means that the edge features of the point cloud map are the same as the edge features of the image. If they cannot overlap, it means that the edge features of the point cloud map and the edge features of the image are different. If the edge features of the two are the same, it means that the type of damage feature of the road section damage in the damage point cloud map information and the damage image information is three-dimensional damage information. Three-dimensional damage information refers to traffic facilities or accidental drops such as water barriers, crash barrels, and vehicle parts on the road. Since these traffic facilities and accidental drops will affect road traffic and even cause traffic accidents, they are called road section damage. Because this type of road section damage is three-dimensional and has geometric features, it is called three-dimensional damage information. If the edge features of the two are different, it means that the type of the road section disease in the disease point cloud information and the disease image information is plane disease information. Plane disease information refers to road defects such as cracks and potholes on the road surface. Since these road defects will increase the wear of passing vehicles and increase the gasoline consumption of passing vehicles, thereby increasing transportation costs and causing economic losses, they are also road section diseases. Since this type of road section disease is planar, it is named plane disease information. Since the geometric features of three-dimensional disease information can be identified by laser radar scanning, edge features can be extracted in the point cloud map, while plane disease information is difficult to identify by laser radar scanning, it is difficult to extract edge features of plane disease information in the point cloud map, or the extracted edge features are incomplete. Therefore, the type of road section disease feature can be determined by comparing edge features. By comparing whether the edge features of the point cloud map are the same as the edge features of the image, the type of road section disease feature can be determined. Using different methods to obtain the location information of road section diseases according to different types of disease features can reduce the amount of calculation and improve the efficiency of locating road section diseases.

[0130] In one embodiment, denoising the disease point cloud image information includes the following steps:

[0131] Calculate the point cloud density of each point in the disease point cloud map information;

[0132] Remove points whose point cloud density is less than the preset point cloud density threshold to obtain the defect point cloud map information after preliminary denoising;

[0133] Perform plane fitting on the defect point cloud image information after preliminary denoising to obtain the defect point cloud plane;

[0134] Calculate the plane distance between each point in the disease point cloud information and the point cloud plane;

[0135] The points whose plane distance is greater than the preset point cloud distance are eliminated to obtain the denoised defect point cloud information.

[0136] In this embodiment, the radius filtering method is used to perform preliminary denoising on the defect point cloud information, and the points with lower density in the defect point cloud information are removed as noise points. It is first assumed that each point in the defect point cloud information contains at least a certain number of neighboring points in the specified radius neighborhood, and the number of neighboring points of each point within the preset search radius is calculated. If the number of neighborhood points is less than the preset threshold number of neighborhood points, the point is regarded as a noise point and removed. After traversing each point in the defect point cloud information, the defect point cloud information after preliminary denoising is obtained, and then the fitting plane method is used to perform local denoising on the defect point cloud information after preliminary denoising, and the least squares method is used to perform plane fitting on the defect point cloud information, and the distance between each point and the plane is calculated. Points exceeding the preset plane distance threshold are extracted to finally obtain the denoised defect point cloud information.

[0137] In one embodiment, obtaining the road section defect location information of the weak signal road section by combining the defect feature type and the road driving model includes the following steps:

[0138] If the defect point cloud image information of the weak signal section is three-dimensional defect information, the center point of the three-dimensional defect information is selected as the three-dimensional defect point;

[0139] Select the center point of the inspection unmanned vehicle as the disease inspection point;

[0140] The spatial distance between the three-dimensional disease point and the disease inspection point in the road driving model is calculated using the Euclidean distance formula;

[0141] Obtain the location information of the unmanned inspection vehicle on the weak signal section based on the road driving model;

[0142] By combining the spatial distance and the location information of the unmanned vehicle, the three-dimensional disease information and the location information of the road section disease in the weak signal section are obtained.

[0143] In this embodiment, if the defect point cloud map information of the weak signal section is three-dimensional defect information, the center point of the three-dimensional defect information is selected as the three-dimensional defect point, and the center point of the inspection unmanned vehicle is selected as the defect inspection point. The purpose of selecting the center point is to facilitate the subsequent calculation of the distance between the three-dimensional defect information and the inspection unmanned vehicle. The spatial distance between the three-dimensional defect point and the defect inspection point is calculated using the Euclidean distance formula. The calculation formula of the Euclidean distance formula is: Among them, (x1, y1, z1) is the coordinate of the defect inspection point in space, and (x2, y2, z2) is the coordinate of the three-dimensional defect point in space.

[0144] Based on the road driving model, the position information of the unmanned inspection vehicle in the weak signal section is obtained, and the three-dimensional disease information is combined with the spatial distance and the position information of the unmanned vehicle to obtain the section disease location information of the weak signal section. Since the road driving model will update the longitude and latitude of the unmanned inspection vehicle in real time, when the spatial distance between the three-dimensional disease information and the unmanned inspection vehicle is known, the longitude and latitude of the three-dimensional disease information can be obtained by using the unmanned inspection vehicle as a reference, that is, the section disease location information of the three-dimensional disease information in the weak signal section.

[0145] In one embodiment, the method further comprises:

[0146] If the defect image information of the weak signal section is plane defect information, the position information of the plane defect information in the road driving model is determined according to the position information of the unmanned vehicle;

[0147] Obtain device parameters of the image acquisition device in the real-time image acquisition module;

[0148] The location information is corrected according to the equipment parameters to obtain the section disease location information of the plane disease information in the weak signal section.

[0149] In this embodiment, if the defect point cloud map information of the weak signal section is plane defect information, the position information of the plane defect information in the road driving model is determined according to the position information of the unmanned vehicle, the position information of the inspection unmanned vehicle when the image acquisition module acquires the plane defect information is obtained, and the distance between the plane defect information and the inspection unmanned vehicle is preliminarily estimated by taking the inspection unmanned vehicle as a reference object, and the position information of the plane defect information in the road driving model is obtained, and the internal and external parameters of the image acquisition device in the image acquisition module are obtained. Taking a high-definition camera as an example, the focal length, pixel size and distortion coefficient of the high-definition camera are obtained, and the position information is corrected by the internal and external parameters of the image acquisition device to obtain the section defect position information of the plane defect information in the weak signal section. In addition, compared with the section defect position information of the three-dimensional defect information in the weak signal section, the section defect position information of the plane defect information in the weak signal section has a certain error, but the error can be ignored. However, the latter has a smaller amount of calculation and higher operation efficiency.

[0150] The present application also discloses a road inspection system based on data fusion and edge computing, including:

[0151] a memory configured to store instructions; and

[0152] The processor is configured to call instructions from the memory and implement the above-mentioned road inspection method based on data fusion and edge computing when executing the instructions.

[0153] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0154] Among them, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the computer device, etc., and the memory can also be a combination of an internal storage unit and an external storage device of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.

[0155] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned road inspection method based on data fusion and edge computing.

[0156] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0157] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of the processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0161] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0162] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory media such as modulated data signals and carrier waves.

[0163] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0164] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A road inspection method based on data fusion and edge computing, characterized in that: Applied to an unmanned inspection vehicle, the unmanned inspection vehicle comprises an edge computing module, a laser radar module and an image acquisition module, the edge computing module is arranged inside the unmanned inspection vehicle, the laser radar module and the image acquisition module are arranged side by side on the top of the unmanned inspection vehicle, and the method comprises the following steps: According to a preset road inspection plan, the inspection unmanned vehicle is controlled to perform a road inspection task along a preset inspection route, wherein the inspection route includes a pre-marked weak signal section; Before the unmanned inspection vehicle enters the weak signal section, the laser radar module is used to continuously collect real-time point cloud information, and the real-time driving information of the unmanned inspection vehicle is obtained at the same time, and the edge computing module is used to preliminarily build a road simulation model and a driving state model based on the real-time point cloud information and the real-time driving information. When the unmanned inspection vehicle enters the weak signal section, the road simulation model and the driving state model are updated in real time; The road simulation model and the driving state model updated in real time are fused in real time to obtain a road driving model; Controlling the unmanned inspection vehicle to continuously collect road section environment information of the weak signal section using the image acquisition module after entering the weak signal section to obtain real-time environment image information; The real-time environmental image information and the real-time point cloud image information are subjected to real-time image recognition using image recognition technology, and the section defect location information of the weak signal section is obtained by combining the image recognition result with the road driving model.

2. The method according to claim 1, characterized in that Before the unmanned inspection vehicle enters the weak signal section, the laser radar module is used to continuously collect real-time point cloud information, and the real-time driving information of the unmanned inspection vehicle is obtained at the same time, and the edge computing module is used to preliminarily build a road simulation model and a driving state model based on the real-time point cloud information and the real-time driving information. When the unmanned inspection vehicle enters the weak signal section, the road simulation model and the driving state model are updated in real time, including the following steps: Before entering the weak signal section, continuously scan the environment of the weak signal section using the laser radar module to obtain real-time point cloud information of the weak signal section; Using the edge computing module to preliminarily construct a road simulation model based on the real-time point cloud information obtained before entering the weak signal road section; Acquire the real-time driving information of the unmanned road inspection vehicle, and use the satellite positioning module installed on the unmanned road inspection vehicle to acquire the satellite positioning information of the unmanned road inspection vehicle in real time; Preliminarily constructing a driving state model based on the real-time driving information and the satellite positioning information; After entering the weak signal section, the road simulation model is updated in real time based on the real-time point cloud information acquired after entering the weak signal section; The driving state model is updated in real time based on the real-time driving information acquired after entering the weak signal section.

3. The method according to claim 2, characterized in that The use of the edge computing module to preliminarily construct a road simulation model based on the real-time point cloud information obtained before entering the weak signal road section includes the following steps: Preprocessing the real-time point cloud information; Segmenting the real-time point cloud map information to obtain road point cloud map information; Fitting the road point cloud information to obtain road feature information of the weak signal section; The edge computing module is used to preliminarily construct a road simulation model based on the road feature information.

4. The method according to claim 2, characterized in that: The real-time updating of the driving state model based on the real-time driving information obtained after entering the weak signal section comprises the following steps: Constructing a position prediction model of the road inspection unmanned vehicle based on a recursive neural network; Using the real-time driving information and the satellite positioning information before entering the weak signal road section as a training set to train the position prediction model; After entering the weak signal section, the driving state model obtained after entering the weak signal section is input into the trained position prediction model to obtain the predicted position of the road inspection unmanned vehicle; The driving state model is continuously updated based on the predicted position.

5. The method according to claim 1, characterized in that The method of using image recognition technology to perform real-time image recognition on the real-time environmental image information and the real-time point cloud image information, and combining the image recognition result with the road driving model to obtain the section defect location information of the weak signal section includes the following steps: Registering the collected real-time environment image information and the real-time point cloud image information; Performing image enhancement on the registered real-time environment image information using an image enhancement algorithm; Extracting a region of interest containing road section disease characteristics from the real-time environmental image information using an image recognition algorithm, wherein the region of interest is characterized as disease image information; Mapping the region of interest from the real-time environmental image information to the real-time point cloud image information at the same time node to obtain disease point cloud image information; Performing edge feature comparison between the defect point cloud information and the defect image information, and obtaining the defect feature type of the weak signal section according to the edge feature comparison result; The road section defect location information of the weak signal section is obtained by combining the defect feature type and the road driving model.

6. The method according to claim 5, characterized in that The step of performing edge feature comparison between the defect point cloud information and the defect image information and obtaining the defect feature type of the weak signal section according to the edge feature comparison result comprises the following steps: Performing denoising on the disease point cloud image information; Extracting the edge points of the diseased point cloud image after denoising based on an adaptive multi-feature fusion method; Clustering the extracted disease edge points and performing fitting to obtain the point cloud edge features of the disease point cloud information; grayscale the disease image information to obtain grayscale disease image information; Extracting all edge points of the grayscale disease image information by using an edge enhancement operator, and integrating all the edge points to obtain image edge features of the grayscale disease image information; Performing edge matching on the edge features of the point cloud image and the edge features of the image, and determining whether the edge features of the point cloud image and the edge features of the image are the same according to the edge matching result; If the edge feature of the point cloud image is the same as the edge feature of the image, determining that the defect point cloud image information of the weak signal section is three-dimensional defect information; If the edge feature of the point cloud image is different from the edge feature of the image, it is determined that the defect image information of the weak signal section is plane defect information.

7. The method according to claim 6, characterized in that The denoising process of the disease point cloud image information comprises the following steps: Calculating the point cloud density of each point in the disease point cloud map information; Removing points whose point cloud density is less than a preset point cloud density threshold to obtain the defect point cloud image information after preliminary denoising; Performing plane fitting on the defect point cloud image information after preliminary denoising to obtain a defect point cloud plane; Calculating the plane distance between each point in the disease point cloud image information and the point cloud plane; The points whose plane distance is greater than the preset point cloud distance are eliminated to obtain the denoised point cloud image information of the defect.

8. The method according to claim 6, characterized in that The step of obtaining the road section defect location information of the weak signal road section by combining the defect feature type and the road driving model comprises the following steps: If the defect point cloud image information of the weak signal section is three-dimensional defect information, then the center point of the three-dimensional defect information is selected as the three-dimensional defect point; Selecting the center point of the inspection unmanned vehicle as the disease inspection point; The spatial distance between the three-dimensional defect point and the defect inspection point in the road driving model is calculated using the Euclidean distance formula; Acquire the unmanned vehicle position information of the unmanned inspection vehicle on the weak signal section based on the road driving model; The spatial distance and the position information of the unmanned vehicle are combined to obtain the section defect position information of the three-dimensional defect information in the weak signal section.

9. The method according to claim 8, characterized in that The method further comprises: If the damage image information of the weak signal road section is plane damage information, determining the position information of the plane damage information in the road driving model according to the position information of the unmanned vehicle; Obtaining device parameters of an image acquisition device in the real-time image acquisition module; The position information is corrected according to the equipment parameters to obtain the section defect position information of the plane defect information in the weak signal section.

10. A road inspection system based on data fusion and edge computing, characterized in that: include: a memory configured to store instructions; as well as A processor is configured to call the instructions from the memory and to implement the road inspection method based on data fusion and edge computing according to any one of claims 1 to 9 when executing the instructions.

Images