Road damage detection method and apparatus, method and apparatus for determining layout position of road sensor, and computer device and storage medium

Abstract

A road damage detection method and apparatus, and a computer device and a storage medium. The detection method comprises: acquiring sensing data information and image information, which are sent by collection boxes, of a road, and a plurality of different sample damage ranges, and on the basis of the sensing data information of the road, determining the current damage range of the road and a damage sensing map of the road (S101); identifying a target damage area of the road on the basis of the image information of the road and the current damage range of the road, and identifying a damage type of the road on the basis of the damage sensing map of the road (S102); acquiring damage sensing data of the target damage area, and on the basis of the damage type of the road and a sample damage range of the damage type at each degree, calculating a damage level corresponding to the damage sensing data of the target damage area (S103); and using the target damage area, the damage level and a damage type as target damage information of the road (S104).

Description

Road damage detection method and device, road sensor layout location determination method and device, computer equipment and storage medium

[0001] Related applications

[0002] This application claims priority to Chinese patent applications numbered 202310740091.1, filed on June 21, 2023, entitled “Road Damage Detection Method, Apparatus, Computer Equipment and Storage Medium,” application numbered 202310745064.3, filed on June 21, 2023, entitled “Method, Apparatus and Computer Equipment for Determining the Location of Road Sensors,” and application numbered 202310745521.9, filed on June 21, 2023, entitled “Road Damage Detection Method, Apparatus, Computer Equipment and Storage Medium,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for detecting road damage, a method and device for determining the location of road sensors, computer equipment, storage media, and computer program products. Background Art

[0004] During their service life, road infrastructure is subjected to the reciprocating effects of vehicle and environmental loads, which can lead to cracking, cracking, and other defects, reducing the road's serviceability. Monitoring the service condition of roads, enabling maintenance teams to repair them at an early stage, is a key way to extend road service life and reduce lifecycle costs. Therefore, detecting road damage is a current research priority.

[0005] Traditional road damage detection methods rely on manual inspection of the road's current state and, based on this inspection, the worker's experience in determining the damage. This approach consumes significant labor costs, and due to varying worker experience, manual judgment can lead to significant deviations in the resulting road damage assessment, resulting in low road damage detection accuracy.

[0006] Furthermore, road health monitoring is a crucial component of road maintenance. To promptly identify potential road defects based on the stresses on the road, sensors are typically embedded within the road to monitor its health.

[0007] Generally speaking, the more sensors installed within a road, the higher the accuracy of road health monitoring. However, deploying too many sensors also increases the cost of building a monitoring system. Therefore, it is necessary to provide a sensor deployment solution that can achieve higher detection accuracy with fewer sensors, thereby reducing the cost of the monitoring system.

[0008] In addition, my country currently ranks first in the world in terms of highway mileage, and a large number of roads need to be maintained and managed. Due to environmental erosion, aging and other factors, highway road damage will occur. Therefore, road damage detection technology has emerged.

[0009] Traditional road damage detection technology usually involves installing cameras on the side of the road to collect road image data. Based on the large amount of collected road image data and convolutional neural networks, image recognition processing is performed on the collected road surface image data to identify road damage in the image data to complete road damage detection.

[0010] However, in traditional road inspection technology, road surface inspection is achieved through road surface image data, which can often only identify road surface diseases and damage. For cement concrete pavements, the most fundamental disease is the void under the road slab. When the road surface is damaged, it is often caused by more serious damage inside the road surface. Therefore, traditional road inspection technology cannot identify internal road diseases and damage, and the accuracy of road damage detection is poor.

[0011] Summary of the Invention

[0012] Based on this, it is necessary to provide a road damage detection method and device, a road sensor deployment location determination method and device, computer equipment, computer-readable storage medium and computer program product to address the above technical issues.

[0013] This application provides a road damage detection method. The method includes:

[0014] Acquiring sensor data information of a road sent by a collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determining a current damage range of the road and a damage sensor map of the road based on the sensor data information of the road; the sample damage ranges include damage ranges of different damage types;

[0015] Based on the image information of the road and the current damage range of the road, a damage location identification strategy is used to identify a target damage area of ​​the road, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road;

[0016] Acquiring damage sensor data of the target damage area, and calculating a damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type;

[0017] The target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road.

[0018] Optionally, the sensor data information of the road includes sensor information of multiple sensors on the road and location information of each of the sensors; and determining the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road includes:

[0019] Based on the sensing information of each sensor and the position information of each sensor, three-dimensional sensor map data of the road is established, and damage map data that meets the road damage condition is screened from the three-dimensional sensor map data of the road, and a range within the three-dimensional sensor map data that contains all the damage map data is used as the current damage range of the road; the damage map data includes the position information of the corresponding pixel points of the damage map data and the sensing information of the corresponding pixel points of the damage map data;

[0020] Based on the position information of each image data within the current damage range and the sensor information of each image data, a damage sensor map corresponding to each image data within the current damage range is established.

[0021] Optionally, identifying a target damaged area of ​​the road by using a damage location identification strategy based on the image information of the road and the current damage range of the road includes:

[0022] Creating three-dimensional image data of the road image information, and identifying a damaged location area of ​​the road image information through a damage image recognition network;

[0023] Establishing a correspondence between the three-dimensional sensor image data and the three-dimensional image data, and identifying a sub-damage range corresponding to a damage location area within the current damage range based on the correspondence;

[0024] Clustering the damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance between each damage map data in each damage map data group and the sub-damage range;

[0025] Filter each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as a sub-damage area, and use all sub-damage areas as the target damage area of ​​the road.

[0026] Optionally, the identifying the damage type of the road through a damage identification network based on the damage sensor map of the road includes:

[0027] Extracting damage feature data of each sub-damage area of ​​the damage sensing map, and inputting each damage feature data into a damage identification network to obtain a sub-damage type corresponding to each damage feature data;

[0028] The sub-damage type corresponding to each damage feature data is used as the damage type of the road.

[0029] Optionally, the calculating the damage level corresponding to the damage sensing data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type includes:

[0030] For each sub-damage region, based on the sample damage ranges of the respective degrees corresponding to the sub-damage types of the sub-damage region, identifying the sample damage range to which the damage sensing data of the sub-damage region belongs, and obtaining the degree of the sub-damage type corresponding to the sub-damage region;

[0031] Based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, the damage level corresponding to the sub-damage area is determined through a damage level classification strategy.

[0032] Optionally, after using the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road, the method further includes:

[0033] For each sub-damage region, determining a damage repair strategy for the sub-damage region based on the sub-damage type of the sub-damage region and the degree of the sub-damage type of the sub-damage region;

[0034] Arrange the maintenance order of each sub-damage area in descending order of the damage level of each sub-damage area to obtain a repair sequence for each sub-damage area; and fill the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area, and send the warning information containing the repair task information and the damage information of the target damage area to the display module.

[0035] Optionally, the method also includes: in response to the user's sensor acquisition system update operation, obtaining the acquisition system update information of each sensor, and sending the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

[0036] Optionally, the method further includes: in response to the user's sensor acquisition task upload operation, generating an acquisition instruction for each sensor, and sending the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

[0037] The present application also provides a road damage detection device, which includes an acquisition module, an identification module, a re-acquisition module, and a determination module.

[0038] an acquisition module configured to acquire sensor data information of the road sent by a collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine a current damage range of the road and a damage sensor map of the road based on the sensor data information of the road; the sample damage ranges include damage ranges of different damage types;

[0039] an identification module configured to identify a target damaged area of ​​the road based on the image information of the road and the current damage range of the road using a damage location identification strategy, and to identify a damage type of the road based on the damage sensor map of the road using a damage identification network;

[0040] a re-acquisition module, configured to acquire damage sensor data of the target damage area and calculate a damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type;

[0041] The determination module is configured to use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as target damage information of the road.

[0042] Optionally, the sensor data information of the road includes sensor information of multiple sensors on the road and location information of each of the sensors.

[0043] The acquisition module is specifically used to: establish three-dimensional sensor map data of the road based on the sensor information of each sensor and the position information of each sensor, and screen damage map data that meets the road damage condition in the three-dimensional sensor map data of the road, and use the range of all damage map data in the three-dimensional sensor map data as the current damage range of the road; the damage map data includes the position information of the pixel points corresponding to the damage map data and the sensor information of the pixel points corresponding to the damage map data; based on the position information of each map data within the current damage range and the sensor information of each map data, establish a damage sensor map corresponding to each map data in the current damage range.

[0044] Optionally, the recognition module is specifically used to: establish three-dimensional map data of the image information of the road, and identify the damage location area of ​​the image information of the road through a damage image recognition network; establish a correspondence between the three-dimensional sensor image data and the three-dimensional map data, and identify the sub-damage range corresponding to the damage location area within the current damage range based on the correspondence; cluster each damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculate the average distance of each damage map data in each damage map data group from the sub-damage range; screen each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as a sub-damage area, and use all sub-damage areas as the target damage area of ​​the road.

[0045] Optionally, the identification module is specifically used to: extract damage feature data of each sub-damage area of ​​the damage sensing map, and input each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data; and use the sub-damage type corresponding to each damage feature data as the damage type of the road.

[0046] Optionally, the re-acquisition module is specifically used to: for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identify the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtain the degree of the sub-damage type corresponding to the sub-damage area; based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, determine the damage level corresponding to the sub-damage area through a damage level classification strategy.

[0047] Optionally, the device further includes: a strategy determination module and a task determination module.

[0048] The strategy determination module is used to determine, for each sub-damage area, a damage repair strategy for the sub-damage area based on the sub-damage type of the sub-damage area and the degree of the sub-damage type of the sub-damage area.

[0049] The task determination module is used to arrange the maintenance order of each sub-damage area in descending order of the damage level of each sub-damage area to obtain a repair sequence for each sub-damage area; and fill the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area, and send the warning information containing the repair task information and the damage information of the target damage area to the display module.

[0050] Optionally, the device further includes an update module.

[0051] The update module is used to respond to the user's sensor acquisition system update operation, obtain the acquisition system update information of each sensor, and send the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

[0052] Optionally, the device further includes an instruction sending module.

[0053] The instruction sending module is used to generate an acquisition instruction for each sensor in response to the user's sensor acquisition task upload operation, and send the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

[0054] The present application also provides a road damage detection system, which includes a cloud platform and a collection box.

[0055] The collection box is communicatively connected to the cloud platform.

[0056] The collection box is used to collect sensor data information of the road and image information of the road.

[0057] The cloud platform is used to obtain multiple different sample damage ranges, and based on the sensor data information of the road, determine the current damage range of the road and the damage sensor map of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, a damage location recognition strategy is used to identify the target damage area of ​​the road, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road; damage sensor data of the target damage area is obtained, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensor data of the target damage area is calculated; the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road.

[0058] The above-mentioned road damage detection method and device obtain sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and multiple different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, the target damage area of ​​the road is identified through a damage position recognition strategy, and based on the damage sensor map of the road, the damage type of the road is identified through a damage identification network; the damage sensor data of the target damage area is obtained, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensor data of the target damage area is calculated; the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road. The target damaged area of ​​the road is determined through sensor data information and road image information. The damage identification network is then used to identify the damage type and damage level of the target damaged area based on multiple sample damage ranges of varying degrees to obtain the target damage information of the road. This eliminates the need for manual inspection to determine the target damaged area, improving the accuracy of determining the location of road damage. The damage type of the target damaged area is then identified, reducing the amount of data processing required to determine road damage information and improving the efficiency of detecting road damage information. Finally, the damage level of the target damaged area is identified through multiple sample damage ranges of varying degrees, improving the accuracy of road damage detection.

[0059] This application also provides a method for determining the placement of road sensors. The method includes:

[0060] Determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road;

[0061] Determining a reference layout point from the layout points corresponding to each of the road conditions, constructing a reference layout point group based on the reference layout point, and taking the layout points corresponding to each of the road conditions that do not belong to the reference layout point group as candidate layout points;

[0062] For each candidate layout point, determining a target road operating condition corresponding to the candidate layout point, determining first operating condition data corresponding to each reference layout point under the target road operating condition, and second operating condition data corresponding to the candidate layout point under the target road operating condition, and determining a correlation coefficient between the candidate layout point and the reference layout point group based on each of the first operating condition data and the second operating condition data, wherein the operating condition data is used to characterize the stress condition of the road at the layout point;

[0063] Selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points;

[0064] The target layout point is added to the reference layout point group to obtain a final target layout point group.

[0065] In one embodiment, determining the layout points corresponding to each road condition from a plurality of preset layout points of the target road includes:

[0066] Construct road models corresponding to the target road under various road conditions respectively;

[0067] For each of the road conditions, the vehicle driving data corresponding to the target road under the road condition is obtained; based on the vehicle driving data and the road model corresponding to the target road under the road condition, the third condition data corresponding to multiple preset layout points of the target road under the road condition are respectively determined; and based on each of the third condition data, the layout point corresponding to the road condition is determined from each of the preset layout points.

[0068] In one embodiment, determining the layout point corresponding to the road working condition from the preset layout points based on the third working condition data includes:

[0069] Sorting the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue;

[0070] Traverse the preset layout point queue, and when the third working condition data corresponding to each of the preset layout points arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each of the preset layout points arranged before the current traversal position as the layout points corresponding to the road working condition.

[0071] In one embodiment, determining the reference layout point from the layout points corresponding to each of the road working conditions includes:

[0072] Determining the number of layout points corresponding to each road condition according to the layout points corresponding to each road condition;

[0073] The layout points corresponding to the largest number of the layout points among the layout points are taken as the reference layout point group.

[0074] In one embodiment, determining the correlation coefficient between the candidate layout points and the reference layout point group based on the first operating condition data and the second operating condition data includes:

[0075] For any of the reference layout points, determining a correlation coefficient between the candidate layout point and the reference layout point based on the first operating condition data and the second operating condition data corresponding to the reference layout point;

[0076] The correlation coefficient between the candidate layout point and the reference layout point group is determined according to the correlation coefficient corresponding to each reference layout point.

[0077] In one embodiment, selecting a target layout point from each candidate layout point according to the correlation coefficient corresponding to each candidate layout point includes:

[0078] For any candidate layout point, if the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, the candidate layout point is used as a target layout point.

[0079] In one embodiment, the method further comprises:

[0080] According to the preset layout point arrangement strategy, a plurality of preset layout points are determined on the target road, and the plurality of preset layout points are evenly distributed on the target road.

[0081] The present application also provides a device for determining the placement of road sensors. The device includes:

[0082] A first determining module is used to determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road;

[0083] a second determining module, configured to determine a reference layout point from the layout points corresponding to each of the road working conditions, construct a reference layout point group based on the reference layout point, and use the layout points corresponding to each of the road working conditions that do not belong to the reference layout point group as candidate layout points;

[0084] a third determination module, configured to determine, for any candidate layout point, a target road operating condition corresponding to the candidate layout point, and determine first operating condition data corresponding to each reference layout point under the target road operating condition, and second operating condition data corresponding to the candidate layout point under the target road operating condition; and determine a correlation coefficient between the candidate layout point and the reference layout point group based on each of the first operating condition data and the second operating condition data, wherein the operating condition data is used to characterize the stress condition of the road at the layout point;

[0085] A selection module, configured to select a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points;

[0086] An adding module is used to add the target layout point to the reference layout point group to obtain a final target layout point group.

[0087] In one embodiment, the first determining module is further configured to:

[0088] Construct road models corresponding to the target road under various road conditions respectively;

[0089] For any of the road conditions, the vehicle driving data corresponding to the target road under the road condition is obtained, and based on the vehicle driving data and the road model corresponding to the target road under the road condition, the third condition data corresponding to multiple preset layout points of the target road under the road condition are respectively determined, and based on each of the third condition data, the layout point corresponding to the road condition is determined from each of the preset layout points.

[0090] In one embodiment, the first determining module is further configured to:

[0091] Sorting the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue;

[0092] Traverse the preset layout point queue, and when the third working condition data corresponding to each of the preset layout points arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each of the preset layout points arranged before the current traversal position as the layout points corresponding to the road working condition.

[0093] In one embodiment, the second determining module is further configured to:

[0094] Determining the number of layout points corresponding to each road condition according to the layout points corresponding to each road condition;

[0095] The layout points corresponding to the largest number of the layout points among the layout points are taken as the reference layout point group.

[0096] In one embodiment, the third determining module is further configured to:

[0097] For any of the reference layout points, determining a correlation coefficient between the candidate layout point and the reference layout point based on the first operating condition data and the second operating condition data corresponding to the reference layout point;

[0098] The correlation coefficient between the candidate layout point and the reference layout point group is determined according to the correlation coefficient corresponding to each reference layout point.

[0099] In one embodiment, the selection module is further configured to:

[0100] For any candidate layout point, if the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, the candidate layout point is used as a target layout point.

[0101] In one embodiment, the apparatus further comprises:

[0102] The fourth determining module is configured to determine a plurality of preset layout points on the target road according to a preset layout point arrangement strategy, wherein the plurality of preset layout points are evenly distributed on the target road.

[0103] The present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above methods when executing the computer program.

[0104] The present application also provides a non-volatile computer-readable storage medium having a computer program stored thereon, which implements any of the above methods when executed by a processor.

[0105] The present application also provides a computer program product, which includes executable instructions, and when the executable instructions are executed by a processor, implements any of the above methods.

[0106] The above-described method and apparatus for determining road sensor placement locations, and computer equipment, select a benchmark point group, then calculate the correlation coefficients between the remaining candidate points and the benchmark point group. If the correlation coefficients meet the requirements, it is determined that each benchmark point in the benchmark point group cannot accurately monitor the road conditions at that candidate point, and the candidate point is added to the benchmark point group to obtain the final target point group. This allows for detection of multiple typical road defects with fewer sensors, reducing detection costs.

[0107] This application also provides a road damage detection method. The method includes:

[0108] Acquiring an acceleration data set and an additional attribute feature data set of a road to be inspected within a preset inspection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be inspected;

[0109] Performing feature extraction on the acceleration data set according to a recursive neural network to obtain acceleration features;

[0110] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0111] The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0112] In one embodiment, the implantable sensor is a vibration acceleration sensor pre-installed inside the road to be inspected, and the step of obtaining an acceleration data set of the road to be inspected within a preset detection period includes:

[0113] Based on a preset sampling frequency and signal length, the vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of ​​the road to be detected within a preset detection period are collected; the vibration acceleration signals are generated by a road panel response generated by a vehicle passing through the road to be detected;

[0114] Performing data preprocessing on each of the vibration acceleration signals, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time;

[0115] An acceleration data set is obtained based on each of the acceleration vectors.

[0116] In one embodiment, obtaining an additional attribute feature dataset of a road to be detected within a preset detection period includes:

[0117] Acquire attribute characteristic data of a road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of an implanted sensor within the preset detection period;

[0118] The attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

[0119] In one embodiment, the hidden layer of the recursive neural network includes multiple hidden layer units, and the feature extraction of the acceleration data set based on the recursive neural network to obtain acceleration features includes: inputting the acceleration data set into a pre-trained recursive neural network, and performing feature extraction on the acceleration vector in the acceleration data set through the multiple hidden layer units included in the hidden layer of the recursive neural network to obtain acceleration features.

[0120] In one embodiment, after inputting the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be inspected, the method further includes:

[0121] Determining a target road management strategy based on the road damage results and the corresponding relationship between the road damage results and the road management strategies;

[0122] Based on the target road management strategy, an instruction is given to perform maintenance management on the road to be inspected.

[0123] In one embodiment, the method further comprises:

[0124] Obtaining a training data sample; the training data sample includes a training acceleration data set, an additional attribute feature data set, and a road damage category label;

[0125] Inputting the training acceleration data set into a recursive neural network, performing feature extraction on the training acceleration data set to obtain acceleration features;

[0126] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0127] Inputting the fused feature vector and the road damage category label into a classification prediction network, performing data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result;

[0128] The loss result of the road damage detection model is determined based on the classification prediction result, the road damage category label and the preset loss function, until the loss result meets the preset model loss condition, and the road damage detection model training is determined to be completed.

[0129] The present application also provides a road damage detection device, which includes an acquisition module, a feature extraction module, a splicing module, and a detection and discrimination module.

[0130] An acquisition module is used to acquire an acceleration data set and an additional attribute feature data set of the road to be inspected within a preset detection period; the acceleration data set is obtained by collecting vibration acceleration signals generated by vehicle loads using an implanted sensor installed inside the road to be inspected;

[0131] A feature extraction module, configured to extract features from the acceleration data set based on a recursive neural network to obtain acceleration features;

[0132] a splicing module, configured to perform feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0133] The detection and discrimination module is used to input the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0134] The present application also provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0135] Acquiring an acceleration data set and an additional attribute feature data set of a road to be inspected within a preset inspection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be inspected;

[0136] Performing feature extraction on the acceleration data set according to a recursive neural network to obtain acceleration features;

[0137] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0138] The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0139] The present application also provides a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0140] Acquiring an acceleration data set and an additional attribute feature data set of a road to be inspected within a preset inspection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be inspected;

[0141] Performing feature extraction on the acceleration data set according to a recursive neural network to obtain acceleration features;

[0142] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0143] The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0144] The present application also provides a computer program product. The computer program product includes executable instructions, and when the executable instructions are executed by a processor, the following steps are implemented:

[0145] Acquiring an acceleration data set and an additional attribute feature data set of a road to be inspected within a preset inspection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be inspected;

[0146] Performing feature extraction on the acceleration data set according to a recursive neural network to obtain acceleration features;

[0147] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0148] The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0149] The above-mentioned road damage detection method, apparatus, computer equipment, storage medium, and computer program product obtain an acceleration dataset and an additional attribute feature dataset for the road to be inspected within a preset detection period. The acceleration dataset is obtained by detecting the vibration acceleration signal generated by the vehicle load using an implantable sensor installed within the road to be inspected. A recursive neural network is used to extract features from the acceleration dataset to obtain acceleration features. The acceleration features are then concatenated with the additional attribute features in the additional attribute feature dataset to obtain a fused feature vector. The fused feature vector is then input into a preset classification and prediction network to obtain a road damage result for the road to be inspected. Using this method, the vibration acceleration signal represents the vibration response generated by the road to be inspected when the vehicle load passes over the road to be inspected. Thus, by processing and analyzing the acceleration data, the condition of the road's internal structure can be detected, and a road damage result corresponding to the road to be inspected can be obtained, thereby improving the accuracy of damage detection for the road to be inspected. BRIEF DESCRIPTION OF THE DRAWINGS

[0150] FIG1 is a schematic flow chart of a road damage detection method according to an embodiment of the present application;

[0151] FIG2 is a schematic diagram of the structure of a road damage detection system according to an embodiment of the present application;

[0152] FIG3 is a flowchart of an example of road damage detection in another embodiment of the present application;

[0153] FIG4 is a structural block diagram of a road damage detection device according to an embodiment of the present application;

[0154] FIG5 is a flow chart of a method for determining the placement of road sensors in one embodiment of the present application;

[0155] FIG6 is a schematic diagram of preset deployment points in one embodiment of the present application;

[0156] FIG7 is a schematic diagram of determining reference layout points in one embodiment of the present application;

[0157] FIG8 is a schematic diagram of determining a target deployment point group in one embodiment of the present application;

[0158] FIG9 is a flow chart of step 102 in one embodiment of the present application;

[0159] FIG10 is a flow chart of step 504 in one embodiment of the present application;

[0160] FIG11 is a flow chart of step 104 in one embodiment of the present application;

[0161] FIG12 is a flow chart of step 106 in one embodiment of the present application;

[0162] FIG13 is a schematic diagram of a process for determining preset deployment points in one embodiment of the present application;

[0163] FIG14 is a schematic diagram of a method for determining the placement of road sensors in one embodiment of the present application;

[0164] FIG15 is a structural block diagram of a device for determining the layout position of a road sensor according to an embodiment of the present application;

[0165] FIG16 is a diagram illustrating an application environment of a road damage detection method according to an embodiment of the present application;

[0166] FIG17 is a flow chart of a road damage detection method according to an embodiment of the present application;

[0167] FIG18 is a schematic diagram of a flow chart of steps for obtaining an acceleration data set in one embodiment of the present application;

[0168] FIG19 is a flow chart showing the steps of obtaining an additional attribute feature dataset in another embodiment of the present application;

[0169] FIG20 is a schematic diagram of the internal structure of a long short-term memory recurrent neural network according to an embodiment of the present application;

[0170] FIG21 is a flowchart illustrating steps for determining a target road management strategy in one embodiment of the present application;

[0171] FIG22 is a flowchart of a method for training a road damage detection model in one embodiment of the present application;

[0172] FIG23 is a flowchart illustrating an exemplary method for detecting road damage applied to a concrete pavement according to an embodiment of the present application;

[0173] FIG24 is a structural block diagram of a road damage detection device according to an embodiment of the present application;

[0174] FIG25 is a diagram showing the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0175] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0176] The road damage detection method provided in an embodiment of the present application is applied to a road infrastructure monitoring system. As shown in FIG2 , the system includes a cloud platform 201 and a collection box 202. The cloud platform controls the collection box 202 to collect sensor data from the road. The collection box 202 collects the sensor data from the road using sensors embedded in multiple fixed locations on the road and transmits the sensor data to the cloud platform 201. The cloud platform 201 determines a target damage area on the road based on the sensor data and image information of the road. Using a damage identification network, the cloud platform 201 identifies the damage type and damage level of the target damage area based on multiple sample damage ranges of varying degrees, thereby obtaining target damage information for the road. This eliminates the need for manual inspection and improves the accuracy of determining the location of road damage by determining the target damage area. Furthermore, identifying the damage type of the target damage area reduces the data processing required to determine road damage information and improves the efficiency of road damage detection. Finally, the cloud platform 201 identifies the damage level of the target damage area based on multiple sample damage ranges of varying degrees, thereby improving the accuracy of road damage detection.

[0177] In one embodiment, as shown in FIG1 , a road damage detection method is provided. The method is described by taking the application of the method to a cloud platform 201 as an example, and includes the following steps S101 to S104 .

[0178] Step S101: Acquire sensor data information of the road, image information of the road, and multiple different sample damage ranges sent by the collection box, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road.

[0179] The sample damage range includes the damage ranges of different damage types.

[0180] In this embodiment, the cloud platform acquires information from multiple sensors embedded in the road, collected by a collection box. The cloud platform obtains sensor information about the road detected by each sensor. The cloud platform also stores the location information of each sensor pre-set on the cloud platform, as well as the sensor information of each sensor, i.e., the sensor information about the road detected by each sensor, as sensor data for the road. The cloud platform acquires image information of the road using images captured by an image acquisition device installed in the collection box. The image acquisition device can be, but is not limited to, a camera, a single-lens reflex camera, or other photographic equipment. The cloud platform then retrieves sensor data for each damage type and severity level from the database, as sample damage ranges of varying severity for the road. Road damage types include faults, cracks, misalignments, voids, collapses, depressions, and potholes. Damage severity is determined based on the damage type, and can be classified as mild, moderate, or severe. For example, a crack of 1 meter corresponds to moderate damage, a misalignment of 10 cm corresponds to severe damage, and a void of 2 mm corresponds to mild damage. The damage range represents the range of the damage sensor data for the road. The sensor data information can be represented in a graphical form, and the collection box collects the sensor data information of each sensor in real time and stores it in real time. The image acquisition device installed in the collection box can collect road images and road videos in real time and store them in real time in the collection box.

[0181] The sensor is a vibration sensor, and the sensing information of the sensor is the vibration frequency and vibration amplitude of the vehicle colliding with the ground within the detection range of the sensor when the vehicle passes through the road.

[0182] The cloud platform then analyzes the current damage area of ​​the road based on the sensor location information and sensor information contained in the road sensor data, and determines the damage sensor map for the roads within that damage area. The specific determination process will be described in detail later.

[0183] In step S102, based on the image information of the road and the current damage range of the road, a damage location recognition strategy is used to identify the target damage area of ​​the road, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road.

[0184] In this embodiment, the cloud platform narrows the current damage range of the road based on road image information and, using a damage location recognition strategy, identifies a target damage area within this current damage range. The target damage area contains multiple sub-damage areas, each of which may be disconnected. The specific recognition process will be described in detail later. The cloud platform then inputs the road damage sensor map into the damage identification network to identify the damage type corresponding to the damage sensor map. The specific data processing process will be described in detail later.

[0185] Step S103 : acquiring damage sensor data of the target damage area, and calculating the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage ranges of each degree of the damage type.

[0186] In this embodiment, the cloud platform obtains damage sensing data of the target damage area based on each sensor. Then, based on the damage type of the road and the sample damage range of each degree of the damage type, the cloud platform determines the damage degree of the damage type corresponding to the target damage area based on the sample damage range corresponding to the damage sensing data of the target damage area, and determines the damage level of the target damage area based on the damage degree of the damage type corresponding to the target damage area. The cloud platform pre-stores the level corresponding to each damage degree of each damage type. The same damage level of different damage types may correspond to different damage levels. For example, when the degree of cracking is moderate damage, the damage level is level 3, when the degree of denting is moderate damage, the damage level is level 1, and when the degree of voiding is moderate damage, the damage level is level 2. The lower the level, the more severe the damage, that is, level 1 is the most severe damage.

[0187] In step S104, the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road.

[0188] In this embodiment, the cloud platform uses the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road.

[0189] Based on the above scheme, the target damaged area of ​​the road is determined through the sensor data information and image information of the road, and the damage type and damage level of the target damaged area are identified based on multiple sample damage ranges of different degrees through a damage identification network to obtain the target damage information of the road. Therefore, the target damaged area can be determined without manual inspection, thereby improving the accuracy of determining the location of road damage. Then, the damage type of the target damaged area is identified, which reduces the data processing volume for determining road damage information and improves the detection efficiency of road damage information. Finally, the damage level of the target damaged area is identified through multiple sample damage ranges of different degrees, thereby improving the accuracy of road damage detection.

[0190] Optionally, the sensor data information of the road includes sensor information from multiple sensors on the road and location information of each sensor. Based on the sensor data information of the road, the current damage range of the road and the damage sensor map of the road are determined, including: establishing three-dimensional sensor map data of the road based on the sensor information of each sensor and the location information of each sensor, screening damage map data that meets the road damage conditions in the three-dimensional sensor map data of the road, and using the range of all damage map data in the three-dimensional sensor map data as the current damage range of the road; the damage map data includes location information of corresponding pixels of the damage map data and sensor information of corresponding pixels of the damage map data; and establishing a damage sensor map corresponding to each map data in the current damage range based on the location information and sensor information of each map data within the current damage range.

[0191] In this embodiment, the cloud platform 201 creates a three-dimensional data map based on the sensor information and location information of each sensor, using a geodetic coordinate system bounded by the range of all sensor data. The cloud platform then populates the three-dimensional data map with the sensor data collected by each sensor and the location information of the three-dimensional data map corresponding to the sensor's location information, thereby generating three-dimensional sensor map data for the road. The three-dimensional sensor map data includes damage data corresponding to multiple pixels in the map data (i.e., sensor information for the pixels corresponding to the map data).

[0192] The cloud platform presets a damage data threshold and selects the image data corresponding to pixels with damage data greater than the threshold from the 3D sensor map data of the road as the damage map data. The cloud platform then uses the range within the 3D sensor map data containing all damage map data as the current damage range of the road. The damage map data includes the position information of the corresponding pixels in the damage map data and the sensor information of the corresponding pixels in the damage map data. Based on the position information and sensor information of each image data within the current damage range, the cloud platform creates a damage sensor map corresponding to each image data in the current damage range according to the distribution of the position information of each damage map data.

[0193] Based on the above scheme, by establishing three-dimensional sensor map data, the current damage range is screened, and based on the damage map data of the damage range, a damage sensor map is established, thereby improving the calculation accuracy of the damage sensor map.

[0194] Optionally, based on the image information of the road and the current damage range of the road, a damage location identification strategy is used to identify the target damage area of ​​the road, including: establishing three-dimensional map data of the image information of the road, and identifying the damage location area of ​​the image information of the road through a damage image recognition network; establishing a correspondence between the three-dimensional sensing map data and the three-dimensional map data, and identifying the sub-damage range corresponding to the damage location area within the current damage range based on the correspondence; clustering each damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance between each damage map data in each damage map data group and the sub-damage range; screening each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and taking the sub-damage range and the range contained in the target damage map data corresponding to the sub-damage range as the sub-damage area; and taking all sub-damage areas as the target damage area of ​​the road.

[0195] In this embodiment, the cloud platform 201 creates three-dimensional image data of the road image information based on a geodetic coordinate system and uses a damage image recognition network to identify damage location regions corresponding to road damage locations in the road image information. The damage image recognition network is a convolutional neural network based on image feature recognition. Based on the image scale and the number of pixels in the three-dimensional sensor image data, the cloud platform performs proportional and pixel-wise processing on the three-dimensional sensor image data. It then establishes a correspondence between the processed three-dimensional image data and the pixels with the same location information in each three-dimensional sensor image data. Based on this correspondence, the cloud platform then determines the damage location region within the current damage range of the three-dimensional sensor image data corresponding to the damage location region in the three-dimensional sensor image data. The cloud platform deems the damage location region within the current damage range as a sub-damage range of the current damage range. The cloud platform then calculates the straight-line distance between each damage image data item within the current damage range and the sub-damage range, and clusters each damage image data item based on its distance to the sub-damage range, resulting in multiple damage image data sets.

[0196] The cloud platform presets a distance threshold and calculates the average distance between each damage map data set and the sub-damage range. The cloud platform then selects target damage map data from the damage map data set corresponding to average distances below the preset distance threshold. The sub-damage range and the range encompassed by the target damage map data corresponding to the sub-damage range are then designated as sub-damage areas. All sub-damage areas are then designated as target damage areas for the road.

[0197] The cloud platform retrieves sample image information for multiple damage types and sample damage image information within each sample image. Then, for each sample image information for each damage type, the cloud platform inputs the sample image information for that damage type and the sample damage image information within that damage type into an initial damage image recognition network, training the initial damage image recognition network's damage recognition parameters for that damage type to obtain a damage image recognition network. The initial damage image recognition network is a convolutional neural network based on image feature recognition.

[0198] Based on the above solution, the current damage range of the road is limited through the image information of the road, and the current damage range is divided into multiple sub-damage areas, thereby improving the accuracy of identifying the damage information corresponding to each sub-damage area of ​​the road.

[0199] Optionally, based on the damage sensing map of the road, the damage type of the road is identified through a damage identification network, including: extracting damage feature data of each sub-damage area of ​​the damage sensing map, and inputting each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data; and using the sub-damage type corresponding to each damage feature data as the damage type of the road.

[0200] In this embodiment, the cloud platform extracts damage feature data for each sub-damage area of ​​the damage sensor map. This damage feature data is the sub-damage sensor map within the damage sensor map corresponding to the sub-damage area. The cloud platform inputs each damage feature data into a damage identification network, which then identifies the sub-damage type corresponding to each damage feature data. The cloud platform then uses the sub-damage type corresponding to each damage feature data as the road damage type. This damage identification network is a reinforcement learning neural network. This network inputs sample damage sensor maps of multiple damage types into an initial reinforcement learning neural network, trains the initial reinforcement learning neural network with the identification parameter range corresponding to each damage type, and generates a damage identification network.

[0201] Based on the above scheme, the damage identification network is used to identify the sub-damage type of each sub-damage area based on the damage feature data of each sub-damage area, thereby improving the efficiency of identifying the damage type of the damage area.

[0202] Optionally, based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensing data of the target damage area is calculated, including: for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identifying the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtaining the degree of the sub-damage type corresponding to the sub-damage area; based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, determining the damage level corresponding to the sub-damage area through a damage level classification strategy.

[0203] In this embodiment, for each sub-damage region, the cloud platform identifies the sample damage range to which the damage sensing data for the sub-damage region belongs based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage region, and obtains the degree of the sub-damage type corresponding to the sub-damage region. The cloud platform then determines the damage level corresponding to the sub-damage region based on the damage level preset for each degree of each damage type on the cloud platform, the sub-damage type corresponding to the sub-damage region, and the degree of the sub-damage type corresponding to the sub-damage region.

[0204] In this embodiment, the cloud platform identifies the damage level of each sub-damage area based on the preset damage level corresponding to the degree of each damage type of the cloud platform, thereby improving the efficiency of identifying the damage level.

[0205] Optionally, after taking the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road, the method further includes: for each sub-damage area, based on the sub-damage type of the sub-damage area and the degree of the sub-damage type of the sub-damage area, determining the damage repair strategy of the sub-damage area; arranging the maintenance order of each sub-damage area in descending order according to the damage level of each sub-damage area to obtain a repair sequence for each sub-damage area; and filling the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area, and sending the warning information containing the repair task information and the damage information of the target damage area to the display module.

[0206] In this embodiment, the cloud platform presets the maintenance method for each damage type and the maintenance resource consumption information corresponding to the different degrees of each damage type. Then, for each self-damaged area, the cloud platform determines the maintenance method of the sub-damaged area based on the sub-damage type of the sub-damaged area, and determines the maintenance resource consumption information of the sub-damaged area based on the degree of the sub-damage type of the sub-damaged area. The cloud platform then uses the maintenance method of the sub-damaged area and the maintenance resource consumption information of the sub-damaged area as the damage repair strategy for the sub-damaged area. For example, the sub-damage type of the sub-damaged area and the degree of the sub-damage type of the sub-damaged area are concave, and the degree of damage of the concave is moderate damage. Based on the maintenance method of each damage type preset in the cloud platform and the maintenance resource demand information corresponding to the different degrees of each damage type, the maintenance method corresponding to the concave is grouting repair, and the maintenance resource consumption information corresponding to the maintenance method of the moderately damaged concave is 0.3 tons / m 3 The cloud platform sorts the repair order of each sub-damage area according to the damage level of each sub-damage area from high to low, and obtains the repair sequence of each sub-damage area. Finally, the cloud platform fills the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area. After obtaining the repair task information, the cloud platform sends the warning information containing the repair task information and the damage information of the target damage area to the display module.

[0207] Based on the above solution, the repair task information of the target damaged area is determined through the damage information of the target damaged area, thereby improving the accuracy of the determined repair task information of the target damaged area.

[0208] Optionally, the method also includes: in response to the user's sensor acquisition system update operation, obtaining the acquisition system update information of each sensor, and sending the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

[0209] In this embodiment, when a user requests to update a sensor's acquisition system, the cloud platform responds to the user's sensor acquisition system update operation and obtains the acquisition system update information for each sensor. The cloud platform then sends the acquisition system update data to the acquisition box. The acquisition box then sends this acquisition system update data to each sensor and controls the sensor to update its current acquisition system data to the updated acquisition system data.

[0210] Based on the above solution, the cloud platform is used to control the collection box to update sensors in real time, avoiding the process of manually updating each sensor and improving the update efficiency of the sensors.

[0211] Optionally, the method also includes: in response to the user's sensor acquisition task upload operation, generating an acquisition instruction for each sensor, and sending the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

[0212] In this embodiment, when a user needs to adjust the sensor's collection task, the cloud platform responds to the user's sensor collection task upload operation, generates a collection instruction for each sensor based on the collection task, and then sends the collection instruction to the collection box; wherein, the collection instruction includes the collection task of each sensor. The collector sends the collection instruction to each sensor separately and controls each sensor to execute the collection task in the collection instruction. The collection task also includes the numerical values ​​corresponding to the heat dissipation and dehumidification working status of the collection box when executing the collection task, as well as the time point when the collection box starts / ends collection. The cloud platform will authenticate different users when different users access the cloud platform. Different users have different permissions to access the cloud platform, and different users can only query their own operation information on the cloud platform.

[0213] In one embodiment, as shown in FIG2 , a road damage detection system is provided. The system includes a cloud platform 201 and a collection box 202 .

[0214] The collection box 202 is communicatively connected to the cloud platform 201; the collection box 202 is used to collect sensor data information of the road; the cloud platform 201 is used to obtain image information of the road, multiple different sample damage ranges, and based on the sensor data information of the road, determine the current damage range of the road and the damage sensor map of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, a damage location recognition strategy is used to identify the target damage area of ​​the road, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road; damage sensor data of the target damage area is obtained, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensor data of the target damage area is calculated; the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road.

[0215] In this embodiment, collection box 202 is in communication with cloud platform 201. Collection box 202 transmits sensor data from each sensor on a road to cloud platform 201. Cloud platform 201 processes the sensor data, road images captured by image acquisition devices, and multiple sample damage ranges of varying degrees stored in a database to obtain target damage information for the road. Collection box 202 includes a download port for users to download historical road sensor data, as well as road images and videos stored therein.

[0216] Among them, the cloud platform 201 transmits control information (i.e., collection system update data information and control instructions) to the collection box 202, and the collection box 202 transmits data information (i.e., road sensor data information and road image information) to the cloud platform 201. The authentication module in the cloud platform 201 is used to store the usage permissions of each user, and the configuration distribution module is used to respond to the user's sensor collection system update operation and obtain the collection system update information of each sensor; the transceiver control module is used to transmit data with the collection box 202, and the storage module is used to store each data information processed by the cloud platform 201; the analysis and processing module is used to execute the content between step S101 and step S102; the diagnosis module is used to execute the content between step S103 and step S104; the display module is used to display the damage information of the target damaged area and the repair task information of the target damaged area; the alarm module is used to execute the task of alerting the user after receiving the early warning information.

[0217] In one embodiment, as shown in FIG3 , a road damage detection example is provided, which includes the following steps S301 to S312 .

[0218] Step S301: Acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges.

[0219] In step S302, based on the sensor information and position information of each sensor in the sensor data, three-dimensional sensor map data of the road is created. Within the three-dimensional sensor map data, damage map data that meets the road damage criteria is screened, and the range within the three-dimensional sensor map data containing all damage map data is defined as the current damage range of the road. In step S303, based on the position information and sensor information of each map data within the current damage range, a damage sensor map spectrum corresponding to each map data within the current damage range is created.

[0220] Step S304: Create three-dimensional image data of the road image information, and identify the damaged location area of ​​the road image information through a damage image recognition network.

[0221] Step S305 : establishing a correspondence between the three-dimensional sensing image data and the three-dimensional image data, and identifying a sub-damage range corresponding to the damage position area within the current damage range based on the correspondence.

[0222] In step S306, the damage map data within the current damage range are clustered according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and the average distance between each damage map data in each damage map data group and the sub-damage range is calculated.

[0223] Step S307: Filter each target damage map data in the damage map data group corresponding to an average distance below a preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as a sub-damage area, and use all sub-damage areas as target damage areas of the road.

[0224] Step S308: extract damage feature data of each sub-damage area of ​​the damage sensing map, and input each damage feature data into the damage identification network to obtain the sub-damage type corresponding to each damage feature data.

[0225] In step S309, the sub-damage type corresponding to each damage feature data is used as the damage type of the road.

[0226] Step S310 , for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identifies the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtains the degree of the sub-damage type corresponding to the sub-damage area.

[0227] Step S311 : Based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, the damage level corresponding to the sub-damage area is determined through a damage level classification strategy.

[0228] In step S312, the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road.

[0229] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0230] Based on the same inventive concept, embodiments of the present application also provide a road damage detection device for implementing the aforementioned road damage detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more road damage detection device embodiments provided below can be found in the aforementioned definitions of the road damage detection method and will not be further elaborated here.

[0231] In one embodiment, as shown in FIG. 4 , a road damage detection device is provided, including: an acquisition module 410 , an identification module 420 , a re-acquisition module 430 , and a determination module 440 .

[0232] Acquisition module 410 is configured to acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and multiple different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road. The sample damage ranges include damage ranges of different damage types.

[0233] The identification module 420 is used to identify the target damaged area of ​​the road based on the image information of the road and the current damage range of the road through a damage location identification strategy, and to identify the damage type of the road based on the damage sensor map of the road through a damage identification network.

[0234] The re-acquisition module 430 is configured to acquire the damage sensor data of the target damage area and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type.

[0235] The determination module 440 is configured to use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as target damage information of the road.

[0236] Optionally, the sensor data information of the road includes sensor information of multiple sensors on the road and location information of each of the sensors.

[0237] The acquisition module 410 is specifically used to: establish three-dimensional sensor map data of the road based on the sensor information of each sensor and the position information of each sensor, and screen damage map data that meets the road damage conditions in the three-dimensional sensor map data of the road, and use the range of all damage map data in the three-dimensional sensor map data as the current damage range of the road; the damage map data includes the position information of the corresponding pixel points of the damage map data and the sensor information of the corresponding pixel points of the damage map data; based on the position information of each map data within the current damage range and the sensor information of each map data, establish a damage sensor map corresponding to each map data in the current damage range.

[0238] Optionally, the recognition module 420 is specifically used to: establish three-dimensional map data of the image information of the road, and identify the damage location area of ​​the image information of the road through a damage image recognition network; establish a correspondence between the three-dimensional sensor image data and the three-dimensional map data, and identify the sub-damage range corresponding to the damage location area within the current damage range based on the correspondence; cluster each damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculate the average distance of each damage map data in each damage map data group from the sub-damage range; screen each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as a sub-damage area, and use all sub-damage areas as the target damage area of ​​the road.

[0239] Optionally, the identification module 420 is specifically used to: extract damage feature data of each sub-damage area of ​​the damage sensing map, and input each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data; and use the sub-damage type corresponding to each damage feature data as the damage type of the road.

[0240] Optionally, the re-acquisition module 430 is specifically used to: for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identify the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtain the degree of the sub-damage type corresponding to the sub-damage area; based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, determine the damage level corresponding to the sub-damage area through a damage level classification strategy.

[0241] Optionally, the device further comprises: a strategy determination module and a task determination module. The strategy determination module is configured to determine, for each sub-damage area, a damage repair strategy for the sub-damage area based on the sub-damage type of the sub-damage area and the degree of the sub-damage type of the sub-damage area.

[0242] The task determination module is used to arrange the maintenance order of each sub-damage area in descending order of the damage level of each sub-damage area to obtain a repair sequence for each sub-damage area; and fill the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area, and send the warning information containing the repair task information and the damage information of the target damage area to the display module.

[0243] Optionally, the device further includes an update module. The update module is configured to, in response to a user's sensor acquisition system update operation, obtain acquisition system update information for each sensor and send the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

[0244] Optionally, the device further includes an instruction sending module. The instruction sending module is configured to generate an acquisition instruction for each sensor in response to a user's sensor acquisition task upload operation, and send the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task for each sensor, and the acquisition instruction is configured to instruct each sensor to execute the acquisition task in the acquisition instruction.

[0245] Each module in the aforementioned road damage detection device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, allowing the processor to call and execute the corresponding operations of each module.

[0246] In one embodiment, as shown in FIG5 , a method for determining the deployment location of road sensors is provided. This embodiment uses the method applied to a terminal as an example. It is understood that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps 102 to 110.

[0247] Step 102 : Determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road.

[0248] In the embodiment of the present application, the target road is a road where sensors need to be deployed, and the preset deployment points refer to location points selected in advance on the target road where sensors can be deployed. As shown in Figure 6, the light gray points in the figure are the preset deployment points for a section of road, and the final target deployment points should be generated from the preset deployment points.

[0249] A road condition refers to the state of a road. For example, a road affected by a certain disease can be considered a road condition, as can a road condition during a certain time period. The layout points corresponding to a road condition are those among the preset layout points that can reflect the stress conditions on the road under that road condition. For example, these points may indicate where the road is subjected to greater stress or where there is a higher probability of vehicles passing through. The specific selection criteria can be determined by those skilled in the art based on actual needs and are not specifically limited in this embodiment of the present application.

[0250] Step 104 , determining a reference layout point from the layout points corresponding to each road condition, constructing a reference layout point group based on the reference layout point, and taking the layout points corresponding to each road condition that do not belong to the reference layout point group as candidate layout points.

[0251] In an embodiment of the present application, in order to make the final number of target layout points smaller, a benchmark layout point group can be determined based on the layout points corresponding to each road condition, and the layout points corresponding to each road condition that do not belong to the benchmark layout point group are used as candidate layout points, and then the candidate layout points are gradually added to the benchmark layout point group to obtain the target layout points that can ultimately reflect the stress conditions of the target road under various road conditions.

[0252] The selection criteria for the reference points in the reference point group can be determined by those skilled in the art based on actual needs. Since it is desirable to have a relatively small number of target points (requiring that the number of reference points initially selected should not be too large) and to improve the efficiency of subsequently adding candidate points (the more candidate points, the longer it takes to add them, which means that the number of reference points initially selected should not be too small), it is necessary to control the number of reference points within a reasonable range.

[0253] For example, if there are overlapping layout points among the layout points corresponding to various road conditions, then the overlapping layout points are the layout points that can simultaneously reflect the stress conditions of the target road under multiple road conditions. This point is likely to belong to the final target layout point; therefore, the overlapping layout points can be used as reference layout points.

[0254] Alternatively, reference points can be selected based on the distribution patterns of the layout points under different road conditions (e.g., distribution positions, distances between distribution centers, etc., where the distribution center can be a point with the smallest sum of distances from all layout points). When there are road conditions with significantly different distribution patterns, some points can be selected from the layout points corresponding to the above road conditions as reference points; when there are road conditions with relatively similar distribution patterns, reference points can be selected based on the overlapping layout points in the two road conditions.

[0255] The above process is illustrated using a specific example. As shown in Figure 7, if there are road conditions A, B, C, D, and E, where the layout points corresponding to road conditions A and D are located on the left side of the road, the layout points corresponding to road conditions B and C are located on the right side of the road, and the layout point corresponding to road condition E is located in the center of the road, then it can be determined that the distribution patterns of A and D, and B and C, are relatively close. A candidate set of benchmark layout points can first be determined based on the road conditions with relatively close distribution patterns. Considering that when the distribution patterns of the layout points are relatively close, the layout points corresponding to one road condition can roughly reflect the road stress conditions in another road condition, the overlapping layout points in the two road conditions can be selected as candidate benchmark layout points.

[0256] The final reference layout point group is then obtained based on the candidate reference layout points determined based on A and D, B and C, and the layout points corresponding to E. Considering that when the distribution patterns of layout points vary greatly, the layout points corresponding to one road condition may not reflect the road stress conditions in another road condition. Therefore, if only the layout points corresponding to one road condition are used as reference layout points, when the candidate layout points of the other road condition are subsequently matched with the reference layout points, it is likely that the candidate layout points will still need to be added to the reference layout point group. Therefore, when determining the reference layout points, some layout points corresponding to the above road conditions can be selected in advance as reference layout points to speed up the subsequent acquisition of the target layout points.

[0257] For example, taking the above example, since B and C correspond to only one candidate benchmark layout point, the candidate benchmark layout point can be added as the final benchmark layout point. A and D correspond to two candidate benchmark layout points. In order to minimize the number of final benchmark layout points, according to the force conditions of the target road at the two candidate benchmark layout points under two road conditions (which can be represented by pressure data, acceleration signal energy data, etc. collected at the layout points, hereinafter referred to as working condition data), one point (for example, a point with a larger average value of working condition data under two road conditions) can be selected from the two candidate benchmark layout points as the benchmark layout point. Similarly, a point can also be selected from the layout points corresponding to E as the benchmark layout point to obtain the final benchmark layout point group.

[0258] After the reference layout point group is determined, the layout points under each road condition that do not belong to the reference layout point group are used as candidate layout points, as shown in FIG7 .

[0259] Step 106: For each candidate layout point, determine the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each benchmark layout point under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition. Based on each first condition data and the second condition data, determine the correlation coefficient between the candidate layout point and the benchmark layout point group, wherein the condition data is used to characterize the stress condition of the road at the layout point.

[0260] In the embodiment of the present application, the target road condition corresponding to the candidate layout point is the road condition that includes the candidate layout point among the layout points. If there are multiple layout points that include the road conditions of the candidate layout point, there will be multiple target road conditions.

[0261] Working condition data refers to any data that can characterize the force conditions of the road at a point, such as the force exerted on the road at that point, the acceleration signal energy data collected at that point over a period of time, the frequency of vehicles passing by that point, and the pressure exerted on the road when the vehicle passes by, etc. The embodiments of this application do not specifically limit this.

[0262] The corresponding working condition data of each reference layout point and candidate layout point in the reference layout point group under the target road working condition can be calculated separately. Based on the first working condition data corresponding to each reference layout point and the second working condition data corresponding to the candidate layout point, the correlation coefficient between the reference layout point group and the candidate layout point can be determined. For example, when the working condition data is a numerical value (such as a pressure value), the correlation coefficient can be made inversely proportional to the smallest difference between the first working condition data and the second working condition data, that is, if the difference between the second working condition data and any first working condition data is small, the correlation coefficient between the candidate layout point and the reference layout point group is larger. When the working condition data is a sequence (such as the energy of the acceleration signal collected over a period of time), the correlation coefficient between each first working condition data and the second working condition data can be calculated according to any algorithm for calculating the correlation coefficient between two sequences (such as the Pearson correlation coefficient, the Spearman correlation coefficient, etc.), and then the largest correlation coefficient among the correlation coefficients is used as the correlation coefficient between the candidate layout point and the reference layout point group. The correlation coefficient may also be calculated in other ways, which are not specifically limited in the embodiments of the present application.

[0263] If a candidate layout point corresponds to multiple target road conditions, the correlation coefficients corresponding to the candidate layout point under multiple target road conditions can be calculated separately, and then the largest correlation coefficient, or the smallest correlation coefficient, or the average of the correlation coefficients, or other numerical values ​​that can characterize the characteristics of multiple correlation coefficients are used as the correlation coefficient corresponding to the candidate layout point. The embodiments of the present application do not make specific limitations on this.

[0264] Step 108 : selecting a target layout point from each candidate layout point according to the correlation coefficient corresponding to each candidate layout point.

[0265] In the embodiments of the present application, candidate layout points with a relatively low correlation coefficient, i.e., those that cannot accurately reflect the road stress conditions at the candidate layout points based on the original reference layout points, can be used as target layout points. For example, a correlation coefficient threshold can be pre-set, and candidate layout points with correlation coefficients less than the correlation coefficient threshold can be used as target layout points. Alternatively, the candidate layout points can be sorted from smallest to largest according to the correlation coefficient, and a preset number of candidate layout points ranked first can be used as target layout points. This embodiment of the present application does not specifically limit this.

[0266] In addition to the correlation coefficient, the working condition data can also be used as a criterion for selecting the target layout points. For example, for the target layout points that have been screened out based on the correlation coefficient, all the layout points under the target road working condition corresponding to the target layout point can be obtained, and the sum of the working condition data of each layout point can be calculated. Then, the first ratio of the working condition data of the target layout point to the sum of the working condition data of each layout point, and the second ratio of the sum of the working condition data of the layout points belonging to the benchmark layout point to the sum of the working condition data of each layout point are calculated. If the first ratio is less than the second ratio and the difference between the first ratio and the second ratio is large, it means that the target layout point is not important for characterizing the stress condition of the target road under the target road working condition, and the target layout point can be deleted. If the first ratio is greater than or equal to the second ratio, or the first ratio is less than the second ratio and the difference between the first ratio and the second ratio is not large, it means that the target layout point is more important, and the target layout point can be retained.

[0267] It should be noted that the above is only an example of selecting target layout points based on the correlation coefficient and operating condition data. In fact, those skilled in the art can also select target layout points based on the correlation coefficient and other data, or can also select target layout points based on the correlation coefficient and operating condition data in other ways, and the embodiments of the present application are not specifically limited to this.

[0268] Step 110: Add the target layout point to the reference layout point group to obtain a final target layout point group.

[0269] In this embodiment of the present application, each target deployment point is added to the reference deployment point group to obtain a final target deployment point group, as shown in Figure 8. After obtaining the target deployment point group, sensors can be deployed at corresponding locations on the target road according to each point in the target deployment point group to monitor the health of the target road.

[0270] The method for determining road sensor placement provided by an embodiment of the present application selects a benchmark placement point group, then calculates the correlation coefficients between the remaining candidate placement points and the benchmark placement point group. If the correlation coefficients meet the requirements, it is determined that the benchmark placement points in the benchmark placement point group cannot accurately monitor the road conditions at the candidate placement point. The candidate placement point is then added to the benchmark placement point group to obtain the final target placement point group. This allows for detection of multiple typical road defects with fewer sensors, reducing detection costs.

[0271] In one embodiment, as shown in FIG. 9 , in step 102 , layout points corresponding to each road condition are determined from a plurality of preset layout points of the target road, including steps 502 and 504 .

[0272] Step 502 : constructing road models corresponding to the target road under various road conditions.

[0273] Step 504, for each road condition, obtain the vehicle driving data corresponding to the target road under the road condition, and determine the third working condition data corresponding to multiple preset layout points of the target road under the road condition based on the vehicle driving data and the road model corresponding to the target road under the road condition, and determine the layout point corresponding to the road condition from each preset layout point based on each third working condition data.

[0274] In the embodiment of the present application, the working condition data can be obtained based on the road model of the target road, and the layout points corresponding to each road working condition can be determined from each preset layout point based on the working condition data. First, a road model of the target road in a normal state can be constructed. The road model should consider the structural parameters and material parameters of each layer of the road (such as the surface layer, base layer, subbase layer, etc.), the boundary conditions of each layer, the size and layout of the force transmission rods and pull rods in the road, etc., and then on this basis, the road model in the normal state is adjusted according to different road working conditions to obtain road models corresponding to different road working conditions. For example, when the road working condition is that the bottom of the road slab corner is hollow, the condition of the bottom of the slab corner being hollow can be simulated at the base layer of the road model in the normal state to obtain the road model under the road working condition.

[0275] The vehicle driving data corresponding to the road working condition refers to the data that can characterize the vehicle conditions on the road, such as the road roughness, vehicle speed, vehicle type, vehicle load, and wheel track distribution probability under the road working condition. The above data can be obtained by monitoring the target road in reality, or typical data obtained by monitoring multiple roads under the road working condition can be used as vehicle driving data. Based on the vehicle driving data and the road model, the force conditions of the road model at the preset layout points can be obtained to simulate the force conditions of the target road at the preset layout points and obtain the working condition data. For example, the vehicle moving load parameters can be obtained based on the above vehicle driving data. The vehicle moving load parameters are used to characterize the force conditions of any point on the road at any time under the road working condition; the vehicle moving load parameters are applied to the road model, and the road model is analyzed using the finite element method to obtain the third working condition data at each preset layout point.

[0276] Since it is necessary to select from the preset layout points a point that can reflect the road stress conditions under the current road working condition, the preset layout points with larger third working condition data can be used as layout points. For example, a third working condition data threshold can be pre-set, and the preset layout points with third working condition data greater than the third working condition data threshold can be used as layout points. Alternatively, the preset layout points can be sorted from largest to smallest according to the third working condition data, and the preset layout points that are ranked first by a preset number of points can be used as layout points. This embodiment of the present application does not specifically limit this.

[0277] The method for determining road sensor placement locations provided in embodiments of the present application constructs a road model for a given road condition, obtains third-condition data for each preset placement point based on vehicle driving data corresponding to the road condition, and then selects placement points based on the third-condition data. Based on the model, it is possible to determine which of the preset placement points best reflects the stress conditions under the given road condition, and then select reference placement points and final target placement points from these points, thereby improving the accuracy of the resulting target placement points.

[0278] In one embodiment, as shown in FIG. 10 , in step 504 , according to each third working condition data, a layout point corresponding to the road working condition is determined from each preset layout point, including: step 602 and step 604 .

[0279] Step 602 : sort the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue.

[0280] Step 604, traverse the preset layout point queue. When the third working condition data corresponding to each preset layout point arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue and use each preset layout point arranged before the current traversal position as the layout point corresponding to the road working condition.

[0281] In this embodiment of the present application, the preset layout points can be sorted from largest to smallest according to the third operating condition data, and the layout points corresponding to the road operating condition can be determined by traversing the preset layout point queue obtained by sorting. The preset strategy refers to the termination condition for selecting the layout points, such as the third operating condition data being less than the third operating condition data threshold, or the number of selected layout points being equal to the number threshold, etc.

[0282] For example, if the preset strategy is that the third operating condition data is less than the third operating condition data threshold, the queue of preset layout points can be traversed and the third operating condition data of the preset layout points arranged before the current traversal position can be compared with the third operating condition data threshold. If all third operating condition data are greater than or equal to the third operating condition data threshold, the traversal continues to the next preset layout point. If there is a third operating condition data that is less than the third operating condition data threshold, the traversal termination condition is triggered, and the preset layout point arranged before the current traversal position is used as the layout point corresponding to the road condition, and the traversal process stops.

[0283] Alternatively, the preset strategy may be that the sum of the third operating condition data of the preset layout points arranged before the current traversal position and the sum of the third operating condition data of all preset layout points is greater than a ratio threshold. The sum of the third operating condition data of all preset layout points can be calculated in advance, and in the process of traversing the queue of preset layout points, the sum of the third operating condition data of each preset layout point arranged before the current traversal position is calculated. When the ratio between this sum and the sum of the third operating condition data of all preset layout points is greater than a ratio threshold (e.g., 15%), the termination condition of the traversal is triggered, the preset layout point arranged before the current traversal position is used as the layout point corresponding to the road operating condition, and the traversal process is stopped.

[0284] The method for determining road sensor deployment locations provided in embodiments of the present application sorts each preset deployment point according to third operating condition data, traverses the resulting queue, and then selects a deployment point based on whether the third operating condition data of each preset deployment point preceding the currently traversed location satisfies a preset strategy. Preset deployment points that best reflect the stress conditions under the current road operating condition can be selected as deployment points for the current road operating condition, and then baseline deployment points and final target deployment points are selected from these points, thereby improving the accuracy of the final target deployment point.

[0285] In one embodiment, as shown in FIG11 , in step 104 , determining a reference layout point from the layout points corresponding to each road condition includes steps 1041 and 1042 .

[0286] Step 1041 : Determine the number of layout points corresponding to each road condition according to the layout points corresponding to each road condition.

[0287] Step 1042: The layout points corresponding to the largest number of layout points among the layout points are used as the reference layout point group.

[0288] In an embodiment of the present application, the layout points with the largest number of layout points among the corresponding layout points in each road condition can be used as the reference layout point group to reduce the number of subsequent matching between the candidate layout points and the reference layout point group.

[0289] If there are multiple road conditions with the largest number of layout points, the method of selecting reference layout points based on the distribution pattern of the layout points in the aforementioned embodiments can be referred to. The similarity between the distribution pattern of the layout points in these road conditions and the distribution pattern of the layout points in other road conditions can be calculated respectively, and the layout points corresponding to the road condition with the highest similarity to the distribution pattern of the layout points in other road conditions can be selected as the reference layout points. The highest similarity to the distribution pattern of the layout points in other road conditions can refer to the highest sum of the similarities between the distribution pattern of the layout points in the road condition and the distribution pattern of the layout points in other road conditions, or the highest average value, etc. This embodiment of the application does not specifically limit this.

[0290] The method for determining the road sensor deployment location provided in an embodiment of the present application uses a group of deployment points with the largest number of deployment points as a reference deployment point group, thereby reducing the number of times candidate deployment points need to be matched with the reference deployment point group and improving the efficiency of obtaining the target deployment point group.

[0291] In one embodiment, as shown in FIG12 , in step 106 , the correlation coefficient between the candidate layout points and the reference layout point group is determined based on the first operating condition data and the second operating condition data, including steps 1061 and 1062 .

[0292] Step 1061 : For any reference layout point, determine the correlation coefficient between the candidate layout point and the reference layout point according to the first operating condition data and the second operating condition data corresponding to the reference layout point.

[0293] Step 1062: Determine the correlation coefficient between the candidate layout points and the reference layout point group based on the correlation coefficient corresponding to each reference layout point.

[0294] In the embodiment of the present application, the correlation coefficient between the second operating condition data and the reference layout point group can be determined based on the correlation coefficient between each first operating condition data and the second operating condition data. Taking the case where the first operating condition data and the second operating condition data are both time series data, which include all acceleration signal energy data collected within one collection cycle, as an example, the correlation coefficient between the first operating condition data and the second operating condition data can be calculated based on the Pearson correlation coefficient (see formula (1)):

[0295] Where X refers to the first operating condition data, Y refers to the second operating condition data, r(X,Y) is the Pearson correlation coefficient, Cov(X,Y) is the covariance of X and Y, Var[X] is the variance of the first operating condition data, and Var[Y] is the variance of the second operating condition data.

[0296] The correlation coefficient between the second operating condition data and the reference layout point group can be determined based on all the calculated correlation coefficients. For example, the smallest correlation coefficient can be used as the correlation coefficient between the second operating condition data and the reference layout point group, the largest correlation coefficient can be used as the correlation coefficient between the second operating condition data and the reference layout point group, or the average of the correlation coefficients can be used as the correlation coefficient between the second operating condition data and the reference layout point group. This embodiment of the present application does not specifically limit this.

[0297] The method for determining the road sensor layout position provided in an embodiment of the present application determines the correlation coefficient between the candidate layout point and the reference layout point group based on the correlation coefficient between each first operating condition data and the second operating condition data. The target layout point can be selected based on the correlation coefficient between the candidate layout point and all the reference layout points. That is, the target layout point is a point that meets the correlation coefficient requirements of all the reference layout points, which can improve the selection accuracy of the target layout point.

[0298] In one embodiment, in step 108, a target layout point is selected from each candidate layout point based on the correlation coefficient corresponding to each candidate layout point, including: for any candidate layout point, if the correlation coefficient corresponding to the candidate layout point is less than the correlation coefficient threshold, the candidate layout point is selected as the target layout point.

[0299] In an embodiment of the present application, when the correlation coefficient corresponding to the candidate layout point is less than the correlation coefficient threshold (a pre-set value, the value of which can be pre-set by a person skilled in the art), it means that each benchmark layout point in the benchmark layout point group cannot accurately reflect the stress conditions of the target road at the candidate layout point, so the candidate layout point can be added to the benchmark layout point group as a target layout point.

[0300] Depending on the way of calculating the correlation coefficient of the candidate layout point, when the correlation coefficient corresponding to the candidate layout point is less than the correlation coefficient threshold, taking the candidate layout point as the target layout point also has different meanings accordingly. For example, when the correlation coefficient corresponding to the candidate layout point is the minimum value of the correlation coefficients between each benchmark layout point and the candidate layout point, taking the candidate layout point as the target layout point means that when there is a benchmark layout point that cannot accurately reflect the stress condition at the candidate layout point, the candidate layout point is added to the benchmark layout point; when the correlation coefficient corresponding to the candidate layout point is the minimum value of the correlation coefficients between each benchmark layout point and the candidate layout point, taking the candidate layout point as the target layout point means that when all benchmark layout points cannot accurately reflect the stress condition at the candidate layout point, the candidate layout point is added to the benchmark layout point. Those skilled in the art can adaptively adjust the standard for calculating the correlation coefficient in the aforementioned embodiment according to the effect to be achieved, and the embodiment of the present application does not make any specific limitation on this.

[0301] The method for determining the deployment position of road sensors provided in an embodiment of the present application takes a candidate deployment point whose correlation coefficient is less than a correlation coefficient threshold as a target deployment point. That is, when the stress condition of the road at the candidate deployment point cannot be accurately detected based on the reference deployment point, the candidate deployment point is taken as the target deployment point. This can improve the accuracy of road health monitoring when sensors are subsequently deployed according to the target deployment point.

[0302] In one embodiment, the above method further includes: determining a plurality of preset layout points on the target road according to a preset layout point arrangement strategy, wherein the plurality of preset layout points are evenly distributed on the target road.

[0303] In the embodiment of the present application, in order to ensure that the preset layout points can cover all points that can characterize the road stress conditions under all road conditions, the preset layout points can be evenly distributed. For example, as shown in FIG13 , the stress conditions of any point on the target road at each moment can be calculated in advance for each road condition using the road model under the road condition and the vehicle driving data, and a candidate point that best characterizes the road stress conditions under the road condition is selected (point 1, point 2, and point 3 on the left side of FIG13 ); after obtaining the candidate points under multiple road conditions, new points can be inserted between the candidate points based on the distribution of the candidate points and the distance between the candidate points to obtain the final preset layout points and ensure that the preset layout points are evenly distributed. For example, the distance between any two candidate points in the direction of travel can be calculated, and the minimum distance can be selected as the longitudinal distance between each preset layout point (d3 in Figure 13). The distance between any two candidate points perpendicular to the direction of travel can be calculated, and the minimum distance can be selected as the lateral distance between each preset layout point (s2 in Figure 13). New points can be evenly inserted between each candidate point based on the longitudinal and lateral distances, as shown on the right side of Figure 13. The shaded points on the right side of Figure 13 are the newly inserted points.

[0304] It should be noted that, as shown in FIG13 , since s3 does not divide s2 and d1 does not divide d3, the spacing between some points may be different from the spacing between other points. In this case, these points can be moved so that the spacing between them and other points is consistent with the spacing between other points; alternatively, these points can remain unchanged, which is not specifically limited in this embodiment of the present application.

[0305] Alternatively, the longitudinal and lateral distances between each preset layout point, as well as the reserved distance between the preset layout point and the road edge, can be pre-set. When it is necessary to determine the preset layout points for a target road, a row of preset layout points is first laid out at the reserved distance from the road edge according to the longitudinal distance, and then a row of preset layout points is laid out at each lateral distance, thereby achieving a uniform arrangement of the preset layout points.

[0306] The method for determining the layout position of road sensors provided in an embodiment of the present application makes each preset layout point evenly distributed when presetting each preset layout point, so that each preset layout point can include a point that can reflect the road stress conditions under various road conditions, thereby improving the determination accuracy of the preset layout points and further improving the accuracy of the final target layout points.

[0307] In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below with reference to specific examples.

[0308] 14 , there is shown a flow chart of a method for determining the placement of road sensors.

[0309] In this embodiment of the present application, target deployment points are sensor deployment points for various road hazards. Road sensor deployment locations can be determined for each concrete slab on the target road. 4 x 5 preset deployment points can be selected on the road in advance, and then the target deployment points can be determined from these preset deployment points.

[0310] A disease-free road model can be established for the target road. This model takes into account the structural parameters, material parameters, and boundary conditions of each road layer. Based on this disease-free model, road models with different road defects can be constructed for each road defect to be detected. Vehicle moving load parameters are calculated using vehicle speed, vehicle load, and wheel track distribution. These vehicle moving load parameters are applied to the road model to determine the acceleration signal energy at each target point.

[0311] The energy of the acceleration signal is |a in Figure 14 i,j (n)| 2 By adding the energy of each acceleration signal at a preset layout point, the sum of the acceleration signal energy corresponding to the preset layout point can be obtained (E in Figure 14 i,j (N in Figure 14 is the total number of acceleration signal energies.) Sort the preset deployment points by their acceleration signal energy sum, from largest to smallest. Traverse the resulting queue, sum the acceleration signal energies corresponding to the preset deployment points preceding the currently traversed position, and calculate the ratio of this sum to the total acceleration signal energies corresponding to all preset deployment points. The first time this ratio exceeds 15%, stop traversing the queue and use the preset deployment point preceding the currently traversed position as the deployment point for this road condition.

[0312] The layout points corresponding to the road condition with the largest number of layout points in each road condition are selected as benchmark layout points, and the layout points that are not benchmark layout points are selected as candidate layout points. The Pearson correlation coefficient between the candidate layout points and each benchmark layout point is calculated. When the correlation coefficient between all benchmark layout points and the candidate layout points is less than 0.8, the candidate layout point is added to the benchmark layout point group as a target layout point, resulting in the final target layout point group.

[0313] The method for determining the deployment location of road sensors provided in the embodiment of the present application uses as few sensors as possible to maximize monitoring, thereby improving detection efficiency and reducing economic costs.

[0314] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0315] Based on the same inventive concept, embodiments of the present application also provide a device for determining the location of a road sensor, for implementing the aforementioned method for determining the location of a road sensor. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for determining the location of a road sensor provided below can be found in the aforementioned method for determining the location of a road sensor, and will not be further elaborated here.

[0316] In one embodiment, as shown in FIG. 15 , a device 1100 for determining a road sensor deployment location is provided, comprising: a first determination module 1102 , a second determination module 1104 , a third determination module 1106 , a selection module 1108 , and an adding module 1100 .

[0317] The first determining module 1102 is configured to determine, from a plurality of preset layout points of the target road, layout points corresponding to each road condition.

[0318] The second determination module 1104 is used to determine the reference layout points from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout points, and use the layout points corresponding to each of the road conditions that do not belong to the reference layout point group as candidate layout points.

[0319] The third determination module 1106 is used to determine the target road condition corresponding to any candidate layout point, and determine the first condition data corresponding to each reference layout point under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group based on each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point.

[0320] The selection module 1108 is configured to select a target layout point from the candidate layout points according to the correlation coefficient corresponding to each candidate layout point.

[0321] The adding module 1110 is configured to add the target layout point to the reference layout point group to obtain a final target layout point group.

[0322] The device for determining road sensor placement locations provided by an embodiment of the present application selects a benchmark placement point group, then calculates the correlation coefficients between the remaining candidate placement points and the benchmark placement point group. If the correlation coefficients meet the requirements, it determines that the road conditions at the candidate placement point cannot be accurately monitored based on each benchmark placement point in the benchmark placement point group, and then adds the candidate placement point to the benchmark placement point group, thereby obtaining the final target placement point group. This allows for detection of multiple typical road defects using fewer sensors, reducing detection costs.

[0323] In one embodiment, the first determination module 1102 is further used to: respectively construct road models corresponding to the target road under various road conditions; for each of the road conditions, obtain vehicle driving data corresponding to the target road under the road condition; and determine third condition data corresponding to multiple preset layout points of the target road under the road condition based on the vehicle driving data and the road model corresponding to the target road under the road condition; and determine the layout point corresponding to the road condition from each of the preset layout points based on each of the third condition data.

[0324] In one embodiment, the first determination module 1102 is further used to: sort each of the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue; traverse the preset layout point queue, and when the third working condition data corresponding to each of the preset layout points arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each of the preset layout points arranged before the current traversal position as the layout point corresponding to the road working condition.

[0325] In one embodiment, the second determination module 1104 is further used to: determine the number of layout points corresponding to each road condition according to the layout points corresponding to each road condition; and use the layout points corresponding to the largest number of layout points among the layout points as the benchmark layout point group.

[0326] In one embodiment, the third determination module 1106 is further used to: for any of the reference layout points, determine the correlation coefficient between the candidate layout point and the reference layout point based on the first operating condition data and the second operating condition data corresponding to the reference layout point; and determine the correlation coefficient between the candidate layout point and the reference layout point group based on the correlation coefficient corresponding to each of the reference layout points.

[0327] In one embodiment, the selection module 1108 is further configured to: for any candidate layout point, if the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, select the candidate layout point as a target layout point.

[0328] In one embodiment, the apparatus further includes a fourth determining module configured to determine a plurality of preset layout points on the target road according to a preset layout point arrangement strategy, wherein the plurality of preset layout points are evenly distributed on the target road.

[0329] Each module in the aforementioned device for determining the location of road sensors may be implemented in whole or in part via software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, allowing the processor to call and execute the corresponding operations of each module.

[0330] In one embodiment, a computer device is provided. The computer device may be a server or a cloud platform, and its internal structure may be as shown in FIG25 . The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external cloud platform via wired or wireless communication. The wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a road damage detection method and a method for determining the location of road sensors. The display screen of the computer device may be a liquid crystal display or an electronic ink display. The input device of the computer device may be a touchscreen layer covering the display screen, buttons, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0331] Those skilled in the art will understand that the structure shown in Figure 25 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0332] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of any one of the above methods when executing the computer program.

[0333] In one embodiment, a non-volatile computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0334] In one embodiment, a computer program product is provided, comprising executable instructions, which, when executed by a processor, implement the steps of any of the above methods.

[0335] One embodiment of the present application provides a road damage detection method that can be applied in an application environment as shown in FIG16 . The method includes an implanted sensor 102 embedded in a road, a roadside acquisition device 104, and a terminal (the remote terminal is not shown in FIG16 ). The implanted sensor 102, the acquisition device 104, and the terminal can communicate with each other via a network. The terminal obtains an acceleration dataset and an additional attribute feature dataset of the road to be detected within a preset detection period. The acceleration dataset is obtained by detecting the vibration acceleration signal generated by the vehicle load using the implanted sensor 102 disposed within the road to be detected. The terminal then extracts features from the acceleration dataset using a recursive neural network to obtain acceleration features. The acceleration features and the additional attribute features in the additional attribute feature dataset are feature-concatenated to obtain a fused feature vector. The terminal then inputs the fused feature vector into a preset classification prediction network to obtain a road damage result for the road to be detected. This allows damage detection within the road to be detected, improving the accuracy of road damage detection.

[0336] The terminals may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, and IoT devices, and the IoT devices may be smart vehicle-mounted devices, etc.

[0337] It is understood that this method can be applied not only to the terminals in the aforementioned application scenarios, but also to servers, and can also be applied to systems including terminals and servers, and implemented through the interaction between the terminals and servers. Alternatively, this application can also be applied to cloud platform systems. Therefore, in addition to the data collection requiring the use of implantable sensors and roadside collection equipment, the specific execution end of this road damage detection method can be any computer device with memory and processor that can perform data processing functions. This application does not limit the type of execution end of this road damage detection method.

[0338] In one embodiment, as shown in FIG17 , this embodiment uses the road damage detection method applied to a terminal as an example. In this embodiment, the method includes the following steps S202 to S208 .

[0339] Step S202: Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period.

[0340] The acceleration data set is obtained by detecting the vibration acceleration signal generated by the vehicle load using an implanted sensor installed inside the road to be tested.

[0341] As shown in Figure 16, using a concrete pavement as an example, multiple implantable sensors are pre-placed within the road section to be inspected. These sensors detect the vibration acceleration signals generated by vehicle loads when a vehicle passes through the inspection area of ​​the road. The number and arrangement of the implantable sensors are not limited. For example, k implantable sensors can be deployed based on the size of the road slabs. These k implantable sensors are placed at different locations within the same road slab, such as at the corners, along the longitudinal edges of the slab along the driving direction, or at the center of the slab. Furthermore, due to the high speed of vehicles, when a vehicle passes through the same slab on the road, the k implantable sensors can capture the vibration acceleration signals generated by the internal structure of the road at the same moment of the vehicle's passage. These vibration acceleration signals are then processed and analyzed to obtain acceleration data at the moment of the vehicle's passage. Consequently, an acceleration dataset is constructed within a preset detection cycle.

[0342] In practice, within a preset detection cycle, the terminal acquires an acceleration dataset detected by the implanted sensor. Furthermore, as the vehicle passes through the detection area of ​​the road to be inspected, the terminal can also collect additional attribute feature data related to the road to be inspected using various other types of sensors. This allows the terminal to acquire an additional attribute feature dataset containing multidimensional data. Based on the acceleration dataset and the additional attribute feature dataset, the road structure of the road to be inspected can be detected.

[0343] Step S204: extract features from the acceleration data set using a recursive neural network to obtain acceleration features.

[0344] In practice, a road damage detection model is pre-deployed in the terminal. This model includes at least a recursive neural network and a classification prediction network. Specifically, after obtaining an acceleration dataset of the road to be inspected, the terminal uses the recursive neural network in the road damage detection model to extract features from the acceleration dataset to obtain acceleration features.

[0345] Optionally, the recursive neural network can be a long short-term memory recursive neural network (LSTM), or a multi-layer recursive neural network, a bidirectional recurrent neural network, etc. The embodiment of the present application does not limit the type of recursive neural network.

[0346] In the embodiment of the present application, the LSTM network is used as an example of a recursive neural network in the road damage detection model. The LSTM network can effectively extract the time series signal features of the time series signal (the vibration acceleration signal contained in the acceleration data set) based on the context information, thereby obtaining the acceleration features corresponding to the acceleration data set to more accurately detect road damage.

[0347] Step S206 , performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector.

[0348] During implementation, the terminal performs a feature concatenation operation on the acceleration features and the additional attribute features in the additional attribute feature dataset to obtain a fused feature vector. Specifically, the road damage detection model may also include a feature fusion layer, which may be a weighted function. Through this feature fusion layer, the acceleration features and the additional attribute features are concatenated to obtain a fused feature vector.

[0349] Step S208: Input the fused feature vector into a preset classification prediction network to obtain a road damage result of the road to be inspected.

[0350] During implementation, the terminal inputs the fused feature vector obtained after feature splicing into the classification prediction network of the road damage detection model, performs data analysis and processing on the fused feature vector through the classification prediction network, and determines the road damage result of the current road to be detected. The road damage result is used to reflect the specific road damage category.

[0351] Specifically, the classification prediction network included in the road damage detection model can be a multilayer perceptron (MLP), a supervised learning model based on a feedforward artificial neural network. This MLP performs prediction processing on the fused feature vector to obtain the probabilities of each road damage classification. The terminal can then select the road damage classification with the highest probability as the road damage result based on the probabilities of each road damage classification result.

[0352] In the above-mentioned road damage detection method, an acceleration dataset and an additional attribute feature dataset are obtained for the road to be inspected within a preset detection period. The acceleration dataset is obtained by collecting the vibration acceleration signal generated by the vehicle load using an implantable sensor installed inside the road to be inspected. A recursive neural network is used to extract features from the acceleration dataset to obtain acceleration features. The acceleration features are then concatenated with the additional attribute features in the additional attribute feature dataset to obtain a fused feature vector. The fused feature vector is then input into a preset classification prediction network to obtain a road damage result for the road to be inspected. Using this method, the vibration acceleration signal represents the vibration response generated by the road to be inspected when the vehicle load passes over the road to be inspected. Therefore, by processing and analyzing the acceleration data, the condition of the road's internal structure can be detected, and the corresponding road damage result for the road to be inspected can be obtained, thereby improving the accuracy of damage detection for the road to be inspected.

[0353] In one embodiment, as shown in FIG18 , the implantable sensor may be a vibration acceleration sensor pre-installed inside the road to be detected. Step S202 acquires an acceleration data set of the road to be detected within a preset detection period, including the following steps S2021 - S2023 .

[0354] Step S2021 : Based on a preset sampling frequency and signal length, the vibration acceleration signals collected by each vibration acceleration sensor when the vehicle passes through the detection area of ​​the road to be detected within a preset detection period are collected.

[0355] The vibration acceleration signal is generated by the road panel response generated by a vehicle passing through the road to be detected.

[0356] Specifically, the terminal collects vibration acceleration signals collected by each vibration acceleration sensor when the vehicle passes through the detection area of ​​the road to be detected within a preset detection period based on a preset sampling frequency and signal length.

[0357] Specifically, because the response frequency of road structures generally does not exceed 100Hz, we configured the vibration accelerometer to collect data at a frequency below 200Hz to reduce the amount of collected data and improve data processing efficiency. Furthermore, on typical highways, the time it takes for a vehicle to pass over each road panel is less than 2 seconds. Therefore, the vibration accelerometer truncates the signal length to 10s, ensuring sufficient response from the road panel.

[0358] Step S2022: performing data preprocessing on each vibration acceleration signal, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time.

[0359] In practice, the terminal preprocesses each vibration acceleration signal and constructs an acceleration vector based on the vibration acceleration signals collected at the same time. For example, if the internal road surface of the road to be inspected is equipped with k sensors, the acceleration data processed by different vibration acceleration sensors at the same time are combined into a k-dimensional acceleration vector.

[0360] Step S2023: Obtain an acceleration data set based on each acceleration vector.

[0361] In implementation, the terminal constructs an acceleration data set based on each acceleration vector generated within a preset detection period.

[0362] In this embodiment, the vibration acceleration signal generated by the road panel response when a vehicle passes by the road to be inspected is collected through a preset sampling frequency and signal length. Thus, an acceleration data set is constructed based on the vibration acceleration signal. The processing and analysis of this acceleration data set can realize the detection of the internal structure of the road.

[0363] In one embodiment, as shown in FIG19 , in addition to collecting vibration acceleration signals using a vibration acceleration sensor, multiple types of sensors may also be used to collect additional attribute feature data from multiple sources. Specifically, step S202 acquires an additional attribute feature dataset of the road to be inspected within a preset inspection period, including steps S402 and S404:

[0364] Step S402 , acquiring attribute characteristic data of the road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of the implanted sensor within the preset detection period.

[0365] The additional attribute feature data may include, but is not limited to, attribute feature data of the road to be detected, attribute feature data of the vehicle, and internal monitoring environment data of the implanted sensor.

[0366] During implementation, the terminal obtains various types of attribute feature data, such as the attribute feature data of the road to be inspected: road structure dimension information (for example, roadbed thickness, base layer thickness, roadbed width, etc.), joint form, etc.; the attribute feature data of each vehicle passing through the road to be inspected within a preset detection period: axle weight, vehicle type, vehicle speed, etc., as well as the internal monitoring environment data of the implanted sensor within the preset detection period: temperature, humidity, etc.

[0367] Specifically, various types of attribute feature data are acquired as follows: Attribute feature data for the road to be inspected can be directly retrieved from the road attribute feature record. That is, based on the current location of the road to be inspected, the terminal can query the road attribute feature record for information such as the road structure dimensions and joint type of the current road to be inspected. Attribute feature data for each vehicle passing on the road to be inspected, as well as internal monitoring environment data of the implanted sensor, can be collected using other types of sensors or directly reported by the implanted sensor. For example, an ultrasonic radar can be installed in the roadside acquisition equipment on the road to be inspected to collect the speed of vehicles passing through the road to be inspected. A weighing machine can also be embedded within the detection area of ​​the road to be inspected to collect vehicle loads, etc. Temperature and humidity sensors can be installed within the acquisition equipment and the implanted sensor to monitor the internal temperature and humidity of the acquisition equipment and the implanted sensor.

[0368] Optionally, the attribute characteristic data of the vehicle are not limited to including: vehicle speed, axle weight, vehicle model, vehicle load, etc., the attribute characteristic data of the road to be inspected are not limited to including road structure dimension information, joint form, material, etc., and the internal monitoring environment data of the implanted sensor are not limited to including temperature, humidity, etc. The embodiment of the present application does not limit the data type of the additional attribute characteristic data.

[0369] Step S404 , performing data cleaning and normalization processing on the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor to obtain an additional attribute feature data set.

[0370] During implementation, the terminal cleans various types of attribute feature data, including attribute feature data for the road to be inspected, attribute feature data for each vehicle, and internal monitoring environment data from implanted sensors. This data eliminates missing values ​​and outliers, and normalizes the cleaned attribute feature data to obtain normalized attribute feature data. Thus, an additional attribute feature dataset is constructed based on the cleaned and normalized attribute feature data.

[0371] The embodiment of the present application adopts a mean-variance normalization method to normalize each attribute feature data. The formula of the mean-variance normalization method is as follows:

[0372] where X i Represents each data in each type of attribute feature data, μ and σ represent the mean and standard deviation of the attribute feature data of this type respectively.

[0373] In this embodiment, an additional attribute feature data set is constructed by collecting multi-source data such as the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor. This additional attribute feature data set can include relevant attribute features for detecting road damage, thereby combining with the acceleration features to achieve multi-dimensional road damage detection.

[0374] In one embodiment, the hidden layer of the recursive neural network in the road damage detection model includes multiple hidden layer units, and step S204 performs feature extraction on the acceleration data set according to the recursive neural network to obtain acceleration features, which specifically includes step S2041.

[0375] In step S2041, the acceleration data set is input into a pre-trained recursive neural network, and the acceleration vector in the acceleration data set is feature extracted by a plurality of hidden layer units included in the hidden layer of the recursive neural network to obtain acceleration features.

[0376] In practice, a recurrent neural network is a neural network with time series synapses. In this application, an LSTM network is selected to process the acceleration data set. Specifically, the acceleration data set is input into the recurrent neural network model in the form of a time sequence. For example, the n k-dimensional acceleration vectors x on the time sequence t are t , as the input data of the LSTM network, the LSTM network contains multiple hidden layer units, as shown in Figure 20, the LSTM network outputs the output result h of the last hidden layer unit. n , as the feature extraction result of the acceleration data, that is, the acceleration feature. The number of hidden layer units is adjusted according to the actual training effect during the model training process, and is not limited in the embodiment of the present application.

[0377] In this embodiment, a recursive neural network is used to extract features from the acceleration dataset and learn the timing signals contained in the acceleration dataset, thereby obtaining acceleration features to better analyze the timing changes contained in the acceleration dataset and more accurately detect road damage.

[0378] In one embodiment, as shown in FIG21 , the method further includes steps S602 and S604 .

[0379] Step S602: Based on the road damage result, a target road management strategy is determined in the corresponding relationship between the road damage result and the road management strategy.

[0380] During implementation, the terminal is pre-configured with a list containing the correspondence between road damage results and road management strategies. Within a preset detection period, the terminal determines the current road damage results of the road to be detected. Then, based on the road damage results, the terminal determines the target road management strategy in the correspondence between each road damage result and the road management strategy.

[0381] Step S604: Based on the target road management strategy, instruct to perform maintenance management on the road to be inspected.

[0382] During implementation, the terminal instructs maintenance management of the road to be inspected based on the target road management strategy. Optionally, the target road management strategy includes generating alarm information and providing road maintenance and management advice information. For example, if the road damage result is road debonding damage, the target road management strategy includes: generating alarm information indicating road debonding damage, and at the same time, providing road maintenance and management advice information for maintaining the road debonding damage (for example, repairing and filling, removing damaged pavement, maintaining roadbed, etc.). The alarm information indicating road debonding damage is used to remind the user that road debonding damage exists within the target detection section of the current road to be inspected, and the road maintenance and management advice information is used to guide the user to complete the corresponding road maintenance.

[0383] In this embodiment, by pre-configuring the correspondence between road damage results and road management strategies, after determining the road damage results of the road to be detected, the target road management strategy for the current road to be detected can be automatically recommended, and then, based on the instructions of the target road management strategy, timely maintenance of the current road to be detected can be achieved.

[0384] In one embodiment, the road damage detection model includes a recursive neural network layer and a classification prediction network layer. Before the road damage detection model is applied, it is necessary to pre-train the model to ensure the accuracy of the model output results. As shown in Figure 22, the method also includes:

[0385] Step S702: Obtain training data samples.

[0386] The training data samples include training acceleration datasets, additional attribute feature datasets, and road damage category labels.

[0387] During implementation, the terminal obtains a training data sample. The additional attribute feature dataset in the training data sample may include, but is not limited to, a vehicle attribute feature dataset, a road attribute feature dataset, and internal monitoring environment data from an implanted sensor. When constructing the training data sample, the training data is divided into a training set, a validation set, and a test set, and labeled. The specific division ratio may be 0.9:0.09:0.01.

[0388] Optionally, the various types of training data contained in the training data sample are similar to the data acquisition process in step 202 in the above embodiment. For example, training acceleration data and internal monitoring environment data of the implanted sensor can be collected through the implanted sensor, vehicle attribute feature data can be collected through the collection device, and road attribute feature data can be obtained through query, etc. Then, the terminal can construct a training data sample based on the acquired training acceleration data, vehicle attribute feature data, road attribute feature data, and internal environment monitoring data of the implanted sensor, etc. The embodiments of the present application will not be described in detail here.

[0389] Optionally, after acquiring each type of training data, data cleaning and normalization processing can be performed on each type of training data. This processing process is similar to step S404 in the above embodiment and will not be described in detail in this embodiment of the present application. Thus, training data samples are constructed based on the training data after data cleaning and normalization processing.

[0390] In step S704, the training acceleration data set is input into a recursive neural network, and feature extraction is performed on the training acceleration data set to obtain acceleration features.

[0391] During implementation, the terminal inputs the training acceleration dataset from the training data samples into a recurrent neural network. The hidden units in the recurrent neural network extract features from the training acceleration dataset to obtain acceleration features. The number of hidden units is adjusted based on the actual training results of the road damage detection model.

[0392] Specifically, when processing training acceleration data, the recurrent neural network adaptively learns the temporal dependencies of sequential data to model and predict acceleration data sequences. During model training, supervised learning can be used to train the model using known annotated acceleration data, thereby extracting features of the acceleration data, such as acceleration value trends, peak values, and duration.

[0393] Step S706 : performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector.

[0394] During implementation, the terminal performs feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector.

[0395] Specifically, the road damage detection model may also include a feature fusion layer, which is used to fuse acceleration features and additional attribute features. The feature fusion layer may be a weighted function with a preset weight coefficient, and the feature splicing of multi-source features is achieved through the weighted function to obtain a fused feature vector.

[0396] In step S708, the fused feature vector and the road damage category label are input into the classification prediction network, and the fused feature vector is processed by the classification prediction network to obtain a classification prediction result.

[0397] During implementation, during the training of the classification prediction network, the terminal inputs the fused feature vector and the road damage category label into the classification prediction network, performs data processing on the fused feature vector through the classification prediction network, and outputs a classification prediction result representing the road damage category. The classification prediction result can be the probability of the corresponding road damage category.

[0398] Specifically, the classification prediction network can be a multilayer perceptron, a feedforward neural network with multiple hidden layers. It can be used for supervised learning tasks. In classification tasks, the MLP can be trained to learn to map input training data samples to predefined category labels. During the training process of this application, except for the last classification layer, which uses the softmax activation function, the activation function of each layer of the multilayer perceptron uses the Relu activation function. The activation function is used to calculate the probability that each input training data belongs to a different road damage category.

[0399] Optionally, the Adam method can be selected to train the classification prediction network. The Adam method can adaptively adjust the learning rate during the gradient descent process to avoid local convergence of the road damage detection model during the model training process.

[0400] Step S710 , determining the loss result of the road damage detection model based on the classification prediction result, the road damage category label, and the preset loss function, until the loss result meets the preset model loss condition, and determining that the road damage detection model training is completed.

[0401] During implementation, the terminal determines the road damage detection model's loss result based on the classification prediction results, the road damage category label, and a preset loss function. Furthermore, the terminal determines whether the road damage detection model is trained based on the loss result and the preset model loss condition.

[0402] Specifically, the final loss function of the road damage detection model can be a cross entropy loss function, and the calculation formula is as follows:

[0403] Where M is the total number of categories, c is the different classification categories (road damage categories), i represents different samples, and p ic is the predicted probability that training data i belongs to classification category c, y ic It has only two values: 0 and 1. When the training data i is actually labeled as c, it is 1, and otherwise it is 0.

[0404] The preset model loss condition may be less than or equal to a preset model loss threshold. If the model loss result does not meet the preset model loss condition, steps S702 to S710 are repeated until the loss result meets the preset model loss condition, and the road damage detection model training is determined to be complete.

[0405] In this embodiment, the road damage detection model is trained using training data samples containing multidimensional training data to obtain a trained road damage detection model. Through the trained road damage detection model, road damage detection based on multidimensional detection data can be implemented.

[0406] In one embodiment, as shown in FIG23 , an example of a road damage detection method applied to a concrete pavement is given, specifically including steps 801 to 805 .

[0407] Step 801: Acquire an additional attribute feature dataset of a road to be detected within a preset detection period.

[0408] Step 802: Acquire an acceleration data set of the road to be inspected within a preset inspection period.

[0409] Step 803: extract features from the acceleration data set using a recursive neural network to obtain acceleration features.

[0410] Step 804 : Concatenate the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector.

[0411] Step 805: Input the fused feature vector into a preset classification prediction network to obtain the road damage result of the concrete pavement.

[0412] As shown in FIG23 , the execution order of step 801 and step 802 may be synchronous.

[0413] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0414] Based on the same inventive concept, embodiments of the present application also provide a road damage detection device for implementing the aforementioned road damage detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more road damage detection device embodiments provided below can be found in the aforementioned definitions of the road damage detection method and will not be further elaborated here.

[0415] In one embodiment, as shown in FIG24 , a road damage detection device 900 is provided, comprising: an acquisition module, a feature extraction module, a splicing module, and a detection and discrimination module.

[0416] The acquisition module 901 is used to obtain the acceleration data set and the additional attribute feature data set of the road to be inspected within a preset detection period; the acceleration data set is obtained by detecting the vibration acceleration signal generated by the vehicle load using an implanted sensor set inside the road to be inspected.

[0417] The feature extraction module 902 is used to extract features from the acceleration data set according to a recursive neural network to obtain acceleration features.

[0418] The splicing module 903 is used to perform feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector.

[0419] The detection and discrimination module 904 is used to input the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0420] In one embodiment, the implantable sensor is a vibration acceleration sensor pre-installed within the road to be inspected. Acquisition module 901 is specifically configured to collect, based on a preset sampling frequency and signal length, vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of ​​the road to be inspected within a preset detection period. The vibration acceleration signals are generated by the road panel response generated by vehicles passing through the road to be inspected.

[0421] The acquisition module 901 is used to perform data preprocessing on each vibration acceleration signal and construct an acceleration vector based on the vibration acceleration signals collected at the same time.

[0422] The acquisition module 901 is configured to obtain an acceleration data set based on each acceleration vector.

[0423] In one embodiment, the acquisition module 901 is specifically used to obtain the attribute feature data of the road to be detected, the attribute feature data of each vehicle passing through the road to be detected within a preset detection period, and the internal monitoring environment data of the implanted sensor within the preset detection period; the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

[0424] In one embodiment, the hidden layer of the recursive neural network includes multiple hidden layer units, and the feature extraction module 902 is specifically used to input the acceleration data set into the pre-trained recursive neural network, and extract features of the acceleration vector in the acceleration data set through the multiple hidden layer units included in the hidden layer of the recursive neural network to obtain acceleration features.

[0425] In one embodiment, the apparatus 900 further includes: a determination module and an indication module.

[0426] The determination module is used to determine a target road management strategy based on the road damage result and in a corresponding relationship between the road damage result and the road management strategy.

[0427] The instruction module is used to instruct maintenance management of the road to be inspected based on the target road management strategy.

[0428] In one embodiment, the apparatus 900 further includes: a training acquisition module, a feature extraction module, a splicing module, a detection and discrimination module, and a training and discrimination module.

[0429] The training acquisition module is used to obtain training data samples. The training data samples include training acceleration datasets, additional attribute feature datasets, and road damage category labels.

[0430] The feature extraction module is used to input the training acceleration data set into the recursive neural network, perform feature extraction on the training acceleration data set, and obtain acceleration features.

[0431] The splicing module is used to perform feature splicing on the acceleration features and the additional attribute features in the additional attribute feature dataset to obtain a fused feature vector.

[0432] The detection and discrimination module is used to input the fused feature vector and road damage category label into the classification prediction network, and perform data processing on the fused feature vector through the classification prediction network to obtain the classification prediction result.

[0433] The training discriminant module is used to determine the loss result of the road damage detection model based on the classification prediction results, road damage category labels and the preset loss function, until the loss result meets the preset model loss condition, and the road damage detection model training is determined to be completed.

[0434] Each module in the aforementioned road damage detection device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, allowing the processor to call and execute the corresponding operations of each module.

[0435] In one embodiment, a computer device is provided, which may be a terminal. Its internal structure may be as shown in FIG25 . The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a road damage detection method. The display screen of the computer device may be a liquid crystal display or an electronic ink display. The input device of the computer device may be a touchscreen covering the display screen, buttons, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0436] Those skilled in the art will understand that the structure shown in Figure 25 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0437] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0438] Acquiring an acceleration data set and an additional attribute feature data set of a road to be inspected within a preset inspection period; the acceleration data set is obtained by collecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be inspected;

[0439] Performing feature extraction on the acceleration data set according to a recursive neural network to obtain acceleration features;

[0440] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0441] The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

[0442] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0443] Based on a preset sampling frequency and signal length, the vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of ​​the road to be detected within a preset detection period are collected; the vibration acceleration signals are generated by a road panel response generated by a vehicle passing through the road to be detected;

[0444] Performing data preprocessing on each of the vibration acceleration signals, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time;

[0445] An acceleration data set is obtained based on each of the acceleration vectors.

[0446] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0447] Acquire attribute characteristic data of a road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of an implanted sensor within the preset detection period;

[0448] The attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

[0449] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0450] The acceleration data set is input into a pre-trained recursive neural network, and acceleration vectors in the acceleration data set are feature extracted using a plurality of hidden layer units included in a hidden layer of the recursive neural network to obtain acceleration features.

[0451] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0452] Determining a target road management strategy based on the road damage results and the corresponding relationship between the road damage results and the road management strategies;

[0453] Based on the target road management strategy, an instruction is given to perform maintenance management on the road to be inspected.

[0454] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0455] Obtaining a training data sample; the training data sample includes a training acceleration data set, an additional attribute feature data set, and a road damage category label;

[0456] Inputting the training acceleration data set into a recursive neural network, performing feature extraction on the training acceleration data set to obtain acceleration features;

[0457] Performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature dataset to obtain a fused feature vector;

[0458] Inputting the fused feature vector and the road damage category label into a classification prediction network, performing data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result;

[0459] The loss result of the road damage detection model is determined based on the classification prediction result, the road damage category label and the preset loss function, until the loss result meets the preset model loss condition, and the road damage detection model training is determined to be completed.

[0460] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0461] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0462] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0463] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A road damage detection method, characterized in that: include: Acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types; Based on the image information of the road and the current damage range of the road, a target damage area of ​​the road is identified through a damage location identification strategy, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road; Acquire damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type; The target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road.

2. The method according to claim 1, characterized in that The sensor data information of the road includes sensor information of multiple sensors of the road and position information of each of the sensors; the current damage range of the road and the damage sensor map of the road are determined based on the sensor data information of the road, including: Based on the sensing information of each sensor and the position information of each sensor, three-dimensional sensing map data of the road is established, and damage map data that meets the road damage condition is screened in the three-dimensional sensing map data of the road, and the range of all damage map data included in the three-dimensional sensing map data is used as the current damage range of the road; the damage map data includes the position information of the corresponding pixel points of the damage map data and the sensing information of the corresponding pixel points of the damage map data; Based on the position information of each image data within the current damage range and the sensor information of each image data, a damage sensor map corresponding to each image data within the current damage range is established.

3. The method according to claim 1 or 2, characterized in that: The method of identifying a target damaged area of ​​the road based on the image information of the road and the current damage range of the road by using a damage location identification strategy includes: Establishing three-dimensional image data of the image information of the road, and identifying the damaged location area of ​​the image information of the road through a damaged image recognition network; Establishing a correspondence between the three-dimensional sensing image data and the three-dimensional image data, and identifying a sub-damage range corresponding to a damage position area within the current damage range based on the correspondence; Clustering the damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance between each damage map data in each damage map data group and the sub-damage range; Filter each target damage map data in the damage map data group corresponding to the average distance below the preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as the sub-damage area, and use all the sub-damage areas as the target damage areas of the road.

4. The method according to any one of claims 1 to 3, characterized in that: The method of identifying the damage type of the road based on the damage sensor map of the road through a damage identification network includes: Extracting damage feature data of each sub-damage area of ​​the damage sensing map, and inputting each damage feature data into a damage identification network to obtain a sub-damage type corresponding to each damage feature data; The sub-damage type corresponding to each damage feature data is used as the damage type of the road.

5. The method according to any one of claims 1 to 4, characterized in that: The calculating the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type includes: For each sub-damage region, based on the sample damage ranges of the degrees corresponding to the sub-damage types of the sub-damage region, the sample damage range to which the damage sensing data of the sub-damage region belongs is identified, and the degree of the sub-damage type corresponding to the sub-damage region is obtained; Based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, the damage A level division strategy is used to determine the damage level corresponding to the sub-damage area.

6. The method according to claim 1, characterized in that: After the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road, the method further includes: For each sub-damage region, determining a damage repair strategy for the sub-damage region based on the sub-damage type of the sub-damage region and the degree of the sub-damage type of the sub-damage region; According to the damage level of each sub-damage area from high to low, the maintenance order of each sub-damage area is arranged to obtain the repair sequence of each sub-damage area; and the damage repair strategy of each sub-damage area is filled into the repair sequence to obtain the repair task information of the target damage area, and the early warning information including the repair task information and the damage information of the target damage area is sent to the display module.

7. The method according to any one of claims 1 to 6, further comprising: In response to the user's sensor acquisition system update operation, acquiring acquisition system update information of each sensor, and sending the acquisition system update data information to the acquisition box; The acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: In response to the user's sensor acquisition task upload operation, generate an acquisition instruction for each sensor, and send the acquisition instruction to the acquisition box; The acquisition instruction includes an acquisition task for each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

9. A road damage detection system, including a cloud platform and a collection box: The collection box is communicatively connected with the cloud platform; The collection box is used to collect sensor data information of the road and image information of the road; The cloud platform is used to obtain multiple different sample damage ranges, and based on the sensor data information of the road, determine the current damage range of the road and the damage sensor map of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, identify the target damage area of ​​the road through a damage location recognition strategy, and based on the damage sensor map of the road, identify the damage type of the road through a damage identification network; obtain the damage sensor data of the target damage area, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, calculate the damage level corresponding to the damage sensor data of the target damage area; use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road.

10. A road damage detection device, comprising: an acquisition module, used to acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types; an identification module, configured to identify a target damaged area of ​​the road through a damage location identification strategy based on the image information of the road and the current damage range of the road, and to identify a damage type of the road through a damage identification network based on a damage sensor map of the road; a re-acquisition module, configured to acquire the damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type; The determination module is used to use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road.

11. A method for determining the location of a road sensor, characterized in that: The method comprises: Determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road; Determine a reference layout point from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout point, and The layout points corresponding to the road condition that do not belong to the reference layout point group are used as candidate layout points; For each candidate layout point, determine the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each reference layout point under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group according to each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point; Selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points; The target layout point is added to the reference layout point group to obtain a final target layout point group.

12. The method according to claim 11, characterized in that The step of determining the layout points corresponding to each road condition from a plurality of preset layout points of the target road includes: Constructing road models corresponding to target roads under various road conditions respectively; For each of the road conditions, the vehicle driving data corresponding to the target road under the road condition is obtained, and based on the vehicle driving data and the road model corresponding to the target road under the road condition, the third condition data corresponding to multiple preset layout points of the target road under the road condition are respectively determined, and based on each of the third condition data, the layout point corresponding to the road condition is determined from each of the preset layout points.

13. The method according to claim 12, characterized in that Determining the layout point corresponding to the road working condition from the preset layout points according to the third working condition data includes: Sorting the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue; Traverse the preset layout point queue, and when the third operating condition data corresponding to each preset layout point arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each preset layout point arranged before the current traversal position as the layout point corresponding to the road operating condition.

14. The method according to any one of claims 11 to 13, characterized in that: The step of determining a reference layout point from the layout points corresponding to each of the road working conditions comprises: According to the layout points corresponding to the road conditions, respectively determine the number of layout points corresponding to the road conditions; The layout points corresponding to the largest number of layout points among the numbers of layout points are taken as the reference layout point group.

15. The method according to any one of claims 11 to 14, characterized in that: The step of determining the correlation coefficient between the candidate layout points and the reference layout point group according to each of the first operating condition data and the second operating condition data includes: For any of the reference layout points, determining a correlation coefficient between the candidate layout point and the reference layout point according to the first operating condition data and the second operating condition data corresponding to the reference layout point; The correlation coefficient between the candidate layout point and the reference layout point group is determined according to the correlation coefficient corresponding to each of the reference layout points.

16. The method according to any one of claims 11 to 15, characterized in that: The selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points comprises: For any of the candidate layout points, when the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, the candidate layout point is used as a target layout point.

17. The method according to any one of claims 11 to 16, characterized in that: The method further comprises: According to the preset layout point arrangement strategy, a plurality of preset layout points are determined on the target road, and the plurality of preset layout points are evenly distributed on the target road.

18. A device for determining the location of a road sensor, characterized in that: The device comprises: A first determination module is used to determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road; The second determination module is used to determine a reference layout point from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout point, and use the layout points that do not belong to the reference layout point group among the layout points corresponding to each of the road conditions as candidate layout points. point; A third determination module is used to determine, for any of the candidate layout points, the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each of the reference layout points under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group according to each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point; A selection module, configured to select a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points; An adding module is used to add the target layout point to the reference layout point group to obtain a final target layout point group.

19. A road damage detection method, characterized in that: The method comprises: Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected; Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features; Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector; The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

20. The method according to claim 19, characterized in that The implantable sensor is a vibration acceleration sensor pre-installed inside the road to be detected, and the acquisition of the acceleration data set of the road to be detected within a preset detection period includes: Based on a preset sampling frequency and signal length, the vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of ​​the road to be detected within a preset detection period are collected; the vibration acceleration signals are generated by a road panel response generated by a vehicle passing through the road to be detected; Performing data preprocessing on each of the vibration acceleration signals, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time; Based on each of the acceleration vectors, an acceleration data set is obtained.

21. The method according to claim 19 or 20, characterized in that The step of obtaining an additional attribute feature data set of the road to be detected within a preset detection period includes: Acquire attribute characteristic data of the road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of the implanted sensor within the preset detection period; The attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

22. The method according to any one of claims 19 to 21, characterized in that: The hidden layer of the recursive neural network includes a plurality of hidden layer units, and the feature extraction of the acceleration data set according to the recursive neural network to obtain acceleration features includes: The acceleration data set is input into a pre-trained recursive neural network, and the acceleration vector in the acceleration data set is feature extracted through a plurality of hidden layer units included in a hidden layer of the recursive neural network to obtain acceleration features.

23. The method according to any one of claims 19 to 22, characterized in that: After inputting the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected, the method further includes: Based on the road damage result, determining a target road management strategy in a corresponding relationship between the road damage result and the road management strategy; Based on the target road management strategy, an instruction is given to perform maintenance management on the road to be inspected.

24. The method according to any one of claims 19 to 23, characterized in that: The method further comprises: Acquire training data samples; the training data samples include a training acceleration data set, an additional attribute feature data set, and a road damage category label; Inputting the training acceleration data set into a recursive neural network, performing feature extraction on the training acceleration data set to obtain acceleration features; Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector; Inputting the fused feature vector and the road damage category label into a classification prediction network, performing data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result; According to the classification prediction result, the road damage category label and the preset loss function, the loss result of the road damage detection model is determined until the loss result meets the preset model loss condition, and it is determined that the road damage detection model training is completed.

25. A road damage detection device, characterized in that: The device comprises: An acquisition module is used to acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected; A feature extraction module, used for extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features; A splicing module, used for performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector; The detection and discrimination module is used to input the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected.

26. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method of any one of claims 1-8, 11-17, 19 to 24 are implemented.

27. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method of any one of claims 1-8, 11-17, 19 to 24 are implemented.

28. A computer program product comprising executable instructions, characterized in that: When the executable instructions are executed by a processor, the steps of the method of any one of claims 1-8, 11-17, 19 to 24 are implemented.

Images