Pavement damage assessment method based on image detection

Abstract

The present invention relates to the field of image data processing technology, and provides a road damage assessment method based on image detection. The method constructs a multimodal perception network, analyzes the backbone network data transmission burden during the execution process, predicts the road data input of image detection, introduces an attention mechanism, and classifies the data set; adopts a trusted data exchange protocol of blockchain, experiences the road status through virtual reality or augmented reality technology, receives alarm notifications, and queries road section information through voice commands. Through dynamic heat maps and virtual reality or augmented reality technology, users can intuitively experience the road status, receive real-time alarms, and query road section information, greatly improving decision-making efficiency and user experience. The present invention not only significantly improves the accuracy and reliability of road damage detection, but also provides strong support for intelligent traffic management and urban planning, and promotes the development of infrastructure construction towards intelligence and refinement.

Description

Technical Field

[0001] The present invention relates to the field of image data technology, and in particular to a road damage assessment method based on image detection. Background Art

[0002] With the rapid progress of society and the acceleration of urbanization, the construction and maintenance of road infrastructure have become an important part of ensuring public safety and promoting economic development. The growing demand for transportation in modern society, the increase in vehicle traffic and the challenges brought by changes in the natural environment have made the problem of road damage more prominent. In order to cope with this situation and ensure the safety and durability of roads, timely and accurate road condition monitoring has become particularly critical. The large amount of image data collected by high-resolution cameras and other sensor devices, combined with advanced algorithm analysis, can achieve efficient identification and quantification of road damage. This technological advancement not only improves detection efficiency and reduces labor costs, but also helps to establish a more scientific and reasonable maintenance plan and promote the development of intelligent transportation systems.

[0003] At present, in the core application of image data processing, the existing road damage assessment methods still have some significant defects. Traditional image processing technology often has difficulty maintaining stable performance when facing complex road environments, such as lighting changes, shadow interference, different types of surface materials, etc., resulting in inaccurate detection results. In addition, the current methods have limited ability to capture subtle damage features, especially in low-contrast or complex texture backgrounds, which are prone to missed detection or false detection. At the same time, most existing solutions rely on preset thresholds or rules to classify damage and lack the ability to adaptively adjust. Moreover, the process of extracting useful information from image data is usually time-consuming and cannot meet the requirements of real-time monitoring, affecting the speed and efficiency of decision-making. Therefore, there is an urgent need for a new technical solution that can overcome the above defects to improve the accuracy and reliability of road damage assessment. Summary of the invention

[0004] Embodiments of the present invention provide a road damage assessment method based on image detection, a computer-readable medium, and an electronic device, which can at least to some extent solve the problem of achieving efficient, accurate, and transparent automatic detection and assessment of road damage by integrating advanced technologies such as multimodal perception networks, edge computing, deep learning, blockchain, and virtual / augmented reality to optimize road maintenance and management. Other features and advantages of the present invention will become apparent from the detailed description below, or may be learned in part through the practice of the present invention.

[0005] According to one aspect of the present invention, a road damage assessment method based on image detection is provided, including: constructing a multimodal perception network, executing the multimodal perception network using edge computing nodes, dispersing computing tasks through a distributed architecture, analyzing the backbone network data transmission burden during the execution process, introducing an automatic calibration enhancement model, and predicting road data input for image detection; identifying the predicted road data input based on a multi-scale feature extraction engine of a deep convolutional neural network and a recursive neural network, training the identified road data input using a multi-layer perceptron architecture, collecting data sets for the trained road data input, introducing an attention mechanism, and classifying the data sets; mapping the classified damage assessment results to a geographic location, combining historical data and trend analysis to enable a network platform to predict road damage conditions, using a trusted data exchange protocol of blockchain, experiencing the road status through virtual reality or augmented reality technology, receiving alarm notifications, and querying road section information through voice commands to complete image detection road damage assessment.

[0006] In the present invention, based on the above scheme, the construction of the multimodal perception network includes capturing the area of ​​road damage based on image detection sensor systems distributed at different locations, and adopting a modular design so that each sensor performs task allocation according to the area of ​​road damage;

[0007] The task allocation includes performing task allocation through a decentralized protocol of the blockchain. Each edge computing node makes a task request to other nodes or accepts a task entrustment according to its own computing power and current workload, and constructs a feedback loop mapping mechanism. In the feedback loop mapping mechanism, each edge computing node maintains real-time communication with the cloud control center to form a closed-loop feedback system.

[0008] When a node detects an abnormal situation, it sends an alarm to the cloud control center and receives instructions from the cloud control center;

[0009] When the number of received instructions continues to increase, it is detected that the road surface is seriously damaged. At this time, the cloud control center dispatches a maintenance team to the scene to deal with the problem.

[0010] In the present invention, based on the aforementioned scheme, the backbone network data transmission burden during the analysis execution process includes deploying an automatic calibration enhancement model on the edge computing node to dynamically adjust image parameters under different lighting conditions, weather conditions and traffic flow;

[0011] When a large number of instructions are received, time series analysis techniques are introduced to optimize the instructions, wherein the optimization includes evaluating the large number of instructions received;

[0012] If it is predicted that the rainy weather will cause blurred images, the image detection sensor system will increase the exposure time or adjust the focal length to provide a clear user interface image. At this time, the working status and image quality of each edge computing node can be viewed through a mobile device or web browser. The health indicators of each node, task progress, and predicted image quality score are displayed on the user interface image to help users make decisions.

[0013] When the image detection sensor system detects that the image quality has deteriorated, it will automatically trigger the intelligent early warning mechanism and send a notification to the staff. At this time, the user will receive the early warning information through voice or text message, and query the fault report and solution suggestions through the interactive interface. At this time, the user will manually adjust the image parameters through the interface to identify road damage under abnormal environmental requirements.

[0014] In the present invention, the identification of the predicted road surface data input includes establishing a spatiotemporal attention model based on the identification results of road surface damage under abnormal environmental requirements, the spatiotemporal attention model includes a time model and a space model, and allocating attention weights based on the two dimensions of the time model and the space model. The attention weights include using a bidirectional long short-term memory network structure to identify the identification results of road surface damage from both the front and back directions of the time model and the space model, and using historical weather information and future trends to build an evolution process model of road surface damage.

[0015] The construction of the evolution process model of the road surface damage includes monitoring the learning progress of the model through a visual interface based on the established feedback loop mapping mechanism, and the user can view the data changes of the evolution process of each road surface damage;

[0016] When it is found that the data has an error with the original data, the user manually marks the error range through the interface. The automatic anomaly generation system based on the adversarial network includes generating damage samples based on the existing annotated data, automatically adding and converting them into labels;

[0017] A hierarchical attention model is introduced, and processing is performed through the hierarchical layers of the hierarchical attention model, the layers include a first layer, a second layer and a third layer, the first layer includes data with an error of range A compared to the original data, the second layer includes data with an error of range B compared to the original data, the third layer includes data with an error of range C compared to the original data, a weakly supervised learning model is adopted, and the road surface data input after recognition is trained using a multi-layer perceptron architecture based on labeled data and unlabeled data.

[0018] In the present invention, based on the aforementioned scheme, the classification of the data set includes introducing an active learning user participation mechanism to optimize the stratified error range. The automatic anomaly generation system will recommend data samples to the user and ask the user to label or confirm. At this time, the user's feedback will be incorporated into the training process of the model, and the parameters of the classifier are manually adjusted through the interface to establish an interactive visualization interface. The user can view the detailed information of the classification results on this interface to improve the sample quantity, confidence distribution and misclassification of each classification result.

[0019] In the present invention, based on the above scheme, the trusted data exchange protocol using blockchain includes, after the sample quantity, confidence distribution and misclassification of each classification result are improved, the damage assessment result and the geographical location are temporally and spatially geocoded, the location information of the damage is recorded, the timestamp and environmental parameters are associated, and the historical damage situation at the same location is longitudinally compared, and a dynamic heat map is generated to display the damage distribution of each road section. The color and intensity of the heat map are adjusted according to the type, severity and development trend of the damage;

[0020] Users can view the latest damage heat map through mobile devices or web browsers. The map will automatically update according to the latest assessment results. At this time, users can select different time ranges and damage types for screening. Combined with the user's current location and destination, the image detection sensor system provides optimal path suggestions;

[0021] The dynamic heat map includes an analysis of the road damage type and severity based on basic color classification;

[0022] When cracks appear on the road surface, a blue color system is used, ranging from light blue to dark blue. The darker the color, the wider and longer the crack.

[0023] When there are potholes on the road, the red color system is used, ranging from pink to dark red. The darker the color, the deeper and larger the pothole.

[0024] When cracks appear on the road surface, the yellow color system is used, ranging from light yellow to golden yellow. The darker the color, the denser the crack network.

[0025] When delamination occurs on the road surface, a green color system is used, ranging from light green to dark green. The darker the color, the larger the delamination area.

[0026] In the present invention, based on the above-mentioned solution, the experience of the road surface status through virtual reality or augmented reality technology includes the user selecting different time periods through a time slider to view the damage status of a certain road section at different time points;

[0027] When the user selects the time period of the past month, the cracks in a certain area are observed to gradually change from light blue to dark blue;

[0028] For risk areas, the image detection sensor system superimposes a trend prediction line on the dynamic heat map to show the development trend in the future and complete the road damage assessment based on image detection.

[0029] According to one aspect of the present invention, a road damage assessment system based on image detection is provided, comprising:

[0030] The multimodal perception network construction module builds a multimodal perception network, uses edge computing nodes to execute the multimodal perception network, disperses computing tasks through a distributed architecture, analyzes the backbone network data transmission burden during the execution process, introduces an automatic calibration enhancement model, and predicts the road surface data input for image detection;

[0031] The multi-layer perceptron architecture training module, which is based on a multi-scale feature extraction engine of deep convolutional neural networks and recurrent neural networks, recognizes the predicted road surface data input, uses the multi-layer perceptron architecture to train the recognized road surface data input, collects data sets for the trained road surface data input, introduces an attention mechanism, and classifies the data sets;

[0032] The pavement damage assessment module based on image detection maps the classified damage assessment results to the geographic location, combines historical data and trend analysis to enable the network platform to predict the pavement damage situation, adopts the trusted data exchange protocol of blockchain, experiences the pavement status through virtual reality or augmented reality technology, receives alarm notifications, and queries the information of the road section through voice commands to complete the pavement damage assessment based on image detection.

[0033] In the present invention, based on the above scheme, the construction of the multimodal perception network includes capturing the area of ​​road damage based on image detection sensor systems distributed at different locations, and adopting a modular design so that each sensor performs task allocation according to the area of ​​road damage;

[0034] The task allocation includes performing task allocation through a decentralized protocol of the blockchain. Each edge computing node makes a task request to other nodes or accepts a task entrustment according to its own computing power and current workload, and constructs a feedback loop mapping mechanism. In the feedback loop mapping mechanism, each edge computing node maintains real-time communication with the cloud control center to form a closed-loop feedback system.

[0035] When a node detects an abnormal situation, it sends an alarm to the cloud control center and receives instructions from the cloud control center;

[0036] When the number of received instructions continues to increase, it is detected that the road surface is seriously damaged. At this time, the cloud control center dispatches a maintenance team to the scene to deal with the problem.

[0037] In the present invention, based on the aforementioned scheme, the backbone network data transmission burden during the analysis execution process includes deploying an automatic calibration enhancement model on the edge computing node to dynamically adjust image parameters under different lighting conditions, weather conditions and traffic flow;

[0038] When a large number of instructions are received, time series analysis techniques are introduced to optimize the instructions, wherein the optimization includes evaluating the large number of instructions received;

[0039] If it is predicted that the rainy weather will cause blurred images, the image detection sensor system will increase the exposure time or adjust the focal length to provide a clear user interface image. At this time, the working status and image quality of each edge computing node can be viewed through a mobile device or web browser. The health indicators of each node, task progress, and predicted image quality score are displayed on the user interface image to help users make decisions.

[0040] When the image detection sensor system detects that the image quality has deteriorated, it will automatically trigger the intelligent early warning mechanism and send a notification to the staff. At this time, the user will receive the early warning information through voice or text message, and query the fault report and solution suggestions through the interactive interface. At this time, the user will manually adjust the image parameters through the interface to identify road damage under abnormal environmental requirements.

[0041] In the present invention, based on the above scheme, the identification of the predicted road surface data input includes the identification results of road surface damage under abnormal environmental requirements, establishing a spatiotemporal attention model, the spatiotemporal attention model includes a time model and a space model, and allocating attention weights based on the two dimensions of the time model and the space model. The attention weights include using a bidirectional long short-term memory network structure to identify the identification results of road surface damage from both the front and back directions of the time model and the space model, and using the historical information and future trends of the weather to build an evolution process model of road surface damage.

[0042] The construction of the evolution process model of the road surface damage includes monitoring the learning progress of the model through a visual interface based on the established feedback loop mapping mechanism, and the user can view the data changes of the evolution process of each road surface damage;

[0043] When it is found that the data has an error with the original data, the user manually marks the error range through the interface. The automatic anomaly generation system based on the adversarial network includes generating damage samples based on the existing annotated data, automatically adding and converting them into labels;

[0044] A hierarchical attention model is introduced, and the hierarchical layering of the hierarchical attention model is used for processing, and the layering is performed from the global to the local of the label. The layering includes a first layer, a second layer and a third layer. The first layer includes data with an error of range A compared with the original data, the second layer includes data with an error of range B compared with the original data, and the third layer includes data with an error of range C compared with the original data. A weakly supervised learning model is adopted, and the road surface data input after recognition is trained using a multi-layer perceptron architecture based on labeled data and unlabeled data.

[0045] In the present invention, based on the aforementioned scheme, the classification of the data set includes introducing an active learning user participation mechanism to optimize the stratified error range. The automatic anomaly generation system will recommend data samples to the user and ask the user to label or confirm. At this time, the user's feedback will be incorporated into the training process of the model, and the parameters of the classifier are manually adjusted through the interface to establish an interactive visualization interface. The user can view the detailed information of the classification results on this interface to improve the sample quantity, confidence distribution and misclassification of each classification result.

[0046] In the present invention, based on the above scheme, the trusted data exchange protocol using blockchain includes, after the sample quantity, confidence distribution and misclassification of each classification result are improved, the damage assessment result and the geographical location are temporally and spatially geocoded, the location information of the damage is recorded, the timestamp and environmental parameters are associated, and the historical damage situation at the same location is longitudinally compared, and a dynamic heat map is generated to display the damage distribution of each road section. The color and intensity of the heat map are adjusted according to the type, severity and development trend of the damage;

[0047] Users can view the latest damage heat map through mobile devices or web browsers. The map will automatically update according to the latest assessment results. At this time, users can select different time ranges and damage types for screening. Combined with the user's current location and destination, the image detection sensor system provides optimal path suggestions;

[0048] The dynamic heat map includes an analysis of the road damage type and severity based on basic color classification;

[0049] When cracks appear on the road surface, a blue color system is used, ranging from light blue to dark blue. The darker the color, the wider and longer the crack.

[0050] When there are potholes on the road, the red color system is used, ranging from pink to dark red. The darker the color, the deeper and larger the pothole.

[0051] When cracks appear on the road surface, the yellow color system is used, ranging from light yellow to golden yellow. The darker the color, the denser the crack network.

[0052] When delamination occurs on the road surface, a green color system is used, ranging from light green to dark green. The darker the color, the larger the delamination area.

[0053] In the present invention, based on the above-mentioned solution, the experience of the road surface status through virtual reality or augmented reality technology includes the user selecting different time periods through a time slider to view the damage status of a certain road section at different time points;

[0054] When the user selects the time period of the past month, the cracks in a certain area are observed to gradually change from light blue to dark blue;

[0055] For risk areas, the image detection sensor system superimposes a trend prediction line on the dynamic heat map to show the development trend in the future and complete the road damage assessment based on image detection.

[0056] According to one aspect of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for assessing road damage based on image detection as described in the above embodiments is implemented.

[0057] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the road damage assessment method based on image detection as described in the above-mentioned embodiments.

[0058] According to one aspect of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the road damage assessment method based on image detection provided in the above-mentioned various optional implementations.

[0059] In the technical solution of the present invention, the present invention realizes efficient, accurate and automatic detection and evaluation of road damage by integrating advanced technologies such as multimodal perception network, edge computing, deep learning, blockchain and virtual / augmented reality. First, the multimodal perception network combined with distributed edge computing greatly reduces the data transmission burden of the backbone network and improves the real-time response speed. Secondly, the multi-scale feature extraction engine based on deep convolutional neural network and recurrent neural network can capture subtle to macro damage features and improve classification accuracy through attention mechanism and weak supervision learning. Furthermore, the trusted data exchange protocol of blockchain is adopted to ensure the security and transparency of data and enhance cross-departmental collaboration. Finally, through dynamic heat map and virtual reality or augmented reality technology, users can intuitively experience the current status of the road surface, receive real-time alarms and query road section information, which greatly improves decision-making efficiency and user experience. Overall, the present invention not only significantly improves the accuracy and reliability of road damage detection, but also provides strong support for intelligent traffic management and urban planning, and promotes the development of infrastructure construction towards intelligence and refinement.

[0060] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0062] Figure 1 The flowchart of a road damage assessment method based on image detection in one embodiment of the present invention is schematically shown.

[0063] Figure 2 The following is a schematic diagram of a road damage assessment system based on image detection in one embodiment of the present invention. DETAILED DESCRIPTION

[0064] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0065] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0066] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0067] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0068] The implementation details of the technical solution of the present invention are described in detail below:

[0069] Figure 1 FIG. 1 is a flow chart showing a method for road damage assessment based on image detection according to an embodiment of the present invention. Figure 1 As shown, the detailed introduction of the road damage assessment method based on image detection is as follows:

[0070] S1, builds a multimodal perception network, uses edge computing nodes to execute the multimodal perception network, disperses computing tasks through a distributed architecture, analyzes the backbone network data transmission burden during the execution process, introduces an automatic calibration enhancement model, and predicts the road surface data input for image detection.

[0071] In one embodiment of the present invention, constructing a multimodal perception network includes capturing the area of ​​road damage based on an image detection sensor system distributed at different locations, and using a modular design so that each sensor performs a task allocation according to the area of ​​road damage;

[0072] Task allocation includes task allocation through the decentralized protocol of blockchain. Each edge computing node makes task requests to other nodes or accepts task delegations based on its own computing power and current workload, and builds a feedback loop mapping mechanism. In the feedback loop mapping mechanism, each edge computing node maintains real-time communication with the cloud control center to form a closed-loop feedback system.

[0073] When a node detects an abnormal situation, it sends an alarm to the cloud control center and receives instructions from the cloud control center;

[0074] When the number of received instructions continues to increase, it is detected that the road surface is seriously damaged. At this time, the cloud control center dispatches a maintenance team to the scene to deal with the problem.

[0075] S1.1, the backbone network data transmission burden during the analysis execution process includes deploying an automatic calibration enhancement model on the edge computing node to dynamically adjust image parameters under different lighting conditions, weather conditions, and traffic flow;

[0076] When a large number of instructions are received, time series analysis technology is introduced to complete the optimization of the instructions, and the optimization includes evaluating the large number of instructions received;

[0077] If it is predicted that the rainy weather will cause blurred images, the image detection sensor system will increase the exposure time or adjust the focal length to provide a clear user interface image. At this time, the working status and image quality of each edge computing node can be viewed through a mobile device or web browser. The health indicators of each node, task progress, and predicted image quality score are displayed on the user interface image to help users make decisions.

[0078] When the image detection sensor system detects that the image quality has deteriorated, it will automatically trigger the intelligent early warning mechanism and send a notification to the staff. At this time, the user will receive the early warning information through voice or text message, and query the fault report and solution suggestions through the interactive interface. At this time, the user will manually adjust the image parameters through the interface to identify road damage under abnormal environmental requirements.

[0079] S2, a multi-scale feature extraction engine based on deep convolutional neural networks and recurrent neural networks, identifies the predicted road surface data input, uses a multi-layer perceptron architecture to train the identified road surface data input, collects data sets for the trained road surface data input, introduces an attention mechanism, and classifies the data sets.

[0080] S2.1, identify the predicted road surface data input, including the identification results of road surface damage under abnormal environmental requirements, establish a spatiotemporal attention model, the spatiotemporal attention model includes a time model and a space model, and allocates attention weights based on the two dimensions of the time model and the space model. The attention weights include using a bidirectional long short-term memory network structure from the front and back directions of the time model and the space model, and simultaneously identify the identification results of road surface damage, and use the historical information and future trends of the weather to build a road surface damage evolution process model;

[0081] Building the evolution process model of pavement damage includes monitoring the learning progress of the model through a visual interface based on the established feedback loop mapping mechanism, and users can view the data changes of the evolution process of each pavement damage;

[0082] When it is found that the data has an error with the original data, the user manually marks the error range through the interface. The automatic anomaly generation system based on the adversarial network includes generating damage samples based on the existing annotated data, automatically adding and converting them into labels;

[0083] A hierarchical attention model is introduced, and the data is processed through the hierarchical stratification of the hierarchical attention model, from the global to the local level of the label. The stratification includes the first layer, the second layer and the third layer. The first layer includes the data with an error in range A compared with the original data, the second layer includes the data with an error in range B compared with the original data, and the third layer includes the data with an error in range C compared with the original data. A weakly supervised learning model is adopted, based on labeled data and unlabeled data, and the road data input after recognition is trained using a multi-layer perceptron architecture.

[0084] Optional, first-level global error range screening, users can see a global view button on the system interface, click it to enter the first-level analysis interface, in this interface, the system will display all data points with A range errors between the data and the original data, for example, assuming the A range error is ±0.5 cm, users can see all crack lengths, pothole depths, etc. within this range. The system provides a slider that allows users to adjust the error range and view the data changes in different error ranges in real time. Users can also select specific areas or sections to further narrow the screening range.

[0085] The second level is medium error range screening. Based on the first level, users can choose to enter the medium error view, which is the second level analysis interface. This level focuses on B range errors, such as errors between ±0.5 and ±1.5 cm. Users can quickly switch to this view by clicking the medium error view button. In this interface, the system highlights all data points that meet the B range error and provides detailed statistics and visual charts. Users can further filter out specific types of damage, such as cracks, potholes, etc., in order to analyze these medium errors in more depth.

[0086] The third level is local error range screening. The third level focuses on C range errors, that is, serious errors greater than ±1.5 cm. Users can enter this view by clicking the local error view button. In this interface, the system will highlight all data points with serious errors and provide detailed damage reports and recommended maintenance measures. For example, the crack length of a certain section of road has expanded from 4.2 cm to 5.8 cm, which exceeds the set error range. The system will automatically mark and generate maintenance suggestions. Users can manually adjust certain parameters such as damage type, location information, etc. through the interactive interface to further optimize the results. In addition, the system will provide historical data comparison to help users understand the development trend of damage in the area.

[0087] Furthermore, when identifying the predicted road surface data input, the system established a spatiotemporal attention model, which combines the temporal model and the spatial model. The bidirectional long short-term memory network structure simultaneously allocates attention weights from the front and back directions of the temporal model and the spatial model. Specifically, the temporal model uses historical weather data and future trend forecasts to analyze the development of damage in different time periods; the spatial model focuses on the damage characteristics in a specific area to ensure that the model can capture subtle changes. For example, in the past year, the number of potholes on a certain section of road increased by 30% during the rainy season, but only increased by 5% on sunny days, indicating that weather conditions have a significant impact on the evolution of damage. Through this two-dimensional attention mechanism, the system can more accurately identify and classify road damage.

[0088] Furthermore, when constructing the evolution process model of road damage, the system monitors the learning progress of the model in real time through a visual interface based on a feedback loop mapping mechanism. Users can view the evolution process of each road damage, including its transition from mild to severe. For example, a specific crack expands from 1 cm to 5 cm in three months. The system will record this change and generate a detailed evolution report. When it is found that the data is different from the original data, the user can manually mark this error range through the interface. For example, if the actual length of a crack is 4.2 cm, but the system predicts it to be 3.8 cm, the user can mark this 0.4 cm error and include it in the training data set. The automatic anomaly generation system based on the adversarial network will generate new damage samples based on the existing labeled data, automatically add labels, and further enrich the training data.

[0089] Preferably, in order to improve the classification accuracy, the system introduces a hierarchical attention mechanism, which divides the data into three levels: the first layer processes minor errors within the error range of ±0.5 cm, the second layer processes medium errors between ±0.5 and ±1.5 cm, and the third layer processes severe errors with an error range greater than ±1.5 cm. Through this hierarchical design, the system can gradually refine the focus of attention to ensure accurate identification of different types of damage. For example, in a certain test, the classification accuracy of the first layer reached 95%, the second layer was 88%, and the third layer was 75%. Using a weakly supervised learning model, the system uses labeled data and unlabeled data for training, which further improves the generalization ability of the model. Finally, the multi-layer perceptron architecture is used to train the road data input after recognition, so that the system can maintain high accuracy and stability in a complex and changing environment. The specific data and its corresponding description in a changing environment are shown in Table 1 below:

[0090] Table 1 Specific data and their corresponding descriptions in a changing environment

[0091]

[0092] Table 1 shows the specific data mentioned in the article and their corresponding descriptions to help you understand each data point more clearly.

[0093] S2.1, classifying the data set includes introducing an active learning user participation mechanism to optimize the stratified error range. The automatic anomaly generation system will recommend data samples to the user and ask the user to label or confirm. At this time, the user's feedback will be incorporated into the model training process. The classifier parameters are manually adjusted through the interface to establish an interactive visualization interface. The user can view the detailed information of the classification results on this interface to improve the sample quantity, confidence distribution and misclassification of each classification result.

[0094] S3, by mapping the classified damage assessment results to the geographic location, combined with historical data and trend analysis, enables the network platform to predict road damage conditions, adopt the blockchain's trusted data exchange protocol, experience the current road conditions through virtual reality or augmented reality technology, receive alarm notifications, query road section information through voice commands, and complete image detection road damage assessment.

[0095] Among them, the trusted data exchange protocol using blockchain includes the spatiotemporal geocoding of the damage assessment results and the geographic location after the sample quantity, confidence distribution and misclassification of each classification result are improved, the location information of the damage is recorded, the timestamp and environmental parameters are associated, and the historical damage situation at the same location is compared longitudinally, and a dynamic heat map is generated to show the damage distribution of each road section. The color and intensity of the heat map are adjusted according to the type, severity and development trend of the damage;

[0096] Users can view the latest damage heat map through mobile devices or web browsers. The map will automatically update according to the latest assessment results. At this time, users can select different time ranges and damage types for screening. Combined with the user's current location and destination, the image detection sensor system provides optimal path suggestions;

[0097] Dynamic heat maps include analysis of road damage type and severity based on basic color classification;

[0098] When cracks appear on the road surface, a blue color system is used, ranging from light blue to dark blue. The darker the color, the wider and longer the crack.

[0099] When there are potholes on the road, the red color system is used, ranging from pink to dark red. The darker the color, the deeper and larger the pothole.

[0100] When cracks appear on the road surface, the yellow color system is used, ranging from light yellow to golden yellow. The darker the color, the denser the crack network.

[0101] When delamination occurs on the road surface, a green color system is used, ranging from light green to dark green. The darker the color, the larger the delamination area.

[0102] Preferably, when filtering by time range, a time slider tool is found on the interface, allowing selection of different time ranges for filtering, for example, the user can select data from the past week, month, or year to observe changes in damage on a specific road section;

[0103] When a time period of the past month is selected, the system will display all damage records within that time period and dynamically show how the cracks gradually change from light blue to dark blue, allowing users to intuitively see the development trend of road damage over time.

[0104] When filtering damage types, in the interface, select the damage type of interest (such as cracks, potholes, fissures, delamination) through the drop-down menu and adjust the displayed color range.

[0105] When the user selects a crack, the system will only display the damaged areas associated with the crack, with the color changing from light blue to dark blue to indicate the severity of the damage.

[0106] Path planning based on the current location and destination. After entering the current location and destination, the system will provide optimal path suggestions based on the current damage heat map data, for example:

[0107] When a primary route has severe cracks or potholes, the system will recommend an alternate route that avoids these high-risk areas.

[0108] The system will also mark all identified high-risk sections on the map and highlight them with different colors to help users quickly understand which sections require special attention.

[0109] Real-time alerts and notifications: When the system detects that the damage on a certain road section has deteriorated sharply, it will immediately send a real-time alert notification to the user. For example, the user may receive a text message or in-app notification informing him that a new severe pothole has appeared on a road section he frequently travels.

[0110] Users can also set custom alert conditions, such as receiving notifications when damage in a specific area reaches a certain severity, so they can prepare for response in advance.

[0111] S3.1, experiencing the current state of the road through virtual reality or augmented reality technology includes the user selecting different time periods through a time slider to view the damage of a certain road section at different time points;

[0112] When the user selects the time period of the past month, the cracks in a certain area are observed to gradually change from light blue to dark blue;

[0113] For risk areas, the image detection sensor system superimposes a trend prediction line on the dynamic heat map to show the development trend in the future and complete the road damage assessment based on image detection.

[0114] Furthermore, when designing virtual reality or augmented reality technology to experience the current state of the road surface, it is appropriate to choose a time period of the past month. This time period can not only fully demonstrate the development process of road damage, but also achieve a good balance between data update frequency and system load. Studies have shown that most minor cracks will expand significantly within a month, with an average expansion rate of about 0.5 to 1 cm per month. For example, in sections with high traffic volume, users can select the time period of the past month through the time slider and clearly observe the process of cracks in a certain area gradually changing from light blue (minor cracks) to dark blue (serious cracks). In addition, for risk areas, the image detection sensor system will superimpose a trend prediction line on the dynamic heat map to show the development trend in the next month. This not only helps users understand the current damage status, but also predicts future evolution, providing a scientific basis for maintenance decisions. Therefore, a one-month time period can capture enough damage changes without causing information overload due to too long data cycles, ensuring the efficiency of the system and the optimization of user experience. The specific time data and its corresponding description are shown in Table 2 below:

[0115]

[0116] Table 2 shows the specific data mentioned in the article and their corresponding descriptions, which helps to more clearly understand each data point and its application in the system.

[0117] In the technical solution of the present invention, the present invention realizes efficient, accurate and automatic detection and evaluation of road damage by integrating advanced technologies such as multimodal perception network, edge computing, deep learning, blockchain and virtual / augmented reality. First, the multimodal perception network combined with distributed edge computing greatly reduces the data transmission burden of the backbone network and improves the real-time response speed. Secondly, the multi-scale feature extraction engine based on deep convolutional neural network and recurrent neural network can capture subtle to macro damage features and improve classification accuracy through attention mechanism and weak supervision learning. Furthermore, the trusted data exchange protocol of blockchain is adopted to ensure the security and transparency of data and enhance cross-departmental collaboration. Finally, through dynamic heat map and virtual reality or augmented reality technology, users can intuitively experience the current status of the road surface, receive real-time alarms and query road section information, which greatly improves decision-making efficiency and user experience. Overall, the present invention not only significantly improves the accuracy and reliability of road damage detection, but also provides strong support for intelligent traffic management and urban planning, and promotes the development of infrastructure construction towards intelligence and refinement.

[0118] The following describes an embodiment of the device of the present invention, which can be used to perform the hysteroscopic-based treatment method in the above-mentioned embodiment of the present invention. It can be understood that the device can be a computer program (including program code) running in a computer device, for example, the device is an application software; the device can be used to perform the corresponding steps in the method provided in the embodiment of the present invention. For details not disclosed in the embodiment of the device of the present invention, please refer to the above-mentioned embodiment of the hysteroscopic-based treatment method of the present invention.

[0119] Reference Figure 2 As shown, a road damage assessment system based on image detection according to an embodiment of the present invention includes:

[0120] A multimodal perception network construction module 310, which constructs a multimodal perception network, uses edge computing nodes to execute the multimodal perception network, disperses computing tasks through a distributed architecture, analyzes the backbone network data transmission burden during the execution process, introduces an automatic calibration enhancement model, and predicts road surface data input for image detection;

[0121] A multi-layer perceptron architecture training module 320, which is based on a multi-scale feature extraction engine of a deep convolutional neural network and a recurrent neural network, identifies the predicted road surface data input, trains the identified road surface data input using a multi-layer perceptron architecture, collects data sets for the trained road surface data input, introduces an attention mechanism, and classifies the data sets;

[0122] The pavement damage assessment module 330 based on image detection maps the classified damage assessment results to the geographic location, combines historical data and trend analysis, enables the network platform to predict the pavement damage situation, adopts the trusted data exchange protocol of blockchain, experiences the pavement status through virtual reality or augmented reality technology, receives alarm notifications, queries the information of the road section through voice commands, and completes the pavement damage assessment based on image detection.

[0123] In the present invention, based on the aforementioned scheme, constructing a multimodal perception network includes capturing the area of ​​road damage based on an image detection sensor system distributed at different locations, and adopting a modular design so that each sensor performs task allocation according to the area of ​​road damage;

[0124] Task allocation includes task allocation through the decentralized protocol of blockchain. Each edge computing node makes task requests to other nodes or accepts task delegations based on its own computing power and current workload, and builds a feedback loop mapping mechanism. In the feedback loop mapping mechanism, each edge computing node maintains real-time communication with the cloud control center to form a closed-loop feedback system.

[0125] When a node detects an abnormal situation, it sends an alarm to the cloud control center and receives instructions from the cloud control center;

[0126] When the number of received instructions continues to increase, it is detected that the road surface is seriously damaged. At this time, the cloud control center dispatches a maintenance team to the scene to deal with the problem.

[0127] In the present invention, based on the aforementioned scheme, the backbone network data transmission burden during the analysis execution process includes deploying an automatic calibration enhancement model on the edge computing node to dynamically adjust image parameters under different lighting conditions, weather conditions and traffic flow;

[0128] When a large number of instructions are received, time series analysis technology is introduced to complete the optimization of the instructions, and the optimization includes evaluating the large number of instructions received;

[0129] If it is predicted that the rainy weather will cause blurred images, the image detection sensor system will increase the exposure time or adjust the focal length to provide a clear user interface image. At this time, the working status and image quality of each edge computing node can be viewed through a mobile device or web browser. The health indicators of each node, task progress, and predicted image quality score are displayed on the user interface image to help users make decisions.

[0130] When the image detection sensor system detects that the image quality has deteriorated, it will automatically trigger the intelligent early warning mechanism and send a notification to the staff. At this time, the user will receive the early warning information through voice or text message, and query the fault report and solution suggestions through the interactive interface. At this time, the user will manually adjust the image parameters through the interface to identify road damage under abnormal environmental requirements.

[0131] In the present invention, based on the above scheme, the predicted road surface data input is identified, including the identification results of road surface damage under abnormal environmental requirements, and a spatiotemporal attention model is established. The spatiotemporal attention model includes a time model and a space model, and attention weights are allocated based on the two dimensions of the time model and the space model. The attention weights include using a bidirectional long short-term memory network structure to identify the identification results of road surface damage from both the front and back directions of the time model and the space model, and using the historical information and future trends of the weather to build an evolution process model of road surface damage.

[0132] Building the evolution process model of pavement damage includes monitoring the learning progress of the model through a visual interface based on the established feedback loop mapping mechanism, and users can view the data changes of the evolution process of each pavement damage;

[0133] When it is found that the data has an error with the original data, the user manually marks the error range through the interface. The automatic anomaly generation system based on the adversarial network generates damage samples based on the existing annotated data, automatically adds and converts them into labels;

[0134] A hierarchical attention model is introduced, and the data is processed through the hierarchical stratification of the hierarchical attention model, from the global to the local level of the label. The stratification includes the first layer, the second layer and the third layer. The first layer includes the data with an error in range A compared with the original data, the second layer includes the data with an error in range B compared with the original data, and the third layer includes the data with an error in range C compared with the original data. A weakly supervised learning model is adopted, based on labeled data and unlabeled data, and the road data input after recognition is trained using a multi-layer perceptron architecture.

[0135] In the present invention, based on the aforementioned scheme, the classification of the data set includes introducing an active learning user participation mechanism to optimize the stratified error range. The automatic anomaly generation system will recommend data samples to the user and ask the user to label or confirm. At this time, the user's feedback will be incorporated into the training process of the model. The parameters of the classifier are manually adjusted through the interface to establish an interactive visualization interface. The user can view the detailed information of the classification results on this interface to improve the sample quantity, confidence distribution and misclassification of each classification result.

[0136] In the present invention, based on the aforementioned scheme, the trusted data exchange protocol using blockchain includes that after the sample quantity, confidence distribution and misclassification of each classification result are improved, the damage assessment result and the geographical location are temporally and spatially geocoded, the location information of the damage is recorded, the timestamp and environmental parameters are associated, and the historical damage situation at the same location is longitudinally compared, and a dynamic heat map is generated to display the damage distribution of each road section. The color and intensity of the heat map are adjusted according to the type, severity and development trend of the damage;

[0137] Users can view the latest damage heat map through mobile devices or web browsers. The map will automatically update according to the latest assessment results. At this time, users can select different time ranges and damage types for screening. Combined with the user's current location and destination, the image detection sensor system provides optimal path suggestions;

[0138] Dynamic heat maps include analysis of road damage type and severity based on basic color classification;

[0139] When cracks appear on the road surface, a blue color system is used, ranging from light blue to dark blue. The darker the color, the wider and longer the crack.

[0140] When there are potholes on the road, the red color system is used, ranging from pink to dark red. The darker the color, the deeper and larger the pothole.

[0141] When cracks appear on the road surface, the yellow color system is used, ranging from light yellow to golden yellow. The darker the color, the denser the crack network.

[0142] When delamination occurs on the road surface, a green color system is used, ranging from light green to dark green. The darker the color, the larger the delamination area.

[0143] In the present invention, based on the aforementioned solution, experiencing the current state of the road surface through virtual reality or augmented reality technology includes the user selecting different time periods through a time slider to view the damage of a certain road section at different time points;

[0144] When the user selects the time period of the past month, the cracks in a certain area are observed to gradually change from light blue to dark blue;

[0145] For risk areas, the image detection sensor system superimposes a trend prediction line on the dynamic heat map to show the development trend in the future and complete the road damage assessment based on image detection.

[0146] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0149] According to one aspect of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0150] As another aspect, the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the hemodialysis patient care management method in the above embodiment.

[0151] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0152] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0153] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the embodiments disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary technical means in the art that are not disclosed in the present invention.

[0154] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A road damage assessment method based on image detection, characterized in that: include: Build a multimodal perception network, use edge computing nodes to execute the multimodal perception network, disperse computing tasks through a distributed architecture, analyze the backbone network data transmission burden during execution, introduce an automatic calibration enhancement model, and predict the road surface data input for image detection; A multi-scale feature extraction engine based on deep convolutional neural networks and recurrent neural networks is used to identify the predicted road surface data input, and a multi-layer perceptron architecture is used to train the identified road surface data input. The trained road surface data input is then collected and the attention mechanism is introduced to classify the data set. By mapping the classified damage assessment results to the geographic location, combining historical data and trend analysis, using the network platform to predict the road damage, using the blockchain's trusted data exchange protocol, experiencing the road status through virtual reality or augmented reality technology, receiving alarm notifications, and querying the road section information through voice commands, the image-based road damage assessment is completed; The identification of the predicted road surface data input includes the identification results of road surface damage based on abnormal environmental requirements, establishing a spatiotemporal attention model, the spatiotemporal attention model includes a time model and a space model, and allocating attention weights based on the two dimensions of the time model and the space model. The attention weights include using a bidirectional long short-term memory network structure to identify the identification results of road surface damage from both the front and back directions of the time model and the space model, and using historical weather information and future trends to build an evolution process model of road surface damage; The construction of the evolution process model of the road surface damage includes monitoring the learning progress of the model through a visual interface based on the established feedback loop mapping mechanism, and the user can view the data changes of the evolution process of each road surface damage; When it is found that the data has an error with the original data, the user manually marks the error range through the interface. The automatic anomaly generation system based on the adversarial network includes generating damage samples based on the existing annotated data, automatically adding and converting them into labels; A hierarchical attention model is introduced, and the hierarchical layering of the hierarchical attention model is used for processing, and the layering is performed from the global to the local of the label. The layering includes a first layer, a second layer and a third layer. The first layer includes data with an error of range A compared with the original data, the second layer includes data with an error of range B compared with the original data, and the third layer includes data with an error of range C compared with the original data. A weakly supervised learning model is adopted, based on labeled data and unlabeled data, and a multi-layer perceptron architecture is used to train the road surface data input after recognition.

2. The road damage assessment method based on image detection according to claim 1 is characterized in that: The multimodal perception network is constructed by capturing the area of ​​road damage based on image detection sensor systems distributed at different locations, and adopting a modular design so that each sensor is assigned a task according to the area of ​​road damage; The task allocation includes performing task allocation through a decentralized protocol of the blockchain. Each edge computing node makes a task request to other nodes or accepts a task entrustment according to its own computing power and current workload, and constructs a feedback loop mapping mechanism. In the feedback loop mapping mechanism, each edge computing node maintains real-time communication with the cloud control center to form a closed-loop feedback system. When a node detects an abnormal situation, it sends an alarm to the cloud control center and receives instructions from the cloud control center; When the number of received instructions continues to increase, it is detected that the road surface is seriously damaged. At this time, the cloud control center dispatches a maintenance team to the scene to deal with the problem.

3. The road damage assessment method based on image detection according to claim 1, characterized in that: The backbone network data transmission burden during the execution of the analysis includes deploying an automatic calibration enhancement model on the edge computing node to dynamically adjust image parameters under different lighting conditions, weather conditions and traffic flow; When a large number of instructions are received, time series analysis techniques are introduced to optimize the instructions, wherein the optimization includes evaluating the large number of instructions received; If it is predicted that the rainy weather will cause blurred images, the image detection sensor system will increase the exposure time or adjust the focal length to provide a clear user interface image. At this time, the working status and image quality of each edge computing node can be viewed through a mobile device or web browser. The health indicators of each node, task progress, and predicted image quality score are displayed on the user interface image to help users make decisions. When the image detection sensor system detects that the image quality has deteriorated, it will automatically trigger the intelligent early warning mechanism and send a notification to the staff. At this time, the user will receive the early warning information through voice or text message, and query the fault report and solution suggestions through the interactive interface. At this time, the user will manually adjust the image parameters through the interface to identify road damage under abnormal environmental requirements.

4. The road damage assessment method based on image detection according to claim 1, characterized in that: The classification of the data set includes introducing an active learning user participation mechanism to optimize the stratified error range. The automatic anomaly generation system will recommend data samples to the user and ask the user to label or confirm. At this time, the user's feedback will be incorporated into the training process of the model. The parameters of the classifier are manually adjusted through the interface to establish an interactive visualization interface. The user can view the detailed information of the classification results on this interface to improve the sample quantity, confidence distribution and misclassification of each classification result.

5. The road damage assessment method based on image detection according to claim 4 is characterized in that: The trusted data exchange protocol using blockchain includes, after the sample quantity, confidence distribution and misclassification of each classification result are improved, the damage assessment results are temporally and spatially geocoded with the geographic location, the location information of the damage is recorded, the timestamp and environmental parameters are associated, and the historical damage situation at the same location is longitudinally compared, and a dynamic heat map is generated to show the damage distribution of each road section. The color and intensity of the heat map are adjusted according to the type, severity and development trend of the damage; Users can view the latest damage heat map through mobile devices or web browsers. The map will automatically update according to the latest assessment results. At this time, users can select different time ranges and damage types for screening. Combined with the user's current location and destination, the image detection sensor system provides optimal path suggestions; The dynamic heat map includes an analysis of the road damage type and severity based on basic color classification; When cracks appear on the road surface, a blue color system is used, ranging from light blue to dark blue. The darker the color, the wider and longer the crack. When there are potholes on the road, the red color system is used, ranging from pink to dark red. The darker the color, the deeper and larger the pothole. When cracks appear on the road surface, the yellow color system is used, ranging from light yellow to golden yellow. The darker the color, the denser the crack network. When delamination occurs on the road surface, a green color system is used, ranging from light green to dark green. The darker the color, the larger the delamination area.

6. The method for road damage assessment based on image detection according to claim 1, characterized in that: The experience of the road condition through virtual reality or augmented reality technology includes the user selecting different time periods through a time slider to view the damage of a certain road section at different time points; When the user selects the time period of the past month, the cracks in a certain area are observed to gradually change from light blue to dark blue; For risk areas, the image detection sensor system superimposes a trend prediction line on the dynamic heat map to show the development trend in the future and complete the road damage assessment based on image detection.

7. The road damage assessment system based on image detection is characterized by: include: The multimodal perception network construction module builds a multimodal perception network, uses edge computing nodes to execute the multimodal perception network, disperses computing tasks through a distributed architecture, analyzes the backbone network data transmission burden during the execution process, introduces an automatic calibration enhancement model, and predicts the road surface data input for image detection; The multi-layer perceptron architecture training module, which is based on a multi-scale feature extraction engine of deep convolutional neural networks and recurrent neural networks, recognizes the predicted road surface data input, uses the multi-layer perceptron architecture to train the recognized road surface data input, collects data sets for the trained road surface data input, introduces an attention mechanism, and classifies the data sets; The pavement damage assessment module based on image detection maps the classified damage assessment results to the geographic location, combines historical data and trend analysis, enables the network platform to predict the pavement damage situation, adopts the blockchain's trusted data exchange protocol, experiences the pavement status through virtual reality or augmented reality technology, receives alarm notifications, and queries the information of the road section through voice commands to complete the pavement damage assessment based on image detection; The identification of the predicted road surface data input includes the identification results of road surface damage based on abnormal environmental requirements, establishing a spatiotemporal attention model, the spatiotemporal attention model includes a time model and a space model, and allocating attention weights based on the two dimensions of the time model and the space model. The attention weights include using a bidirectional long short-term memory network structure to identify the identification results of road surface damage from both the front and back directions of the time model and the space model, and using historical weather information and future trends to build an evolution process model of road surface damage; The construction of the evolution process model of the road surface damage includes monitoring the learning progress of the model through a visual interface based on the established feedback loop mapping mechanism, and the user can view the data changes of the evolution process of each road surface damage; When it is found that the data has an error with the original data, the user manually marks the error range through the interface. The automatic anomaly generation system based on the adversarial network includes generating damage samples based on the existing annotated data, automatically adding and converting them into labels; A hierarchical attention model is introduced, and the hierarchical layering of the hierarchical attention model is used for processing, and the layering is performed from the global to the local of the label. The layering includes a first layer, a second layer and a third layer. The first layer includes data with an error of range A compared with the original data, the second layer includes data with an error of range B compared with the original data, and the third layer includes data with an error of range C compared with the original data. A weakly supervised learning model is adopted, based on labeled data and unlabeled data, and a multi-layer perceptron architecture is used to train the road surface data input after recognition.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the road damage assessment method based on image detection as claimed in any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the road damage assessment method based on image detection as described in any one of claims 1 to 6.

Images