Cloud-edge collaborative road inspection system, method, equipment and medium

Abstract

The embodiment of the invention provides a cloud-edge collaborative road inspection system, method, equipment and medium, and the system comprises an edge calculation box which is used for collecting road images and position information, and obtaining first road damage information through the reasoning of a first reasoning model; detecting working state parameters of the box; and uploading the road image, the position information, the first road damage information and the working state parameters to a cloud server, receiving a returned working state adjustment instruction and a road damage reasoning evaluation index, and adjusting a first reasoning model according to the returned working state adjustment instruction and the returned road damage reasoning evaluation index. The cloud server is used for reasoning the information uploaded by the box through a second reasoning model to obtain second road damage information and obtaining historical road damage information from a knowledge base; evaluating the first road damage information according to the road damage reasoning evaluation index to obtain an evaluation result; further obtaining a working state adjusting instruction; and issuing the instruction and the index to an edge computing box. The box algorithm and the working state performance can be improved through the cloud model, so that the two are dynamically balanced to adapt to complex road condition changes.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and smart transportation technology, and in particular to a cloud-edge collaborative road inspection system, a cloud-edge collaborative road inspection method, an electronic device and a computer-readable storage medium. Background Art

[0002] The road inspection system based on edge computing boxes is a system that uses edge computing technology to achieve road inspection and monitoring. By deploying edge computing boxes near the road, real-time data collection, processing and analysis are achieved, improving the real-time, low latency, high efficiency and reliability of the system. The system can be applied to multiple scenarios such as road monitoring, traffic management, environmental monitoring, emergency response, etc., providing powerful decision support and management capabilities.

[0003] At present, the road inspection system based on edge computing boxes has difficulty in balancing model performance and box performance, and the set algorithm configuration parameters and box working parameters cannot automatically adapt to complex road conditions. In addition, due to the limited computing power of edge computing boxes, it is difficult to further improve the detection accuracy of built-in lightweight models. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a cloud-edge collaborative road inspection system, a cloud-edge collaborative road inspection method, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problems, an embodiment of the present invention discloses a cloud-edge collaborative road inspection system, including: an edge computing box, a cloud server;

[0006] The edge computing box is used to collect a road image and corresponding position information, and perform road damage inference on the road image through a first inference model to obtain first road damage information; detect a working state parameter of the edge computing box; upload the road image, the position information, the first road damage information and the working state parameter to the cloud server; receive a working state adjustment instruction and a road damage inference evaluation index sent by the cloud server, adjust the working state parameter according to the working state adjustment instruction, and adjust the first inference model according to the road damage inference evaluation index;

[0007] The cloud server is used to perform road damage inference on the road image uploaded by the edge computing box through a second inference model to obtain second road damage information; obtain historical road damage information from a preset knowledge base according to the location information; evaluate the first road damage information according to a road damage inference evaluation index based on the second road damage information and the historical road damage information queried from the knowledge base to obtain an evaluation result; obtain a working state adjustment instruction according to the working state parameter of the edge computing box, the road damage inference evaluation index and the evaluation result; and send the working state adjustment instruction and the road damage inference evaluation index to the edge computing box.

[0008] Optionally, the cloud server is used to calculate the optimization decision of the edge computing box according to the working status parameters of the edge computing box, the road hazard reasoning evaluation indicators and evaluation results through a reinforcement learning algorithm, and generate a working status adjustment instruction according to the optimization decision.

[0009] Optionally, the cloud server is further used to update the historical road hazard information corresponding to the location information in the knowledge base according to the location information and the second road hazard information of the road image corresponding to the location information.

[0010] Optionally, the edge computing box is arranged in a road inspection vehicle.

[0011] In a second aspect, an embodiment of the present invention further discloses a cloud-edge collaborative road inspection method, including:

[0012] Collecting a road image, and performing road damage inference on the road image through a first inference model to obtain first road damage information; detecting a working state parameter of an edge computing box; uploading the road image, the first road damage information, and the working state parameter to a cloud server; receiving a working state adjustment instruction and a road damage inference evaluation index sent by the cloud server, adjusting the working state parameter according to the working state adjustment instruction, and adjusting the first inference model according to the road damage inference evaluation index;

[0013] According to the road image uploaded by the edge computing box, the second inference model performs road damage inference to obtain second road damage information; historical road damage information is obtained from a preset knowledge base; based on the second road damage information and the historical road damage information queried from the knowledge base, the first road damage information is evaluated according to the road damage inference evaluation index to obtain an evaluation result; according to the working state parameters of the edge computing box, the road damage inference evaluation index and the evaluation result, a working state adjustment instruction is obtained; and the working state adjustment instruction and the road damage inference evaluation index are sent to the edge computing box.

[0014] Optionally, obtaining a working state adjustment instruction according to the working state parameter of the edge computing box, the road damage reasoning evaluation index and the evaluation result includes:

[0015] According to the working state parameters of the edge computing box, the road hazard reasoning evaluation index and the evaluation results, the optimization decision of the edge computing box is calculated through the reinforcement learning algorithm, and a working state adjustment instruction is generated according to the optimization decision.

[0016] Optionally, the acquiring of historical road hazard information from a preset knowledge base further includes:

[0017] The historical road hazard information corresponding to the location information in the knowledge base is updated according to the location information and the second road hazard information of the road image corresponding to the location information.

[0018] Optionally, the edge computing box is arranged in a road inspection vehicle.

[0019] In the third aspect, an embodiment of the present invention further discloses an electronic device, comprising: a processor, a memory, and a computer program stored on the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the cloud-edge collaborative road inspection method as described above are implemented.

[0020] In a fourth aspect, an embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cloud-edge collaborative road inspection method as described above are implemented.

[0021] The embodiments of the present invention include the following advantages:

[0022] In an embodiment of the present invention, a cloud-edge collaborative road inspection system includes an edge computing box and a cloud server; the edge computing box is used to collect road images and corresponding location information, and perform road damage inference through a first inference model to obtain first road damage information; detect the working state parameters of the box; upload the road image, location information, first road damage information and working state parameters to the cloud server; receive the working state adjustment instruction and road damage inference evaluation index sent by the cloud server and adjust the first inference model accordingly; the cloud server is used to perform road damage inference on the road image uploaded by the box through a second inference model to obtain second road damage information, and obtain historical road damage information from a preset knowledge base according to the location information; accordingly, the first road damage information is evaluated according to the road damage inference evaluation index to obtain an evaluation result; the working state adjustment instruction is obtained according to the box working state parameters, road damage inference evaluation index and evaluation result; the working state adjustment instruction and road damage inference evaluation index are sent to the edge computing box. On the one hand, it can dynamically adapt to complex road conditions and achieve a balance between algorithm performance and box performance, and on the other hand, the road inspection algorithm performance is improved through the cloud large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a structural diagram of a cloud-edge collaborative road inspection system provided by an embodiment of the present invention;

[0024] Figure 2 It is a structural diagram of another cloud-edge collaborative road inspection system provided by an embodiment of the present invention;

[0025] Figure 3 It is a flowchart of the steps of setting up a cloud-edge collaborative road inspection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] One of the core concepts of the embodiments of the present invention is to design a cloud-edge collaborative system architecture, upload the built-in model inference results of the road inspection edge computing box and the box working parameter results to the cloud, and then evaluate the small model results of the edge computing box through the cloud large model and the built-in knowledge base, and provide the optimal working parameters of the box and the small model through the intelligent decision-making module, and send the decision instructions to the edge computing box. On the one hand, it can dynamically adapt to complex road conditions and achieve a balance between algorithm performance and box performance. On the other hand, the road inspection algorithm performance is improved through the cloud large model.

[0028] Reference Figure 1 , shows a structural diagram of a cloud-edge collaborative road inspection system provided by an embodiment of the present invention, and the system may specifically include:

[0029] The edge computing box 10 is used to collect a road image and corresponding position information, and perform road damage inference on the road image through a first inference model to obtain first road damage information; detect a working state parameter of the edge computing box; upload the road image, the position information, the first road damage information and the working state parameter to the cloud server 11; receive a working state adjustment instruction and a road damage inference evaluation index sent by the cloud server 11, adjust the working state parameter according to the working state adjustment instruction, and adjust the first inference model according to the road damage inference evaluation index;

[0030] The road inspection system based on edge computing boxes is a system that uses edge computing technology to achieve road inspection and monitoring. Edge computing boxes are computing devices deployed at the edge of the network that have data processing, storage and analysis capabilities, and can perform real-time processing and decision-making close to the data source. Edge computing boxes are commonly used in the fields of Internet of Things, industrial automation, intelligent transportation, smart cities, etc. to reduce data transmission delays, improve system response speed and reduce network bandwidth requirements. The road inspection system is a system used to monitor and manage road conditions. Through various sensors, cameras, data acquisition equipment and analysis tools, it monitors the operating status of the road in real time, and promptly detects and handles problems such as road damage, traffic congestion, and environmental pollution.

[0031] The edge computing box has the following main functions: data acquisition, through the sensor interface, real-time collection of various data, such as images, videos, sensor data, etc.; data processing, local data processing and analysis, such as image recognition, data analysis, fault detection, etc.; data storage, the collected data and processing results are stored in local or remote storage devices for subsequent analysis and query; data transmission, through the communication unit, the processing results and important data are transmitted to the central server or cloud for further processing and analysis; remote control, allowing users to remotely control the edge computing box through the central server or user interface, such as adjusting sensor parameters, restarting the device, etc.

[0032] The road inspection system based on edge computing box usually includes the following main components: edge computing box, a computing device deployed near the road, responsible for real-time data collection, processing and analysis; sensors, various sensors installed on the road, such as cameras, radars, lidars, environmental sensors, etc., used to collect data such as road conditions, traffic flow, environmental parameters, etc.; communication network, a communication network connecting the edge computing box and the central server, such as 4G / 5G, Wi-Fi, etc., for data transmission and remote control; central server, a computing device deployed in the data center, responsible for data storage, analysis, management and decision support.

[0033] The road inspection system based on the edge computing box can be applied to the following scenarios: road monitoring, real-time monitoring of road conditions, detection of road damage, traffic congestion, environmental pollution and other problems, and timely taking measures; traffic management, through real-time traffic flow statistics and predictions, optimize traffic signal control and improve road traffic efficiency; environmental monitoring, real-time monitoring of environmental parameters around the road, such as air quality, noise level, etc., to provide environmental management support; emergency response, rapid response and handling of emergencies such as traffic accidents and natural disasters to reduce losses.

[0034] However, the computing resources of edge computing boxes are usually limited, making it difficult to handle complex computing tasks and large-scale data; the storage capacity of edge computing boxes is usually limited, making it difficult to store large amounts of data and processing results; the communication bandwidth of edge computing boxes is usually limited, making it difficult to transmit large amounts of data and processing results; edge computing boxes are usually deployed in various complex environments and are easily affected by factors such as weather, light, and electromagnetic interference, affecting the accuracy of data collection and processing.

[0035] In some embodiments of the present invention, the edge computing box 10 and the cloud server 11 are combined for analysis. The inspection system mainly inspects the road damage. Road damage refers to the damage and defects caused by various factors during the long-term use of the road. Road damage not only affects the service life and safety of the road, but may also cause traffic accidents and traffic congestion.

[0036] Road damage usually includes the following types: cracks, such as longitudinal cracks, transverse cracks, mesh cracks, etc., which are usually caused by temperature changes, material aging, load and other factors; potholes, depressions or potholes on the road surface, which are usually caused by material damage, water erosion, load and other factors; rutting, deep grooves or rutting on the road surface, which are usually caused by long-term vehicle driving, material wear, poor drainage and other factors; looseness, looseness or peeling on the road surface, which are usually caused by material aging, water erosion, load and other factors; settlement, sinking or settlement on the road surface, which are usually caused by unstable foundation, water erosion, load and other factors; water accumulation, water accumulation on the road surface, which are usually caused by poor drainage, material damage, design defects and other factors. These road damages not only affect the flatness of the road, but may also cause unstable vehicle driving and traffic accidents.

[0037] Through reasonable design and optimization, the detection, repair and prevention of road hazards can improve the service life and safety of roads and ensure the safety and smoothness of roads. The inspection system of the embodiment of the present invention first performs real-time data collection through the edge computing box 10, including taking images of the road through a camera, and collecting corresponding location information of images and GPS data through the built-in GPS (Global Positioning System) module of the box, and performs a road hazard reasoning through the first reasoning model built into the edge computing box 10, such as the yolov5 (You Only Look Once version 5) model, to predict the road hazard situation, that is, to obtain the first road hazard information, and timely process and analyze the data to reduce data transmission delay.

[0038] YOLOv5 is a single-stage target detection algorithm that can complete the target detection task through a single forward propagation. The core idea of ​​YOLOv5 is to transform the target detection task into a regression problem, and to achieve target detection by predicting the target's bounding box and category probability. YOLOv5 uses an efficient network structure and training method, has high detection speed and accuracy, and is suitable for real-time target detection tasks. YOLOv5 is widely used in intelligent transportation, security monitoring, industrial automation, medical imaging and other fields, with the advantages of fast speed, high accuracy, easy to use, and strong flexibility. Through reasonable design and optimization, YOLOv5 can improve the efficiency and accuracy of target detection and adapt to different application scenarios.

[0039] At the same time, the temperature, CPU occupancy, memory occupancy, camera frame rate and other working parameters of the edge computing box 10 are monitored. The working parameters of the edge computing box also determine the accuracy of data collection and affect the subsequent algorithm reasoning results. Therefore, it is necessary to upload the collected road image and the corresponding location information, the inferred first road damage information and the relevant working status parameters of the edge computing box 10 to the cloud server 11. Through the intelligent analysis of the cloud server 11, the corresponding working status adjustment instructions and road damage reasoning evaluation indicators are fed back to the edge computing box 10, and the working status parameters are adjusted according to the working status adjustment instructions, and the built-in first reasoning model is adjusted according to the road damage reasoning evaluation indicators.

[0040] Road damage reasoning evaluation indicators are used to evaluate the performance and accuracy of road inspection systems in detecting and identifying road damage, traffic congestion, environmental pollution and other problems. These indicators can help developers and operation and maintenance personnel understand the system's detection capabilities, false positive rates and overall performance, so as to optimize and improve them. Common road hazard reasoning evaluation indicators include: accuracy, which refers to the ratio between the number of road hazards correctly detected by the system and the total number of road hazards detected by the system; recall, which refers to the ratio between the number of road hazards correctly detected by the system and the total number of road hazards that actually exist. The higher the recall, the fewer road hazards missed by the system; precision and accuracy refer to the ratio between the number of road hazards correctly detected by the system and the total number of road hazards detected by the system; F1 score, which is the harmonic mean of precision and recall, and comprehensively considers precision and recall to evaluate the overall performance of the system; mAP, mean average precision (mAP, Mean Average Precision) is a comprehensive indicator used to evaluate the performance of target detection algorithms. mAP combines recall and precision, and considers performance under different detection thresholds; there are also false positive rate, missed positive rate, etc. Road damage reasoning evaluation indicators are used to evaluate the performance and accuracy of road inspection systems in detecting and identifying problems such as road damage, traffic congestion, and environmental pollution. By analyzing these indicators, we can understand the system's detection capability, false detection rate, and overall performance, and thus optimize and improve it.

[0041] The cloud server 11 is used to perform road damage inference on the road image uploaded by the edge computing box through a second inference model to obtain second road damage information; obtain historical road damage information from a preset knowledge base according to the location information; evaluate the first road damage information according to a road damage inference evaluation index based on the second road damage information and the historical road damage information queried from the knowledge base to obtain an evaluation result; obtain a working state adjustment instruction according to the working state parameter of the edge computing box, the road damage inference evaluation index and the evaluation result; and send the working state adjustment instruction and the road damage inference evaluation index to the edge computing box.

[0042] Cloud servers play an important role in the cloud-edge collaborative system, responsible for data storage, processing, analysis and management. For example, data storage, the data collected by the edge and the processing results are stored in the cloud for subsequent analysis and query; data processing, large-scale data processing and analysis in the cloud, such as data cleaning, feature extraction, model training, etc.; data analysis, using machine learning and deep learning algorithms to analyze the data stored in the cloud and provide decision support; model training, training machine learning and deep learning models in the cloud to improve the accuracy and robustness of the model; model deployment, deploying the trained model to the edge to achieve real-time data processing and decision-making; remote control, allowing users to remotely control edge devices through cloud servers, such as adjusting sensor parameters, restarting devices, etc.

[0043] Cloud servers have powerful computing capabilities and can handle large-scale data and complex computing tasks, making them suitable for data analysis and model training. They have large-capacity storage devices that can store large amounts of data and processing results to facilitate subsequent analysis and query. They have high-bandwidth network interfaces that can quickly transmit large amounts of data and processing results, making them suitable for real-time data processing and decision-making. They support a variety of operating systems and application software, and users can flexibly adjust system configuration and operation according to their needs to adapt to different application scenarios. They have strict security measures, such as data encryption, access control, and security audits, to ensure data security and privacy protection.

[0044] In some embodiments of the present invention, road damage inference is performed again based on the road image uploaded from the edge computing box 10 using a large cloud model, i.e., a second inference model. The obtained inference result, i.e., the second road damage information, can be used as a benchmark reference for calibrating and evaluating the small model built into the cloud edge computing box 10. However, for comprehensive evaluation, it is also necessary to refer to the historical road damage information of the corresponding road position in the preset knowledge base. For example, there was a large road damage in a certain position A before, but the road damage is gone in the inferred first road damage information. It may be necessary to pay attention to whether the inference situation of this position A is incorrect.

[0045] In combination with the second road damage information inferred by the cloud server 11 and the historical road damage information queried from the knowledge base, the first road damage information is evaluated according to the road damage inference evaluation index in the previous article to obtain the evaluation results. The cloud server 11 then generates a working state adjustment instruction acting on the edge computing box 10 based on these results, and sends the working state adjustment instruction to the edge computing box 10 to optimize the working state parameters. For example, if the image clarity is not high, the camera frame rate can be adjusted, or if such a high-definition image is not needed, the camera frame rate can be reduced to reduce the pressure on the edge computing box; and at the same time, the road damage inference evaluation index is sent to adjust the small model algorithm of the edge computing box 10 for correction.

[0046] Reference Figure 2 , shows a structural diagram of another cloud-edge collaborative road inspection system provided by an embodiment of the present invention, and the system may specifically include:

[0047] Edge computing box 20, including:

[0048] Data acquisition module 201, used to acquire road images and corresponding location information;

[0049] A first algorithm reasoning module 202, configured to perform road hazard reasoning on the road image using a first reasoning model to obtain first road hazard information;

[0050] A detection module 203, used to detect the working status parameters of the edge computing box;

[0051] A first communication module 204, used to upload the road image, the location information, the first road hazard information and the working status parameter to the cloud server 21;

[0052] In some embodiments of the present invention, the edge computing box 20 is disposed in a road inspection vehicle.

[0053] A road inspection vehicle is a vehicle specially used for road inspection and maintenance. It is equipped with various sensors, cameras, data acquisition equipment and analysis tools to monitor the operation status of roads in real time, timely discover and deal with problems such as road damage, traffic congestion, and environmental pollution. It uses various sensors and cameras to collect data such as road conditions, traffic flow, and environmental parameters in real time. First, the collected data is transmitted to the data processing center through data acquisition equipment and communication networks; data processing and analysis are performed in the data processing center, such as image recognition, traffic flow statistics, and environmental parameter analysis; machine learning and deep learning algorithms are used to realize image recognition, traffic flow prediction, fault detection and other functions; real-time data collection and transmission are realized through various sensors and communication networks; big data technology is used to store, analyze and mine the collected data to provide decision support; the remote control center is used to store, analyze and manage data, providing powerful computing and storage capabilities. In the embodiment of the present invention, the edge computing box 20 is the device that realizes its main technology, which collects video streams through cameras, collects the latitude and longitude running trajectories of vehicle-mounted equipment through GPS units, and then infers through models, and transmits data to cloud servers 21, etc.

[0054] The cloud server 21 includes:

[0055] A second algorithm reasoning module 210 is used to perform road damage reasoning on the road image uploaded by the edge computing box through a second reasoning model to obtain second road damage information;

[0056] A knowledge base module 211 is used to query historical road hazard information according to the location information;

[0057] In some embodiments of the present invention, the knowledge base module 211 is further used to update the historical road hazard information corresponding to the location information in the knowledge base according to the location information and the second road hazard information of the road image corresponding to the location information.

[0058] The knowledge base module 211 uses a structured database and GPS information as a unit to query and update historical road damage information. For example, the location information and road damage are stored correspondingly. Now the edge computing box 20 has detected the road damage at position A, but in the original knowledge base, there may be no road damage at position A. The knowledge base can be updated to provide a more accurate reference for subsequent calibration and evaluation.

[0059] An algorithm evaluation module 212 is used to evaluate the first road hazard information according to the road hazard reasoning evaluation index based on the second road hazard information and the historical road hazard information queried from the knowledge base to obtain an evaluation result;

[0060] An intelligent decision-making module 213, configured to obtain a working state adjustment instruction according to the working state parameters of the edge computing box, the road hazard reasoning evaluation index and the evaluation result;

[0061] In some embodiments of the present invention, the intelligent decision-making module 213 is used to calculate the optimization decision of the edge computing box according to the working state parameters of the edge computing box, the road hazard reasoning evaluation indicators and the evaluation results through a reinforcement learning algorithm, and generate a working state adjustment instruction according to the optimization decision.

[0062] Reinforcement Learning (RL) is a machine learning method that learns to take the best action strategy in a specific environment to maximize the cumulative reward through the interaction between the agent and the environment. Reinforcement learning algorithms usually include core concepts such as state, action, reward, strategy, and value function. They are based on the agent optimizing its own behavior strategy through continuous trial and error and learning in the interaction with the environment to maximize the cumulative reward. Reinforcement learning is widely used in game AI, robot control, autonomous driving, recommendation systems, financial transactions and other fields, and has the advantages of adaptability, generalization, robustness, flexibility and so on.

[0063] The intelligent decision-making module 213 calculates the decisions that optimize various aspects of the edge computing box 20 based on the working status parameters of the edge computing box 20, the road hazard reasoning evaluation indicators and the evaluation results, through the reinforcement learning algorithm, and converts these optimization decisions into instructions that can adjust the working status, such as the video acquisition frame rate, the box small model detection threshold, the road hazard tracking threshold, etc.

[0064] The second communication module 214 is used to send the working state adjustment instruction and the road damage reasoning evaluation index to the edge computing box.

[0065] The second communication module 214 sends the working state adjustment instruction, cloud reasoning results and road damage reasoning evaluation indicators to the edge computing box to optimize the box-side algorithm reasoning results and the box working state. However, the evaluation results are often large in data. If the network speed is not fast enough or the memory is insufficient, you can choose not to send the evaluation results, and only send the working state adjustment instruction and road damage reasoning evaluation indicators.

[0066] The embodiment of the present invention provides a cloud-edge collaborative road inspection system, including an edge computing box and a cloud server; the edge computing box is used to collect road images and corresponding location information, and perform road damage inference through a first inference model to obtain first road damage information; detect the working state parameters of the box; upload the road image, location information, first road damage information and working state parameters to the cloud server; receive the working state adjustment instruction and road damage inference evaluation index sent by the cloud server and adjust the first inference model accordingly; the cloud server is used to perform road damage inference on the road image uploaded by the box through a second inference model to obtain second road damage information, and obtain historical road damage information from a preset knowledge base according to the location information; accordingly, the first road damage information is evaluated according to the road damage inference evaluation index to obtain an evaluation result; the working state adjustment instruction is obtained according to the box working state parameters, road damage inference evaluation index and evaluation result; the working state adjustment instruction and road damage inference evaluation index are sent to the edge computing box. On the one hand, it can dynamically adapt to complex road conditions and achieve a balance between algorithm performance and box performance, and on the other hand, the road inspection algorithm performance is improved through the cloud large model.

[0067] Reference Figure 3 , shows a flow chart of the steps of a cloud-edge collaborative road inspection system provided by an embodiment of the present invention. The method may specifically include the following steps:

[0068] Step 301, collecting a road image, and performing road damage inference on the road image through a first inference model to obtain first road damage information; detecting a working state parameter of an edge computing box; uploading the road image, the first road damage information, and the working state parameter to a cloud server; receiving a working state adjustment instruction and a road damage inference evaluation index sent by the cloud server, adjusting the working state parameter according to the working state adjustment instruction, and adjusting the first inference model according to the road damage inference evaluation index;

[0069] In some embodiments of the present invention, step 502 may include:

[0070] The edge computing box is arranged in a road inspection vehicle.

[0071] Step 302: Based on the road image uploaded by the edge computing box, the second inference model performs road damage inference to obtain second road damage information; historical road damage information is obtained from a preset knowledge base; based on the second road damage information and the historical road damage information queried from the knowledge base, the first road damage information is evaluated according to the road damage inference evaluation index to obtain an evaluation result; based on the working state parameters of the edge computing box, the road damage inference evaluation index and the evaluation result, a working state adjustment instruction is obtained; and the working state adjustment instruction and the road damage inference evaluation index are sent to the edge computing box.

[0072] In some embodiments of the present invention, step 302 may include:

[0073] The historical road hazard information corresponding to the location information in the knowledge base is updated according to the location information and the second road hazard information of the road image corresponding to the location information.

[0074] In some embodiments of the present invention, step 302 may further include:

[0075] According to the working state parameters of the edge computing box, the road hazard reasoning evaluation index and the evaluation results, the optimization decision of the edge computing box is calculated through the reinforcement learning algorithm, and a working state adjustment instruction is generated according to the optimization decision.

[0076] In one embodiment, the specific steps are as follows: first, the edge computing box is initialized, the video stream is collected through the camera, and the longitude and latitude running trajectory of the vehicle-mounted equipment is collected through the GPS unit; the video stream is cut, and the road damage data is inferred through the edge computing box small model, including the road damage type, road damage pixel position, road damage length, and road damage area; through the edge computing box monitoring module, the box working monitoring data is collected, including CPU, GPU utilization, working temperature, memory occupancy, camera acquisition frame rate / resolution, etc.; based on the edge computing box monitoring data collected in the previous step, combined with the edge computing box small model algorithm working hyperparameters (including road damage detection threshold, road damage tracking threshold, etc.), the data is uploaded to the cloud server intelligent analysis feedback system through the edge computing box communication module; after the cloud server intelligent analysis feedback system receives the box side communication data, it re-infers the uploaded image based on the cloud road damage detection large model to obtain the cloud large model inference data, including The information includes the road hazard type, road hazard pixel location, road hazard length, and road hazard area; then, according to the GPS longitude and latitude reported by the edge computing box, historical road hazard data is queried based on the built-in database of the knowledge base module, and the data in the knowledge base is updated based on the new cloud-based large model inference data; the large model inference results in the knowledge base are used as the evaluation benchmark to calculate the box small model road hazard inference evaluation indicators, including road hazard recall rate, road hazard accuracy, mAP, etc., to check whether there are any false alarms or omissions; based on the working status data uploaded on the edge computing box side and combined with the edge computing box algorithm evaluation results, the box side optimization decision is obtained through the reinforcement learning algorithm in the intelligent decision-making module, and converted into the working status adjustment instructions on the edge computing box side, including video acquisition frame rate, box small model detection threshold, road hazard tracking threshold, etc.; the cloud server's instruction adjustment data, the road hazard inference evaluation indicators of the algorithm comparison results, and the evaluation results are sent to the edge computing box side through the communication module for status adjustment; the above steps can be repeated for repeated calibration.

[0077] The embodiment of the present invention provides a cloud-edge collaborative road inspection method. After uploading the built-in model inference results of the road inspection edge box and the box working parameter results to the cloud, the small model results of the edge box are evaluated and corrected based on the joint analysis of the cloud large model and the built-in knowledge base system, and the optimal working parameters of the box and the small model are provided through the intelligent decision-making module, and the decision instructions are sent to the edge box to realize the cloud-edge collaborative road inspection full life cycle implementation process. The method proposed in this patent can be combined with complex real-time road conditions, and the working parameters and algorithm judgment parameters of the edge device can be adaptively adjusted to achieve a balance between the algorithm performance and the box calculation complexity; it solves the problem of low algorithm accuracy of the edge device itself, and the calculation accuracy of the edge device can be significantly improved through the cloud large model and the external knowledge base system.

[0078] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0079] An embodiment of the present invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned cloud-edge collaborative road inspection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0080] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned cloud-edge collaborative road inspection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0081] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0082] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0083] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0086] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0087] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0088] The above is a detailed introduction to a cloud-edge collaborative road inspection system and a cloud-edge collaborative road inspection method provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A cloud-edge collaborative road inspection system, characterized in that: include: Edge computing box, cloud server; The edge computing box is used to collect road images and corresponding location information, and perform road hazard inference on the road images through a first inference model to obtain first road hazard information; Detecting the working state parameters of the edge computing box; uploading the road image, the location information, the first road hazard information and the working state parameters to the cloud server; receiving the working state adjustment instruction and the road hazard reasoning evaluation index sent by the cloud server, adjusting the working state parameters according to the working state adjustment instruction, and adjusting the first reasoning model according to the road hazard reasoning evaluation index; The cloud server is used to perform road damage inference on the road image uploaded by the edge computing box through a second inference model to obtain second road damage information; and obtain historical road damage information from a preset knowledge base according to the location information; Based on the second road hazard information and the historical road hazard information retrieved from the knowledge base, the first road hazard information is evaluated according to the road hazard reasoning evaluation index to obtain an evaluation result; According to the working state parameters of the edge computing box, the road damage reasoning evaluation index and the evaluation result, a working state adjustment instruction is obtained; and the working state adjustment instruction and the road damage reasoning evaluation index are sent to the edge computing box.

2. The system according to claim 1, characterized in that The cloud server is used to calculate the optimization decision of the edge computing box according to the working state parameters of the edge computing box, the road hazard reasoning evaluation indicators and the evaluation results through a reinforcement learning algorithm, and generate a working state adjustment instruction according to the optimization decision.

3. The system according to claim 1, characterized in that The cloud server is further configured to update the historical road hazard information corresponding to the location information in the knowledge base according to the location information and the second road hazard information of the road image corresponding to the location information.

4. The system according to claim 1, characterized in that The edge computing box is arranged in a road inspection vehicle.

5. A cloud-edge collaborative road inspection method, characterized in that: include: Collecting a road image, and performing road hazard inference on the road image using a first inference model to obtain first road hazard information; Detecting the working state parameters of the edge computing box; uploading the road image, the first road damage information and the working state parameters to the cloud server; receiving the working state adjustment instruction and the road damage reasoning evaluation index sent by the cloud server, adjusting the working state parameters according to the working state adjustment instruction, and adjusting the first reasoning model according to the road damage reasoning evaluation index; According to the road image uploaded by the edge computing box, the second inference model performs road damage inference to obtain second road damage information; Obtain historical road damage information from a preset knowledge base; Based on the second road hazard information and the historical road hazard information retrieved from the knowledge base, the first road hazard information is evaluated according to the road hazard reasoning evaluation index to obtain an evaluation result; According to the working state parameters of the edge computing box, the road damage reasoning evaluation index and the evaluation result, a working state adjustment instruction is obtained; and the working state adjustment instruction and the road damage reasoning evaluation index are sent to the edge computing box.

6. The method according to claim 1, characterized in that The obtaining of a working state adjustment instruction according to the working state parameter of the edge computing box, the road hazard reasoning evaluation index and the evaluation result includes: According to the working state parameters of the edge computing box, the road hazard reasoning evaluation index and the evaluation result, the optimization decision of the edge computing box is calculated through the reinforcement learning algorithm, and the working state adjustment instruction is generated according to the optimization decision.

7. The method according to claim 1, characterized in that The step of obtaining historical road hazard information from a preset knowledge base further includes: The historical road hazard information corresponding to the location information in the knowledge base is updated according to the location information and the second road hazard information of the road image corresponding to the location information.

8. The method according to claim 1, characterized in that: The edge computing box is arranged in a road inspection vehicle.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the cloud-edge collaborative road inspection method as described in any one of claims 5-8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the cloud-edge collaborative road inspection method as described in any one of claims 5-8 are implemented.

Images