A mobile robot inspection method, electronic device, storage medium, and product

Abstract

The present application provides a mobile robot inspection method, an electronic device, a storage medium, and a product, including: obtaining grid map data, where the grid map data includes annotation information of each grid; the annotation information includes any one of a grid to be inspected, a grid that does not need to be inspected, and a grid with a low-probability disease; removing the grids that do not need to be inspected, and performing path planning based on the grids to be inspected and the grids with a low-probability disease to obtain the current inspection path; when moving to a grid with a low-probability disease, determining whether the number of times passing through the grid with a low-probability disease reaches a detection threshold, and when the detection threshold is reached, collecting environmental image data; when moving to a grid to be inspected, directly collecting environmental image data; and determining whether there is a disease in the corresponding grid according to the collected environmental image data and updating the annotation information. This is used to improve the efficiency and flexibility of road disease inspection.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent road inspection, and more specifically, to a mobile robot inspection method, electronic equipment, storage medium and product. Background Art

[0002] With the rapid development of transportation and the continuous increase in traffic volume, the frequency and load of road use have increased significantly, and the resulting road surface disease problems have become increasingly prominent, including cracks, potholes, and settlement. Road surface diseases not only affect the service life of roads, but also threaten driving safety and increase the risk of traffic accidents. Therefore, timely and accurate detection of road surface diseases is crucial to ensuring road quality and traffic safety.

[0003] In the existing technology, manual inspections rely on manpower and are inefficient, while intelligent inspections mostly use fixed routes or pre-set paths to inspect each checkpoint or area one by one. Inspecting each area one by one regardless of whether there is a disease will waste a lot of time and lead to low inspection efficiency, especially in large areas or areas with uneven disease distribution. It is impossible to dynamically adjust the key inspection areas according to the actual distribution of diseases, and the flexibility is low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a mobile robot inspection method, electronic equipment, storage medium and computer program product to improve the efficiency and flexibility of road defect inspection.

[0005] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a mobile robot inspection method, comprising:

[0007] Obtain grid map data, the grid map data including annotation information of each grid; the annotation information includes any one of a must-check grid, a grid that does not need to be checked, and a grid with low probability of disease; eliminate the grid that does not need to be checked, perform path planning based on the must-check grid and the grid with low probability of disease, and obtain a current inspection path; when traveling to the grid with low probability of disease, determine whether the number of times passing through the grid with low probability of disease reaches a detection threshold, and when the detection threshold is reached, collect environmental image data; when traveling to the must-check grid, directly collect environmental image data; determine whether the corresponding grid has a disease based on the collected environmental image data, and update the annotation information.

[0008] In one embodiment, it further includes: judging in real time whether the annotation information of each grid has changed. If it has not changed, continue to execute the patrol task according to the current patrol path; if it has changed, take the current grid as the starting point, re - execute path planning to obtain an optimized path, and execute the patrol task according to the optimized path until the current patrol task is completed.

[0009] In one embodiment, the annotation information includes the reward value of each grid. The reward value of the mandatory inspection grid is greater than the reward value of the low - probability disease grid, and the reward value of the end - point grid is greater than the reward value of the mandatory inspection grid. The path planning based on the mandatory inspection grid and the low - probability disease grid to obtain the current patrol path includes: numbering each grid and setting a reward value for each grid; initializing the value function corresponding to each grid according to the reward values of the mandatory inspection grid and the low - probability disease grid; iteratively updating the value function of each grid according to the reward value of each grid and the attenuation coefficient until a preset stop condition is met; generating the current patrol path of the mobile robot according to the updated value function.

[0010] In one embodiment, generating the current patrol path of the mobile robot according to the updated value function includes: determining the grid where the current mobile robot is located as the current grid; identifying all candidate grids adjacent to the current grid and evaluating the value functions of all the candidate grids; selecting the candidate grid with the highest value function as the next target grid; if there are multiple candidate grids with the same value function, randomly select one of them as the next target grid; after the mobile robot travels to the next target grid, repeat the above steps until the end - point grid is reached.

[0011] In one embodiment, the value function of each grid is calculated by the following formula:

[0012]

[0013] where is the function value of the grid numbered ; is the number of the grid that can be walked to next when the mobile robot is in the grid numbered ; is the reward value obtained when the mobile robot travels to the next grid; is the attenuation coefficient; the value 0.25 means that each grid has 4 grids in the front, back, left, and right, representing the normalized weight of each grid.

[0014] In one embodiment, when traveling to the low - probability disease grid, after determining whether the number of times passing through the low - probability disease grid reaches the detection threshold, the method further includes: if the number of times passing through the low - probability disease grid does not reach the detection threshold, no disease identification is performed on the low - probability disease grid, the number of times the current grid is passed through is recorded as incremented by 1, and it is determined whether the end grid is reached; if the end grid is not reached, continue to move forward to the next grid.

[0015] In one embodiment, according to the collected environmental image data, determining whether there is a disease in the corresponding grid and updating the annotation information includes: pre - processing the environmental image data to obtain pre - processed data; inputting the pre - processed data into a trained disease identification model to obtain the disease probability value output by the disease identification model; according to the disease probability value, determining that the current grid is a mandatory inspection grid, a grid that does not need to be inspected, or a low - probability disease grid, and updating the annotation information corresponding to the current grid.

[0016] In one embodiment, pre - processing the environmental image data to obtain pre - processed data includes: extracting the odd - numbered frame images of the environmental image data and splicing the odd - numbered frame images to obtain the pre - processed data.

[0017] In one embodiment, the disease identification model includes a ResNet sub - model. Inputting the pre - processed data into the trained disease identification model to obtain the disease probability value output by the disease identification model includes: for each odd - numbered frame image, extracting the first feature image corresponding to each odd - numbered frame image; extracting the second feature image of the pre - processed data through the ResNet sub - model; splicing the second feature image and the first feature image corresponding to each odd - numbered frame image to obtain a fused feature image; after performing convolution and pooling operations on the fused feature image, inputting it into the softmax function to obtain the disease probability value output by the softmax function.

[0018] According to the second aspect of the present application, there is provided an electronic device, including: a processor; a memory for storing processor - executable instructions; wherein, the processor is configured to execute the mobile robot inspection method described in the above - mentioned embodiments.

[0019] According to the third aspect of the present application, there is provided a computer - readable storage medium, the storage medium stores a computer program, and the computer program can be executed by a processor to complete the mobile robot inspection method described in the above - mentioned embodiments.

[0020] According to the fourth aspect involved in the present application, there is provided a computer program product including computer programs / instructions which, when executed by a processor, implement the mobile robot inspection method described in the above embodiments.

[0021] For the mobile robot inspection method, electronic device, storage medium and product provided in the above embodiments, through rasterization processing, mandatory inspection grids, non-inspection grids and low-probability disease grids are set, the inspection frequency of low-probability disease grids is reduced, ineffective inspections of non-inspection grids are avoided, duplicate inspections are reduced, and the inspection efficiency is significantly improved. Moreover, during the inspection process, the mobile robot can collect, process and feedback disease information in real time, dynamically adjust the inspection strategy according to real-time data, and update the annotation information of the grids, which can better cope with the dynamic changes in the distribution of diseases and improve the flexibility of inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic diagram of the overall structure of the mobile robot inspection system provided in the embodiments of the present application;

[0024] Figure 2 It is a first flowchart of the mobile robot inspection method provided in the embodiments of the present application;

[0025] Figure 3 It is a second flowchart of the mobile robot inspection method provided in the embodiments of the present application;

[0026] Figure 4 It is a schematic diagram of the overall operation process of the mobile robot inspection provided in the embodiments of the present application;

[0027] Figure 5 It is a flowchart of the execution path planning provided in the embodiments of the present application;

[0028] Figure 6 It is a flowchart of generating the current inspection path provided in the embodiments of the present application;

[0029] Figure 7 It is a flowchart of identifying the disease situation provided in the embodiments of the present application;

[0030] Figure 8 It is a flowchart of obtaining the disease probability value provided in the embodiments of the present application.

[0031] Icons: 100 - System platform; 200 - Mobile robot. Detailed implementation manners

[0032] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0033] Figure 1 It is a schematic diagram of the overall structure of the mobile robot inspection system provided by the embodiment of the present application. As Figure 1 shown, the system includes: a system platform 100 and a mobile robot 200.

[0034] The system platform 100 is used to input and manage road network structure data, perform rasterization processing on the drivable area and assign numbers, and set and adjust the annotation information of the rasters at the same time; it is also responsible for generating a dynamic inspection path, interacting with the mobile robot in terms of data, and receiving the disease data fed back by the mobile robot 200 to generate an inspection report. Among them, the road network structure refers to a road system composed of various roads that are interconnected and intertwined in a mesh distribution within a certain area. Rasterization processing refers to dividing the area into rectangular grids.

[0035] The mobile robot 200 is used to execute the inspection task according to the path planned by the system platform 100, collect road surface image or video data in real time, and identify diseases by processing the data through a disease recognition algorithm; it can also adjust the path in real time according to the change of the annotation information of the rasters.

[0036] The operation personnel perform initial settings through the system platform 100, including inputting the road network structure through the system platform, marking the drivable area, and setting the annotation information of the rasters; the operation personnel can also adjust the annotation information of the rasters at any time according to the actual situation, monitor the inspection progress of the mobile robot 200, and consult the inspection report generated by the system platform 100 to formulate maintenance decisions.

[0037] The system platform 100, the mobile robot 200, and the operation personnel cooperate with each other to jointly achieve efficient and intelligent road surface disease inspection.

[0038] Among them, the annotation information of the rasters includes: mandatory inspection rasters, non - inspection - required rasters, and low - probability disease rasters. Only during the first run, the operation personnel input the road network structure and mark the drivable area of the mobile robot, and mark some mandatory inspection rasters and non - inspection - required rasters according to the actual situation. During subsequent non - first runs, the operation personnel can add or delete such rasters at any time. After setting the annotation information, the mobile robot performs path planning and executes the inspection task.

[0039] Figure 2 It is the first flow schematic diagram of the mobile robot inspection method provided by the embodiment of the present application; as Figure 2As shown in the figure, a mobile robot inspection method includes the following steps S1 - S4:

[0040] Step S1: Obtain grid map data, where the grid map data includes the annotation information of each grid; the annotation information includes any one of the grids to be inspected, the grids that do not need to be inspected, and the grids with low - probability diseases.

[0041] Specifically, the system platform obtains the grid map data from the database, where each grid is labeled as a grid to be inspected, a grid that does not need to be inspected, or a grid with low - probability diseases, for subsequent path planning and execution of inspection tasks. The grids to be inspected are grids that must be inspected pre - set by the operation personnel according to the high - incidence areas of diseases, known disease points, or potential high - risk areas. These areas usually need to be inspected first to ensure road safety. The grids that do not need to be inspected are grids set by the operation personnel according to historical data, experience judgment, or areas where disease risks have been confirmed to be absent. These areas can be skipped during the current inspection to save time and resources. The grids with low - probability diseases are grids set based on the relatively low potential risk of disease occurrence.

[0042] Step S2: Eliminate the grids that do not need to be inspected, and based on the grids to be inspected and the grids with low - probability diseases, perform path planning to obtain the current inspection path.

[0043] Specifically, in the path - planning stage, the system platform first eliminates the grids labeled as not needing to be inspected from the grid map data because these grids are considered to have no disease risks or do not require key attention in the current inspection task. Subsequently, the system performs a path - planning algorithm based on the remaining grids to be inspected and the grids with low - probability diseases. The path - planning algorithm comprehensively considers the distribution and priority of these grids to generate the current inspection path of the mobile robot, ensuring that the robot can efficiently cover all areas to be inspected, thereby optimizing the inspection efficiency and reducing unnecessary inspection operations. It should be noted that the above steps S1 and S2 can be executed by the system platform and sent to the mobile robot after generating the current inspection path. In another embodiment, the above steps S1 and S2 can also be executed by the mobile robot, where the mobile robot obtains the grid map data from the system platform and generates the current inspection path.

[0044] Step S3: When moving to a grid with low - probability diseases, determine whether the number of times passing through the grid with low - probability diseases reaches the detection threshold. When the detection threshold is reached, collect environmental image data; when moving to a grid to be inspected, directly collect environmental image data.

[0045] Specifically, when the mobile robot moves to a grid with a low - probability disease, the system will determine whether the number of times the grid has been passed reaches a preset detection threshold. If the number of passes reaches the detection threshold, the acquisition of environmental image data will be triggered. Exemplarily, the detection threshold is set to 10, and the inspection is carried out once every 10 inspections, thus improving the inspection efficiency.

[0046] If the threshold is not reached, the acquisition will be skipped and the inspection task will continue to be executed. For the grids that must be inspected, regardless of the number of passes, the mobile robot directly performs the acquisition of environmental image data each time to ensure the timely detection and recording of diseases in high - risk areas.

[0047] Step S4: According to the acquired environmental image data, determine whether there is a disease in the corresponding grid and update the annotation information.

[0048] Specifically, according to the acquired environmental image data, the system analyzes the image through a disease recognition algorithm to determine whether there is a disease in the current grid. If the recognition result shows that a disease exists, the grid will be marked as a grid that must be inspected, and the specific information of the disease (such as location, time, picture, etc.) will be recorded. If the recognition result shows that there is no disease, the grid will be marked as a grid that does not need to be inspected. If the recognition result shows a low - probability disease, it will be marked as a grid with a low - probability disease. Finally, the system stores the updated annotation information in the database to dynamically adjust the path planning and inspection strategy in subsequent inspection tasks.

[0049] The above - mentioned method flexibly sets the key areas and non - key areas of the inspection according to actual needs, improves the pertinence of the inspection, avoids unnecessary inspections of areas that do not need to be inspected, reduces resource waste, improves the inspection efficiency and flexibility, and enhances the automation degree of the inspection.

[0050] In one embodiment, Figure 3 is the second process schematic diagram of the mobile robot inspection method provided by the embodiment of the present application. As Figure 3 shown, the mobile robot inspection method further includes the following steps S5 - S7:

[0051] Step S5: Continuously determine whether the annotation information of each grid has changed.

[0052] Step S6: If there is no change, continue to execute the inspection task according to the current inspection path.

[0053] Step S7: If there is a change, take the current grid as the starting point, re - execute the path planning to obtain an optimized path, and execute the inspection task according to the optimized path until the current inspection task is completed.

[0054] Specifically, during the inspection process, the system monitors the annotation information of each grid in real time to determine whether it has changed. The changes in the annotation information may include the adjustment of the grid type by the operation personnel (such as changing a grid from "low - probability disease grid" to "must - inspect grid"), or the dynamic update of the grid type based on the disease recognition result (such as changing from "no - need - to - inspect grid" to "must - inspect grid").

[0055] If, during the inspection process, the system determines that the annotation information of the grid has not changed, the mobile robot will continue to execute the task according to the currently planned inspection path. The mobile robot will sequentially inspect each grid according to the established path order without re - planning the path, thus ensuring the continuity and efficiency of the inspection task. If the system detects a change in the annotation information of the grid (for example, a grid is marked as a must - inspect grid, or a must - inspect grid is adjusted to a no - need - to - inspect grid), then the mobile robot will take the currently located grid as the starting point and re - execute the path planning. The system will dynamically generate a new optimized path according to the updated grid annotation information and continue to execute the inspection task according to the new path. This process ensures that the inspection path can adapt to the changes in grid information in real time, improving the flexibility and pertinence of the inspection, and finally completing the current inspection task.

[0056] The following specifically describes the overall operation process of the inspection robot.

[0057] Figure 4 It is a schematic diagram of the overall operation process of the mobile robot inspection provided by the embodiment of this application. As Figure 4 shown, it includes the following steps 100 - step 800:

[0058] Step 100: Start the mobile robot.

[0059] Step 200: Obtain the grid map data and execute path planning. For specific content, see the above steps S1 - S2, which will not be elaborated here.

[0060] Step 300: The mobile robot travels one grid according to the planned path.

[0061] Step 400: Determine whether the grid is a low - probability disease grid. If it is, execute Step 500; if not, execute Step 600.

[0062] Step 500: Determine whether the number of times passing through this grid reaches the inspection threshold. If it does, execute Step 600. For specific content, see the above step S3, which will not be elaborated here; otherwise, execute Step 800.

[0063] Step 600: Extract the video data captured by the camera.

[0064] Step 700: Identify the presence of diseases.

[0065] The captured video is processed by the disease recognition process, the disease image recognition algorithm is executed, and the disease recognition result is processed accordingly. For details, see the above step S4.

[0066] If the disease is identified, step 710 is executed, the mobile robot marks it as a must-check grid, records the grid number, and executes step 730 to store the corresponding data in the database, record the identification time, the grid location of the disease, intercept the disease image, and save the disease video data and image. If the algorithm determines that the disease possibility is extremely low, step 720 is executed to mark it as a low-probability disease grid, and step 730 is executed to store the corresponding data in the database. If the algorithm determines that there is no disease, step 800 is directly executed.

[0067] Step 800: Determine whether the vehicle has reached the destination grid.

[0068] Determine whether the mobile robot has reached the end grid. If yes, execute step 810, return to the origin, and generate an inspection report. The report contains the grid number that must be inspected, the grid number with defects, the corresponding recording time, the defect location (if any), the defect picture (if any), and the defect video (if any) for subsequent maintenance and repair work. At this point, the process ends.

[0069] If the destination grid is not reached, step 820 is executed to determine whether the grid database has changed. If not, return to step 300 and execute the subsequent steps in sequence; if there is a change, step 830 is executed to remove the grids that have been traveled, return to step 200 and execute the subsequent steps in sequence.

[0070] In some embodiments, Figure 5 A schematic diagram of a process flow for executing path planning provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, in step S2, path planning is performed according to the must-inspect grids and the low-probability disease grids to obtain the current inspection path, including the following steps S21-S24:

[0071] Step S21: number each grid and set a reward value for each grid.

[0072] Step S22: Initialize the value function corresponding to each grid according to the reward values of the must-inspect grids and the low-probability disease grids.

[0073] Step S23: iteratively update the value function of each grid according to the reward value and attenuation coefficient of each grid until a preset stop condition is met.

[0074] Step S24: Generate the current inspection path of the mobile robot according to the updated value function.

[0075] Specifically, each grid in the raster map is uniquely encoded for identification and reference in path planning. The annotation information includes the reward value of each grid. Among them, the reward value of the grid that must be inspected is greater than that of the grid with low - probability diseases, and the reward value of the end - point grid is greater than that of the grid that must be inspected. This reward mechanism can effectively guide the robot to preferentially select the grid that must be inspected and obtain the highest reward when reaching the end point. Exemplarily, it is set that when the mobile robot travels to the grid that must be inspected, it can obtain a reward value of 10 points, and when it travels to other grids, it can obtain a reward value of 0 points. When it travels to the end - point grid, it can obtain a reward of 20 points.

[0076] Based on these reward values, the system initializes the value function of each grid. The value function u(i) represents the expected cumulative reward from the current grid i to reach the end point according to the optimal strategy. At initialization, the value function values of all grids are usually set to 0, and then gradually optimized through an iterative update formula until a preset stop condition is reached. Then, based on the updated value function, the current inspection path of the mobile robot is generated. This process provides the basis for dynamic programming of the mobile robot, enabling it to select the optimal path according to the value function, preferentially inspect the grids that must be inspected, and reasonably arrange the inspection of grids with low - probability diseases.

[0077] In some embodiments, the above - mentioned execution of the path - planning algorithm specifically includes the following steps S8001 - S8020:

[0078] Step S8001, Initialize the value function: Suppose there are N grids in total. Define the function u(i) to represent the function value of the grid numbered i, and initialize u(i) of all grids to 0. Provide an initial value evaluation for each grid.

[0079] Step S8002, Set list variables: , which is used to store the updated function value corresponding to each grid . Create a data structure to store the updated value function values of each grid, facilitating subsequent iterative updates and making the code clearer and easier to understand.

[0080] Step S8003, Let the variable , and initialize the variable i to 1. Define a variable i to track the number of the currently processed grid, used to control the loop of the initialization process to ensure that the initialization process proceeds in order.

[0081] Step S8004, Put the value 0 into The tail of the variable. Initialize the value function u(i) of the current grid i to 0 to ensure that the value function of each grid starts from 0, avoiding undefined or inconsistent states. This provides a consistent starting point for subsequent iterative updates.

[0082] Step S8005, loop to initialize all grids. If the variable , then increment the value of by 1, execute step S8004 and continue to execute sequentially. If , then execute step S8006. Through the loop, initialize the value function of each grid to 0 one by one, avoiding missing any grid.

[0083] Step S8006, the function values corresponding to all grids are initialized. This provides a stable and consistent starting point for subsequent iterative updates.

[0084] In one embodiment, the value function of each grid is calculated using the following formula:

[0085]

[0086] where, is the function value of the grid numbered ; is the number of the grid that can be walked to next when the mobile robot is in the grid numbered , that is, the number of the grid where the mobile robot is located after moving forward, backward, left or right; is the reward value obtained when the mobile robot travels to the next grid; is the attenuation coefficient; the value 0.25 means that each grid has 4 grids in the front, back, left and right, representing the normalized weight of each grid.

[0087] Specifically, the integrated reward value is a value set according to the experiment, which represents the reward obtained by the mobile robot when it travels to this grid. As the subsequent iteration progresses, grids with higher integrated rewards are more likely to obtain higher values. The integrated reward value can be any value greater than 0. The larger the integrated reward, the faster the convergence, but too large may also lead to non-convergence. By setting reasonable reward values and optimizing the path planning, the inspection efficiency and accuracy can be improved.

[0088] Among them, the attenuation coefficient can be set according to the actual situation to balance the influence of the current reward and the future reward. The attenuation coefficient can be set between 0.5 and 1. Exemplarily, the default value of the attenuation coefficient is 0.9, indicating that when planning the path, the influence of the next multiple steps, such as 40 steps, is considered. This value is a parameter obtained from the experiment.

[0089] Step S8007. Define the stop condition: Define a variable as a very small value, such as , to provide a stop condition for determining whether the algorithm converges. This avoids infinite iteration, ensures that the algorithm converges within a reasonable time, provides a clear termination condition for the algorithm, and ensures the stability and efficiency of the algorithm.

[0090] Step S8008. Set the old value list , and create a data structure to store the value function values of the previous iteration.

[0091] Step S8009. Copy the new value to the old value, and copy the content of to to provide a reference value for the next iteration.

[0092] Step S8010. Initialize the iteration variable: Let the variable , and define a variable n to track the grid number currently being processed to ensure that the iteration process proceeds in order.

[0093] Step S8011. Initialize the temporary variable: Let the variable , and define a temporary variable to store the updated value of the current grid.

[0094] Step S8012. Find the neighboring grids: Find the encoding of the neighboring grids when the mobile robot is at grid number . Assume that the encoding of the neighboring grid is . Exemplarily, the maximum number of grids directly adjacent to a grid is 4, which are the numbers of the grids where the robot is located after moving forward, backward, left, or right, to determine all possible next positions of the current grid.

[0095] Step S8013. Initialize the neighboring grid counter: Let the variable , and define a variable j to track the neighboring grid number currently being processed.

[0096] Step S8014. Calculate the value update of the neighboring grids.

[0097] Calculate , and assign this value to the variable .

[0098] Among them, is the reward value obtained when traveling from the current grid n to the neighboring grid Cj; d is the attenuation coefficient; is the neighboring grid The current function value, where 0.25 is the weight for each direction. The optimal value of the current grid is evaluated through the weighted sum of the reward value and the future value. Ensure that the value function of each grid can be updated based on the values of its neighboring grids, and gradually approach the optimal path through iterative updates.

[0099] Step S8015, loop through all neighboring grids: If , then increment the value of by 1, execute step S8014 and continue to execute in sequence, otherwise execute step S8016. Ensure that the value updates of all neighboring grids are calculated.

[0100] Step S8016, update the value of the current grid: Assign the value of to .

[0101] Step S8017, loop through all grids: If , then increment the value of by 1, execute step S8011 and continue to execute in sequence, otherwise execute step S8018.

[0102] Step S8018, calculate the change in the value function. It is calculated using the following formula:

[0103]

[0104] That is, assign the calculated value to the variable , subtract the corresponding elements of and and take the absolute value to evaluate the difference between the value function value of the current iteration and the value function value of the previous iteration. Provide a quantitative method to evaluate the convergence of the algorithm, which helps to determine whether the algorithm has approached the optimal solution.

[0105] Step S8019, determine whether to converge: Find the maximum value in and compare it with . If this value is greater than , then execute step S8020 and continue to execute in sequence, otherwise execute step S8009 and then continue to execute in sequence. Used to determine whether the algorithm has converged to the optimal solution according to the set stop condition.

[0106] Step S8020, the path optimization process is completed, and the scores of each grid are obtained.

[0107] In one embodiment, Figure 6 is the schematic flowchart of the process for generating the current inspection path provided by the embodiment of the present application, as shown in Figure 6As shown in the figure, step S24: Generate the current inspection path of the mobile robot according to the updated value function, including the following steps S241 - S244:

[0108] Step S241: Determine the grid where the current mobile robot is located as the current grid.

[0109] Specifically, during the inspection process, the mobile robot determines the grid where its current position is located in real time and marks this grid as the "current grid". This step ensures that the robot always knows its position in the road network, providing a basis for subsequent path selection.

[0110] Step S242: Identify all candidate grids adjacent to the current grid and evaluate the value functions of all candidate grids.

[0111] Specifically, if the current mobile robot is in the grid numbered , find all reachable adjacent grids around the grid (4 options are available, namely forward, backward, left shift, and right shift). For each candidate grid, the robot calls its corresponding value function value, which reflects the potential value of moving from the current grid to the candidate grid and is dynamically updated based on the reward value and the decay coefficient.

[0112] Step S243: Select the candidate grid with the highest value function as the target grid for the next step; if there are multiple candidate grids with the same value function, randomly select one of them as the target grid for the next step.

[0113] Specifically, find the value of the candidate grid in the variable , and find the grid corresponding to the maximum value, which is the target grid that the mobile robot will travel to in the next step. If there are multiple candidate grids with the same maximum value, then randomly select a grid to travel to. This selection strategy ensures that the robot always moves in the optimal direction, and at the same time introduces randomness when there are multiple optimal choices to avoid path fixation.

[0114] Step S244: After the mobile robot travels to the target grid of the next step, repeat the above steps until the end grid is reached.

[0115] The mobile robot moves according to the selected target grid and repeats steps S241 - S243 after arrival until the end grid is reached. This loop process ensures that the robot can dynamically adjust the path, respond to changes in the grid value function in real time, and finally complete the inspection task.

[0116] The above steps guide the mobile robot to optimize the path in real time during the inspection process by dynamically evaluating and selecting the candidate grid with the highest value function, ensuring that the inspection task is completed efficiently and flexibly.

[0117] In one embodiment, step S3 further includes: if the number of times passing through the low - probability disease grid does not reach the detection threshold, no disease identification is performed on the low - probability disease grid, the passing - through times of the current grid are recorded and incremented by 1, and it is determined whether the end grid is reached; if the end grid is not reached, continue to move forward to the next grid.

[0118] That is, if the number of times the mobile robot passes through the low - probability disease grid does not reach the preset detection threshold, no disease identification is performed on this grid. Instead, the passing - through times of the current grid are incremented by 1, and it is determined whether the end grid has been reached. If the end grid has not been reached, the robot continues to move forward along the established path to the next grid until the detection threshold is met or the inspection task is completed. Checking the low - probability disease grid according to the inspection threshold can avoid over - inspection or missed inspection, improve the efficiency and accuracy of the inspection, and help to reasonably allocate resources.

[0119] In one embodiment, Figure 7 is a schematic flow diagram for identifying disease conditions provided by an embodiment of the present application. As Figure 7 shown, step S4 includes the following steps S41 - S43:

[0120] Step S41: Pre - process the environmental image data to obtain the pre - processed data.

[0121] In one embodiment, the data pre - processing includes: extracting the odd - numbered frame images of the environmental image data and splicing the odd - numbered frame images to obtain the pre - processed data. Extracting odd - numbered frames can reduce the data volume, but still retain sufficient information for disease identification, so as to reduce the computational amount and improve the operation efficiency of the algorithm.

[0122] Exemplarily, input 1 - second video data, a total of 24 frame pictures. Assume that the size of each frame picture is . Only extract the odd - numbered frames (the 1st, 3rd, 5th... 23rd frames), a total of 12 frames. Combine the pictures of the odd - numbered frames into a group and represent them with a tensor. Then the shape and size of the tensor are . The data pre - processing operation is completed, which is the result after processing.

[0123] Step S42: Input the pre - processed data into the trained disease identification model to obtain the disease probability value output by the disease identification model.

[0124] Among them, the trained disease identification model contains sub - models. The entire model has the following parameters:

[0125] The size is of , ... A total of 12 weight parameters. The size is of 、 ... There are a total of 12 offset parameters.

[0126] The input channel is set to 13, the output channel is set to 3, and the convolutional kernel size is set to , and the sliding window step size is set to , and the padding mode is set to of the convolutional kernel .

[0127] The input channel is set to 3, the output channel is set to 1, and the convolutional kernel size is set to , and the sliding window step size is set to , and the padding mode is set to of the convolutional kernel .

[0128] The size is of and the size is of .

[0129] Among them, represents all weight parameters in the sub-models, and represents all bias parameters in the ResNet sub-models. The input parameters need to be adjusted to a size suitable for the input image size. The output parameters need to be adjusted so that the size of the output data is . The designs of the convolutional kernel sizes of conv1 and conv2 are interrelated and can be adjusted to a suitable size according to experimental measurements.

[0130] In one embodiment, Figure 8 is a schematic flowchart of the process for obtaining the disease probability value provided by the embodiment of the present application. As Figure 8 shown, in step S42, the preprocessed data is input into the trained disease recognition model to obtain the disease probability value output by the disease recognition model, which specifically includes the following steps S421 - S424:

[0131] Step S421: For each odd-frame image, extract the first feature image corresponding to each odd-frame image.

[0132] Step S422: Extract the second feature image of the preprocessed data through the ResNet sub-model.

[0133] Step S423: Concatenate the second feature image and the first feature image corresponding to each odd-frame image to obtain a fused feature image.

[0134] Step S424: After performing convolution and pooling operations on the fused feature image, input it into the softmax function to obtain the disease probability value output by the softmax function.

[0135] Specifically, during the execution of the disease recognition algorithm, first, for each frame of the collected environmental image data, extract its corresponding first feature image to capture the basic feature information in the image. Subsequently, use the ResNet sub-model to perform deep feature extraction on the preprocessed image data to generate a second feature image, and utilize the powerful convolutional neural network structure of ResNet to extract high-level features. Then, splice the second feature image with the first feature image of each odd-frame image to form a fused feature image. This feature fusion method can combine low-level and high-level features to more comprehensively describe the disease information in the image. Finally, perform convolution and pooling operations on the fused feature image to further extract and compress features, and then input it into the softmax function to obtain the probability value of the presence of the disease, thereby achieving accurate judgment of the disease. This process combines image preprocessing, deep learning models, and feature fusion technologies, significantly improving the efficiency and accuracy of disease recognition.

[0136] Step S43: According to the disease probability value, determine whether the current grid is a grid that must be inspected, a grid that does not need to be inspected, or a grid with a low probability of disease, and update the annotation information corresponding to the current grid.

[0137] Specifically, according to the probability value output by the disease recognition model, the system evaluates the disease risk of the current grid and updates the annotation information of the current grid accordingly. If the disease probability value output by the model is higher than the preset high-risk threshold (for example, the probability of the presence of the disease is greater than 0.8), then mark the current grid as a grid that must be inspected to ensure key inspection of this area during subsequent patrols; if the disease probability value is lower than the preset low-risk threshold (for example, the probability of the presence of the disease is less than 0.1), then mark the current grid as a grid that does not need to be inspected to reduce unnecessary repeated inspections and optimize the patrol efficiency; if the disease probability value is between the high-risk threshold and the low-risk threshold, then mark the current grid as a grid with a low probability of disease, and dynamically adjust the inspection frequency of this grid according to the preset inspection threshold (such as checking once every 10 patrols). In this way, the system can update the annotation information of the grid in real time, dynamically adjust the patrol strategy, and ensure that the mobile robot can more efficiently and accurately identify and process disease areas during subsequent patrols.

[0138] If, during the driving process of the mobile robot, the operator adds or deletes a grid, after the mobile robot completes the inspection of the current grid, it will immediately re-plan the path. When re-planning the path, the already traveled grids will be excluded to ensure that the path planning is based on the latest grid data.

[0139] In some embodiments, the above-mentioned disease recognition algorithm specifically includes the following steps S9001 - S9015:

[0140] Step S9001: The tensor shape size of the input data is , represented by .

[0141] Organize the data into a format that the model can process. The model can receive this tensor as input and start forward propagation. Among them, the tensor is a commonly used data structure in deep learning and can conveniently represent multi-dimensional data.

[0142] Step S9002: Initialize a counter i for processing data frame by frame, and let .

[0143] Step S9003: Process the data of the i-th frame and calculate features.

[0144] The calculation formula is as follows: .

[0145] Extract the features of the data through the linear combination of weights and biases.

[0146] Step S9004: Process all 12 frames of data frame by frame through a loop: If , let increase the value of by 1, and return to step S9003 to execute, and then continue to execute in the order of steps. Otherwise, execute step S9005 to ensure that all frame data is processed.

[0147] Step S9005: Complete the feature extraction of all frames to obtain 12 first feature images: Obtain a total of 12 results, and the shape size of each result is . The feature map of each frame represents the features of that frame and is used for subsequent disease recognition.

[0148] Step S9006: Use the ResNet model to process the input data to obtain the output of the ResNet model. That is, use the model to process , and can obtain results. The formula is as follows:

[0149]

[0150] Extract higher-level second feature images through the ResNet model.

[0151] Step S9007: Concatenate the second feature image and the first image feature.

[0152] That is, The results are concatenated with to obtain a tensor with a shape and size of which is named the variable. By concatenation, features from different sources can be combined to improve the recognition ability of the model and the accuracy of the algorithm.

[0153] Step S9008: Perform a convolution operation on the concatenated features.

[0154] Use the convolution kernel to perform a convolution operation on that is

[0155]

[0156] a tensor with a shape and size of can be obtained .

[0157] Step S9009: Perform a max pooling operation on the convolved features.

[0158] Use a MaxPooling operation with a size of and a sliding window stride set to to perform a max pooling operation on the result of Step S9008, obtaining the output tensor, that is

[0159]

[0160] The shape and size of the tensor is . The max pooling operation can reduce the size of the feature map while retaining important feature information.

[0161] Step S9010: Perform a convolution operation on the max-pooled features.

[0162] Use the convolution kernel to perform a convolution operation on as follows:

[0163]

[0164] a tensor with a size of can be obtained .

[0165] Step S9011: Perform a max pooling operation on the convolved features.

[0166] Use a size of and a sliding window stride set to The MaxPooling operation performs max pooling on the result of step S9010 to obtain the output tensor:

[0167]

[0168] tensor has a shape and size of .

[0169] The max pooling operation further reduces the size of the feature map while retaining important feature information.

[0170] Step S9012: Reshape the tensor to a shape and size of . Flatten the feature map into a one-dimensional vector for input to the fully connected layer.

[0171] Step S9013: Perform a linear combination of the flattened feature vector with the weights and biases of the fully connected layer.

[0172] Perform the operation according to the following formula and method:

[0173]

[0174] The final disease probability is output through the fully connected layer, obtaining a vector with a shape of (2), representing the probability of the disease existing.

[0175] Step S9014: Perform a softmax operation on the output of the fully connected layer.

[0176] Using the softmax function to operate on the tensor yields:

[0177]

[0178] Its size is , representing the probability of the disease existing. The softmax function can convert the output into a probability distribution, facilitating subsequent classification decisions, thereby achieving efficient disease recognition and recording.

[0179] Exemplarily, the model outputs a vector, such as [0.1, 0.9]. Each element in this vector represents the probability of a class. Assume the first element (0.1) represents the probability that the disease does not exist, and the second element (0.9) represents the probability that the disease exists. Compare the magnitudes of the elements in the probability vector and find the index corresponding to the element with the largest value. The element with the largest value is 0.9, and the corresponding index is 1. An index of 1 indicates that the disease exists.

[0180] Step S9015, the forward propagation of the model ends, and the result is output.

[0181] Through the above steps, the solution of the present application can achieve efficient inspection of road surface diseases, ensure that diseases can be discovered and recorded in time, and at the same time, through dynamic path planning and real-time data processing, improve the flexibility and adaptability of the system. These steps ensure the efficiency and accuracy of the entire inspection process and are applicable to actual road surface disease inspection tasks.

[0182] The embodiment of the present application also provides an electronic device, which may include a processor and a memory for storing processor-executable instructions. The processor is configured to implement the steps of the mobile robot inspection method in any of the above embodiments.

[0183] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by a processor to complete the steps of the mobile robot inspection method provided by the present disclosure.

[0184] The embodiment of the present application also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the mobile robot inspection method provided by the present disclosure are implemented.

[0185] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative.

[0186] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0187] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0188] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A mobile robot inspection method, characterized in that, Including: Obtain raster map data, where the raster map data includes annotation information for each raster; The annotation information includes any one of mandatory inspection rasters, non-inspection rasters, and low-probability disease rasters; Eliminate the non-inspection rasters, and perform path planning based on the mandatory inspection rasters and low-probability disease rasters to obtain the current inspection path; When moving to the low-probability disease raster, determine whether the number of times passing through the low-probability disease raster reaches the detection threshold. When the detection threshold is reached, collect environmental image data; when moving to the mandatory inspection raster, directly collect environmental image data; Based on the collected environmental image data, determine whether there is a disease in the corresponding raster and update the annotation information; Continuously judge whether the annotation information of each raster changes; If there is no change, continue to perform the inspection task according to the current inspection path; If there is a change, take the current raster as the starting point, re-perform path planning to obtain an optimized path, and perform the inspection task according to the optimized path until the current inspection task is completed; Among them, the step of determining whether there is a disease in the corresponding raster and updating the annotation information based on the collected environmental image data includes: If it is determined that there is a disease in the raster and the disease is a high-risk disease, update the annotation information of the raster to a mandatory inspection raster; if it is determined that there is a disease in the raster and the disease is a low-risk disease, update the annotation information of the raster to a low-probability disease raster; if it is determined that there is no disease in the raster, update the annotation information of the raster to a non-inspection raster; Among them, the environmental image data is road surface image data.

2. The method according to claim 1, characterized in that, The annotation information includes the reward value of each raster, the reward value of the mandatory inspection raster is greater than the reward value of the low-probability disease raster, and the reward value of the end raster is greater than the reward value of the mandatory inspection raster; The step of performing path planning based on the mandatory inspection rasters and low-probability disease rasters to obtain the current inspection path includes: Number each raster and set a reward value for each raster; Initialize the value function corresponding to each raster according to the reward values of the mandatory inspection rasters and low-probability disease rasters; Iteratively update the value function of each raster according to the reward value of each raster and the attenuation coefficient until the preset stop condition is met; Generate the current inspection path of the mobile robot according to the updated value function.

3. The method according to claim 2, characterized in that, The step of generating the current inspection path of the mobile robot according to the updated value function includes the following steps: S241: Determine the raster where the current mobile robot is located as the current raster; S242: Identify all candidate rasters adjacent to the current raster and evaluate the value functions of all the candidate rasters; S243: Select the candidate raster with the highest value function as the next target raster; if there are multiple candidate rasters with the same value function, randomly select one of them as the next target raster; S244: After the mobile robot travels to the next target raster, repeat the above steps S241 to S243 until the end raster is reached.

4. The method according to claim 2, wherein The value function of each raster is calculated using the following formula: Among them, is the function value of the grid numbered ; is the grid number that can be walked to next when the mobile robot is at the grid numbered ; is the reward value obtained when the mobile robot travels to the next grid; is the attenuation coefficient; the value 0.25 means that each grid has 4 grids in front, behind, left, and right, representing the normalized weight of each grid.

5. The method according to claim 1, characterized in that, When it travels to the low - probability disease grid, after determining whether the number of times passing through the low - probability disease grid reaches the detection threshold, the method further includes: If the number of times passing through the low - probability disease grid does not reach the detection threshold, do not perform disease identification on the low - probability disease grid, record that the number of times the current grid is passed through is incremented by 1, and determine whether the end grid is reached; If the end grid is not reached, continue to move forward to the next grid.

6. The method according to claim 1, wherein The method of judging whether there is a disease in the corresponding grid according to the collected environmental image data and updating the annotation information includes: Pre - process the environmental image data to obtain pre - processed data; Input the pre - processed data into the trained disease identification model to obtain the disease probability value output by the disease identification model; According to the disease probability value, determine that the current grid is a mandatory inspection grid, a grid that does not need to be inspected, or a low - probability disease grid, and update the annotation information corresponding to the current grid.

7. The method according to claim 6, characterized in that, The pre - processing of the environmental image data to obtain pre - processed data includes: Extract the odd - numbered frame images of the environmental image data and splice the odd - numbered frame images to obtain the pre - processed data.

8. The method according to claim 6, characterized in that, The disease identification model includes a ResNet sub - model. The input of the pre - processed data into the trained disease identification model to obtain the disease probability value output by the disease identification model includes: For each odd - numbered frame image, extract the first feature image corresponding to each odd - numbered frame image; Extract the second feature image of the pre - processed data through the ResNet sub - model; Splice the second feature image and the first feature image corresponding to each odd - numbered frame image to obtain a fused feature image; After performing convolution and pooling operations on the fused feature image, input it into the softmax function to obtain the disease probability value output by the softmax function.

9. An electronic device, characterized in that, The electronic device includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the mobile robot inspection method according to any one of claims 1 - 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program can be executed by the processor to complete the mobile robot inspection method according to any one of claims 1 - 8.

11. A computer program product, characterized in that, Including a computer program / instructions, when the computer program / instructions are executed by the processor, the mobile robot inspection method according to any one of claims 1 - 8 is implemented.

Images