An intelligent inspection method and system of a leg-foot type robot combined with a unmanned aerial vehicle

Abstract

The present application relates to the field of industrial inspection, in particular to a kind of intelligent inspection method and system of leg-foot type robot combined with unmanned aerial vehicle.A kind of intelligent inspection system of leg-foot type robot combined with unmanned aerial vehicle, comprising: global perspective acquisition module, environmental data analysis module, detection action screening module, task allocation module and inspection progress monitoring module.The present application dynamically calculates the ideal detection frequency of each device by the change of the environment of the device and the historical performance of the device, combined with fault risk value and environmental data, and screens out the most suitable detection action set according to each detection result.Through this method, the detection action of each device is no longer fixed, but is optimized and adjusted according to actual needs, thereby avoiding unnecessary repeated detection, reducing the inspection time, and improving the accuracy and efficiency of equipment monitoring.

Description

Technical Field

[0001] This invention relates to the field of industrial inspection, specifically to an intelligent inspection method and system for a legged robot combined with an unmanned aerial vehicle (UAV). Background Technology

[0002] With the improvement of industrial automation and the increasing complexity of industrial equipment, traditional manual inspection methods are gradually failing to meet the requirements of efficiency, safety and accuracy. Especially in large facilities or complex environments, there are many types of equipment and complex inspection tasks. Manual inspection is not only inefficient, but also prone to omissions and failure to detect potential faults in time, which can lead to equipment failure and safety accidents.

[0003] In recent years, the combination of drones and ground robots (especially legged robots) for inspection has been gradually applied. Drones can provide a wide field of view, quickly scan a large area, obtain the location information of equipment, and even detect abnormal signals of equipment, such as overheating. Legged robots, on the other hand, can perform more precise close-range inspections, perform complex tasks such as thermal imaging, vibration detection, and gas sampling, and adapt to complex terrain and environment.

[0004] Existing collaborative inspection systems of drones and ground robots mostly rely on predetermined inspection frequencies and fixed inspection tasks, lacking intelligent task allocation and dynamic environmental adaptability. Therefore, how to dynamically adjust the inspection frequency and inspection actions based on the equipment's historical data, current environment, and real-time status, and intelligently allocate robot resources using optimization algorithms, has become a key issue in improving inspection efficiency and accuracy.

[0005] To address this issue, this invention proposes a combined UAV-based legged robot intelligent inspection method and system to achieve efficient and accurate equipment inspection tasks. Summary of the Invention

[0006] This invention dynamically calculates the ideal detection frequency for each device by combining changes in the device's environment and the device's historical performance with fault risk values ​​and environmental data. Based on the results of each detection, it selects the most suitable set of detection actions. In this way, the detection actions of each device are no longer fixed, but are optimized and adjusted according to actual needs, thereby avoiding unnecessary repeated detection, reducing inspection time, and improving the accuracy and efficiency of equipment monitoring.

[0007] A method for intelligent inspection using a legged robot in conjunction with a drone includes:

[0008] Before any legged robot performs an inspection task, a drone provides a global perspective to obtain the position of the equipment to be inspected and the legged robot within the target inspection area. A thermal imager is used to perform thermal imaging scans on the equipment to be inspected in the target inspection area. Environmental data of the target inspection area is obtained. Based on the environmental data, the fault risk value of the equipment is determined, and the ideal inspection frequency of all equipment is calculated.

[0009] For each device to be tested, the testing actions performed include thermal imaging, visible light imaging, near-field lidar scanning, vibration sampling, gas sampling, and tactile confirmation; among them, visible light imaging and near-field lidar scanning are fixed testing actions, while thermal imaging, vibration sampling, gas sampling, and tactile confirmation are variable testing actions.

[0010] After any device to be tested completes a test, based on the device's most recent test result, determine the variable test actions that it needs to perform in the next test, and obtain the set of test actions for the device accordingly. Then, calculate the test time for the device, and finally record the device number, the set of test actions, and the test time.

[0011] Based on the ideal detection frequency of the devices to be tested and the detection time of all devices to be tested, the ideal number of legged robots to be deployed, the set of target devices to be tested by each legged robot and its inspection path are calculated by the swarm optimization algorithm; the corresponding number of legged robots are started, and each robot performs the detection of the corresponding set of target devices in sequence according to the inspection path, and returns to the starting point to complete one inspection task;

[0012] During the inspection missions of the legged robots, the drone periodically acquires the location and inspection progress of all legged robots performing inspection missions according to a preset inspection cycle, and detects whether there are new obstacles on the inspection path. If so, obstacle removal measures are implemented. Based on the inspection time, the ideal inspection progress of each legged robot is calculated. For any legged robot, if its actual inspection progress does not reach the ideal inspection progress and the progress deviation exceeds the preset deviation threshold, emergency detection measures are triggered.

[0013] Preferably, based on the environmental data, the fault risk value of the equipment in the target inspection area is determined, and the specific operation is as follows:

[0014] Acquire several historical detection records of the target inspection area. Each historical detection record contains the fault detection result of a device under inspection and the environmental data when the fault occurred. The fault detection result is a binary tag, where 1 indicates that a fault exists and 0 indicates that it is normal. The environmental data includes temperature, humidity, wind speed, rainfall and light intensity.

[0015] The Z-score standardization method is used to standardize all environmental data to obtain standardized environmental data.

[0016] Use the elbow rule to determine the optimal number of clusters;

[0017] The standardized environmental data is clustered using the K-means clustering algorithm. The specific steps are as follows:

[0018] Step 1: Randomly select K samples as initial cluster centers;

[0019] Step 2: For each historical detection record, calculate its Euclidean distance to each cluster center and assign it to the nearest cluster;

[0020] Step 3: Recalculate the mean of all standardized environmental data in each cluster and use this mean as the new cluster center;

[0021] Step 4: Repeat steps 2 and 3 until the cluster centers no longer change, at which point the algorithm converges;

[0022] Count the number of faulty devices in each cluster, divide the number by the total number of devices in the cluster, and obtain the probability of failure for that cluster.

[0023] Based on the current environmental data, calculate the Euclidean distance between the current environmental data and the center of each cluster, assign it to the nearest cluster, and use the failure probability of this cluster as the failure risk value of the current environmental data.

[0024] Preferably, the ideal detection frequency for each device to be tested is calculated, and the specific operation is as follows:

[0025] Calculate the failure probability for all clusters; compare the failure probability corresponding to the current environmental data with the failure probability for all clusters, and calculate the frequency selection coefficient according to the following rules:

[0026] If the probability of failure corresponding to the current environmental data is equal to the lowest probability of failure among all clusters, then the frequency selection coefficient is 0; if the probability of failure corresponding to the current environmental data is equal to the highest probability of failure among all clusters, then the frequency selection coefficient is 1; for environmental data between the lowest and highest probabilities of failure, the frequency selection coefficient is mapped according to the proportion of the probability of failure within the linear interval to obtain the corresponding value, which is between 0 and 1.

[0027] Set the highest and lowest detection frequencies, and map the frequency selection coefficient between the lowest and highest detection frequencies using the following method:

[0028] When the frequency selection factor is 0, the ideal detection frequency of the device is the lowest detection frequency; when the frequency selection factor is 1, the ideal detection frequency of the device is the highest detection frequency; for other frequency selection factors, the ideal detection frequency of the device is calculated by linear interpolation.

[0029] Preferably, based on the most recent detection result of the device, the variable detection actions that need to be performed in the next detection are determined, and the set of detection actions of the device is obtained accordingly. The specific operation is as follows:

[0030] Visible light imaging and near-range lidar scanning are used as fixed detection actions, and a set of detection actions is added to each detection.

[0031] The detection dataset is obtained after the device under test was last subjected to each of the variable detection actions. The detection data corresponding to thermal imaging temperature measurement is the surface temperature of the device under test.

[0032] The detection data corresponding to vibration sampling is the RMS vibration value of the device under test.

[0033] The detection data corresponding to gas sampling consists of the gas concentrations of several target gases.

[0034] The tactile confirmation corresponds to the peak force and displacement.

[0035] For screening thermal imaging temperature measurement, perform the following operations:

[0036] The surface temperatures of all similar devices to the device under test are most recently acquired, and the average surface temperature of this type of device is calculated. At the same time, based on the thermal imaging scan results of the UAV, the temperature of the area where all similar devices to the device under test are located is calculated, and the average imaging temperature is calculated.

[0037] Calculate the degree of deviation between the current surface temperature of the device under test and the average surface temperature to obtain a first degree of deviation; calculate the degree of deviation between the temperature of the area where the device under test is located and the average imaging temperature to obtain a second degree of deviation;

[0038] If either the first deviation degree or the second deviation degree is greater than the preset deviation threshold, then thermal imaging temperature measurement is added to the detection action set of the current device under test; otherwise, no operation is performed.

[0039] To filter vibration samples, perform the following operations:

[0040] If the vibration RMS of the most recent sample is greater than twice the standard deviation of the historical vibration mean of the device, then the vibration sampling is added to the detection action set; if vibration sampling was not performed in the last two tests, then the vibration sampling also needs to be added to the detection action set.

[0041] To screen gas samples, perform the following operations:

[0042] If the concentration of any gas sampled most recently is greater than the corresponding warning threshold, then the gas sampling will be added to the detection action set; if the device under test has not performed gas sampling in the last two tests, then the gas sampling will also be added to the detection action set.

[0043] For tactile confirmation screening, perform the following operations:

[0044] If the peak force obtained from the most recent tactile confirmation exceeds the preset threshold and the displacement exceeds the warning displacement threshold, then the tactile confirmation will be added to the detection action set of the current device to be tested.

[0045] Preferably, the ideal number of legged robots to be deployed, the set of target detection devices that each legged robot should be responsible for, and their inspection paths are calculated by a swarm optimization algorithm, specifically a particle swarm optimization algorithm.

[0046] Preferably, the ideal number of legged robots to be deployed, the set of target detection devices that each legged robot should be responsible for, and their inspection paths are calculated using a particle swarm optimization algorithm. The specific operation is as follows:

[0047] Step 1: Read the set of detection actions and corresponding dwell time of each device in the next inspection, and calculate the detection time of the device; based on the passable cost grid provided by the UAV, use the shortest path search to calculate the path length between any two device observation positions, and convert it into walking time; the inspection time of one inspection is the detection time of all devices in the path, the walking time between each adjacent device, and the walking time from the last device back to the first device.

[0048] Step 2: Set the following constraints:

[0049] The time constraint is that the time for a single inspection must not exceed the ideal inspection interval;

[0050] Coverage and exclusivity constraints require that all devices must be allocated, and each device can only be allocated to one robot;

[0051] The path constraint is that the inspection path of each robot is a closed path;

[0052] Step 3: Determine the initial lower bound of the number of devices to be deployed: Calculate the average time spent on device inspection and the average travel time between the device and the nearest neighbor device; by dividing the ideal inspection interval by the sum of the inspection time and travel time of a single device, the upper limit of the number of devices per robot can be obtained, and then the initial number of devices to be deployed can be calculated.

[0053] Step 4: Set up particles and initial population. Particles include the number of robots activated, the equipment set of each robot, and the inspection order; cluster the devices based on their spatial distribution to generate initial particles.

[0054] Step 5: Calculate the inspection time for each path. If the time exceeds the ideal detection interval, the particle is not feasible. Among the feasible particles, prioritize them based on the number of activated particles, the total remaining time, and the travel distance.

[0055] Step 6: Generate new particles through the particle update operator, and adjust the number of activated particles until a feasible solution is obtained;

[0056] Step 7: Output the ideal number of robots to be deployed, the equipment set, and the inspection path, and generate execution instructions for each robot.

[0057] Preferably, the rules for setting the inspection cycle are as follows:

[0058] Based on the wind speed and rainfall data obtained from the environmental data before the legged robot performs the inspection task, maximum wind speed and maximum rainfall limits are set respectively. The wind speed and rainfall are normalized based on the maximum wind speed and maximum rainfall limits, and the average value of the normalized wind speed and rainfall is calculated as the inspection cycle control value.

[0059] Set the inspection cycle control range, including the shortest inspection cycle and the longest inspection cycle, and linearly map the inspection cycle control value to the inspection cycle control range to obtain the inspection cycle in actual application;

[0060] Emergency testing measures specifically include:

[0061] If any legged robot fails to reach the ideal inspection progress during the inspection process and its deviation exceeds the preset deviation threshold, then select several devices from the target devices that the robot has not yet completed its inspection, remove them from the inspection task, and mark the removed devices as spare devices; count the numbers and locations of all spare devices, and start several additional legged robots to perform inspection tasks on the spare devices to ensure that all devices complete their tasks within the remaining inspection time;

[0062] The specific method for selecting and removing equipment is as follows: based on the degree of deviation in the inspection progress, calculate the number of equipment that the legged robot needs to remove. This number is obtained by multiplying the degree of deviation by the total number of target detection equipment of the current robot and rounding up; based on the distance between the equipment and the robot, remove the equipment sequentially starting from the equipment farthest from the current robot.

[0063] A legged robotic intelligent inspection system combining unmanned aerial vehicles (UAVs) includes:

[0064] The global view acquisition module is used to obtain the position of the equipment to be inspected and the legged robot in the target inspection area by providing a global view through the UAV before any legged robot performs an inspection task, and to perform thermal imaging scanning on the equipment to be inspected in the target inspection area through the thermal imager.

[0065] The environmental data analysis module is used to acquire environmental data of the target inspection area, determine the fault risk value of the equipment based on the environmental data, and calculate the ideal detection frequency of all equipment.

[0066] The detection action filtering module is used to determine the variable detection actions that need to be performed in the next detection based on the most recent detection result of any device to be detected after it has completed a detection. Based on this, the module obtains the set of detection actions for the device, calculates the detection time for the device, and finally records the device number, the set of detection actions, and the detection time.

[0067] The task allocation module is used to calculate the ideal number of legged robots to be deployed, the set of target inspection devices that each legged robot should be responsible for, and their inspection paths, based on the ideal inspection frequency of the devices to be inspected and the inspection time of all devices to be inspected, using a swarm optimization algorithm. The corresponding number of legged robots are then started to perform inspections on the corresponding set of target inspection devices in sequence according to the inspection path, and return to the starting point to complete one inspection task.

[0068] The inspection progress monitoring module is used to periodically acquire the position and inspection progress of all legged robots performing inspection tasks according to a preset inspection cycle during the inspection of the legged robots. It also detects whether there are new obstacles on the inspection path. If so, it performs obstacle removal measures. Based on the inspection time, it calculates the ideal inspection progress for each legged robot. If the actual inspection progress of any legged robot does not reach the ideal inspection progress and the progress deviation exceeds the preset deviation threshold, it triggers emergency detection measures.

[0069] The present invention has the following advantages:

[0070] 1. This invention dynamically calculates the ideal detection frequency for each device by combining changes in the device's environment and the device's historical performance with fault risk values ​​and environmental data, and selects the most suitable set of detection actions based on each detection result. In this way, the detection actions of each device are no longer fixed, but are optimized and adjusted according to actual needs, thereby avoiding unnecessary repeated detection, reducing inspection time, and improving the accuracy and efficiency of equipment monitoring.

[0071] 2. This invention intelligently allocates robot tasks through a swarm optimization algorithm, optimizing inspection paths and robot resource utilization. Based on the ideal detection frequency and detection time of each device, the swarm optimization algorithm can calculate the ideal number of robots to be deployed and allocate a suitable set of devices and inspection paths to each robot, ensuring that the robot can complete the inspection task within the specified time. Through this optimization process, the reasonable allocation of robot tasks and path optimization are ensured, minimizing robot energy consumption and improving inspection efficiency. Attached Figure Description

[0072] Figure 1 This is a schematic diagram of the structure of the legged robot intelligent inspection system using a combined unmanned aerial vehicle (UAV) in an embodiment of the present invention. Detailed Implementation

[0073] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this invention.

[0074] Example 1: A method for intelligent inspection of a legged robot combined with an unmanned aerial vehicle (UAV), comprising:

[0075] Before any legged robot performs an inspection task, a drone provides a global perspective to obtain the position of the equipment to be inspected and the legged robot within the target inspection area. The drone then performs thermal imaging scanning of the equipment to be inspected within the target inspection area using a thermal imager.

[0076] In this invention, the drone, equipped with various sensors (such as visible light cameras, thermal imagers, and lidar), provides a global view of the target inspection area, enabling the system to acquire the position and status of all devices to be inspected and inspection robots within a large area in real time. This process not only acquires the location information of the devices but also uses thermal imagers to perform thermal scanning on the devices, quickly identifying potential overheating problems and thus discovering potential equipment failure risks in advance.

[0077] Acquire environmental data of the target inspection area, determine the fault risk value of the equipment in the target inspection area based on the environmental data, and calculate the ideal inspection frequency of all equipment to be inspected;

[0078] To more accurately determine the risk of equipment failure, this invention not only relies on the equipment's historical failure records but also analyzes the environmental data of the current inspection area. The environmental data includes temperature, humidity, gas concentration, wind speed, and light intensity, which are collected and transmitted to the system in real time. Different environmental conditions have a significant impact on the operating status of equipment, and some equipment has a higher risk of failure in extreme environments such as high temperature, high humidity, and low temperature. Through real-time analysis of this environmental data, the system can better determine the risk factors in the environment in which the equipment is located.

[0079] For each device under inspection, the inspection actions include thermal imaging, visible light imaging, near-range lidar scanning, vibration sampling, gas sampling, and tactile confirmation. Visible light imaging and near-range lidar scanning are fixed inspection actions, while thermal imaging, vibration sampling, gas sampling, and tactile confirmation are variable inspection actions. In this invention, visible light imaging and near-range lidar scanning are set as fixed inspection actions primarily because the defect types corresponding to these two inspections are typically appearance defects and structural defects, which are sudden and unpredictable. Appearance defects (such as cracks, corrosion, and wear) and structural defects (such as deformation and instability) often do not exhibit a clear time-varying pattern during equipment use; they may suddenly appear at any point in time and are closely related to environmental changes or the equipment's operating condition. Therefore, visible light imaging must be performed for each inspection. Optical imaging and near-range lidar scanning ensure that appearance or structural defects of equipment can be captured at any time. These two detection actions have fixed timing and must be performed during each inspection to ensure timely detection and recording of all potential defects. In contrast, thermal imaging, vibration sampling, gas sampling, and tactile verification are considered variable detection actions because these inspections typically involve functional defects of the equipment, and these defects usually occur gradually over time. For example, overheating problems (thermal imaging), vibration problems (vibration sampling), and gas leakage problems (gas sampling) usually worsen gradually over time, and these problems generally only become apparent after the equipment has been running for a long time. Therefore, the execution of these variable actions is closely related to the equipment's historical performance, operating status, and environmental changes, and does not need to be performed during every inspection.

[0080] After any device under test completes a test, based on the device's most recent test result, the system determines the variable test actions that need to be performed in the next test, and obtains the set of test actions for that device. The system then calculates the test time for that device and finally records the device's number, the set of test actions, and the test time. After each test task is completed, the system records the execution results of each test action, such as temperature changes, vibration amplitude, and gas concentration. This data provides crucial information for assessing the device's health status. Through comprehensive analysis of these test results, the system determines whether certain variable test actions need to be re-executed in the next test. For example, if the system detects that the surface temperature is close to or exceeds a preset threshold after thermal imaging temperature measurement, it may need to perform thermal imaging temperature measurement again to confirm whether the device is continuously overheating. Similarly, for vibration sampling, if the device's vibration amplitude exceeds historical standards, it may need to perform vibration detection again in the next round of testing to further monitor the device's status.

[0081] Based on the ideal detection frequency of the devices to be tested and the detection time of all devices to be tested, the ideal number of legged robots to be deployed, the set of target devices to be tested by each legged robot and its inspection path are calculated by the swarm optimization algorithm; the corresponding number of legged robots are started, and each robot performs the detection of the corresponding set of target devices in sequence according to the inspection path, and returns to the starting point to complete one inspection task;

[0082] During the inspection tasks performed by the legged robots, the drone periodically acquires the position and inspection progress of all legged robots performing inspection tasks according to a preset inspection cycle, and detects whether there are new obstacles on the inspection path. If so, obstacle removal measures are implemented. Based on the inspection time, the ideal inspection progress of each legged robot is calculated (this progress is based on the robot's nominal walking speed and the inspection time of each device to be inspected. The ideal position that the robot should reach is calculated based on the current inspection time and the robot's inspection path). For any legged robot, if its actual inspection progress does not reach the ideal inspection progress and the progress deviation exceeds the preset deviation threshold, emergency detection measures are triggered.

[0083] This invention solves the problems of low efficiency and delayed fault detection in traditional equipment inspection methods. By combining environmental data, historical equipment performance, and group optimization algorithms, it dynamically adjusts the detection frequency and actions of the equipment, optimizes robot resource allocation, and ensures efficient and accurate inspection through path planning. This technical solution is applicable to various complex industrial scenarios, especially in environments such as large photovoltaic power plants, wind farms, and oil and gas pipelines. It can improve the real-time performance, accuracy, and resource utilization of equipment inspection, reduce the risk of faults, and enhance production safety.

[0084] Based on the environmental data, the fault risk value of the equipment in the target inspection area is determined. The specific operation is as follows:

[0085] Acquire several historical detection records of the target inspection area. Each historical detection record contains the fault detection result of a device under inspection and the environmental data when the fault occurred. The fault detection result is a binary tag, where 1 indicates that a fault exists and 0 indicates that it is normal. The environmental data includes temperature, humidity, wind speed, rainfall and light intensity.

[0086] The Z-score standardization method is used to standardize all environmental data to obtain standardized environmental data.

[0087] The standardized formula is:

[0088]

[0089] in, These are the raw values ​​of the environmental data. The mean of this feature. The standard deviation of this feature; each environmental feature is standardized individually to ensure that the mean of the data is 0 and the standard deviation is 1.

[0090] The elbow rule is used to determine the optimal number of clusters, and the specific steps are as follows:

[0091] For different cluster numbers K (e.g., from 1 to 10), the K-means clustering algorithm is used to calculate the sum of squared errors (SSE) for each K value. The sum of squared errors (SSE) is the sum of the squared distances from each data point to its cluster center after clustering, representing the density of the data point with its cluster center. The calculation formula is:

[0092]

[0093] in, The indicator function represents the data point. Does it belong to a cluster? , , It is a cluster The center It represents the total number of samples.

[0094] Plot the SSE values ​​for different K values, with the horizontal axis representing the K value and the vertical axis representing the SSE value. When the SSE decreases to an inflection point, the rate of change begins to slow down. This inflection point is the elbow point, and the corresponding K value is the optimal number of clusters.

[0095] The standardized environmental data is clustered using the K-means clustering algorithm. The specific steps are as follows:

[0096] Step 1: Randomly select K samples as initial cluster centers;

[0097] Step 2: For each historical detection record, calculate its Euclidean distance to each cluster center and assign it to the nearest cluster;

[0098] Step 3: Recalculate the mean of all standardized environmental data in each cluster and use this mean as the new cluster center;

[0099] Step 4: Repeat steps 2 and 3 until the cluster centers no longer change, at which point the algorithm converges;

[0100] Count the number of faulty devices in each cluster, divide the number by the total number of devices in the cluster, and obtain the probability of failure for that cluster.

[0101] Based on the current environmental data, calculate the Euclidean distance between the current environmental data and the center of each cluster, assign it to the nearest cluster, and use the failure probability of this cluster as the failure risk value of the current environmental data.

[0102] The ideal detection frequency for each device to be tested is calculated as follows:

[0103] Calculate the failure probability for all clusters; compare the failure probability corresponding to the current environmental data with the failure probability for all clusters, and calculate the frequency selection coefficient according to the following rules:

[0104] If the probability of failure corresponding to the current environmental data is equal to the lowest probability of failure among all clusters, then the frequency selection coefficient is 0; if the probability of failure corresponding to the current environmental data is equal to the highest probability of failure among all clusters, then the frequency selection coefficient is 1; for environmental data between the lowest and highest probabilities of failure, the frequency selection coefficient is mapped according to the proportion of the probability of failure within the linear interval to obtain the corresponding value, which is between 0 and 1.

[0105] Set the highest and lowest detection frequencies, and map the frequency selection coefficient between the lowest and highest detection frequencies using the following method:

[0106] When the frequency selection factor is 0, the ideal detection frequency of the device is the lowest detection frequency; when the frequency selection factor is 1, the ideal detection frequency of the device is the highest detection frequency; for other frequency selection factors, the ideal detection frequency of the device is calculated by linear interpolation.

[0107] Based on the device's most recent detection result, determine the variable detection actions that need to be performed in the next detection, and obtain the device's set of detection actions accordingly. The specific operation is as follows:

[0108] Visible light imaging and near-range lidar scanning are used as fixed detection actions, and a set of detection actions is added to each detection.

[0109] The detection dataset is obtained after the device under test was last subjected to each of the variable detection actions. The detection data corresponding to thermal imaging temperature measurement is the surface temperature of the device under test.

[0110] The detection data corresponding to vibration sampling is the RMS vibration value of the device under test.

[0111] The detection data corresponding to gas sampling consists of the gas concentrations of several target gases.

[0112] The tactile confirmation corresponds to the peak force and displacement.

[0113] For screening thermal imaging temperature measurement, perform the following operations:

[0114] The surface temperatures of all similar devices to the device under test are most recently acquired, and the average surface temperature of this type of device is calculated. At the same time, based on the thermal imaging scan results of the UAV, the temperature of the area where all similar devices to the device under test are located is calculated, and the average imaging temperature is calculated.

[0115] Calculate the degree of deviation between the current surface temperature of the device under test and the average surface temperature (by calculating the ratio of the current surface temperature of the device to the average surface temperature, and then calculating the difference between the ratio and 1) to obtain the first degree of deviation; calculate the degree of deviation between the temperature of the area where the device under test is located and the average imaging temperature (by calculating the ratio of the temperature of the area where the device is located to the average imaging temperature, and then calculating the difference between the ratio and 1) to obtain the second degree of deviation;

[0116] If either the first deviation degree or the second deviation degree is greater than the preset deviation threshold, then thermal imaging temperature measurement will be added to the detection action set of the current device under test (i.e., the current device under test needs to perform thermal imaging temperature measurement in the next test); otherwise, no operation will be performed.

[0117] To filter vibration samples, perform the following operations:

[0118] If the vibration RMS of the most recent sample is greater than twice the standard deviation of the historical vibration mean of the device (i.e., the vibration amplitude of the current device is significantly greater than the historical average), then the vibration sampling will be added to the detection action set (i.e., the device under test needs to perform vibration sampling in the next test); if vibration sampling has not been performed in the last two tests, then the vibration sampling also needs to be added to the detection action set.

[0119] To screen gas samples, perform the following operations:

[0120] If the concentration of any gas sampled most recently is greater than the corresponding warning threshold, then gas sampling will be added to the detection action set (i.e., the device under test needs to perform gas sampling in the next test); if the device under test has not performed gas sampling in the last two tests, then gas sampling will also be added to the detection action set.

[0121] For tactile confirmation screening, perform the following operations:

[0122] If the peak force obtained from the most recent tactile confirmation exceeds the preset threshold and the displacement exceeds the warning displacement threshold, then the tactile confirmation will be added to the detection action set of the current device to be tested.

[0123] The ideal number of legged robots to be deployed, the set of target detection devices that each legged robot should be responsible for, and their inspection paths are calculated using a swarm optimization algorithm, specifically a particle swarm optimization algorithm.

[0124] The ideal number of legged robots, the set of target detection devices that each legged robot should be responsible for, and their inspection paths are calculated using the particle swarm optimization algorithm. The specific operation is as follows:

[0125] Step 1: Read the set of detection actions and corresponding dwell times of each device in the next inspection, add up the dwell times of each action to get the inspection time of the device; based on the passable cost grid provided by the UAV, use the shortest reachable path search to obtain the path length between any two observation positions of the device, and convert it into walking time using the nominal walking speed calibrated by the legged robot; for any candidate inspection path, the inspection time is defined as the sum of the detection time of all devices in the path, the sum of the walking time between each adjacent device, and the sum of the walking time from the last device back to the first device, and the return to the first device is the criterion for the end of an inspection;

[0126] Step 2: Set the following constraints:

[0127] The time constraint is that the inspection time of any candidate inspection path does not exceed the ideal inspection interval corresponding to the ideal detection frequency.

[0128] The coverage and exclusivity constraints stipulate that all devices must be allocated and the same device can only be allocated to one legged robot.

[0129] The path constraint is that the inspection path of each legged robot is a closed path;

[0130] Step 3: Determine the initial lower bound of the deployment quantity: Calculate the average detection time of all devices; and calculate the average walking time between all devices and their nearest neighbor devices; divide the ideal detection interval by the sum of the average detection time and average walking time of a single device, and round down to obtain the upper limit of the number of devices that a single legged robot can cover in one round; divide the total number of devices by the upper limit of the number of devices and round up to obtain the initial lower bound of the deployment quantity, and use this lower bound as the initial activation quantity;

[0131] Step 4: Set up the particles and the initial population. The content of each particle includes the number of legged robots enabled, the set of devices corresponding to each legged robot, and the inspection order of each device in the set of devices (the first device is regarded as the start and end point).

[0132] Given the initial number of devices, several device sets are obtained by clustering based on the spatial distribution of the devices. Within each set, closed paths are generated by sorting by nearest neighbors. Multiple initial particles are formed by swapping devices and reversing path segments several times.

[0133] Step 5: Calculate the inspection time for each path; if the inspection time for any path exceeds the ideal detection interval, the particle is marked as infeasible; within the set of feasible particles, score and compare according to the following priority: prioritize the solution with fewer activated particles; if the number of activated particles is the same, compare the total remaining time of all paths (the remaining time of a single path is the ideal detection interval minus the inspection time of that path), and prioritize the one with the smaller total remaining time; if they are still the same, compare the total travel distance of all paths, and prioritize the one with the shorter total travel distance.

[0134] Step Six: Use position swapping and path segment reversal as the discretization update operators, and combine individual optimality and global optimality to guide the generation of new particles; the population size, number of iterations and stopping conditions are set using a preset fixed parameter table; if no feasible particles are obtained with the current number of activated particles, the number of activated particles will be increased by one after stopping, and steps four and five will be re-executed until a feasible solution is obtained;

[0135] Step 7: Output the ideal number of robots to be deployed, the set of target detection devices for each legged robot, and the inspection path; based on the above output results, generate an execution instruction for each legged robot that includes the device number sequence, arrival order, estimated arrival time, inspection time, and return-to-start action.

[0136] The rules for setting the inspection cycle are as follows:

[0137] Based on the wind speed and rainfall data obtained from the environmental data before the legged robot performs the inspection task, maximum wind speed and maximum rainfall limits are set respectively. The wind speed and rainfall are normalized based on the maximum wind speed and maximum rainfall limits, and the average value of the normalized wind speed and rainfall is calculated as the inspection cycle control value.

[0138] Set the inspection cycle control range, including the shortest inspection cycle and the longest inspection cycle, and linearly map the inspection cycle control value to the inspection cycle control range to obtain the inspection cycle in actual application;

[0139] Emergency testing measures specifically include:

[0140] If any legged robot fails to reach the ideal inspection progress during the inspection process and its deviation exceeds the preset deviation threshold, then select several devices from the target devices that the robot has not yet completed its inspection, remove them from the inspection task, and mark the removed devices as spare devices; count the numbers and locations of all spare devices, and start several additional legged robots to perform inspection tasks on the spare devices to ensure that all devices complete their tasks within the remaining inspection time;

[0141] The specific method for selecting and removing equipment is as follows: based on the degree of deviation in the inspection progress, calculate the number of equipment that the legged robot needs to remove. This number is obtained by multiplying the degree of deviation by the total number of target detection equipment of the current robot and rounding up; based on the distance between the equipment and the robot, remove the equipment sequentially starting from the equipment farthest from the current robot.

[0142] Example 2: A legged robot intelligent inspection system combining unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, it includes:

[0143] The global view acquisition module is used to obtain the position of the equipment to be inspected and the legged robot in the target inspection area by providing a global view through the UAV before any legged robot performs an inspection task. The UAV performs thermal imaging scanning on the equipment to be inspected in the target inspection area through a thermal imager.

[0144] The environmental data analysis module is used to acquire environmental data of the target inspection area, determine the fault risk value of the equipment in the target inspection area based on the environmental data, and calculate the ideal detection frequency of all equipment to be inspected.

[0145] The detection action filtering module is used to determine the variable detection actions that need to be performed in the next detection based on the most recent detection result of any device to be detected after it has completed a detection. Based on this, the module obtains the set of detection actions for the device, calculates the detection time for the device, and finally records the device number, the set of detection actions, and the detection time.

[0146] The task allocation module is used to calculate the ideal number of legged robots to be deployed, the set of target inspection devices that each legged robot should be responsible for, and their inspection paths, based on the ideal inspection frequency of the devices to be inspected and the inspection time of all devices to be inspected, using a swarm optimization algorithm. The corresponding number of legged robots are then started to perform inspections on the corresponding set of target inspection devices in sequence according to the inspection path, and return to the starting point to complete one inspection task.

[0147] The inspection progress monitoring module is used to periodically acquire the position and inspection progress of all legged robots performing inspection tasks according to a preset inspection cycle during the inspection of the legged robots. It also detects whether there are new obstacles on the inspection path. If so, it performs obstacle removal measures. Based on the inspection time, it calculates the ideal inspection progress for each legged robot. If the actual inspection progress of any legged robot does not reach the ideal inspection progress and the progress deviation exceeds the preset deviation threshold, it triggers emergency detection measures.

[0148] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A method for intelligent inspection of a legged robot combined with an unmanned aerial vehicle (UAV), characterized in that, include: Before any legged robot performs an inspection task, a drone provides a global perspective to obtain the position of the equipment to be inspected and the legged robot within the target inspection area. A thermal imager is used to perform thermal imaging scans on the equipment to be inspected in the target inspection area. Environmental data of the target inspection area is obtained. Based on the environmental data, the fault risk value of the equipment is determined, and the ideal inspection frequency of all equipment is calculated. For each device to be tested, the testing actions performed include thermal imaging, visible light imaging, near-field lidar scanning, vibration sampling, gas sampling, and tactile confirmation; among them, visible light imaging and near-field lidar scanning are fixed testing actions, while thermal imaging, vibration sampling, gas sampling, and tactile confirmation are variable testing actions. After any device to be tested completes a test, based on the device's most recent test result, determine the variable test actions that it needs to perform in the next test, and obtain the set of test actions for the device accordingly. Then, calculate the test time for the device, and finally record the device number, the set of test actions, and the test time. Based on the ideal detection frequency of the devices to be tested and the detection time of all devices to be tested, the ideal number of legged robots to be deployed, the set of target devices to be tested by each legged robot and its inspection path are calculated by the swarm optimization algorithm; the corresponding number of legged robots are started, and each robot performs the detection of the corresponding set of target devices in sequence according to the inspection path, and returns to the starting point to complete one inspection task; During the inspection missions of the legged robots, the drone periodically acquires the location and inspection progress of all legged robots performing inspection missions according to a preset inspection cycle, and detects whether there are new obstacles on the inspection path. If so, obstacle removal measures are implemented. Based on the inspection time, the ideal inspection progress of each legged robot is calculated. For any legged robot, if its actual inspection progress does not reach the ideal inspection progress and the progress deviation exceeds the preset deviation threshold, emergency detection measures are triggered.

2. The intelligent inspection method for a legged robot combined with an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Based on the environmental data, the fault risk value of the equipment in the target inspection area is determined. The specific operation is as follows: Acquire several historical detection records of the target inspection area. Each historical detection record contains the fault detection result of a device under inspection and the environmental data when the fault occurred. The fault detection result is a binary tag, where 1 indicates that a fault exists and 0 indicates that it is normal. The environmental data includes temperature, humidity, wind speed, rainfall and light intensity. The Z-score standardization method is used to standardize all environmental data to obtain standardized environmental data. Use the elbow rule to determine the optimal number of clusters; The standardized environmental data is clustered using the K-means clustering algorithm. The specific steps are as follows: Step 1: Randomly select K samples as initial cluster centers; Step 2: For each historical detection record, calculate its Euclidean distance to each cluster center and assign it to the nearest cluster; Step 3: Recalculate the mean of all standardized environmental data in each cluster and use this mean as the new cluster center; Step 4: Repeat steps 2 and 3 until the cluster centers no longer change, at which point the algorithm converges; Count the number of faulty devices in each cluster, divide the number by the total number of devices in the cluster, and obtain the probability of failure for that cluster. Based on the current environmental data, calculate the Euclidean distance between the current environmental data and the center of each cluster, assign it to the nearest cluster, and use the failure probability of this cluster as the failure risk value of the current environmental data.

3. The intelligent inspection method for a legged robot combined with an unmanned aerial vehicle (UAV) according to claim 2, characterized in that, The ideal detection frequency for each device to be tested is calculated as follows: Calculate the failure probability for all clusters; compare the failure probability corresponding to the current environmental data with the failure probability for all clusters, and calculate the frequency selection coefficient according to the following rules: If the probability of failure corresponding to the current environmental data is equal to the lowest probability of failure among all clusters, then the frequency selection coefficient is 0; if the probability of failure corresponding to the current environmental data is equal to the highest probability of failure among all clusters, then the frequency selection coefficient is 1; for environmental data between the lowest and highest probabilities of failure, the frequency selection coefficient is mapped according to the proportion of the probability of failure within the linear interval to obtain the corresponding value, which is between 0 and 1. Set the highest and lowest detection frequencies, and map the frequency selection coefficient between the lowest and highest detection frequencies using the following method: When the frequency selection factor is 0, the ideal detection frequency of the device is the lowest detection frequency; when the frequency selection factor is 1, the ideal detection frequency of the device is the highest detection frequency; for other frequency selection factors, the ideal detection frequency of the device is calculated by linear interpolation.

4. The intelligent inspection method for a legged robot combined with an unmanned aerial vehicle (UAV) according to claim 3, characterized in that, Based on the device's most recent detection result, determine the variable detection actions that need to be performed in the next detection, and obtain the device's set of detection actions accordingly. The specific operation is as follows: Visible light imaging and near-range lidar scanning are used as fixed detection actions, and a set of detection actions is added to each detection. The detection dataset is obtained after the device under test was last subjected to each of the variable detection actions. The detection data corresponding to thermal imaging temperature measurement is the surface temperature of the device under test. The detection data corresponding to vibration sampling is the RMS vibration value of the device under test. The detection data corresponding to gas sampling consists of the gas concentrations of several target gases. The tactile confirmation corresponds to the peak force and displacement. For screening thermal imaging temperature measurement, perform the following operations: The surface temperatures of all similar devices to the device under test are most recently acquired, and the average surface temperature of this type of device is calculated. At the same time, based on the thermal imaging scan results of the UAV, the temperature of the area where all similar devices to the device under test are located is calculated, and the average imaging temperature is calculated. Calculate the degree of deviation between the current surface temperature of the device under test and the average surface temperature to obtain a first degree of deviation; calculate the degree of deviation between the temperature of the area where the device under test is located and the average imaging temperature to obtain a second degree of deviation; If either the first deviation degree or the second deviation degree is greater than the preset deviation threshold, then thermal imaging temperature measurement is added to the detection action set of the current device under test; otherwise, no operation is performed. To filter vibration samples, perform the following operations: If the vibration RMS of the most recent sample is greater than twice the standard deviation of the historical vibration mean of the device, then the vibration sampling is added to the detection action set; if vibration sampling was not performed in the last two tests, then the vibration sampling also needs to be added to the detection action set. To screen gas samples, perform the following operations: If the concentration of any gas sampled most recently is greater than the corresponding warning threshold, then the gas sampling will be added to the detection action set; if the device under test has not performed gas sampling in the last two tests, then the gas sampling will also be added to the detection action set. For tactile confirmation screening, perform the following operations: If the peak force obtained from the most recent tactile confirmation exceeds the preset threshold and the displacement exceeds the warning displacement threshold, then the tactile confirmation will be added to the detection action set of the current device to be tested.

5. The intelligent inspection method for a legged robot combined with an unmanned aerial vehicle (UAV) according to claim 4, characterized in that, The ideal number of legged robots to be deployed, the set of target detection devices that each legged robot should be responsible for, and their inspection paths are calculated using a swarm optimization algorithm, specifically a particle swarm optimization algorithm.

6. The intelligent inspection method for a legged robot combined with an unmanned aerial vehicle (UAV) according to claim 5, characterized in that, The ideal number of legged robots, the set of target detection devices that each legged robot should be responsible for, and their inspection paths are calculated using the particle swarm optimization algorithm. The specific operation is as follows: Step 1: Read the set of detection actions and corresponding dwell time for each device in the next inspection, and calculate the inspection time of the device; Based on the passable cost grid provided by the UAV, use the shortest path search to calculate the path length between any two device observation positions, and convert it into travel time; The inspection time of one inspection is the inspection time of all devices in the path, the travel time between each adjacent device, and the travel time from the last device back to the first device. Step 2: Set the following constraints: The time constraint is that the time for a single inspection must not exceed the ideal inspection interval; Coverage and exclusivity constraints require that all devices must be allocated, and each device can only be allocated to one robot; The path constraint is that the inspection path of each robot is a closed path; Step 3: Determine the initial lower bound of the number of devices to be deployed: Calculate the average time spent on device inspection and the average travel time between the device and the nearest neighbor device; by dividing the ideal inspection interval by the sum of the inspection time and travel time of a single device, the upper limit of the number of devices per robot can be obtained, and then the initial number of devices to be deployed can be calculated. Step 4: Set up particles and initial population. Particles include the number of robots activated, the equipment set of each robot, and the inspection order; cluster the devices based on their spatial distribution to generate initial particles. Step 5: Calculate the inspection time for each path. If the time exceeds the ideal detection interval, the particle is not feasible. Among the feasible particles, prioritize them based on the number of activated particles, the total remaining time, and the travel distance. Step 6: Generate new particles through the particle update operator, and adjust the number of activated particles until a feasible solution is obtained; Step 7: Output the ideal number of robots to be deployed, the equipment set, and the inspection path, and generate execution instructions for each robot.

7. The intelligent inspection method for a legged robot combined with an unmanned aerial vehicle (UAV) according to claim 6, characterized in that, The rules for setting the inspection cycle are as follows: Based on the wind speed and rainfall data obtained from the environmental data before the legged robot performs the inspection task, maximum wind speed and maximum rainfall limits are set respectively. The wind speed and rainfall are normalized based on the maximum wind speed and maximum rainfall limits, and the average value of the normalized wind speed and rainfall is calculated as the inspection cycle control value. Set the inspection cycle control range, including the shortest inspection cycle and the longest inspection cycle, and linearly map the inspection cycle control value to the inspection cycle control range to obtain the inspection cycle in actual application; Emergency testing measures specifically include: If any legged robot fails to reach the ideal inspection progress during the inspection process and its deviation exceeds the preset deviation threshold, then select several devices from the target devices that the robot has not yet completed its inspection, remove them from the inspection task, and mark the removed devices as spare devices; count the numbers and locations of all spare devices, and start several additional legged robots to perform inspection tasks on the spare devices to ensure that all devices complete their tasks within the remaining inspection time; The specific method for selecting and removing equipment is as follows: based on the degree of deviation in the inspection progress, calculate the number of equipment that the legged robot needs to remove. This number is obtained by multiplying the degree of deviation by the total number of target detection equipment of the current robot and rounding up; based on the distance between the equipment and the robot, remove the equipment sequentially starting from the equipment farthest from the current robot.

8. A legged robot intelligent inspection system combining unmanned aerial vehicles (UAVs), characterized in that, The system is applied to the intelligent inspection method of a legged robot using a combined unmanned aerial vehicle (UAV) as described in any one of claims 1-7, comprising: The global view acquisition module is used to obtain the position of the equipment to be inspected and the legged robot in the target inspection area by providing a global view through the UAV before any legged robot performs an inspection task, and to perform thermal imaging scanning on the equipment to be inspected in the target inspection area through the thermal imager. The environmental data analysis module is used to acquire environmental data of the target inspection area, determine the fault risk value of the equipment based on the environmental data, and calculate the ideal detection frequency of all equipment. The detection action filtering module is used to determine the variable detection actions that need to be performed in the next detection based on the most recent detection result of any device to be detected after it has completed a detection. Based on this, the module obtains the set of detection actions for the device, calculates the detection time for the device, and finally records the device number, the set of detection actions, and the detection time. The task allocation module is used to calculate the ideal number of legged robots to be deployed, the set of target inspection devices that each legged robot should be responsible for, and their inspection paths, based on the ideal inspection frequency of the devices to be inspected and the inspection time of all devices to be inspected, using a swarm optimization algorithm. The corresponding number of legged robots are then started to perform inspections on the corresponding set of target inspection devices in sequence according to the inspection path, and return to the starting point to complete one inspection task. The inspection progress monitoring module is used to periodically acquire the position and inspection progress of all legged robots performing inspection tasks according to a preset inspection cycle during the inspection of the legged robots. It also detects whether there are new obstacles on the inspection path. If so, it performs obstacle removal measures. Based on the inspection time, it calculates the ideal inspection progress for each legged robot. If the actual inspection progress of any legged robot does not reach the ideal inspection progress and the progress deviation exceeds the preset deviation threshold, it triggers emergency detection measures.

Images