An adaptive power inspection robot and an inspection method

Abstract

The application provides an adaptive power inspection robot and an inspection method, and belongs to the field of intelligent robots. The robot comprises: a multi-modal perception data space-time normalization fusion module for generating a multi-dimensional device perception tensor; a device and task characteristic quantization module for calculating a device health quantization value, an environmental operation difficulty coefficient and a task urgency quantization value; a robot body capability state coding module for generating a robot body capability state vector; and a coupling field gradient decision module for inputting the above-mentioned quantization values, the state vector and features dynamically extracted from the perception tensor into a pre-trained coupling field model, outputting a global behavior gradient field, and decoupling into a four-wheel chassis motion instruction and a mechanical arm operation instruction. The application realizes deep real-time cooperation of movement and operation in the inspection process, solves the problems of low efficiency, slow response and poor adaptability of traditional inspection robots caused by the fragmentation of movement and operation, and improves the accuracy and safety of intelligent inspection of substations.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot technology, and in particular to an adaptive power inspection robot and inspection method. Background Technology

[0002] With the advancement of smart grid construction, the demand for intelligent and unmanned substation inspections is becoming increasingly urgent. Currently, power inspection robots are gradually being applied in substation environments. These robots are typically equipped with intelligent sensors such as visible light and infrared sensors, enabling them to move along preset routes and perform image acquisition and temperature detection on equipment. Existing technical solutions mostly employ a serial control architecture of perception, planning, and execution. This means that the robot is first controlled to move to the vicinity of the target equipment through a path planning algorithm, and then the robotic arm is controlled to perform the specified operation. While this approach achieves a certain degree of automated inspection, it treats the robot's movement and operation as two independent, sequentially executed stages.

[0003] However, the aforementioned existing technical solutions have significant drawbacks: in the complex and dynamic substation environment, the robot's movement and operational tasks are closely linked and mutually restrictive, and the serial execution mode leads to low overall inspection efficiency and insufficient real-time response. For example, when emergency equipment malfunctions require immediate attention, the robot cannot simultaneously prepare operating tools or adjust its observation posture during movement; when facing complex terrain, the deployment and operation of the robotic arm may affect the robot's movement stability. Existing technologies lack an integrated decision-making mechanism that can deeply integrate environmental conditions, task requirements, and robot capabilities, and collaboratively generate movement and operational commands in real time, making it difficult to meet the high-efficiency and high-safety intelligent inspection requirements of substations.

[0004] Therefore, this invention proposes an adaptive power inspection robot and inspection method. Summary of the Invention

[0005] This invention provides an adaptive power inspection robot and inspection method. By constructing a coupled field decision core, it solves the problems of the separation of movement and operation links and poor coordination in traditional inspection robots, and realizes the integrated perception and integrated collaborative control of the environment, task and body state.

[0006] This invention provides an adaptive power inspection robot, comprising:

[0007] The multimodal sensing data spatiotemporal normalization fusion module is used to synchronously acquire visible light images, infrared thermal images, laser point clouds and ultrasonic data of substation equipment, and perform spatiotemporal synchronization alignment. For each substation equipment, a corresponding multi-dimensional equipment sensing tensor is generated. The multi-dimensional equipment sensing tensor includes local visible light image data blocks, infrared temperature image data blocks and three-dimensional laser point cloud data fragments of the corresponding substation equipment in a unified coordinate system.

[0008] The device and task feature quantification module is used to calculate and output device health quantification values ​​and environmental operation difficulty coefficients based on multi-dimensional device perception tensors; and to parse background instructions and alarm signals to output task urgency quantification values.

[0009] The robot body capability state encoding module is used to encode the robot's spatial position, attitude, remaining battery power, manipulator joint load and temperature in real time, and generate the robot body capability state vector.

[0010] The coupled field gradient decision module is used to input the equipment health quantification value, environmental operation difficulty coefficient, task urgency quantification value, robot body capability state vector, and the correlation features dynamically extracted from the multi-dimensional equipment perception tensor into the pre-trained coupled field model, and output the global behavior gradient field; wherein the global behavior gradient field is decoupled into the real-time motion control command of the quadruped chassis and the precise operation control command of the robotic arm end effector.

[0011] Preferably, the device and task characteristic quantification module includes:

[0012] The Equipment Health Quantification Submodule is used to analyze the visible light image data block and infrared temperature image data block in the multi-dimensional equipment perception tensor, identify instrument reading deviations and abnormal temperature rise areas, and calculate and output the equipment health quantification value that characterizes the degree of equipment abnormality.

[0013] The environmental operation difficulty coefficient calculation submodule is used to analyze the three-dimensional laser point cloud data fragments in the multi-dimensional equipment perception tensor, assess the density of obstacles around the equipment, the unevenness of the ground and the narrowness of the passage, and calculate and output the environmental operation difficulty coefficient.

[0014] The task urgency quantification submodule is used to parse background commands and alarm signals, combine them with preset inspection procedure priorities, and calculate and output a task urgency quantification value.

[0015] Preferred, including:

[0016] The intelligent retrieval and dynamic matching module of the inspection case database is used to perform the following processes:

[0017] Based on the current task's equipment health quantification value, environmental operation difficulty coefficient, and task urgency quantification value, and the corresponding feature value of each historical case in the historical inspection case library, calculate the multidimensional feature similarity.

[0018] Select multiple historical cases with multidimensional feature similarity higher than a preset threshold to form the optimal reference case set;

[0019] Based on the equipment health quantification value, task urgency quantification value, environmental operation difficulty coefficient, and multi-dimensional feature similarity with all historical cases in the optimal reference case set, the comprehensive inspection priority score of the current task is obtained through weighted calculation.

[0020] The comprehensive inspection priority score is added as a new decision feature to the set of quantitative features output by the equipment and task feature quantification module.

[0021] Preferably, the coupled field gradient decision module includes:

[0022] The dynamic cost map construction submodule is used to generate a dynamic cost map based on the three-dimensional laser point cloud data fragments in the multi-dimensional device perception tensor, by integrating device location information, and marking the location of obstacles, passage costs, and observation value for different devices.

[0023] The task-capability matching degree calculation submodule is used to quantitatively analyze the current capabilities corresponding to the task requirements and the robot's own capability state vector, and output a feasible task weight vector that identifies the current feasibility of each task.

[0024] The gradient field analysis and behavior instruction generation submodule is used to take the dynamic cost map and feasible task weight vector as the real-time input of the coupled field model. The coupled field model comprehensively calculates the mobile energy consumption, operational risk, and task completion benefits, and analyzes the behavior gradient direction with the optimal comprehensive evaluation index as the global behavior gradient field. Based on the global behavior gradient field, the quadruped chassis gait sequence and the robotic arm joint trajectory are generated.

[0025] Preferred options also include:

[0026] An emergency response collaborative mapping module based on dynamic reference cases is used to execute the following when equipment malfunction is detected:

[0027] Local image features, temperature field features, and 3D point cloud features of the faulty device are extracted from the multi-dimensional device perception tensor and concatenated with the robot's own capability state vector to generate a unique fault-robot collaborative context code.

[0028] Calculate the similarity between the fault-robot collaborative scenario code and the case codes in the historical emergency response case library, and return the top K cases with the highest similarity as the scenario similarity reference case set;

[0029] By analyzing the robot approach path and robotic arm operation sequence recorded in the reference case set with similar scenarios, and using the current coupled field model, multi-dimensional device perception tensor and robot body capability state vector as constraints, a new collaborative handling trajectory adapted to the current scenario is planned.

[0030] Preferably, the process of generating fault-robot collaborative context codes specifically includes:

[0031] Obtain the latest equipment health quantification value and environmental operation difficulty coefficient of the faulty equipment from the equipment and task characteristic quantification module;

[0032] The equipment health quantification value, environmental operation difficulty coefficient and the local visual features, temperature field features and three-dimensional point cloud features of the faulty equipment extracted from the multi-dimensional equipment perception tensor are normalized and vectorized and spliced ​​together to form a fault feature vector.

[0033] The fault feature vector is concatenated with the current robot capability state vector to generate a fault-robot cooperative scenario code.

[0034] Preferably, it also includes a risk field coupling and active guidance module for human-machine collaborative operation, including:

[0035] The operator's gestures, tool types, and relative positional relationship with the target equipment are identified in real time by visual and depth sensors, and an operation intention vector is generated by parsing.

[0036] Based on the spatial location information of the device and obstacle information in the multi-dimensional device perception tensor, combined with the robot's real-time motion state and the identified operator's position, a three-dimensional spatial risk field is constructed; the risk value of each point in the three-dimensional spatial risk field is calculated by weighted superposition of the electric shock risk coefficient and the collision risk coefficient.

[0037] Based on the risk distribution along the operator's path in the three-dimensional risk field, navigation instructions to avoid high-risk areas are generated; at the same time, the robot is calculated to move to the optimal barrier pose that can provide physical safety shielding for the operator, and the robot is controlled to move to the optimal barrier pose.

[0038] Preferably, the coupled field model is trained and optimized using a deep reinforcement learning framework, and the reward function during the training process comprehensively considers task completion, movement energy consumption, operational risk and task switching overhead.

[0039] After training, the internal parameters of the coupled field model are periodically fine-tuned using task execution performance data recorded during actual inspections.

[0040] Preferably, the robot body capability state encoding module further includes:

[0041] The joint state safety margin sensing submodule is used to sense the flexible deformation of the quadrupedal joints and each joint of the robotic arm in real time through embedded sensors, output load and temperature in real time, and calculate the margin between the current state of each joint and the joint design safety threshold, as joint state safety margin data.

[0042] When generating the robot's body capability state vector, joint state safety margin data is included, so that when the coupled field gradient decision module plans actions, it will prioritize the use of joints with high margins and impose restrictions on the range of motion and output force of joints with low margins.

[0043] This invention provides an inspection method, comprising:

[0044] The system synchronously acquires visible light images, infrared thermal images, laser point clouds, and ultrasonic data from substation equipment and performs spatiotemporal synchronization alignment. For each substation device, a corresponding multi-dimensional device perception tensor is generated. The multi-dimensional device perception tensor includes local visible light image data blocks, infrared temperature image data blocks, and three-dimensional laser point cloud data fragments of the corresponding substation equipment in a unified coordinate system.

[0045] Based on the multi-dimensional device perception tensor, calculate and output the device health quantification value and the environmental operation difficulty coefficient; and use it to parse background instructions and alarm signals to output the task urgency quantification value.

[0046] Real-time encoding of the robot's spatial position, posture, remaining battery power, robotic arm joint load and temperature, generating a robot's capability state vector;

[0047] The equipment health quantification value, environmental operation difficulty coefficient, task urgency quantification value, robot body capability state vector, and the correlation features dynamically extracted from the multi-dimensional equipment perception tensor are all input into the pre-trained coupled field model, and the global behavior gradient field is output. The global behavior gradient field is decoupled into the real-time motion control command of the quadruped chassis and the precise operation control command of the robotic arm end effector.

[0048] The beneficial effects of this invention compared to existing technologies are as follows: By constructing a decision-making paradigm that couples the environment, task, and robot body, it fundamentally overcomes the core defects of existing inspection robots, which suffer from fragmented movement and operation, slow response, and poor global coordination due to the serial architecture of perception, planning, and execution. Its technical effect lies in the unified encoding and deep fusion of multimodal perception data, real-time task requirements, and the robot's own state. Using a pre-trained coupled field model, the optimal global behavioral gradient field is calculated in a single, collaborative manner and directly decoupled into real-time control commands for chassis movement and robotic arm operation. This achieves seamless connection, deep coordination, and dynamic optimal response of the robot's movement and operation in complex substation environments, significantly improving the overall efficiency, adaptability, and intelligence level of visual inspection operations.

[0049] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a diagram illustrating the core system architecture and data flow of the adaptive power inspection robot in this embodiment of the invention.

[0053] Figure 2 This is an interaction diagram of the system extended function modules in an embodiment of the present invention. Detailed Implementation

[0054] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0055] like Figure 1 As shown, this invention provides an embodiment of an adaptive power inspection robot, comprising:

[0056] The multimodal sensing data spatiotemporal normalization fusion module is used to synchronously acquire visible light images, infrared thermal images, laser point clouds and ultrasonic data of substation equipment, and perform spatiotemporal synchronization alignment. For each substation equipment, a corresponding multi-dimensional equipment sensing tensor is generated. The multi-dimensional equipment sensing tensor includes local visible light image data blocks, infrared temperature image data blocks and three-dimensional laser point cloud data fragments of the corresponding substation equipment in a unified coordinate system.

[0057] The device and task feature quantification module is used to calculate and output device health quantification values ​​and environmental operation difficulty coefficients based on multi-dimensional device perception tensors; and to parse background instructions and alarm signals to output task urgency quantification values.

[0058] The robot body capability state encoding module is used to encode the robot's spatial position, attitude, remaining battery power, manipulator joint load and temperature in real time, and generate the robot body capability state vector.

[0059] The coupled field gradient decision module is used to input the equipment health quantification value, environmental operation difficulty coefficient, task urgency quantification value, robot body capability state vector, and the correlation features dynamically extracted from the multi-dimensional equipment perception tensor into the pre-trained coupled field model, and output the global behavior gradient field; wherein the global behavior gradient field is decoupled into the real-time motion control command of the quadruped chassis and the precise operation control command of the robotic arm end effector.

[0060] In this embodiment, substation equipment refers to electrical equipment deployed in a substation that requires regular inspection, status monitoring, or emergency operation, such as transformers, circuit breakers, disconnect switches, instrument transformers, surge arresters, and various control cabinets and instrument panels.

[0061] In this embodiment, visible light images are images of the device's exterior captured by a visible light camera, used to identify device nameplates, instrument pointer readings, digital displays, indicator light status, and physical anomalies. Infrared thermal images are temperature images of the infrared radiation distribution on the device surface captured by an infrared thermal imager, used to detect thermal defects such as localized overheating and uneven temperature distribution. Laser point clouds are data sets composed of a large number of three-dimensional spatial coordinate points obtained through lidar scanning, accurately describing the three-dimensional geometry and spatial position of the device and its surrounding environment. Ultrasonic data are precise distance information based on sound wave reflection collected by an ultrasonic sensor, used for close-range ranging of the robot's surface and collision avoidance sensing during delicate operations.

[0062] In this embodiment, spatiotemporal synchronization alignment means marking a unified acquisition time for all sensor data streams using hardware synchronization signals or software timestamps, and using pre-calibrated position and attitude parameters of each sensor relative to the robot body to convert all data into a unified spatial coordinate system with the robot or a fixed point as the origin, ensuring that data from different sensors describing the same spatial object at the same time can be accurately corresponded and superimposed.

[0063] In this embodiment, generating a corresponding multi-dimensional device perception tensor for each substation device refers to defining a three-dimensional region of interest centered on the device within the spatiotemporally aligned fused data stream, based on the substation map or the device's location identified in real time. This region is then used to extract and encapsulate raw or pre-processed data from different sensors, forming a structured multi-dimensional data object. Specifically, this tensor includes: an image pixel matrix (visible light image data block) of a rectangular region centered on the device, extracted from the aligned visible light video stream; a temperature value matrix (infrared temperature image data block) of the same spatial region, extracted from the aligned infrared thermal image video stream; and all three-dimensional coordinate points falling within this three-dimensional region of interest, along with their possible accompanying reflection intensity information, extracted from the aligned laser point cloud (three-dimensional laser point cloud data fragments).

[0064] In this embodiment, the equipment health quantification value refers to a comprehensive numerical score. It is obtained by instrument identification and reading analysis of visible light image data blocks and comparing them with standard values ​​or historical normal values ​​to obtain deviations. At the same time, the temperature distribution of infrared temperature image data blocks is analyzed to identify whether there are abnormal temperature rise areas exceeding the threshold. Finally, these deviations and abnormalities are combined with preset weight rules to calculate a scalar value. The level of this value directly reflects the comprehensive abnormality or failure risk level of the equipment.

[0065] In this embodiment, the environmental operation difficulty coefficient is a quantitative evaluation value. It is calculated by analyzing three-dimensional laser point cloud data fragments, calculating the density distribution of the point cloud to assess the density of obstacles, analyzing the height variance of ground points to assess the flatness of the ground, and measuring the size of the effective passage space around the equipment to assess the narrowness of the passage. These geometric features are combined and calculated through a predefined evaluation model. The higher the coefficient, the greater the environmental challenge faced by the robot in arriving at and stably operating the equipment.

[0066] In this embodiment, the task urgency quantification value is a number that represents the priority of task execution. It is generated by parsing the background instructions from the substation monitoring system or the alarm signals automatically issued (usually with labels such as prompt, warning, and serious level), and combining them with the inspection procedures pre-existing in the robot system (which define the baseline priorities of different equipment types and different inspection items). The value directly corresponds to the urgency of the task that needs to be responded to and processed.

[0067] In this embodiment, real-time encoding of the robot's spatial position, posture, remaining battery power, manipulator joint load, and temperature to generate a robot's physical capability state vector refers to continuously collecting and processing the following information: the robot's position and three-dimensional posture angles in the global coordinate system obtained through laser SLAM or visual odometry; the remaining battery power and instantaneous power consumption read by the battery management system; and the real-time load and temperature rise data read by the force / torque sensors and temperature sensors built into each joint of the manipulator. These physical quantities are then normalized (e.g., the battery power is converted to a percentage, and the load is converted to a percentage of the rated load) and arranged in a fixed order into a one-dimensional array, thereby forming a comprehensive, structured, and machine-readable state snapshot that characterizes the robot's current physical capabilities and operational abilities.

[0068] In this embodiment, the dynamically extracted relevance features from the multi-dimensional device perception tensor mean that, during the decision-making process, not all possible features are extracted at once. Instead, based on the current decision objective (e.g., whether to plan a path or perform a switching operation), the algorithm selectively locates and calculates the most relevant information from the multi-dimensional device perception tensor in real time. For example, when the decision requires determining whether it is safe to pass through a certain area, the point cloud density features of that area are dynamically extracted; when the decision requires operating a certain knob, the precise pixel coordinates and three-dimensional spatial coordinates of the knob in the image are dynamically extracted. These features are calculated on demand, improving the system's flexibility and efficiency.

[0069] In this embodiment, the pre-trained coupled-field model refers to a machine learning model (usually a deep neural network) that is fully trained using deep reinforcement learning algorithms in a simulated substation digital twin environment or a large amount of historical operational data before the robot is actually deployed. During training, the model learns through trial and error, gradually adjusting its internal parameters to the optimal level, thereby learning to output behavioral decision-making strategies that maximize long-term cumulative rewards (such as the number of tasks completed, efficiency, and safety) under given complex environmental conditions, task requirements, and its own capability constraints. This trained model is then solidified and integrated into the robot's real-time control system.

[0070] In this embodiment, the global behavior gradient field refers to a core output generated by the pre-trained coupled field model after receiving all input information (quantized features, ontological state, dynamic features) at the current moment through internal calculations. It is a mathematical field defined on the robot's integrated state space, where each point corresponds to a possible state, and the direction of the gradient vector at that point indicates the optimal joint improvement direction that the robot should follow when simultaneously adjusting its movement (chassis) and manipulation (robotic arm) from the current state. It represents the intent or trend of high-level decision-making, rather than a specific sequence of actions.

[0071] In this embodiment, the global behavioral gradient field is decoupled into real-time motion control commands for the quadruped chassis and precise operation control commands for the robotic arm's end effector. This means that the robot's low-level motion controller receives this high-level gradient field as input. The controller internally includes kinematics calculation and trajectory planning algorithms, capable of decomposing the joint optimization direction indicated by the gradient vector in real time and transforming it into two sets of independently executable yet coordinated low-level commands: one set sends a sequence of desired angle, angular velocity, or torque commands to the joint actuators of the quadruped chassis, controlling the overall movement of the robot; the other set sends desired trajectory, posture, or force commands to the servo actuators of the robotic arm's joints and the end effector (such as grippers or tool heads), controlling the robotic arm to perform precise operations. The specific process is as follows:

[0072] Kinematics calculations are used to establish the mapping relationship between the high-level joint optimization direction and the low-level joint space changes. The robot control system stores its complete kinematic model, including the overall kinematics of the quadruped chassis (describing the relationship between the body pose and the angles of each leg joint) and the forward / inverse kinematics of the robotic arm (describing the relationship between the end effector pose and the angles of each arm joint). When the coupled field model outputs a global behavior gradient field, this field essentially defines the instantaneous optimal direction (i.e., a high-dimensional velocity vector) for how the body pose and the end effector pose should coordinately change under the robot's current integrated pose (including chassis position / attitude and robotic arm configuration). The core task of kinematics calculation is to decompose this high-dimensional pose space velocity vector into two specific sets of joint space velocity vectors through inverse kinematics calculations: one set corresponding to all driven joints of the quadruped chassis (e.g., 12 or 16 joints), and the other set corresponding to all driven joints of the robotic arm (e.g., 6 or 7 joints). This process ensures that the cooperative relationship in the high-level intent is accurately translated into the cooperative requirements of the low-level joint motion.

[0073] The trajectory planning algorithm is responsible for smoothly and feasiblely transforming the aforementioned instantaneous joint space velocity vectors into a time-continuous sequence of low-level control commands. This process includes the following steps:

[0074] Starting from the current joint position and using the calculated joint space velocity vector as the initial motion direction, and combining the robot's dynamic constraints (such as maximum joint velocity, acceleration, and torque limits) and environmental constraints (such as obstacle avoidance requirements from coupled field decisions), an algorithm (such as polynomial interpolation and online optimization) is used to plan a smooth and dynamically feasible joint angle trajectory within a short time window (e.g., 0.5 seconds in the future). This generates two independent joint angle-time curves for the chassis joint and the robotic arm joint, respectively.

[0075] Although the trajectories are generated independently, the planner ensures that the two trajectories are synchronized in time and that their execution results satisfy the cooperative objectives implicit in the high-level gradient environment. For example, if the gradient direction requires the robot to extend its arm to touch a target while moving, the planner ensures that the time when the chassis moves to the specified position precisely matches or has a specified phase relationship with the time when the end effector of the arm reaches the target point.

[0076] The planned joint angle trajectory is converted into real-time servo commands that can be directly sent to each joint actuator through position control, speed control, or torque control modes. For a quadruped chassis, this is usually the desired angle or torque sequence sent to each leg joint actuator; for a robotic arm, in addition to the servo commands sent to each arm joint, depending on the operation type (such as gripping or pressing), opening and closing commands or force control reference signals for the end effector may also be generated simultaneously.

[0077] Through the close collaboration of the aforementioned kinematic calculation and trajectory planning algorithms, the high-level, abstract joint optimization direction can be decoupled in real time and accurately and transformed into two sets of low-level joint motion commands and end-effector operation commands that are synchronized in time, coordinated in space, and feasible in dynamics, thereby converting intelligent decisions into the robot's physical actions without loss.

[0078] To accurately quantify multi-source sensing data into key decision features and provide structured, computable input for subsequent intelligent decision-making, a device and task feature quantification module is proposed, including:

[0079] The Equipment Health Quantification Submodule is used to analyze the visible light image data block and infrared temperature image data block in the multi-dimensional equipment perception tensor, identify instrument reading deviations and abnormal temperature rise areas, and calculate and output the equipment health quantification value that characterizes the degree of equipment abnormality.

[0080] The environmental operation difficulty coefficient calculation submodule is used to analyze the three-dimensional laser point cloud data fragments in the multi-dimensional equipment perception tensor, assess the density of obstacles around the equipment, the unevenness of the ground and the narrowness of the passage, and calculate and output the environmental operation difficulty coefficient.

[0081] The task urgency quantification submodule is used to parse background commands and alarm signals, combine them with preset inspection procedure priorities, and calculate and output a task urgency quantification value.

[0082] In this embodiment, analyzing the visible light image data block and infrared temperature image data block in the multi-dimensional device perception tensor, identifying instrument reading deviations and abnormal temperature rise areas, and calculating and outputting a device health quantification value that characterizes the degree of device abnormality refers to: performing image recognition processing on the visible light image data block, extracting the pointer instrument angle or digital instrument value, and comparing it with the normal operating parameter range of the device to calculate the reading deviation value; simultaneously, performing temperature analysis on the infrared temperature image data block, identifying pixel areas whose temperature is significantly higher than other areas of the device or exceeds a preset safety threshold, defining them as abnormal temperature rise areas; and weighting and fusing the severity of the reading deviation with the area, temperature peak, and other characteristics of the abnormal temperature rise area to generate a comprehensive device health quantification value, which intuitively reflects the overall abnormality level of the device in numerical form.

[0083] In this embodiment, the analysis of three-dimensional laser point cloud data fragments in the multi-dimensional device perception tensor assesses the density of obstacles around the device, ground unevenness, and the narrowness of passages, and calculates the output environmental operation difficulty coefficient. This involves: rasterizing or statistically processing the point cloud data to calculate the number of laser points per unit volume, thereby quantifying obstacle density; assessing ground unevenness by fitting a ground plane and calculating the height variance of each point to the fitted plane; assessing the narrowness of passages by analyzing the three-dimensional dimensions of the passable space between the device body and the nearest obstacle in the point cloud; and normalizing and weighting the above three evaluation indicators (obstacle density, ground unevenness, and passage narrowness) to output a quantified environmental operation difficulty coefficient. The higher the coefficient, the greater the environmental challenge for the robot to perform approach and operation tasks.

[0084] In this embodiment, parsing background instructions and alarm signals, combined with preset inspection procedure priorities, calculates and outputs a quantitative value for task urgency. This involves: parsing the instruction text or alarm signal code from the substation monitoring background to identify the task type (e.g., routine inspection, special inspection, fault investigation) and alarm level (e.g., information, warning, severe); simultaneously, querying the inspection procedure database pre-existing in the robot system to obtain the baseline priority defined in the procedure for that task type; dynamically adjusting the baseline priority according to the alarm level (e.g., increasing the priority of a severe alarm); and finally calculating a quantitative value for task urgency through a predefined mapping or calculation formula. This value directly determines the task's ranking position in the execution queue. The predefined mapping or calculation formula is a built-in, deterministic calculation rule used to integrate the basic importance of the task and the severity of the real-time alarm into a unified urgency score.

[0085] The rule works as follows: the system pre-sets a base score for each task type (such as daily inspections, special inspections, and emergency investigations), representing its importance under normal circumstances. Secondly, the system also sets a level coefficient for each alarm level (such as general alert, requiring attention, and critical alarm), with a higher coefficient indicating a greater urgency of the alarm.

[0086] The essence of the calculation formula is to adjust the task's base score upwards based on the current alarm level coefficient. Specifically:

[0087] If the alarm received is a general notification type, the urgency score of the task will be roughly equal to its base score.

[0088] If the alarm received is one that requires attention, the urgency score of the task will be increased significantly from its base score.

[0089] If a critical alert is received, the urgency score of the task will receive a very large boost, usually exceeding the base score of all regular tasks, thus allowing it to be prioritized immediately.

[0090] To introduce a dynamic matching mechanism based on historical cases and improve the adaptability and accuracy of inspection task scheduling, the following measures are proposed:

[0091] The intelligent retrieval and dynamic matching module of the inspection case database is used to perform the following processes:

[0092] Based on the current task's equipment health quantification value, environmental operation difficulty coefficient, and task urgency quantification value, and the corresponding feature value of each historical case in the historical inspection case library, calculate the multidimensional feature similarity.

[0093] Select multiple historical cases with multidimensional feature similarity higher than a preset threshold to form the optimal reference case set;

[0094] Based on the equipment health quantification value, task urgency quantification value, environmental operation difficulty coefficient, and multi-dimensional feature similarity with all historical cases in the optimal reference case set, the comprehensive inspection priority score of the current task is obtained through weighted calculation.

[0095] The comprehensive inspection priority score is added as a new decision feature to the set of quantitative features output by the equipment and task feature quantification module.

[0096] In this embodiment, the historical inspection case library is a structured database that stores complete records of all inspection tasks successfully performed by the robot in the past. Each record (i.e., a case) not only includes multi-dimensional device perception tensor data collected during task execution, calculated device health quantification values, environmental operation difficulty coefficients, task urgency quantification values, etc., but also records the robot's final operation sequence, movement path, total task completion time, and efficiency results of whether the operation was successful or not.

[0097] In this embodiment, the corresponding feature value of a historical case refers to the feature data of the same type as the current task to be decided, which is pre-calculated and archived for each stored historical case in the historical inspection case database. Specifically, it includes: the equipment health quantification value calculated when the historical case was executed, the environmental operation difficulty coefficient assessed at that time, and the corresponding task urgency quantification value.

[0098] In this embodiment, multidimensional feature similarity is calculated based on the current task's equipment health quantification value, environmental operation difficulty coefficient, and task urgency quantification value, and the corresponding feature values ​​of each historical case in the historical inspection case library. This means that the three feature values ​​of the current task are combined into a multidimensional feature vector, and the corresponding three feature values ​​of each historical case are also combined into a feature vector. By calculating the similarity metric between the current task's feature vector and the feature vector of each historical case (such as the reciprocal of Euclidean distance, cosine similarity, etc.), a one-dimensional or multi-dimensional similarity value is obtained, which is used to quantify the degree of closeness between the current scene and each historical case in the multidimensional feature space.

[0099] In this embodiment, the preset threshold is a pre-defined numerical threshold used to determine whether historical cases selected from the historical inspection case library are sufficiently similar to the current task. When the calculated multidimensional feature similarity is greater than or equal to the preset threshold, the historical case is considered to be highly relevant to the current task scenario and can be selected into the subsequent set of reference cases.

[0100] In this embodiment, the comprehensive inspection priority score for the current task is obtained through weighted calculation based on the equipment health quantification value, task urgency quantification value, environmental operation difficulty coefficient, and multidimensional feature similarity with all historical cases in the optimal reference case set. This score determines the initial weights of the three indicators—equipment health quantification value (reflecting equipment status), task urgency quantification value (reflecting the urgency of the task itself), and environmental operation difficulty coefficient (reflecting environmental complexity)—when calculating the priority. A dynamic adjustment factor related to historical experience is introduced—the average multidimensional feature similarity between the current task and all cases in the optimal reference case set (a set of highly similar cases selected from a pool of data). The higher the similarity, the more reliable the historical success experience, which can enhance or fine-tune the effectiveness of the initial weights. Through a weighted calculation model (for example, multiplying the equipment health quantification value by its weight on the impact on system safety, multiplying the task urgency quantification value by its weight on the timeliness requirement, multiplying the environmental operation difficulty coefficient by its negative weight on the operational feasibility, and then combining the weighted sum of these three factors with the average similarity factor), a single comprehensive inspection priority score is finally output, which represents the degree to which the current task should be prioritized among all pending tasks.

[0101] In this embodiment, the comprehensive inspection priority score is added as a new decision feature to the quantitative feature set output by the equipment and task feature quantification module. This means that the original output of the module includes a set of basic quantitative features such as equipment health quantification values, environmental operational difficulty coefficients, and task urgency quantification values. After introducing the inspection case library for dynamic matching, the newly generated comprehensive inspection priority score, as a high-level decision feature representing the urgency and importance of the task globally, is added to this quantitative feature set. This allows the subsequent coupled field gradient decision module, when calculating the global behavioral gradient, to make a more comprehensive and optimized decision not only based on the equipment's own abnormal conditions and environmental difficulty, but also by explicitly considering task priority information calibrated by historical experience.

[0102] To construct a dynamic cost map and task-capability matching mechanism, and to achieve behavior gradient parsing and instruction generation based on global optimization, a coupled field gradient decision module is proposed, including:

[0103] The dynamic cost map construction submodule is used to generate a dynamic cost map based on the three-dimensional laser point cloud data fragments in the multi-dimensional device perception tensor, by integrating device location information, and marking the location of obstacles, passage costs, and observation value for different devices.

[0104] The task-capability matching degree calculation submodule is used to quantitatively analyze the current capabilities corresponding to the task requirements and the robot's own capability state vector, and output a feasible task weight vector that identifies the current feasibility of each task.

[0105] The gradient field analysis and behavior instruction generation submodule is used to take the dynamic cost map and feasible task weight vector as the real-time input of the coupled field model. The coupled field model comprehensively calculates the mobile energy consumption, operational risk, and task completion benefits, and analyzes the behavior gradient direction with the optimal comprehensive evaluation index as the global behavior gradient field. Based on the global behavior gradient field, the quadruped chassis gait sequence and the robotic arm joint trajectory are generated.

[0106] In this embodiment, based on 3D laser point cloud data fragments in the multi-dimensional device perception tensor, and by fusing device location information, a dynamic cost map is generated that labels obstacle locations, passage costs, and observation values ​​for different devices. This involves: performing 3D rasterization processing on the 3D laser point cloud data fragments to divide the environment into voxel grids; analyzing each voxel to identify areas with dense point cloud data and abrupt height changes, and labeling these as obstacle locations; simultaneously, combining known device location information (i.e., 3D coordinates of devices obtained from maps or identification results) to clearly mark the spatial location of each device in the cost map; furthermore, calculating a passage cost value for each passable voxel based on the voxel's terrain slope, surface roughness, and distance from obstacles; and assigning an observation value score to each device location based on device type and current task requirements; finally, generating a dynamically updated spatial map that integrates obstacle locations, passage cost grids, and device observation value labels.

[0107] In this embodiment, the passage cost is a quantifiable value used to characterize the estimated comprehensive costs incurred by the robot when moving from a certain location to an adjacent location or staying at that location, including energy consumption, time cost, and stability risk. The specific calculation considers the slope of the ground at that location (affecting energy consumption), roughness (affecting movement smoothness and speed), and the distance between that location and the nearest obstacle (affecting safety margin and path tortuosity).

[0108] In this embodiment, the observation value of different devices is a dynamically adjusted score based on task objectives and device status, used to quantify the expected benefits of observing or operating a particular device. The calculation of this value takes into account the device type (e.g., critical main devices have a higher value than auxiliary devices), the current device health quantification value (abnormal devices have a higher value than normal devices), and task requirements (devices that the task requires to be operated have a higher value). A higher value means that the robot should prioritize planning observation or operation paths toward that device.

[0109] In this embodiment, the equipment location information refers to the precise three-dimensional coordinates of each piece of equipment in the substation under a unified world coordinate system. This information comes from a pre-recorded digital map of the substation, or is acquired and updated in real time through robot real-time perception and recognition technology (such as equipment recognition and positioning based on point clouds and images).

[0110] In this embodiment, the quantitative analysis of task requirements and the corresponding current capabilities in the robot's capability state vector outputs a feasible task weight vector that identifies the current feasibility of each task. This involves: analyzing the requirements of all tasks to be executed, such as the required positioning accuracy, the force and accuracy required for the robotic arm operation, and the estimated task time; comparing and calculating the matching degree of these requirements with the current capability parameters represented in the robot's capability state vector (such as remaining battery power, joint load margin, estimated positioning accuracy, and the current reachable workspace of the robotic arm); and calculating a feasibility weight between 0 and 1 for each task, where a higher value indicates that the robot is more capable of successfully completing the task in its current state. The feasibility weights of all tasks are arranged in order to form the feasible task weight vector.

[0111] In this embodiment, the current feasibility of each task is a specific component in the aforementioned feasible task weight vector. It is a quantified scalar value that comprehensively reflects the degree to which the robot's own capabilities and resource status (such as power, load, and manipulator flexibility) meet all the requirements for executing the task at the current moment. A feasibility of 1 indicates that the execution conditions are fully met, while a feasibility close to 0 indicates that the current conditions are severely insufficient and execution is difficult.

[0112] In this embodiment, the dynamic cost map and feasible task weight vector are used as instantaneous inputs to the coupled field model. The coupled field model comprehensively calculates movement energy consumption, operational risk, and task completion benefits, and parses the optimal behavior gradient direction as the global behavior gradient field. Based on the global behavior gradient field, a quadruped chassis gait sequence and robotic arm joint trajectory are generated. This means that after receiving the dynamic cost map (containing environmental passage cost and equipment benefit information) and feasible task weight vector (containing feasibility information for each task), the coupled field model performs multi-objective optimization calculations internally. It evaluates the robot's comprehensive performance in terms of movement energy consumption (derived from the cost map), operational risk (such as the risk of collision with obstacles or charged equipment), and potential task completion benefits (combining feasible weights and observational value) when adopting different combinations of movement and operation. The model finally outputs a global behavior gradient field, which indicates the optimization direction with respect to the robot's overall pose (including chassis position, posture, and robotic arm configuration) that maximizes the instantaneous improvement of the comprehensive evaluation index of benefit, cost, and risk at the current position and state. The underlying motion planner, based on this gradient direction, uses inverse kinematics and trajectory optimization algorithms to calculate a temporally continuous and dynamically feasible sequence of quadrupedal chassis gait parameters and the angular trajectories of each joint of the robotic arm that can change in this direction.

[0113] To achieve collaborative trajectory planning for emergency response based on dynamic case matching and contextualized coding, the following additional measures are proposed:

[0114] An emergency response collaborative mapping module based on dynamic reference cases is used to execute the following when equipment malfunction is detected:

[0115] Local image features, temperature field features, and 3D point cloud features of the faulty device are extracted from the multi-dimensional device perception tensor and concatenated with the robot's own capability state vector to generate a unique fault-robot collaborative context code.

[0116] Calculate the similarity between the fault-robot collaborative scenario code and the case codes in the historical emergency response case library, and return the top K cases with the highest similarity as the scenario similarity reference case set;

[0117] By analyzing the robot approach path and robotic arm operation sequence recorded in the reference case set with similar scenarios, and using the current coupled field model, multi-dimensional device perception tensor and robot body capability state vector as constraints, a new collaborative handling trajectory adapted to the current scenario is planned.

[0118] In this embodiment, detecting equipment abnormality means that the equipment health quantification value output by the equipment and task feature quantification module exceeds the preset safety threshold, or that the immediate visual characteristics of the fault, such as open flame, smoke, and severe deformation, are directly identified through real-time analysis of the visible light or infrared images in the multi-dimensional equipment perception tensor, thereby triggering the emergency response process.

[0119] In this embodiment, dynamic reference cases refer to one or more historical successful handling cases that are not pre-fixed or universal emergency plans, but rather retrieved in real time from a historical emergency handling case database based on the current specific fault-robot collaborative scenario code. These cases are most similar to the current situation in terms of fault characteristics and robot capability status. These cases serve as dynamic reference templates for the current planned handling strategy.

[0120] In this embodiment, local image features, temperature field features, and 3D point cloud features of the faulty device are extracted from the multi-dimensional device perception tensor and concatenated with the robot's capability state vector to generate a unique fault-robot collaborative context code. This involves: extracting a depth visual feature vector describing the appearance of the fault (e.g., cracking, ablation) from the corresponding visible light image data block centered on the faulty device; extracting a thermal image feature vector describing abnormal temperature distribution (e.g., hotspot shape, temperature gradient) from its infrared temperature image data block; and extracting a point cloud geometric feature vector describing changes in physical structure (e.g., component displacement, missing parts) from its 3D laser point cloud data fragment. Simultaneously, the current robot capability state vector is obtained. These three fault feature vectors are sequentially concatenated with the robot state vector and processed through an encoding network or hash function to ultimately generate a compact, high-dimensional digital vector, i.e., the fault-robot collaborative context code. This code uniquely represents which fault and which robot state handles this combined context.

[0121] In this embodiment, the historical emergency response case library is a dedicated data storage module, where each record stores a successful emergency response instance. Each case includes the following core information: the fault-robot collaborative scenario code generated when the case occurred, the robot's final approach path point sequence, the robotic arm's action sequence (including end-effector trajectory, operational force, etc.), and the post-response effect evaluation (e.g., whether the open flame was extinguished). This case library is dynamically updated; successful responses will generate new cases and be added to the library.

[0122] In this embodiment, calculating the similarity between the fault-robot collaborative scenario code and the scenario codes in the historical emergency response case library means treating the currently generated fault-robot collaborative scenario code as a query vector and the scenario code stored in each case in the historical case library as a target vector. By calculating metrics such as cosine similarity or Euclidean distance between vectors, the closeness between the current scenario and each historical scenario in the multi-dimensional feature space is quantified. Higher similarity indicates greater reference value of the handling experience from the historical case for the current scenario.

[0123] In this embodiment, case encoding refers to the fault-robot collaborative scenario encoding corresponding to each historical successful emergency response case stored in the historical emergency response case database. It is the digital fingerprint of the case in the feature space.

[0124] In this embodiment, the robot approach path and robotic arm operation sequence recorded in the scenario-similar reference case set refer to the motion data actually executed by the robot when the case occurred, which is directly read from the retrieved scenario-similar reference case set. The robot approach path is a sequence of three-dimensional coordinate points traversed by the robot chassis as it moves from its starting position to an operable position; the robotic arm operation sequence is the trajectory of the angle changes of each joint and the sequence of action commands from the end effector when the robotic arm completes a specific operation (such as spraying fire extinguishing agent or using an insulating rod).

[0125] In this embodiment, the robot approach path and robotic arm operation sequence recorded in a set of similar scenario reference cases are analyzed. Using the current coupled field model, multi-dimensional device perception tensor, and robot capability state vector as constraints, a new collaborative handling trajectory adapted to the current scenario is replanned. This means using the paths and operation sequences of similar reference cases as high-quality initial solutions or reference templates. The current actual environmental state (represented by the current multi-dimensional device perception tensor, which may include obstacle distributions different from historical cases), the robot's current capability state (represented by the current robot capability state vector, such as potentially lower battery power), and the latest decision preferences implied by the current coupled field model learned online are all used as optimization constraints. A trajectory optimization algorithm is used to locally adjust and replan the reference path and operation sequence, such as avoiding newly appearing obstacles, selecting a more energy-efficient movement method based on the current battery power, or adjusting the operation intensity based on the model's reassessment of risk. This generates a new collaborative handling trajectory that both draws on historical successes and is fully adapted to the current real-time scenario.

[0126] To generate highly discriminative fault-robot cooperative context codes through multi-source feature fusion and concatenation encoding, the specific process for generating fault-robot cooperative context codes includes:

[0127] Obtain the latest equipment health quantification value and environmental operation difficulty coefficient of the faulty equipment from the equipment and task characteristic quantification module;

[0128] The equipment health quantification value, environmental operation difficulty coefficient and the local visual features, temperature field features and three-dimensional point cloud features of the faulty equipment extracted from the multi-dimensional equipment perception tensor are normalized and vectorized and spliced ​​together to form a fault feature vector.

[0129] The fault feature vector is concatenated with the current robot capability state vector to generate a fault-robot cooperative scenario code.

[0130] In this embodiment, obtaining the latest equipment health quantification value and environmental operation difficulty coefficient of the faulty equipment from the equipment and task feature quantification module means that when the system detects an equipment anomaly and triggers the emergency response procedure, it immediately sends a query request to the equipment and task feature quantification module. The module responds to the request by re-running or calling the cached equipment health quantification value calculation process and environmental operation difficulty coefficient calculation process for the faulty equipment based on the multi-dimensional equipment perception tensor at the current moment (or the most recently updated one), and outputs the quantification value that accurately reflects the degree of equipment anomaly and the quantification value of the current environmental complexity at this moment. This ensures that the features used for encoding are up-to-date and strictly correspond to the abnormal state.

[0131] In this embodiment, the equipment health quantification value, environmental operation difficulty coefficient, and local visual features, temperature field features, and 3D point cloud features of the faulty equipment extracted from the multi-dimensional equipment perception tensor are normalized and vectorized and concatenated to form a fault feature vector. This involves normalizing the acquired equipment health quantification value and environmental operation difficulty coefficient (e.g., scaling them to the 0-1 range) and converting them into single-element vectors. Local visual features (such as depth feature vectors describing the appearance of the fault extracted through convolutional neural networks), temperature field features (such as spatial distribution statistical feature vectors of abnormal hot areas), and 3D point cloud features (such as geometric descriptor vectors describing structural deformation) of the faulty equipment are extracted from the multi-dimensional equipment perception tensor, and these high-dimensional feature vectors are also subjected to length normalization or standardization. Following a predetermined order (e.g., equipment health quantification value vector, environmental operation difficulty coefficient vector, local visual feature vector, temperature field feature vector, 3D point cloud feature vector), all processed feature vectors are concatenated end-to-end along the dimensional direction to form a single, high-dimensional fault feature vector. This vector integrates the quantitative indicators of the fault with multi-modal perception details.

[0132] In this embodiment, the fault feature vector and the current robot capability state vector are concatenated a second time to generate a fault-robot cooperative context code. This means that the robot capability state vector at the current moment is obtained in real time from the robot capability state encoding module. The comprehensive fault feature vector formed in the previous step and the current robot capability state vector are used as two independent input blocks. These two vectors are concatenated in a fixed order (e.g., fault feature vector first, robot state vector second) to form a higher-dimensional combined vector. This combined vector can be directly used as the final fault-robot cooperative context code, or it can be further processed through a fully connected layer or encoder network for dimensionality reduction and feature fusion to generate a more compact and representative encoded vector. This encoding simultaneously encapsulates key information about what fault occurred (What), who is responsible for handling it, and whether it can currently handle it (Who & How).

[0133] To construct a three-dimensional spatial risk field and achieve active guidance and protection, thereby improving the safety and efficiency of human-machine collaborative operations, a risk field coupling and active guidance module that also includes human-machine collaborative operation is proposed, comprising:

[0134] The operator's gestures, tool types, and relative positional relationship with the target equipment are identified in real time by visual and depth sensors, and an operation intention vector is generated by parsing.

[0135] Based on the spatial location information of the device and obstacle information in the multi-dimensional device perception tensor, combined with the robot's real-time motion state and the identified operator's position, a three-dimensional spatial risk field is constructed; the risk value of each point in the three-dimensional spatial risk field is calculated by weighted superposition of the electric shock risk coefficient and the collision risk coefficient.

[0136] Based on the risk distribution along the operator's path in the three-dimensional risk field, navigation instructions to avoid high-risk areas are generated; at the same time, the robot is calculated to move to the optimal barrier pose that can provide physical safety shielding for the operator, and the robot is controlled to move to the optimal barrier pose.

[0137] In this embodiment, the operation intention vector is generated by real-time identification of the operator's gestures, tool types, and relative positional relationship with the target device using visual and depth sensors. This involves: capturing the operator's hand postures and movement sequences using the robot's visual sensors (such as an RGB camera); parsing the intended actions (such as turning left or pressing a button) using a gesture recognition algorithm; determining the type of tool held by the operator (such as an insulating rod or a wrench) through image recognition; and accurately calculating the three-dimensional spatial coordinates and orientation of the operator's body and tool tip relative to the target device (such as a knife switch or a switch) using 3D point cloud data acquired by a depth sensor (such as an RGB-D camera). Finally, this identified discrete information (gesture category encoding, tool type encoding, and relative position vector) is digitized and concatenated or mapped into a structured one-dimensional digital vector, namely the operation intention vector, which represents the planned operation content of the operator.

[0138] In this embodiment, the device spatial location information and obstacle information in the multi-dimensional device perception tensor refer to two types of key spatial information that can be directly read or indirectly derived from the currently maintained multi-dimensional device perception tensor. Device spatial location information refers to the coordinates of the three-dimensional bounding box or key points of each electrical device in a unified world coordinate system, corresponding to the tensor index or determined through registration of point cloud data within the tensor. Obstacle information refers to the three-dimensional position and contour information of objects occupying space other than the target device (such as temporary fences, engineering vehicles, and other equipment bases) identified by processing three-dimensional laser point cloud data fragments.

[0139] In this embodiment, the robot's real-time motion state and the identified operator's position refer to: the robot's real-time motion state includes the robot chassis's instantaneous position coordinates, velocity vector, and attitude angle in a global coordinate system, typically provided by odometry, IMU, and SLAM systems. The identified operator's position refers to the three-dimensional coordinates of the operator's torso or helmet in a unified world coordinate system, sensed and calculated in real-time by the aforementioned sensors.

[0140] In this embodiment, a three-dimensional spatial risk field is constructed based on the spatial location information of the device and obstacles in the multi-dimensional device perception tensor, combined with the robot's real-time motion state and the identified operator's position. The risk value of each point in the three-dimensional spatial risk field is calculated by weighted superposition of the electric shock risk coefficient and the collision risk coefficient. This means that the robot's working space is discretized into a three-dimensional mesh. For each mesh cell, two risk components are calculated: the electric shock risk coefficient, calculated using a distance-risk decay function (the closer the distance, the higher the coefficient) based on the distance between the mesh cell and the nearest energized device (based on device location information and the energized status in the device log); and the collision risk coefficient, calculated using a distance and velocity correlation function, comprehensively considering the distance between the mesh cell and static obstacles (based on obstacle information), the distance to the predicted trajectory of the dynamic robot itself, and the distance to the operator's current position and possible direction of movement. These two coefficients are normalized and weighted summed according to preset weights to obtain the final comprehensive risk value of the mesh cell. The risk values ​​of all mesh cells constitute a three-dimensional scalar field—the three-dimensional spatial risk field.

[0141] In this embodiment, the electric shock risk coefficient is a numerical value used to quantify the risk of electric shock accidents that may occur due to the proximity of a grid cell to energized equipment. Its calculation primarily relies on the Euclidean distance between the grid cell and the known safety boundary of the energized equipment; the closer the distance, the more non-linearly the coefficient increases (e.g., the risk function is defined piecewise according to safety distance thresholds for different voltage levels set by safety regulations). The collision risk coefficient is a numerical value used to quantify the risk of physical collisions occurring to the grid cell. Its calculation comprehensively considers the distance to static obstacles within the cell, the proximity to the space area that the moving robot may occupy in the next control cycle, and the distance to the operator's current and predicted positions.

[0142] In this embodiment, generating navigation instructions to avoid high-risk areas based on the risk distribution along the operator's path in a three-dimensional spatial risk field means: simulating or predicting the possible movement path of the operator from their current position to the target operating position in the three-dimensional spatial risk field; analyzing the risk value of each grid cell traversed by the path to identify high-risk sections where the risk value exceeds the safety threshold; and generating specific navigation instructions. These instructions can be visual cues (such as highlighting a safe path on the AR interface) or voice reminders delivered to the operator via augmented reality (AR) glasses or voice broadcasts, guiding the operator to actively adjust their movement route and avoid high-risk areas.

[0143] In this embodiment, calculating the optimal barrier pose for the robot to move to provide physical safety shielding for the operator means treating the robot itself as a movable barrier within a three-dimensional risk field. The optimization objective is to find the position and orientation (i.e., pose) of the robot chassis such that, when the robot moves and remains stationary at this pose, it creates effective physical shielding between the operator and one or more major risk sources (such as electrical equipment or moving machinery), while minimizing the robot's own movement costs and introduced new risks (such as the robot blocking an escape route). This is typically transformed into a constrained optimization problem, and the solved target robot pose is the optimal barrier pose. Subsequently, the robot autonomously plans its path and moves to this pose.

[0144] To enable decision-making models to continuously self-optimize and adapt to the environment through reinforcement learning and online fine-tuning, a coupled field model is proposed for training and optimization using a deep reinforcement learning framework. The reward function during training comprehensively considers task completion, mobile energy consumption, operational risk, and task switching overhead.

[0145] After training, the internal parameters of the coupled field model are periodically fine-tuned using task execution performance data recorded during actual inspections.

[0146] In this embodiment, the coupled-field model is trained and optimized using a deep reinforcement learning framework. The reward function during training comprehensively considers task completion, movement energy consumption, operational risk, and task switching overhead. Specifically, the model is trained in a simulator simulating a substation environment, employing deep reinforcement learning algorithms (such as Proximal Policy Optimization (PPO) or Deep Deterministic Policy Gradient (DDPG)). At each training time step, the model outputs an action (i.e., the global behavior gradient) based on current environmental observations (simulated multi-dimensional equipment perception tensor, task requirements, and robot state). After the simulator executes the specific movement and operation corresponding to this action, it calculates an immediate reward value and feeds it back to the model. This reward function is designed as a weighted sum of multiple sub-items, aiming to guide the model to learn a strategy that efficiently completes the task (positive reward) while conserving energy, ensuring safety, and avoiding ineffective switching (negative penalty). Through numerous rounds of trial and error, the model iteratively updates its neural network parameters, eventually converging to an optimized strategy that maximizes long-term cumulative rewards.

[0147] In this embodiment, the reward function refers to a mathematical function in the deep reinforcement learning training framework used to evaluate the gain or loss of an agent (i.e., the coupled-field model) after taking a certain action in a certain state. It directly defines the objective that the model needs to optimize. In this embodiment, it is a scalar function, typically a linear or nonlinear weighted combination of its components: R = w1 Task completion rate -w2 Mobile power consumption - W3 Operational risk - W4 Task switching overhead, where w1, w2, w3, and w4 are preset positive weight coefficients.

[0148] In this embodiment, task completion rate refers to the positive reward component in the reward function used to measure the robot's performance on the current specified task. Its specific calculation depends on the task type: for inspection tasks, it might be the proportion of successfully reading and accurately identifying instruments; for temperature measurement tasks, it might be the reward for successfully acquiring a complete temperature image of a specified area; for switching operation assistance, it might be the reward for the number of steps correctly completed. It motivates the model to move towards task completion.

[0149] In this embodiment, mobility energy consumption refers to the negative component of the reward function used to penalize the robot for the energy consumed during movement. It is typically proportional to the estimated work done (or the integral of current consumed) by the motors at each joint of the quadruped chassis within a time step or motion cycle. It encourages the model to plan more energy-efficient and effective movement paths and gaits.

[0150] In this embodiment, operational risk refers to the negative component in the reward function used to penalize the robot for potential safety risks during operation. Its calculation comprehensively considers: the safe distance between the robotic arm's end effector and electrical equipment or personnel (the risk increases if the distance is too close), whether the force applied by the operating tool is within a safe range, and whether the current action is within a known high-risk area. It guides the model to prioritize safe operating methods and paths while completing the task.

[0151] In this embodiment, task switching overhead refers to the negative component of the reward function used to penalize the efficiency loss caused by unnecessary or excessive switching between different tasks. For example, when the model interrupts an unfinished task (such as halfway through an inspection) to perform another task, a penalty term is generated, the magnitude of which may be related to the progress of the interrupted task, the estimated time required for repositioning, and posture adjustment. It encourages the model to maintain the continuity of task execution where possible and reduce unnecessary interruptions.

[0152] In this embodiment, after training, the internal parameters of the coupled field model are periodically fine-tuned using task execution performance data recorded during actual inspections. This means that after the initial simulation training is completed and the model is deployed to the real robot, the system continuously runs and records data from each actual task execution. This data includes input features before task execution, the model's output decisions, and the final actual task completion results (such as actual time consumption, energy consumption, and success rate). Every so often (e.g., daily or weekly), this newly collected real-world data is used to perform additional supervised learning or offline reinforcement learning fine-tuning on the deployed coupled field model with a small learning rate. The goal of this fine-tuning is to make the model's decisions better adapt to the dynamic characteristics of the real environment (such as real friction and light changes) and situations that may not have been fully modeled in the simulation, thereby achieving online adaptation and continuous performance improvement of the model.

[0153] like Figure 2 As shown, to introduce joint safety margin perception and realize dynamic motion protection and resource optimization scheduling based on the robot's body state, the proposed robot body capability state encoding module also includes:

[0154] The joint state safety margin sensing submodule is used to sense the flexible deformation of the quadrupedal joints and each joint of the robotic arm in real time through embedded sensors, output load and temperature in real time, and calculate the margin between the current state of each joint and the joint design safety threshold, as joint state safety margin data.

[0155] When generating the robot's body capability state vector, joint state safety margin data is included, so that when the coupled field gradient decision module plans actions, it will prioritize the use of joints with high margins and impose restrictions on the range of motion and output force of joints with low margins.

[0156] In this embodiment, the real-time sensing of the flexible deformation, output load, and temperature of the four-legged joints and the robotic arm joints via embedded sensors means that strain gauges, torque sensors, and temperature sensors are integrated inside each motion joint of the robot (including the hip, knee, and ankle joints of the four legs and the rotational joints of the robotic arm). Strain gauges measure minute deformations of joint connectors or transmission components, indirectly reflecting the degree of joint flexibility; torque sensors directly measure the real-time torque or force output by the joint motor output shaft or reducer; and temperature sensors monitor the real-time operating temperature of the joint actuator or reducer. This sensor data is collected and preprocessed by the microcontroller built into the joint and then uploaded in real-time to the robot's central control system.

[0157] In this embodiment, calculating the margin between the current state of each joint and its design safety threshold refers to the central control system receiving sensor data (deformation ε, load L, temperature T) from each joint. For each joint, the system pre-stores its design safety threshold, including the maximum allowable deformation ε. max Maximum continuous load L max and maximum safe operating temperature T max The safety margin (S) of joint conditions is typically calculated using either the weakest link effect or a weighted average approach. A typical calculation method is to calculate the deformation margin S separately. ε =1-(ε / ε max ), load margin S L =1-(L / L max Temperature margin S T =1-(T / T max Then, the minimum value among the three is taken as the comprehensive safety margin of the joint, i.e., S = min(S ε ,S L ,S T The margin value S ranges from 0 to 1. S=1 indicates that the joint is in a completely ideal state, S close to 0 indicates that a certain indicator of the joint is close to the safety limit, and S=0 or negative indicates that the limit has been exceeded.

[0158] In this embodiment, when generating the robot's body capability state vector, joint state safety margin data is included. This allows the coupled field gradient decision module to prioritize joints with high safety margins when planning actions, and impose limitations on the motion amplitude and output force of joints with low safety margins. Specifically, during the construction of the robot's body capability state vector, in addition to conventional states such as position, attitude, and battery level, the calculated comprehensive safety margin (S) of each joint is added to the vector as a separate state component, following the joint order. When the coupled field gradient decision module makes decisions based on this state vector, its internal learning mechanism or preset optimization constraints recognize this safety margin information. During the parsing of the global behavior gradient field and the final generation of joint-level control commands (gait sequences, joint trajectories), the decision logic tends to allocate a higher proportion of motion tasks and loads to joints with high safety margins and healthy states. Simultaneously, for joints with low safety margins, the decision logic automatically imposes soft limitations, such as reducing the expected range of motion (amplitude limitation), reducing the maximum output torque command (output force limitation), or assigning them a smaller share of the load in multi-joint collaborative tasks. This enables dynamic load balancing and preventative protection based on the real-time health status of joints, extending the robot's lifespan and improving its overall safety during task execution.

[0159] This invention provides an embodiment of an inspection method, comprising:

[0160] The system synchronously acquires visible light images, infrared thermal images, laser point clouds, and ultrasonic data from substation equipment and performs spatiotemporal synchronization alignment. For each substation device, a corresponding multi-dimensional device perception tensor is generated. The multi-dimensional device perception tensor includes local visible light image data blocks, infrared temperature image data blocks, and three-dimensional laser point cloud data fragments of the corresponding substation equipment in a unified coordinate system.

[0161] Based on the multi-dimensional device perception tensor, calculate and output the device health quantification value and the environmental operation difficulty coefficient; and use it to parse background instructions and alarm signals to output the task urgency quantification value.

[0162] Real-time encoding of the robot's spatial position, posture, remaining battery power, robotic arm joint load and temperature, generating a robot's capability state vector;

[0163] The equipment health quantification value, environmental operation difficulty coefficient, task urgency quantification value, robot body capability state vector, and the correlation features dynamically extracted from the multi-dimensional equipment perception tensor are all input into the pre-trained coupled field model, and the global behavior gradient field is output. The global behavior gradient field is decoupled into the real-time motion control command of the quadruped chassis and the precise operation control command of the robotic arm end effector.

[0164] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. An adaptive power line inspection robot, characterized in that, include: The multimodal sensing data spatiotemporal normalization fusion module is used to synchronously acquire visible light images, infrared thermal images, laser point clouds and ultrasonic data of substation equipment, and perform spatiotemporal synchronization alignment. For each substation equipment, a corresponding multi-dimensional equipment sensing tensor is generated. The multi-dimensional equipment sensing tensor includes local visible light image data blocks, infrared temperature image data blocks and three-dimensional laser point cloud data fragments of the corresponding substation equipment in a unified coordinate system. The device and task feature quantification module is used to calculate and output device health quantification values ​​and environmental operation difficulty coefficients based on multi-dimensional device perception tensors; and to parse background instructions and alarm signals to output task urgency quantification values. The robot body capability state encoding module is used to encode the robot's spatial position, attitude, remaining battery power, manipulator joint load and temperature in real time, and generate the robot body capability state vector. The coupled field gradient decision module is used to input the device health quantification value, environmental operation difficulty coefficient, task urgency quantification value, robot body capability state vector, and the associated features dynamically extracted from the multi-dimensional device perception tensor into the pre-trained coupled field model, and output the global behavior gradient field. The global behavior gradient field is decoupled into real-time motion control commands for the quadruped chassis and precise operation control commands for the robotic arm's end effector. The coupled field gradient decision module includes a dynamic cost map construction submodule, which is used to generate a dynamic cost map that marks the location of obstacles, the cost of passage, and the observation value of different devices, based on the three-dimensional laser point cloud data fragments in the multi-dimensional device perception tensor and by fusing device location information. The task-capability matching degree calculation submodule is used to quantitatively analyze the current capabilities corresponding to the task requirements and the robot's own capability state vector, and output a feasible task weight vector that identifies the current feasibility of each task. The gradient field analysis and behavior instruction generation submodule is used to take the dynamic cost map and feasible task weight vector as the real-time input of the coupled field model. The coupled field model comprehensively calculates the mobile energy consumption, operational risk, and task completion benefits, and analyzes the behavior gradient direction with the optimal comprehensive evaluation index as the global behavior gradient field. Based on the global behavior gradient field, the quadruped chassis gait sequence and the robotic arm joint trajectory are generated.

2. The adaptive power inspection robot according to claim 1, characterized in that, The device and task characteristic quantification module includes: The Equipment Health Quantification Submodule is used to analyze the visible light image data block and infrared temperature image data block in the multi-dimensional equipment perception tensor, identify instrument reading deviations and abnormal temperature rise areas, and calculate and output the equipment health quantification value that characterizes the degree of equipment abnormality. The environmental operation difficulty coefficient calculation submodule is used to analyze the three-dimensional laser point cloud data fragments in the multi-dimensional equipment perception tensor, assess the density of obstacles around the equipment, the unevenness of the ground and the narrowness of the passage, and calculate and output the environmental operation difficulty coefficient. The task urgency quantification submodule is used to parse background commands and alarm signals, combine them with preset inspection procedure priorities, and calculate and output a task urgency quantification value.

3. The adaptive power inspection robot according to claim 1, characterized in that, include: The intelligent retrieval and dynamic matching module of the inspection case database is used to perform the following processes: Based on the current task's equipment health quantification value, environmental operation difficulty coefficient, and task urgency quantification value, and the corresponding feature value of each historical case in the historical inspection case library, calculate the multidimensional feature similarity. Select multiple historical cases with multidimensional feature similarity higher than a preset threshold to form the optimal reference case set; Based on the equipment health quantification value, task urgency quantification value, environmental operation difficulty coefficient, and multi-dimensional feature similarity with all historical cases in the optimal reference case set, the comprehensive inspection priority score of the current task is obtained through weighted calculation. The comprehensive inspection priority score is added as a new decision feature to the set of quantitative features output by the equipment and task feature quantification module.

4. The adaptive power inspection robot according to claim 1, characterized in that, Also includes: An emergency response collaborative mapping module based on dynamic reference cases is used to execute the following when equipment malfunction is detected: Local image features, temperature field features, and 3D point cloud features of the faulty device are extracted from the multi-dimensional device perception tensor and concatenated with the robot's own capability state vector to generate a unique fault-robot collaborative context code. Calculate the similarity between the fault-robot collaborative scenario code and the case codes in the historical emergency response case library, and return the top K cases with the highest similarity as the scenario similarity reference case set; By analyzing the robot approach path and robotic arm operation sequence recorded in the reference case set with similar scenarios, and using the current coupled field model, multi-dimensional device perception tensor and robot body capability state vector as constraints, a new collaborative handling trajectory adapted to the current scenario is planned.

5. The adaptive power inspection robot according to claim 4, characterized in that, The process of generating fault-robot collaborative context coding specifically includes: Obtain the latest equipment health quantification value and environmental operation difficulty coefficient of the faulty equipment from the equipment and task characteristic quantification module; The equipment health quantification value, environmental operation difficulty coefficient and the local visual features, temperature field features and three-dimensional point cloud features of the faulty equipment extracted from the multi-dimensional equipment perception tensor are normalized and vectorized and spliced ​​together to form a fault feature vector. The fault feature vector is concatenated with the current robot capability state vector to generate a fault-robot cooperative scenario code.

6. The adaptive power inspection robot according to claim 1, characterized in that, It also includes a risk field coupling and active guidance module for human-machine collaborative operation, including: The operator's gestures, tool types, and relative positional relationship with the target equipment are identified in real time by visual and depth sensors, and an operation intention vector is generated by parsing. Based on the spatial location information of the device and obstacle information in the multi-dimensional device perception tensor, combined with the robot's real-time motion state and the identified operator's position, a three-dimensional spatial risk field is constructed; the risk value of each point in the three-dimensional spatial risk field is calculated by weighted superposition of the electric shock risk coefficient and the collision risk coefficient. Based on the risk distribution along the operator's path in the three-dimensional risk field, navigation instructions to avoid high-risk areas are generated; at the same time, the robot is calculated to move to the optimal barrier pose that can provide physical safety shielding for the operator, and the robot is controlled to move to the optimal barrier pose.

7. The adaptive power inspection robot according to claim 1, characterized in that, The coupled field model is trained and optimized using a deep reinforcement learning framework. The reward function during the training process comprehensively considers task completion, movement energy consumption, operational risk, and task switching overhead. After training, the internal parameters of the coupled field model are periodically fine-tuned using task execution performance data recorded during actual inspections.

8. The adaptive power inspection robot according to claim 1, characterized in that, The robot's capability state coding module also includes: The joint state safety margin sensing submodule is used to sense the flexible deformation of the quadrupedal joints and each joint of the robotic arm in real time through embedded sensors, output load and temperature in real time, and calculate the margin between the current state of each joint and the joint design safety threshold, as joint state safety margin data. When generating the robot's body capability state vector, joint state safety margin data is included, so that when the coupled field gradient decision module plans actions, it will prioritize the use of joints with high margins and impose restrictions on the range of motion and output force of joints with low margins.

9. An inspection method for an adaptive power line inspection robot, characterized in that, include: The system synchronously acquires visible light images, infrared thermal images, laser point clouds, and ultrasonic data from substation equipment and performs spatiotemporal synchronization alignment. For each substation device, a corresponding multi-dimensional device perception tensor is generated. The multi-dimensional device perception tensor includes local visible light image data blocks, infrared temperature image data blocks, and three-dimensional laser point cloud data fragments of the corresponding substation equipment in a unified coordinate system. Based on the multi-dimensional device perception tensor, calculate and output the device health quantification value and the environmental operation difficulty coefficient; and use it to parse background instructions and alarm signals to output the task urgency quantification value. Real-time encoding of the robot's spatial position, posture, remaining battery power, robotic arm joint load and temperature, generating a robot's capability state vector; The equipment health quantification value, environmental operation difficulty coefficient, task urgency quantification value, robot body capability state vector, and the correlation features dynamically extracted from the multi-dimensional equipment perception tensor are all input into the pre-trained coupled field model, and the global behavior gradient field is output. The global behavioral gradient field is decoupled into real-time motion control commands for the quadruped chassis and precise operation control commands for the robotic arm's end effector, including: Based on the 3D laser point cloud data fragments in the multi-dimensional device perception tensor, and by integrating device location information, a dynamic cost map is generated that marks the location of obstacles, the cost of passage, and the observation value of different devices. Quantitatively analyze the current capabilities corresponding to the task requirements and the robot's own capability state vector, and output a feasible task weight vector that identifies the current feasibility of each task. The dynamic cost map and feasible task weight vector are used as real-time inputs to the coupled field model. The coupled field model comprehensively calculates the mobile energy consumption, operational risk, and task completion benefits, and analyzes the optimal behavior gradient direction of the comprehensive evaluation index as the global behavior gradient field. Based on the global behavior gradient field, the quadruped chassis gait sequence and the robotic arm joint trajectory are generated.

Images