Man-machine interaction safety control method and system for power distribution network emergency inspection quadruped robot

Abstract

A method and system for human-machine interaction safety control of a quadruped robot for emergency inspection of power distribution networks includes: acquiring interactive commands issued by operators to the robot and extracting intent data sets; constructing a spatiotemporal three-dimensional grid map of the environment; based on the intent data sets, performing action path planning through a path planning algorithm to generate an initial execution path sequence; conducting multi-dimensional safety risk quantification assessment of the path points in the initial execution path sequence and calculating a comprehensive risk index; if the comprehensive risk index of all path points is lower than a safety threshold, the initial execution path sequence is converted into commands and executed; otherwise, the cost function of the path planning algorithm is changed, and an alternative path sequence is regenerated; if the comprehensive risk index of all path points in the alternative path sequence is lower than a safety threshold, the alternative execution path sequence is converted into commands and executed; otherwise, an early warning is issued. This invention achieves efficient and safe operation in complex and dangerous environments.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, and more specifically, relates to a human-machine interaction safety control method and system for a quadruped robot for emergency inspection of power distribution networks. Background Technology

[0002] With the deepening of smart grid construction, the reliance on automation and intelligent equipment for distribution network operation and maintenance is increasing. Quadruped robots, with their excellent adaptability to unstructured terrain, high mobility, and multi-degree-of-freedom operation potential, are gradually becoming an important vehicle for emergency inspection of distribution networks in complex urban environments. These robots can perform tasks in areas where traditional wheeled or tracked platforms cannot pass, such as narrow alleys, stairs, mud, or gravel, significantly improving fault response speed and operational coverage. Under the framework of human-machine collaborative operation, the current mainstream control mode mainly relies on preset command sequences or remote teleoperation, and its core logic is based on the assumption that the operator has complete control over the environment and the task.

[0003] Emergency inspection scenarios in power distribution networks place extremely high demands on the safety, real-time performance, and semantic understanding capabilities of human-machine interaction. Natural language, as the most intuitive way for humans to express commands, has not yet been effectively integrated into existing robot control systems. Current systems generally use buttons, joysticks, or structured command-line input, which not only results in low interaction efficiency but also easily leads to misoperation in emergency situations involving high voltage, strong electromagnetic interference, or obstructed vision. More critically, existing control architectures lack deep semantic analysis and risk prediction mechanisms for commands. When operators, due to nervousness or lack of information, issue vague or high-risk commands such as "approach the 10kV switchgear for inspection" or "return after crossing the work area," the robot cannot combine real-time environmental perception, its own status, and power grid safety regulations to conduct autonomous risk assessment. It often mechanically executes commands, leading to intrusion into energized restricted areas, collisions with high-voltage equipment, or interference with on-site personnel, posing serious safety hazards.

[0004] Existing technologies also have significant shortcomings in environmental perception and safety control. Although some systems have introduced LiDAR and visual sensors for obstacle detection, their identification and modeling of static obstacles, dynamic obstacles (such as mobile maintenance vehicles and workers), and terrain undulations remain at the basic detection stage, lacking accurate prediction of dynamic target trajectories and quantitative assessment of terrain traversability. Meanwhile, robot self-status monitoring is mostly limited to single threshold alarms such as battery level or joint overheating, failing to construct a comprehensive safety status model integrating multi-source operational indicators. At the command execution level, systems typically adopt a binary decision of "full execution" or "full rejection," lacking the ability to intelligently correct medium-risk commands—neither dynamically adjusting the path, speed, or gait to avoid risks while ensuring the task intent remains unchanged, nor adaptively adjusting the perception frequency and motion parameters according to terrain complexity and obstacle threat levels. These deficiencies collectively make it difficult for existing quadruped robots to achieve "efficient, flexible, and safe" coordinated control in power grid emergency scenarios, urgently requiring an intelligent safety control method and system that deeply integrates natural language understanding, multimodal environmental perception, dynamic risk quantification, and adaptive command correction. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a human-machine interaction safety control method and system for a quadruped robot for emergency inspection of power distribution networks.

[0006] The present invention adopts the following technical solution.

[0007] The first aspect of this invention proposes a human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks, specifically as follows: The robot acquires interactive commands issued by the operator, performs semantic parsing on the commands, and extracts the task intent, operation object, and execution parameters contained in the commands to form an intent data set. Based on the sensors on the robot and the pre-set digital twin map data of the power distribution network environment, a spatiotemporal three-dimensional grid map of the environment is constructed, which includes equipment information, topological relationships and real-time dynamic obstacles. Based on the intent data set, an action path is planned in the spatiotemporal 3D grid map of the environment using a path planning algorithm to generate an initial execution path sequence; By combining the environmental spatiotemporal 3D raster map, a multi-dimensional safety risk quantification assessment is performed on each path point in the initial execution path sequence, and a comprehensive risk index is calculated. If the comprehensive risk index of all path points is lower than the set safety threshold, the initial execution path sequence is converted into robot drive instructions and executed. Otherwise, the cost function of the path planning algorithm is changed, the action path is re-planned, an alternative path sequence is generated, and the comprehensive risk index of each path point in the alternative path sequence is calculated. If the comprehensive risk index of all path points in the alternative path sequence is lower than the set safety threshold, the alternative execution path sequence is converted into robot drive instructions and executed; otherwise, an early warning is issued.

[0008] Preferably, the step of semantically parsing the interactive instructions and extracting the task intent, operation object, and execution parameters contained in the instructions to form an intent data set specifically involves: The robot collects speech signals using an omnidirectional microphone array, and uses a pre-trained deep neural network acoustic and language model to recognize the speech signals and convert them into a text sequence. Then, a natural language understanding model pre-trained using a corpus in the field of power distribution network inspection is used to perform entity recognition and intent classification on the text sequence, extracting the core verbs in the instructions as task intents. The core verbs are intent verbs set in the corpus in the field of power distribution network inspection. The names or numbers of the power distribution equipment are identified as the operation objects, and the distance, angle and / or inspection mode contained in the instructions are identified as execution parameters.

[0009] Preferably, the interactive instructions also include gestures. The depth camera mounted on the robot's head captures a sequence of gesture images of the operator. A three-dimensional convolutional neural network model is used to identify the gesture image sequence and classify it into predefined types, including no gesture instruction, pointing, stop, confirmation, and cancellation. If the gesture instruction is pointing, the three-dimensional spatial coordinates of the end of the gesture are bound to the operation object in the voice instruction. If it is stop, confirmation, or cancellation, the robot is controlled to perform the corresponding instruction.

[0010] Preferably, the construction of an environmental spatiotemporal three-dimensional grid map containing device semantic information, topological relationships, and real-time dynamic obstacles specifically includes: The robot acquires 3D point cloud data of the surrounding environment through LiDAR, acquires depth images of the scene through binocular depth cameras, acquires the robot's posture and angular velocity information through inertial measurement unit, and acquires the robot's global position coordinates through positioning module. The robot's six-degree-of-freedom pose is estimated and optimized in real time using a tightly coupled LiDAR-inertial-visual odometry method. The point cloud data of consecutive frames are stitched together to generate a real-time local 3D environment point cloud map, in which the positions of obstacles are marked. Load a pre-built digital twin map of the power distribution network environment. The digital twin map contains equipment information of all devices and electrical connection topology between devices. The equipment information includes equipment type, 3D model, equipment number, voltage level and functional area division. Equipment types include electrical equipment, non-electrical static equipment and non-electrical dynamic equipment. The equipment information of dynamic equipment also includes the equipment movement path. The point cloud registration algorithm is used to align the local 3D environment point cloud map with the digital twin map. The device information and electrical connection topology between devices in the digital twin map are marked on the corresponding point cloud clusters in the aligned local 3D environment point cloud map. The obstacles on the corresponding point cloud clusters are matched with specific devices, and other unmatched obstacles are identified as personnel. An octree data structure is used to voxelize the labeled local 3D environmental point cloud map to construct a spatiotemporal 3D raster map of the environment.

[0011] Preferably, the path points include three-dimensional coordinates, robot angles, and robot speeds. The robot's current six-degree-of-freedom pose and current speed are concatenated to form the initial path point. Target points are generated based on the target object and execution parameters. The target point, along with a set endpoint orientation angle and a set robot speed when reaching the target object, is concatenated to form the termination path point. The endpoint orientation angle and robot speed are set based on the task intent. An improved A* algorithm is used to generate path points from the initial path point to the termination path point as the initial execution path sequence. The cost function of the A* algorithm at candidate path point n is the actual cumulative cost from the initial path point to candidate path point n plus the heuristic cost from candidate path point n to the termination path point. When searching for the next path point, the A* algorithm prioritizes searching for the candidate path point with the smallest cost function. For the current path point, candidate path points include all adjacent voxel points of the local three-dimensional environment point cloud map.

[0012] Preferably, during the search, the A* algorithm searches for the location of the path point. The robot's angle and speed do not affect the cost function. The robot's speed is the base speed set in the current inspection mode divided by the set speed coefficient multiplied by the terrain coefficient of the corresponding path point. When the safety penalty item of the corresponding path point exceeds the set penalty item threshold, the set speed coefficient is 2; otherwise, it is 1. The robot angle is set to the tangent direction from the previous path point to the corresponding path.

[0013] Preferably, the actual cumulative cost from the initial path point to the candidate path point n is: the actual cumulative cost from the initial path point to the current path point plus the single-step cost from the current path point to the candidate path point n. The single-step cost from the current path point to the candidate path point n is the Euclidean distance between the position corresponding to the current path point and the position corresponding to the candidate path point n multiplied by the terrain coefficient. The terrain coefficient is set according to the voxel ground slope of the candidate path point n. When the voxel ground slope is greater than the set angle, it is the first coefficient; otherwise, it is the second coefficient. The first coefficient is greater than the second coefficient. The heuristic cost from candidate path point n to the termination path point is the Euclidean distance between the position corresponding to candidate path point n and the position corresponding to the termination path point.

[0014] Preferably, the step of combining the environmental spatiotemporal three-dimensional raster map to perform a multi-dimensional safety risk quantification assessment on each path point in the initial execution path sequence, and calculating a comprehensive risk index, is as follows: For each electrical device, calculate the difference between the minimum distance set for the corresponding type of electrical device and the actual distance between each path point of the robot and the corresponding electrical device, divided by the minimum distance set for the corresponding type of electrical device. Select the largest division result among all electrical devices, multiply the largest division result by the set risk index as the exponent, and the natural base e as the base, and perform exponential operation. The result is the electrical proximity risk value of the corresponding path point. Calculate the product of the ground slip coefficient and the robot tilt angle at the corresponding path point. When the product is greater than a preset threshold, the instability risk value of the corresponding path point is the product multiplied by the instability coefficient; otherwise, it is a fixed value. Calculate the collision probability of each type of non-electric obstacle at the corresponding path point according to the type of non-electric obstacle, and select the highest collision probability as the collision risk value of the corresponding path point. The types of non-electric obstacles include people, non-electric static equipment and non-electric dynamic equipment. The comprehensive risk index for the corresponding path point is obtained by weighting and summing the electrical proximity risk value, instability risk value, and collision risk value according to the set weights.

[0015] Preferably, the step of calculating the collision probability of each type of non-electric obstacle according to the type of non-electric obstacle specifically involves: The square of the minimum actual distance between each path point of the robot and all personnel is multiplied by the negative number of the personnel collision coefficient as the exponent, and the natural base e is used as the base. The result is the personnel collision risk value. The square of the minimum actual distance between each path point of the robot and all non-electrical dynamic devices is multiplied by the negative of the collision coefficient of the non-electrical dynamic devices as the exponent, and the natural base e is used as the base. The result is the collision risk value of the non-electrical dynamic devices. The difference between the set minimum collision distance and the minimum actual distance between each path point of the robot and all non-electrical stationary devices is used as the non-electrical stationary device difference value. If the non-electrical stationary device difference value is greater than or equal to 0, the non-electrical stationary device difference value is multiplied by the set non-electrical stationary device collision coefficient to obtain the non-electrical stationary device collision risk value; otherwise, the non-electrical stationary device collision risk value is 0.

[0016] Preferably, the cost function of changing the path planning algorithm is specifically: A safety penalty term is introduced into the cost function. The single-step cost edge from the current path point to the candidate path point n is the Euclidean distance between the position corresponding to the current path point and the position corresponding to the candidate path point n, multiplied by the terrain coefficient, and then multiplied by the safety penalty term. The safety penalty is calculated by multiplying the minimum distance between all electrical devices and path point n by the penalty coefficient and then adding 1.

[0017] The second aspect of this invention proposes a human-machine interaction safety control system for a quadruped robot used in emergency inspection of a power distribution network using the method described in the first aspect of this invention. The system includes a pre-defined intent data set module, an environment map generation module, an initial execution path sequence generation module, a comprehensive risk index calculation module, and a path verification module. Specifically: The intent data set module describes how to acquire interactive instructions given by operators to the robot, perform semantic parsing on the interactive instructions, and extract the task intent, operation object, and execution parameters contained in the instructions to form an intent data set. Environmental map generation module: Based on the sensors on the robot and the pre-set digital twin map data of the power distribution network environment, construct a spatiotemporal three-dimensional grid map of the environment that includes equipment information, topological relationships and real-time dynamic obstacles; Initial execution path sequence generation module: Based on the intent data set, it performs action path planning in the spatiotemporal 3D raster map of the environment through a path planning algorithm to generate an initial execution path sequence; Comprehensive Risk Index Calculation Module: Combines the environmental spatiotemporal 3D raster map to perform multi-dimensional safety risk quantification assessment on each path point in the initial execution path sequence, and calculates the comprehensive risk index; Path verification module: If the overall risk index of all path points is lower than the set safety threshold, the initial execution path sequence is converted into robot drive instructions and executed; otherwise, the cost function of the path planning algorithm is changed, the action path is re-planned, an alternative path sequence is generated, and the overall risk index of each path point in the alternative path sequence is calculated. If the overall risk index of all path points in the alternative path sequence is lower than the set safety threshold, the alternative execution path sequence is converted into robot drive instructions and executed; otherwise, an early warning is issued.

[0018] A third aspect of the invention provides an apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor performing steps of the human-machine interaction safety control method for a quadrupedal robot used in emergency inspection of a power distribution network as described in the first aspect of the invention.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, uses the steps of the human-machine interaction safety control method for a quadrupedal robot for emergency inspection of power distribution networks described in the first aspect of the present invention.

[0020] The beneficial effect of the present invention is that, compared with the prior art, the present invention can understand and execute complex and ambiguous instructions given by natural language and gestures, which significantly improves the naturalness of human-computer interaction and emergency response efficiency. By integrating real-time sensor data with a digital twin map of the power distribution network, a three-dimensional spatiotemporal grid map of the environment with semantic information is constructed, enabling the robot to have a profound understanding of the environment. A multi-dimensional safety risk quantification assessment mechanism enables the robot to conduct proactive and pre-emptive safety assessments before executing any instructions. By establishing a closed-loop process of safety intervention and interactive feedback, this invention does not simply stop the task when a risk is detected, but can intelligently generate alternative safety solutions to ensure the smooth completion of the task and achieve efficient and safe operation in complex and dangerous environments. Attached Figure Description

[0021] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0023] like Figure 1 As shown, Embodiment 1 of the present invention proposes a human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks, specifically as follows: The robot acquires interactive commands issued by the operator, performs semantic parsing on the commands, and extracts the task intent, operation object, and execution parameters contained in the commands to form an intent data set. Based on the sensors on the robot and the pre-set digital twin map data of the power distribution network environment, a spatiotemporal three-dimensional grid map of the environment is constructed, which includes equipment information, topological relationships and real-time dynamic obstacles. Based on the intent data set, an action path is planned in the spatiotemporal 3D grid map of the environment using a path planning algorithm to generate an initial execution path sequence; By combining the environmental spatiotemporal 3D raster map, a multi-dimensional safety risk quantification assessment is performed on each path point in the initial execution path sequence, and a comprehensive risk index is calculated. If the comprehensive risk index of all path points is lower than the set safety threshold, the initial execution path sequence is converted into robot drive instructions and executed. Otherwise, the cost function of the path planning algorithm is changed, the action path is re-planned, an alternative path sequence is generated, and the comprehensive risk index of each path point in the alternative path sequence is calculated. If the comprehensive risk index of all path points in the alternative path sequence is lower than the set safety threshold, the alternative execution path sequence is converted into robot drive instructions and executed; otherwise, an early warning is issued.

[0024] In this preferred embodiment, the step of semantically parsing the interactive instructions and extracting the task intent, operation object, and execution parameters contained in the instructions to form an intent data set specifically involves: The robot collects speech signals using an omnidirectional microphone array, and uses a pre-trained deep neural network acoustic and language model to recognize the speech signals and convert them into a text sequence. Then, a natural language understanding model pre-trained using a corpus in the field of power distribution network inspection is used to perform entity recognition and intent classification on the text sequence, extracting the core verbs in the instructions as task intents. The core verbs are intent verbs set in the corpus in the field of power distribution network inspection. The names or numbers of the power distribution equipment are identified as the operation objects, and the distance, angle and / or inspection mode contained in the instructions are identified as execution parameters.

[0025] It should be noted that the omnidirectional microphone array consists of eight high-sensitivity electret condenser microphones, evenly distributed in a ring on the top of the robot's torso, with a sampling frequency of 48 kHz and a quantization depth of 16 bits. The depth camera uses structured light, with a frame rate of 30 frames per second, a resolution of 640 x 480 pixels, a depth measurement range of 0.5 meters to 5 meters, and an accuracy of ±2 mm. The speech recognition model adopts an encoder-decoder architecture based on an attention mechanism. The encoder consists of a six-layer convolutional neural network and a bidirectional long short-term memory network stacked together, while the decoder is a two-layer autoregressive recurrent neural network. The output layer is connected to a language model trained on a corpus based on power distribution network inspection. This language model includes the contextual probability distribution of professional terms such as power distribution transformers, circuit breakers, disconnect switches, busbars, and cable joints, ensuring the accuracy of the recognition results in professional contexts. For example, in this embodiment, the recognized output text sequence is "Please approach the 10 kV No. 1 transformer at 0.8 meters for infrared temperature measurement." Subsequently, the text sequence was fed into a natural language understanding model, which was fine-tuned on a corpus consisting of power distribution network safety regulations, inspection logs, and operation manuals, based on a pre-trained Chinese language model architecture. The model performed a named entity recognition task, identifying "10 kV No. 1 transformer" as the object of operation, and performed an intent classification task, determining that the task intent corresponding to the verb "approach" was to approach the target equipment, and parsing out the implicit execution parameters 0.8 meters and infrared temperature measurement mode.

[0026] In this preferred embodiment, the interactive instructions also include gestures. The depth camera mounted on the robot's head captures a sequence of gesture images of the operator. A three-dimensional convolutional neural network model is used to identify the gesture image sequence and classify it into predefined types, including no gesture instruction, pointing, stop, confirmation, and cancellation. If the gesture instruction is pointing, the three-dimensional spatial coordinates of the end of the gesture are bound to the operation object in the voice instruction. If it is stop, confirmation, or cancellation, the robot is controlled to perform the corresponding instruction.

[0027] It should be noted that the timestamps of the speech signal and the gesture image sequence are aligned using a hardware synchronization signal to ensure the consistency of multimodal data in the time dimension. The acquired raw speech signal is pre-amplified, bandpass filtered, and automatically gained control processed before being sent to the subsequent speech recognition module; the gesture image sequence undergoes background segmentation, human key point detection, and hand region cropping preprocessing to form a standardized gesture input sequence. The three-dimensional convolutional neural network model consists of five three-dimensional convolutional layers and three fully connected layers.

[0028] In this preferred embodiment, the construction of an environmental spatiotemporal three-dimensional raster map containing device semantic information, topological relationships, and real-time dynamic obstacles specifically involves: The robot acquires 3D point cloud data of the surrounding environment through LiDAR, acquires depth images of the scene through binocular depth cameras, acquires the robot's posture and angular velocity information through inertial measurement unit, and acquires the robot's global position coordinates through positioning module. The robot's six-degree-of-freedom pose is estimated and optimized in real time using a tightly coupled LiDAR-inertial-visual odometry method. The point cloud data of consecutive frames are stitched together to generate a real-time local 3D environment point cloud map, in which the positions of obstacles are marked. Load a pre-built digital twin map of the power distribution network environment. The digital twin map contains equipment information of all devices and electrical connection topology between devices. The equipment information includes equipment type, 3D model, equipment number, voltage level and functional area division. Equipment types include electrical equipment, non-electrical static equipment and non-electrical dynamic equipment. The equipment information of dynamic equipment also includes the equipment movement path. The point cloud registration algorithm is used to align the local 3D environment point cloud map with the digital twin map. The device information and electrical connection topology between devices in the digital twin map are marked on the corresponding point cloud clusters in the aligned local 3D environment point cloud map. The obstacles on the corresponding point cloud clusters are matched with specific devices, and other unmatched obstacles are identified as personnel. An octree data structure is used to voxelize the labeled local 3D environmental point cloud map to construct a spatiotemporal 3D raster map of the environment.

[0029] It should be noted that the LiDAR is a 16-line mechanical rotating type with a scanning frequency of 10 Hz, an angular resolution of 0.2 degrees, and an effective ranging range of 0.1 meters to 100 meters. A binocular depth camera simultaneously acquires scene depth images, the inertial measurement unit outputs data from a three-axis accelerometer and a three-axis gyroscope at a sampling frequency of 200 Hz, and the global positioning system module outputs latitude, longitude, and altitude coordinates with sub-meter positioning accuracy. All sensor data, after being timestamped, are input into a tightly coupled state estimation algorithm engine of LiDAR-inertial-vision. This engine uses a sliding window optimization framework with a window length of two seconds, containing the most recent twenty frames of LiDAR point cloud data, four hundred frames of inertial measurement unit data, and 60 frames of depth images. The algorithm estimates the robot's six-degree-of-freedom pose in real time by minimizing the weighted sum of the LiDAR point-to-plane residual, the visual reprojection error, and the inertial pre-integration residual. The pose estimation results are used to transform the continuous frames of LiDAR point cloud to the global coordinate system and perform dynamic point cloud filtering to remove point clouds generated by moving obstacles such as pedestrians or vehicles. The filtered static point cloud is fused with the depth image to generate a high-density local 3D environment point cloud map.

[0030] The point cloud registration algorithm is an iterative nearest-point algorithm. First, it extracts planar, cylindrical and edge features from the local point cloud and matches them with corresponding features in the digital twin map. Then, it calculates the initial transformation matrix and refines the alignment accuracy through iterative optimization, ultimately achieving millimeter-level spatial alignment.

[0031] The system associates obstacles on the corresponding point cloud clusters with specific equipment, for example, labeling the point cloud cluster as "10kV No. 1 Transformer - Electrical Equipment". The fused point cloud map is then fed into an octree-based map builder, which recursively partitions the space with an initial voxel resolution of 0.1 meters. Each leaf node voxel stores the occupancy probability, equipment semantic label, equipment operating parameters (such as voltage level and temperature status), and safety attributes (such as whether it is energized or a restricted area). Dynamic obstacles, such as dynamic equipment or personnel, are assigned a temporary occupancy status. The final constructed spatiotemporal 3D raster map of the environment is stored in memory in an octree structure, supporting fast neighborhood queries and path planning access.

[0032] In this preferred embodiment, the action path planning in the spatiotemporal 3D raster map using a path planning algorithm specifically involves: The path points include 3D coordinates, robot angles, and robot velocities. The initial path point is obtained by concatenating the robot's current six-DOF pose and current velocity. Target points are generated based on the target object and execution parameters. The final path point is obtained by concatenating the target point with a set endpoint orientation angle and a set robot velocity when reaching the target object. The endpoint orientation angle and robot velocity are set based on the task intent. An improved A* algorithm is used to generate path points from the initial path point to the final path point as the initial execution path sequence. The cost function of the A* algorithm for candidate path point n is the actual cumulative cost from the initial path point to candidate path point n plus the heuristic cost from candidate path point n to the final path point. When searching for the next path point, the A* algorithm prioritizes searching for the candidate path point with the smallest cost function. For the current path point, candidate path points include all adjacent voxel points of the local 3D environment point cloud map.

[0033] In this preferred embodiment, during the search, the A* algorithm searches for the location of the path point. The robot's angle and speed do not affect the cost function. The robot's speed is the set baseline speed in the current inspection mode divided by the set speed coefficient multiplied by the terrain coefficient of the corresponding path point. When the safety penalty item of the corresponding path point exceeds the set penalty item threshold, the set speed coefficient is 2; otherwise, it is 1. The robot angle is set to the tangent direction from the previous path point to the corresponding path.

[0034] In this embodiment, the actual cumulative cost from the initial path point to the candidate path point n is: the actual cumulative cost from the initial path point to the current path point plus the single-step cost from the current path point to the candidate path point n. The single-step cost from the current path point to the candidate path point n is the Euclidean distance between the position corresponding to the current path point and the position corresponding to the candidate path point n multiplied by the terrain coefficient. The terrain coefficient is set according to the voxel ground slope of the candidate path point n. When the voxel ground slope is greater than the set angle, it is the first coefficient; otherwise, it is the second coefficient. The first coefficient is greater than the second coefficient. The heuristic cost from candidate path point n to the termination path point is the Euclidean distance between the position corresponding to candidate path point n and the position corresponding to the termination path point.

[0035] In this preferred embodiment, the step of combining the environmental spatiotemporal three-dimensional raster map to perform a multi-dimensional safety risk quantification assessment on each path point in the initial execution path sequence and calculating a comprehensive risk index is as follows: For each electrical device, calculate the difference between the minimum distance set for the corresponding type of electrical device and the actual distance between each path point of the robot and the corresponding electrical device, divided by the minimum distance set for the corresponding type of electrical device. Select the largest division result among all electrical devices, multiply the largest division result by the set risk index as the exponent, and the natural base e as the base, and perform exponential operation. The result is the electrical proximity risk value of the corresponding path point. Calculate the product of the ground slip coefficient and the robot tilt angle at the corresponding path point. When the product is greater than a preset threshold, the instability risk value of the corresponding path point is the product multiplied by the instability coefficient; otherwise, it is a fixed value. Calculate the collision probability of each type of non-electric obstacle at the corresponding path point according to the type of non-electric obstacle, and select the highest collision probability as the collision risk value of the corresponding path point. The types of non-electric obstacles include people, non-electric static equipment and non-electric dynamic equipment. The comprehensive risk index for the corresponding path point is obtained by weighting and summing the electrical proximity risk value, instability risk value, and collision risk value according to the set weights.

[0036] Specifically, the electromagnetic radiation intensity limit for the 10 kV busbar is 100 microtesla, the minimum distance is set at 0.7 meters, the ground slippage coefficient is estimated through depth camera texture analysis, and the robot's tilt angle is calculated by the inertial measurement unit with a preset threshold of 0.3. The weights of the electrical proximity risk value, instability risk value, and collision risk value are set according to the inspection mode. In this embodiment, infrared detection belongs to the conventional inspection mode, and the electrical proximity risk value, instability risk value, and collision risk value are set to 0.6, 0.2, and 0.2, respectively.

[0037] In this preferred embodiment, the step of calculating the collision probability of each type of non-electric obstacle according to the type of non-electric obstacle specifically involves: The square of the minimum actual distance between each path point of the robot and all personnel is multiplied by the negative number of the personnel collision coefficient as the exponent, and the natural base e is used as the base. The result is the personnel collision risk value. The square of the minimum actual distance between each path point of the robot and all non-electrical dynamic devices is multiplied by the negative of the collision coefficient of the non-electrical dynamic devices as the exponent, and the natural base e is used as the base. The result is the collision risk value of the non-electrical dynamic devices. The difference between the set minimum collision distance and the minimum actual distance between each path point of the robot and all non-electrical stationary devices is used as the non-electrical stationary device difference value. If the non-electrical stationary device difference value is greater than or equal to 0, the non-electrical stationary device difference value is multiplied by the set non-electrical stationary device collision coefficient to obtain the non-electrical stationary device collision risk value; otherwise, the non-electrical stationary device collision risk value is 0.

[0038] It should be noted that the minimum distances set for different types of electrical equipment and the minimum collision distances are all stored in the preset distribution network safety knowledge graph.

[0039] In this preferred embodiment, the change in the cost function of the path planning algorithm specifically refers to: A safety penalty term is introduced into the cost function. The single-step cost edge from the current path point to the candidate path point n is the Euclidean distance between the position corresponding to the current path point and the position corresponding to the candidate path point n, multiplied by the terrain coefficient, and then multiplied by the safety penalty term. The safety penalty is calculated by multiplying the minimum distance between all electrical devices and path point n by the penalty coefficient and then adding 1.

[0040] If a safe path is replanned, the human-machine interface feedback module renders a 3D scene on a high-resolution display, highlighting the original risky path in red and the alternative safe path in green, and pops up a confirmation dialog box. The voice synthesis module announces: "The instruction has a safety risk; a safe path has been replanned." The system waits for the operator to click "Confirm" or "Cancel" via the touchscreen. If the operator confirms, the alternative path is converted into a low-level drive instruction, controlling the quadruped robot to execute; if canceled, the task terminates. If replanning fails (e.g., the target point is inside a high-voltage isolation zone), the system announces: "The instruction cannot be executed while meeting safety constraints; the task has been canceled."

[0041] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0042] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0043] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks, characterized in that: The robot acquires interactive commands issued by the operator, performs semantic parsing on the commands, and extracts the task intent, operation object, and execution parameters contained in the commands to form an intent data set. Based on the sensors on the robot and the pre-set digital twin map data of the power distribution network environment, a spatiotemporal three-dimensional grid map of the environment is constructed, which includes equipment information, topological relationships and real-time dynamic obstacles. Based on the intent data set, an action path is planned in the spatiotemporal 3D grid map of the environment using a path planning algorithm to generate an initial execution path sequence; By combining the environmental spatiotemporal 3D grid map, a multi-dimensional safety risk quantification assessment is performed on each path point in the initial execution path sequence, and a comprehensive risk index is calculated. If the comprehensive risk index of all path points is lower than the set safety threshold, the initial execution path sequence is converted into robot drive instructions and executed. Otherwise, the cost function of the path planning algorithm is changed, the action path planning is redone, an alternative path sequence is generated, and the comprehensive risk index of each path point in the alternative path sequence is calculated. If the comprehensive risk index of all path points in the alternative path sequence is lower than the set safety threshold, the alternative execution path sequence is converted into robot drive instructions and executed; otherwise, an early warning is issued.

2. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 1, characterized in that: The semantic parsing of the interactive instructions, extracting the task intent, operation object, and execution parameters contained in the instructions to form an intent data set, specifically involves: The robot collects speech signals using an omnidirectional microphone array, and uses a pre-trained deep neural network acoustic and language model to recognize the speech signals and convert them into a text sequence. Then, a natural language understanding model pre-trained using a corpus in the field of power distribution network inspection is used to perform entity recognition and intent classification on the text sequence, extracting the core verbs in the instructions as task intents. The core verbs are intent verbs set in the corpus in the field of power distribution network inspection. The names or numbers of the power distribution equipment are identified as the operation objects, and the distance, angle and / or inspection mode contained in the instructions are identified as execution parameters.

3. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 2, characterized in that: The interactive commands also include gestures. The depth camera mounted on the robot's head captures a sequence of gesture images of the operator. A three-dimensional convolutional neural network model is used to identify the gesture image sequence and classify it into predefined types. These types include no gesture command, pointing, stop, confirmation, and cancellation. If the gesture command is pointing, the three-dimensional spatial coordinates of the end of the gesture are bound to the operation object in the voice command. If it is stop, confirmation, or cancellation, the robot is controlled to perform the corresponding command.

4. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 3, characterized in that: The construction of an environmental spatiotemporal 3D raster map, which includes device semantic information, topological relationships, and real-time dynamic obstacles, specifically involves: The robot acquires 3D point cloud data of the surrounding environment through LiDAR, acquires depth images of the scene through binocular depth cameras, acquires the robot's posture and angular velocity information through inertial measurement unit, and acquires the robot's global position coordinates through positioning module. The robot's six-degree-of-freedom pose is estimated and optimized in real time using a tightly coupled LiDAR-inertial-visual odometry method. The point cloud data of consecutive frames are stitched together to generate a real-time local 3D environment point cloud map, in which the positions of obstacles are marked. Load a pre-built digital twin map of the power distribution network environment. The digital twin map contains equipment information of all devices and electrical connection topology between devices. The equipment information includes equipment type, 3D model, equipment number, voltage level and functional area division. Equipment types include electrical equipment, non-electrical static equipment and non-electrical dynamic equipment. The equipment information of dynamic equipment also includes the equipment movement path. The point cloud registration algorithm is used to align the local 3D environment point cloud map with the digital twin map. The device information and electrical connection topology between devices in the digital twin map are marked on the corresponding point cloud clusters in the aligned local 3D environment point cloud map. The obstacles on the corresponding point cloud clusters are matched with specific devices, and other unmatched obstacles are identified as personnel. An octree data structure is used to voxelize the labeled local 3D environmental point cloud map to construct a spatiotemporal 3D raster map of the environment.

5. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 4, characterized in that: The process of planning action paths in a spatiotemporal 3D raster map using a path planning algorithm specifically involves: The path points include 3D coordinates, robot angles, and robot velocities. The initial path point is obtained by concatenating the robot's current six-DOF pose and current velocity. Target points are generated based on the target object and execution parameters. The final path point is obtained by concatenating the target point with a set endpoint orientation angle and a set robot velocity when reaching the target object. The endpoint orientation angle and robot velocity are set based on the task intent. An improved A* algorithm is used to generate path points from the initial path point to the final path point as the initial execution path sequence. The cost function of the A* algorithm for candidate path point n is the actual cumulative cost from the initial path point to candidate path point n plus the heuristic cost from candidate path point n to the final path point. When searching for the next path point, the A* algorithm prioritizes searching for the candidate path point with the smallest cost function. For the current path point, candidate path points include all adjacent voxel points of the local 3D environment point cloud map.

6. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 5, characterized in that: During the search, the A* algorithm searches for the location of the path point. The robot's angle and speed do not affect the cost function. The robot's speed is the base speed set in the current inspection mode divided by the set speed coefficient multiplied by the terrain coefficient of the corresponding path point. When the safety penalty of the corresponding path point exceeds the set penalty threshold, the set speed coefficient is 2; otherwise, it is 1. The robot angle is set to the tangent direction from the previous path point to the corresponding path.

7. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 5, characterized in that: The actual cumulative cost from the initial path point to the candidate path point n is: the actual cumulative cost from the initial path point to the current path point plus the single-step cost from the current path point to the candidate path point n. The single-step cost from the current path point to the candidate path point n is the Euclidean distance between the position corresponding to the current path point and the position corresponding to the candidate path point n multiplied by the terrain coefficient. The terrain coefficient is set according to the voxel ground slope of the candidate path point n. When the voxel ground slope is greater than the set angle, it is the first coefficient; otherwise, it is the second coefficient. The first coefficient is greater than the second coefficient. The heuristic cost from candidate path point n to the termination path point is the Euclidean distance between the position corresponding to candidate path point n and the position corresponding to the termination path point.

8. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 7, characterized in that: The method involves combining the environmental spatiotemporal three-dimensional raster map to perform a multi-dimensional safety risk quantification assessment on each path point in the initial execution path sequence, and calculating a comprehensive risk index, specifically as follows: For each electrical device, calculate the difference between the minimum distance set for the corresponding type of electrical device and the actual distance between each path point of the robot and the corresponding electrical device, divided by the minimum distance set for the corresponding type of electrical device. Select the largest division result among all electrical devices, multiply the largest division result by the set risk index as the exponent, and the natural base e as the base, and perform exponential operation. The result is the electrical proximity risk value of the corresponding path point. Calculate the product of the ground slip coefficient and the robot tilt angle at the corresponding path point. When the product is greater than a preset threshold, the instability risk value of the corresponding path point is the product multiplied by the instability coefficient; otherwise, it is a fixed value. Calculate the collision probability of each type of non-electric obstacle at the corresponding path point according to the type of non-electric obstacle, and select the highest collision probability as the collision risk value of the corresponding path point. The types of non-electric obstacles include people, non-electric static equipment and non-electric dynamic equipment. The comprehensive risk index for the corresponding path point is obtained by weighting and summing the electrical proximity risk value, instability risk value, and collision risk value according to the set weights.

9. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 8, characterized in that: The collision probability of each type of non-electric obstacle is calculated separately according to the type of non-electric obstacle, specifically as follows: The square of the minimum actual distance between each path point of the robot and all personnel is multiplied by the negative number of the personnel collision coefficient as the exponent, and the natural base e is used as the base. The result is the personnel collision risk value. The square of the minimum actual distance between each path point of the robot and all non-electrical dynamic devices is multiplied by the negative of the collision coefficient of the non-electrical dynamic devices as the exponent, and the natural base e is used as the base. The result is the collision risk value of the non-electrical dynamic devices. The difference between the set minimum collision distance and the minimum actual distance between each path point of the robot and all non-electrical stationary devices is used as the non-electrical stationary device difference value. If the non-electrical stationary device difference value is greater than or equal to 0, the non-electrical stationary device difference value is multiplied by the set non-electrical stationary device collision coefficient to obtain the non-electrical stationary device collision risk value; otherwise, the non-electrical stationary device collision risk value is 0.

10. The human-machine interaction safety control method for a quadruped robot for emergency inspection of power distribution networks according to claim 6, characterized in that: The cost function for changing the path planning algorithm is specifically as follows: A safety penalty term is introduced into the cost function. The single-step cost edge from the current path point to the candidate path point n is the Euclidean distance between the position corresponding to the current path point and the position corresponding to the candidate path point n, multiplied by the terrain coefficient, and then multiplied by the safety penalty term. The safety penalty is calculated by multiplying the minimum distance between all electrical devices and path point n by the penalty coefficient and then adding 1.

11. A human-machine interaction safety control system for a quadrupedal robot used in emergency inspection of power distribution networks according to any one of claims 1-10, comprising a pre-defined intent data set module, an environment map generation module, an initial execution path sequence generation module, a comprehensive risk index calculation module, and a path verification module, characterized in that: The intent data set module describes how to acquire interactive instructions given by operators to the robot, perform semantic parsing on the interactive instructions, and extract the task intent, operation object, and execution parameters contained in the instructions to form an intent data set. Environmental map generation module: Based on the sensors on the robot and the pre-set digital twin map data of the power distribution network environment, construct a spatiotemporal three-dimensional grid map of the environment that includes equipment information, topological relationships and real-time dynamic obstacles; Initial execution path sequence generation module: Based on the intent data set, it performs action path planning in the spatiotemporal 3D raster map of the environment through a path planning algorithm to generate an initial execution path sequence; Comprehensive Risk Index Calculation Module: Combines the environmental spatiotemporal 3D raster map to perform multi-dimensional safety risk quantification assessment on each path point in the initial execution path sequence, and calculates the comprehensive risk index; Path verification module: If the overall risk index of all path points is lower than the set safety threshold, the initial execution path sequence will be converted into robot drive instructions and executed. Otherwise, the cost function of the path planning algorithm is changed, the action path planning is redone, an alternative path sequence is generated, and the comprehensive risk index of each path point in the alternative path sequence is calculated. If the comprehensive risk index of all path points in the alternative path sequence is lower than the set safety threshold, the alternative execution path sequence is converted into robot drive instructions and executed; otherwise, an early warning is issued.

12. An apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor performing steps of the human-machine interaction safety control method for a quadrupedal robot used in any one of claims 1 to 10 for emergency inspection of power distribution networks.

13. A computer-readable storage medium storing a computer program that, when executed by a processor, uses the steps of the human-machine interaction safety control method for a quadrupedal robot for emergency inspection of power distribution networks according to any one of claims 1 to 10.

Images