A power inspection robot control method and device, terminal equipment and storage medium

Abstract

The application discloses a power inspection robot control method and device, terminal equipment and storage medium, and belongs to the technical field of intelligent control. The method comprises the following steps: when a natural language instruction is monitored, a preset power inspection vector library is checked to generate an inspection instruction; the inspection instruction is analyzed according to a same preset meta-skill set corresponding to each robot to be inspected, a preset substation scene semantic map and a device operation mapping association table, an ordered meta-skill action sequence is generated, the ordered meta-skill action sequence is matched with a target inspection space position in the semantic map, a current control instruction generated by the ordered meta-skill action sequence and an expected execution result is sent to the target inspection robot; visual perception data and execution state data when the target inspection robot executes an action are received, whether the execution result matches the expected execution result is checked in a preset multi-modal visual model, if yes, a next control instruction is issued, and if not, a termination instruction is issued. Through implementation of the application, the problems of high cost and poor adaptability in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to a control method, device, terminal equipment, and storage medium for a power inspection robot. Background Technology

[0002] Traditionally, power sector inspections are done manually. However, given that most power facilities are located in remote areas and it is inconvenient for staff to move frequently between inspection sites, it is necessary to use robots or other automated equipment to replace manual labor in inspection tasks.

[0003] The autonomy of power line inspection robots is a challenge in the industry. Under current conditions, although robots can execute preset basic actions with a high success rate under specific initial conditions, the question of when to trigger these actions remains unresolved. For example, while robots have the ability to toggle switches and press designated buttons, manual commands are still required to press the buttons. This process necessitates that human operators be familiar with the principles of robot motion control and complex structured instruction interfaces, failing to truly liberate human labor. Furthermore, it places high demands on both the human operator's knowledge of power operations and their robot operation skills, resulting in high overall costs and poor applicability. Summary of the Invention

[0004] This invention provides a control method, device, terminal equipment, and storage medium for a power inspection robot, which can effectively solve the problems of high overall control cost and poor applicability of existing technologies.

[0005] An embodiment of the present invention provides a control method for a power inspection robot, comprising: Upon detecting natural language commands, the commands are validated for compliance based on a pre-set power inspection vector library, and inspection commands are generated. The inspection instructions are parsed based on the same preset meta-skill set corresponding to each robot to be inspected, the preset substation scene semantic map, and the preset equipment operation mapping association table. An ordered meta-skill action sequence is generated, and the expected execution result corresponding to the ordered meta-skill action sequence is determined. Based on the ordered element skill action sequence, match all target inspection space locations in the substation scene semantic map, and determine the appropriate target inspection robot based on the initial target inspection space location and the current state of each robot to be inspected. The current control command is generated based on the ordered element skill action sequence and the expected execution result, and then sent to the target inspection robot so that the target inspection robot can perform the corresponding inspection action. It receives visual perception data and execution status data in real time during the inspection process of the target inspection robot; Based on visual perception data and execution status data, the system analyzes the data in a preset multimodal vision model to verify whether the execution result of the current inspection action matches the expected execution result. If yes, the system continues to issue control instructions for the next inspection action; otherwise, it issues a termination instruction.

[0006] Furthermore, natural language instructions include: a first inspection instruction in the form of voice input by the user, a second inspection instruction in the form of text input by the user, or a third inspection instruction generated based on an inspection document; Based on a pre-defined power inspection vector library, the natural language commands are validated for compliance, and inspection commands are generated, including: Determine the type of the natural language instruction. If the natural language instruction is the first inspection instruction, convert the first inspection instruction into a text instruction to obtain the text instruction to be verified. When the natural language instruction is the second inspection instruction, the text content in the second inspection instruction is formatted to obtain the text instruction to be verified. When the natural language instruction is the third inspection instruction, the full text content of the inspection document in the third inspection instruction is extracted, and the inspection requirement information is extracted from the full text content based on the preset power inspection vector library. The text instruction to be verified is generated based on the inspection requirement information. The system retrieves structured inspection requirements that match the text instructions to be verified from the preset power inspection vector library. Based on these structured inspection requirements, it verifies the operational compliance and safety of the text instructions to be verified and generates inspection instructions.

[0007] Furthermore, the pre-defined semantic map of the substation scene includes: substation equipment information, substation obstacle marking information, and inspection and access map; Based on the same preset meta-skill set corresponding to each robot to be inspected, a preset substation scene semantic map, and a preset equipment operation mapping association table, the inspection instructions are parsed to generate an ordered meta-skill action sequence, including: The inspection instructions are semantically parsed to extract the substation equipment to be inspected, the operation type, and the work requirements. Based on the substation equipment to be inspected, the corresponding substation equipment information is matched from the preset substation scene semantic map to obtain the equipment number, equipment coordinates and location information of the substation equipment to be inspected. Based on the equipment coordinates and location information of the substation equipment to be inspected, the corresponding inspection access map and substation obstacle marking information are extracted from the substation scene semantic map. Based on the corresponding inspection access map and substation obstacle marking information, the basic parameters of the navigation path of the robot to be inspected from its current position to the substation equipment to be inspected are determined. Based on the preset equipment operation mapping association table, match the corresponding operation points and status points of the substation equipment to be inspected, and determine the operation objects and operation specifications required to complete the inspection command based on the corresponding operation points and status points. Based on the same preset meta-skill set corresponding to each robot to be inspected, and based on the meta-skill set, the inspection instructions are broken down into single-step meta-skill actions ordered according to the execution logic, based on the operation point, state point, basic parameters of the navigation path, operation object, and operation specifications, generating an ordered meta-skill action sequence.

[0008] Furthermore, based on the ordered meta-skill action sequence, all target inspection spatial locations are matched in the substation scene semantic map, including: Extract the substation equipment to be inspected based on the ordered element skill action sequence; Based on semantic map matching of substation scenarios, the spatial coordinates of the operation points corresponding to the substation equipment to be inspected are determined, and the spatial coordinates of the corresponding operation points are used as the target inspection spatial location of the target operation points.

[0009] Furthermore, the current status of the robot to be inspected includes: current location, current battery level, current operational health status, and current task occupancy status; The appropriate target inspection robot is determined based on the initial target inspection spatial location and the current state of each robot to be inspected, including: Based on the initial target inspection space position, the travel distance between each robot to be inspected and the target operation point is calculated according to the current position of the robot to be inspected. The comprehensive adaptation value is calculated based on the travel distance, current location, current battery level, current operating health status, current task occupancy status, and corresponding preset weight coefficients. The robot to be inspected with the highest overall fit value will be selected as the target robot for inspection.

[0010] Furthermore, based on the ordered meta-skill action sequence and the expected execution result, a current control command is generated and sent to the target inspection robot, including: For each single-step skill action in the ordered meta-skill action sequence, based on the execution logic of the ordered meta-skill action sequence, according to the robot parameters of the target inspection robot and the preset communication protocol, the current single-step skill action and the corresponding expected execution result are formatted and the current control command is generated. The current control command is sent to the target inspection robot so that the current control command triggers the target inspection robot to perform the corresponding inspection action.

[0011] Furthermore, based on visual perception data and execution status data, analysis is performed in a preset multimodal visual model to verify whether the execution result of the current inspection action matches the corresponding expected execution result, including: The target state point is located based on visual perception data, and the corresponding text label information is determined based on the location. The text label information and the target state point are matched and aligned across modally using a preset multimodal visual model. After the cross-modal matching and alignment, the real-time running status of the target state point is identified. The execution status data is analyzed to determine the completion rate of the current inspection action of the target inspection robot and the equipment operating parameters. The real-time operating status of the target status point, the completion rate of the current inspection action, and the equipment operating parameters are used as the execution result of the current inspection action. The execution result is compared with the expected execution result corresponding to the current inspection action to determine whether the deviation between the execution result and the expected execution result is less than the preset fault tolerance threshold.

[0012] As an improvement to the above solution, another embodiment of the present invention provides a power inspection robot control device, comprising: The inspection instruction generation module is used to perform instruction compliance verification on natural language instructions based on a preset power inspection vector library when natural language instructions are detected, and then generate inspection instructions. The ordered meta-skill action sequence generation module is used to parse the inspection instructions based on the same preset meta-skill set corresponding to each robot to be inspected, the preset substation scene semantic map, and the preset equipment operation mapping association table, generate ordered meta-skill action sequences, and determine the expected execution results corresponding to the ordered meta-skill action sequences. The target inspection robot determination module is used to match all target inspection spatial locations in the substation scene semantic map according to the ordered element skill action sequence, and determine the appropriate target inspection robot according to the initial target inspection spatial location and the current state of each robot to be inspected. The current control command generation module is used to generate current control commands based on the ordered meta-skill action sequence and the expected execution result, and send the current control commands to the target inspection robot so that the target inspection robot can perform the corresponding inspection action; The execution data receiving module is used to receive visual perception data and execution status data of the target inspection robot in real time during the inspection process. The execution result judgment module is used to analyze the visual perception data and execution status data in a preset multimodal vision model to verify whether the execution result of the current inspection action matches the corresponding expected execution result. If yes, the control command for the next inspection action is issued; otherwise, a termination command is issued.

[0013] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a power inspection robot control method as described in the above embodiments.

[0014] Another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the power inspection robot control method described in the above embodiment.

[0015] By implementing this invention, at least the following beneficial effects are achieved: This invention provides a control method, device, terminal equipment, and storage medium for a power inspection robot. The method directly monitors natural language commands, uses a pre-set power inspection vector library for semantic compliance verification to generate robot-understandable inspection commands, and then parses these commands using a pre-set substation scenario semantic map and equipment operation mapping association table to generate specific ordered meta-skill action sequences. This solves the problem of when commands are triggered, and the generated ordered meta-skill action sequences replace manually issued command buttons, solving the problem of when the robot presses a button. Finally, the execution result is verified based on a multimodal visual model. The execution result is analyzed based on visual perception data and execution status data, compared with the expected execution result corresponding to the ordered meta-skill action sequence, and a decision is made to continue issuing the next control command or terminate the process. This allows the robot to complete tasks autonomously without real-time human confirmation. Meanwhile, by receiving natural language commands, even non-professional power inspection workers can command robots without understanding the principles of robot motion control and complex operating interfaces, greatly reducing the threshold for hiring and training costs. Workers with different levels of power operation knowledge and robot operation skills do not need to memorize specific operation buttons, but can generate control commands through the semantic map of the substation scene and the equipment operation mapping table, thereby completing the inspection task and improving the applicability of robot control. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a power inspection robot control method according to an embodiment of the present invention; Figure 2 This is an interactive diagram of a user client, an inspection robot, and a computing server provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a power inspection robot control device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 To address the problems of high overall control costs and poor applicability in existing technologies, an embodiment of the present invention provides a flowchart of a power inspection robot control method, including: S1. Upon detecting a natural language command, perform a compliance check on the natural language command based on the preset power inspection vector library, and generate an inspection command. Specifically, natural language instructions are commands input by users in everyday human language to trigger inspection tasks. These are the input sources for this method and include three categories: voice, text, and structured instructions extracted from documents. The power inspection vector library is a vector database adapted to the RAG retrieval enhancement generation system. It is constructed from documents such as power system safety operation regulations, inspection operation standards, equipment operation and maintenance specifications, and substation management rules, after text segmentation and semantic encoding. It serves as the compliance basis for instruction compliance verification and action sequence generation. Instruction compliance verification involves checking the legality of the input natural language instructions, their scenario adaptability, and operational safety to ensure that the instructions comply with power industry operation standards and prevent safety accidents caused by violations. Inspection instructions are standardized, machine-parsable structured inspection task instructions generated after passing compliance verification.

[0019] Specifically, in this embodiment, the execution entity is an independently deployed computing server, rather than the robot itself. The robot and the computing server maintain 24-hour real-time communication via 5G or industrial Ethernet to solve the problem of insufficient computing power of the robot itself.

[0020] To illustrate, it is necessary to pre-build a power inspection vector library. Power documents such as the implementation details of substation operation management standards and electrical operation management details are parsed and divided into blocks. The block text is then vector-encoded using an open-source text semantic encoding model and stored in a vector database to build a power inspection vector library adapted to the RAG system.

[0021] To illustrate, the computing server monitors natural language commands from user client apps or periodic task schedulers in real time. Once a natural language command is detected, compliance verification and command generation are performed. First, using RAG retrieval enhancement generation technology, the input natural language command is encoded into text and its cosine similarity is calculated with vectors in the power inspection vector library to retrieve matching power safety operation regulations and inspection operation standards. Then, based on the retrieval results, the LLM large language model verifies whether the operation of the natural language command complies with power safety regulations, is suitable for the substation scenario, and whether there are any safety risks. If the verification is successful, standardized and structured inspection commands are generated; if the verification fails, a non-compliance prompt is returned.

[0022] To illustrate, the inspection robot needs to pre-develop basic skills such as navigation, posture adjustment, button pressing, knob rotation, and high-definition visual perception. The robot must be equipped with a SLAM navigation algorithm that supports pre-mapping, real-time localization, and dynamic path planning to ensure accurate navigation. After a termination command is issued upon failure of verification, an abnormal alarm signal must be sent to the user client simultaneously, along with the abnormal location, action information, and real-time visual data. Simultaneously, the robot should be triggered to retreat to the standby charging station, awaiting human intervention. Once the entire sequence of actions is completed and all verifications pass, the robot must send task completion feedback to the user client, simultaneously pushing the entire inspection data, visual records, and execution logs, and autonomously return to the standby point.

[0023] Specifically, the inspection robot can cover almost all inspection and maintenance tasks in a substation by performing at least the following actions: activating circuit breakers, deactivating circuit breakers, closing circuit breakers, opening circuit breakers, switching remote terminals, switching local terminals, turning on air conditioning, turning off air conditioning, raising the temperature, lowering the temperature, turning on lights, turning off lights, infrared temperature measurement, brush cleaning, external inspection of partial discharge instruments, hanging signs, removing signs, engaging and disengaging pressure plates, testing with voltage testers, installing and removing grounding wires, setting up and dismantling fences, absorbing static electricity, moving toolboxes, positioning and navigation, opening and closing cabinet doors, grabbing temperature guns, grabbing brushes, grabbing partial discharge instruments, grabbing signs, releasing temperature guns, releasing brushes, releasing partial discharge instruments, releasing signs, aligning the main camera, bringing the hand camera closer, indoor navigation and movement, and adjusting the robot's posture.

[0024] Preferably, the natural language instructions include: a first inspection instruction in the form of voice input by the user, a second inspection instruction in the form of text input by the user, or a third inspection instruction generated based on an inspection document; Based on a pre-defined power inspection vector library, the natural language commands are validated for compliance, and inspection commands are generated, including: Determine the type of the natural language instruction. If the natural language instruction is the first inspection instruction, convert the first inspection instruction into a text instruction to obtain the text instruction to be verified. When the natural language instruction is the second inspection instruction, the text content in the second inspection instruction is formatted to obtain the text instruction to be verified. When the natural language instruction is the third inspection instruction, the full text content of the inspection document in the third inspection instruction is extracted, and the inspection requirement information is extracted from the full text content based on the preset power inspection vector library. The text instruction to be verified is generated based on the inspection requirement information. The system retrieves structured inspection requirements that match the text instructions to be verified from the preset power inspection vector library. Based on these structured inspection requirements, it verifies the operational compliance and safety of the text instructions to be verified and generates inspection instructions.

[0025] Specifically, the first inspection command is a temporary, voice-based inspection command input by the user in real time via a mobile app, computer client, or other terminal, suitable for sudden inspections and temporary maintenance scenarios. The second inspection command is a temporary, text-based inspection command input by the user in real time via a client, suitable for precise operations and complex task scenarios. The third inspection command is an automatically generated, periodically executed inspection command based on uploaded periodic inspection requirement documents (or inspection plans), such as daily or weekly inspections, suitable for routine daily inspection scenarios.

[0026] To illustrate, if the detected natural language instruction is a first inspection instruction in speech form, an open-source speech-to-text model is used to convert the speech instruction into a standardized text instruction. After the user confirms that the content is correct, the text instruction to be verified is obtained. If the detected natural language instruction is a second inspection instruction in text form, semantic denoising, format regularization, and special symbol filtering are performed on the text content, and it is uniformly converted into a standardized text format that the model can recognize, resulting in the text instruction to be verified. If the detected natural language instruction is a third inspection instruction generated based on an inspection document, OCR technology is used to extract the full text content of the inspection requirement document in PDF or image format. Using an LLM large model combined with a power inspection vector library, structured inspection requirement information such as inspection location, inspection equipment, inspection frequency, and inspection requirements is extracted from the text to generate a standardized text instruction to be verified. Finally, using RAG technology, the power safety regulations and inspection operation standards that match the text instruction to be verified are retrieved from the power inspection vector library. Based on the retrieval results, the operation compliance and safety of the text instruction to be verified are checked. If the verification is successful, a standardized inspection instruction is generated. If the verification fails, a non-compliance prompt and modification suggestions are returned to the client.

[0027] In a preferred embodiment of the present invention, maintenance personnel discover an abnormal temperature in the Section II AC incoming cabinet at the remote monitoring center and send a voice command via a mobile APP: Perform infrared temperature measurement on the Section II AC incoming cabinet. The system converts the voice command into a text command through a voice-to-text model. After confirmation by the maintenance personnel, the command enters the compliance verification stage. After the verification is passed, a standardized inspection command is generated, and the subsequent inspection process is executed.

[0028] In a preferred embodiment of the present invention, maintenance personnel input text commands through a computer client to inspect the status of indicator lights on all switchgear in the 35kV power distribution room at 9:00 AM every day. The system performs format regularization and semantic noise reduction on the text content to obtain the text command to be verified. After passing the compliance verification, a standardized inspection command is generated.

[0029] In a preferred embodiment of the present invention, maintenance personnel upload the document of the daily inspection management method for the 35kV distribution room of substation D. The system extracts the full text content of the document through OCR technology, and extracts the structured requirements for a daily connectivity inspection of the AC incoming line equipment of section II through the LLM large model combined with the power inspection vector library. The system generates a text instruction to be verified. After passing the compliance verification, a standardized periodic inspection instruction is generated and automatically triggered at 8:00 am every day.

[0030] This implementation covers three major inspection scenarios: temporary voice commands, text commands, and periodic document commands, meeting the needs of all substation inspection task types. Corresponding preprocessing procedures were developed for different types of commands to ensure that all input commands can be converted into standardized text that is recognizable by the model, improving the method's compatibility and robustness. Supporting multiple input methods including voice, text, and document, maintenance personnel do not need to learn complex command formats; they can trigger inspection tasks simply by speaking or uploading inspection plans, further lowering the operational threshold. Simultaneously, all types of commands must undergo compliance verification by the power inspection vector library, preventing the execution of illegal commands from the source and ensuring the security of inspection operations.

[0031] S2. Based on the same preset meta-skill set corresponding to each robot to be inspected, the preset substation scene semantic map, and the preset equipment operation mapping association table, the inspection instructions are parsed to generate an ordered meta-skill action sequence and determine the expected execution result corresponding to the ordered meta-skill action sequence. Specifically, the robots to be inspected are candidate inspection robots, which are all power inspection robots deployed in substation scenarios that possess inspection capabilities and can be scheduled. Basic meta-skills development must be completed in advance. Robot parameters are fundamental parameters characterizing the hardware capabilities and software functions of the inspection robot, including the robot's executable meta-skill set, communication protocol, motion parameters, load capacity, and visual perception capabilities. These are the core basis for scheduling robots and generating action sequences. Meta-skills, also called meta-actions, are the smallest action units that engineers have pre-developed and that the robot can stably execute without human intervention. They are the foundation for the robot to perform inspection tasks and include navigation, posture adjustment, button pressing, knob rotation, and infrared temperature measurement.

[0032] Specifically, the real-time location refers to the inspection robot's real-time spatial coordinates within the substation scene semantic map, calculated and reported in real-time by the robot's onboard SLAM navigation algorithm. The current status represents the inspection robot's real-time operating status, including remaining battery power, operational health, task occupancy, and fault status, serving as the core selection criterion for robot scheduling. The substation scene semantic map is a pre-generated, semantically labeled map of the substation scene generated through LiDAR scanning. It forms the spatial basis for robot navigation and equipment localization, including substation equipment information, obstacle markers, and inspection access routes. Figure 3 The main modules are as follows: The Equipment Operation Mapping Association Table is a pre-built database table storing the correspondence between substation equipment to be inspected and operation points / status points, establishing a many-to-many mapping relationship between equipment and operation ports / status monitoring ports. The Ordered Meta-Skill Action Sequence is a set of single-step meta-skill actions that the robot can directly execute, ordered according to the power operation execution logic after the inspection instructions are broken down. It serves as the direct basis for the robot to execute inspection tasks. The Expected Execution Result is the standard state that the target status point should reach after the operation, corresponding one-to-one with the single-step meta-skill action. It serves as the benchmark for verifying the single-step action execution result, such as the circuit breaker indicator light illuminating and the knob rotating to the local position.

[0033] In a schematic representation, a substation scene is scanned and mapped using LiDAR, generating a semantic map of the substation scene with semantic information. This map marks the name, number, and spatial coordinates of the equipment to be inspected, the location of obstacles, the standby points of charging piles, and the inspection route. Simultaneously, an equipment operation mapping association table is constructed, establishing a many-to-many mapping relationship between equipment, operation points, and status points. For example, {Equipment: Section II AC incoming line, Operation point: 2HL1 trip button, Status point: 2HL1 trip indicator light}.

[0034] To illustrate, the computing server acquires the robot parameters, real-time location, and current status of all robots to be inspected within the substation in real time. Then, it performs semantic parsing of standardized inspection instructions using an LLM large language model to extract the substation equipment to be inspected, operation type, and work requirements. Next, it obtains the coordinates and location of the target equipment by combining the substation scene semantic map, obtains the corresponding operation points and status points by combining the equipment operation mapping association table, and retrieves the compliant operation process from the power inspection vector library by combining the set of executable meta-skills of the robot. The inspection instructions are then broken down into single-step meta-skill actions ordered by execution logic to generate an ordered meta-skill action sequence. Finally, for each single-step meta-skill action in the sequence, the corresponding expected execution result is matched to establish a one-to-one correspondence between single-step actions and expected results.

[0035] Preferably, the preset substation scene semantic map includes: substation equipment information, substation obstacle marking information, and inspection access map; Based on the same preset meta-skill set corresponding to each robot to be inspected, a preset substation scene semantic map, and a preset equipment operation mapping association table, the inspection instructions are parsed to generate an ordered meta-skill action sequence, including: The inspection instructions are semantically parsed to extract the substation equipment to be inspected, the operation type, and the work requirements. Based on the substation equipment to be inspected, the corresponding substation equipment information is matched from the preset substation scene semantic map to obtain the equipment number, equipment coordinates and location information of the substation equipment to be inspected. Based on the equipment coordinates and location information of the substation equipment to be inspected, the corresponding inspection access map and substation obstacle marking information are extracted from the substation scene semantic map. Based on the corresponding inspection access map and substation obstacle marking information, the basic parameters of the navigation path of the robot to be inspected from its current position to the substation equipment to be inspected are determined. Based on the preset equipment operation mapping association table, match the corresponding operation points and status points of the substation equipment to be inspected, and determine the operation objects and operation specifications required to complete the inspection command based on the corresponding operation points and status points. Based on the same preset meta-skill set corresponding to each robot to be inspected, and based on the meta-skill set, the inspection instructions are broken down into single-step meta-skill actions ordered according to the execution logic, based on the operation point, state point, basic parameters of the navigation path, operation object, and operation specifications, generating an ordered meta-skill action sequence.

[0036] Specifically, the substation equipment information consists of the complete basic information of the substation equipment to be inspected, stored in the substation scene semantic map. This includes the equipment name, unique equipment number, spatial coordinates, location in the distribution room, and equipment type. The substation obstacle marking information is the information stored in the substation scene semantic map regarding obstacles that will affect the robot's navigation, including the location, size, and type of obstacles such as rodent barriers, distribution cabinets, fences, and doors. The inspection passage map is the route map stored in the substation scene semantic map that the robot can traverse, marking information such as the width, slope, door connectivity, and room accessibility of the passage route; it forms the basis for robot path planning. The navigation path basic parameters are the core parameters of the robot's navigation path from its current position to the target equipment, calculated based on the inspection passage map and obstacle information. These parameters include the starting coordinates, ending coordinates, travel distance, and route inflection points.

[0037] To illustrate, when equipment changes or obstacles are adjusted in a substation scenario, the semantic map of the substation scenario can be quickly updated by rescanning the local area with LiDAR, without the need for full scenario reconstruction.

[0038] In a schematic manner, the standardized inspection instructions are semantically decomposed using an LLM (Large Language Model) to extract key information such as the substation equipment to be inspected, operation type, work requirements, and execution time. Based on the extracted names and labels of the substation equipment to be inspected, a matching process is performed in a pre-defined substation scene semantic map to obtain the unique equipment label, spatial coordinates, and substation area information of the substation equipment, thus clarifying the specific location of the target equipment. Then, based on the spatial coordinates and area of ​​the target equipment, the corresponding inspection route map and obstacle marker information are extracted from the substation scene semantic map. Using path planning algorithms such as RRT (Road Traffic Response) to calculate the traversable route from the current position to the target equipment, the basic parameters of the navigation path are determined. Next, based on a pre-defined equipment operation mapping association table, all operation points and status points corresponding to the substation equipment to be inspected are matched to clarify the specific objects to be operated (such as buttons and knobs) and the status objects to be verified (such as indicator lights and meters) to complete the inspection task. At the same time, the corresponding operation specifications are retrieved from the power inspection vector library. Then, based on the robot parameters of the robot to be inspected, the set of meta-skills that the robot can stably execute is obtained. Based on the power compliance operation specifications, and combined with the basic parameters of operation points, status points, and navigation paths, the inspection instructions are decomposed into single-step meta-skill actions that are strictly ordered according to the power operation logic through the LLM large model, and finally a complete ordered meta-skill action sequence is generated.

[0039] By implementing this embodiment, based on the semantic map of the substation scene with semantic information, the spatial location of the operation point of the substation equipment can be accurately located, providing a precise spatial reference for robot navigation and operation; it can generate action sequences adapted to the corresponding robot according to the meta-skill set of different robots, without the need to redevelop logic for different robots, and has extremely strong adaptability.

[0040] S3. Match all target inspection space locations in the substation scene semantic map according to the ordered element skill action sequence, and determine the appropriate target inspection robot according to the initial target inspection space location and the current state of each robot to be inspected. Specifically, the target inspection location is the spatial coordinate of the operation point corresponding to the equipment to be inspected, which serves as the target reference point for robot navigation and scheduling. The target inspection robot is a dedicated inspection robot selected by the scheduling logic to perform this inspection task.

[0041] Specifically, the port used to operate or control a device is called an operation point, and the port used to view the current status of the device is called a status point. An operation point can be a switch, button, knob, or gate. A status point can be an instrument, indicator light, knob status, gate open / closed status, or the operating position of additional detection devices such as an infrared thermometer. The relationship between device and operation point, and between device and status point, is a typical many-to-many relationship.

[0042] Schematic, all substation equipment to be inspected is extracted from the ordered meta-skill action sequence. Matching is performed based on the substation scene semantic map to obtain the spatial coordinates of the operation point corresponding to each piece of substation equipment to be inspected, which serves as the target inspection spatial location for the target operation point. Then, using the target inspection spatial location as a reference, the travel distance between each robot to be inspected and the target point is calculated. Combining the robot's remaining battery power, operational health status, idle status, and whether it possesses the meta-skills required to execute the current action sequence, the most suitable target inspection robot is selected based on preset scheduling logic. This embodiment supports full-scene scheduling of single or multiple robots.

[0043] Preferably, matching all target inspection spatial locations in the substation scene semantic map based on the ordered meta-skill action sequence includes: Extract the substation equipment to be inspected based on the ordered element skill action sequence; Based on semantic map matching of substation scenarios, the spatial coordinates of the operation points corresponding to the substation equipment to be inspected are determined, and the spatial coordinates of the corresponding operation points are used as the target inspection spatial location of the target operation points.

[0044] Specifically, spatial coordinate matching involves extracting the substation equipment to be inspected and matching it against a pre-defined semantic map of the substation scene to determine the precise spatial coordinates of all operation points corresponding to the equipment. The target inspection spatial location is then determined by using the spatial coordinates of the matched operation points as the target inspection spatial location for this inspection task, providing a precise target reference for subsequent robot scheduling and navigation.

[0045] Indicatively, from the generated ordered meta-skill action sequence, the equipment to be inspected in this inspection task is extracted as Section II AC incoming line; based on the equipment label of Section II AC incoming line, matching is performed in the substation scene semantic map to determine the spatial coordinates (X,Y,Z) of the corresponding trip button operation point; the spatial coordinates of this operation point are used as the target inspection spatial location for this inspection task, providing a benchmark for subsequent robot scheduling.

[0046] By implementing this embodiment, the spatial coordinates of the operation point are directly used as the target inspection space location, rather than the overall coordinates of the equipment, ensuring that the target point for robot scheduling and navigation is the actual location that needs to be operated, thus avoiding operation failures caused by navigation deviations.

[0047] Preferably, the current status of the robot to be inspected includes: current location, current battery level, current operating health status, and current task occupancy status; The appropriate target inspection robot is determined based on the initial target inspection spatial location and the current state of each robot to be inspected, including: Based on the initial target inspection space position, the travel distance between each robot to be inspected and the target operation point is calculated according to the current position of the robot to be inspected. The comprehensive adaptation value is calculated based on the travel distance, current location, current battery level, current operating health status, current task occupancy status, and corresponding preset weight coefficients. The robot to be inspected with the highest overall fit value will be selected as the target robot for inspection.

[0048] Specifically, based on a defined initial target inspection location, the actual travel distance from each robot's current position to the target operation point is calculated using a path planning algorithm, rather than the straight-line distance, to ensure the accuracy of the distance calculation. Robot screening and scheduling combines the current position, current battery level, current operational health status, and current task occupancy status of each robot to be inspected to select the most suitable target inspection robot from all candidate robots.

[0049] In a preferred embodiment of the present invention, the robot with the shortest travel distance can be selected via distance-first scheduling logic; the robot with the highest remaining battery power can be selected via battery-first scheduling logic; and the robot with the best operating health status can be selected via optimal scheduling logic. The scheduling logic has been extensively described in existing technologies, and this embodiment will not provide a detailed explanation of each scheduling logic. If the task involves multiple target inspection locations, multiple robots can be scheduled to perform the inspection task in parallel, improving inspection efficiency.

[0050] By implementing this embodiment, multiple dimensions such as the robot's execution capability, travel distance, current battery level, current operational health status, and current task occupancy status are comprehensively considered, and the selected target robot is fully adapted to this inspection task; the actual travel distance rather than the straight-line distance is used as the scheduling benchmark, which fits the actual situation of the complex indoor scene of the substation, and the scheduling results are more practical.

[0051] S4. Generate the current control command based on the ordered element skill action sequence and the expected execution result, and send the current control command to the target inspection robot so that the target inspection robot can perform the corresponding inspection action; Specifically, the current control command is a control command that encapsulates the single-step meta-skill action and the corresponding expected execution result according to the robot communication protocol, and can be directly sent to the robot for execution.

[0052] Schematic, based on the execution order of the ordered meta-skill action sequence, the single-step meta-skill action to be executed is selected, and combined with the corresponding expected execution result, it is formatted and encapsulated according to the communication protocol and robot parameters of the target inspection robot to generate the current single-step control instruction; then, through the real-time communication link between the computing server and the robot, the current control instruction is sent to the target inspection robot to trigger the robot to execute the corresponding single-step inspection action; this embodiment adopts a single-step sending and single-step verification mode to ensure that each action is verified before the next step is executed, avoiding safety accidents caused by continuous erroneous operations.

[0053] Preferably, generating a current control command based on the ordered meta-skill action sequence and the expected execution result, and sending the current control command to the target inspection robot, includes: For each single-step skill action in the ordered meta-skill action sequence, based on the execution logic of the ordered meta-skill action sequence, according to the robot parameters of the target inspection robot and the preset communication protocol, the current single-step skill action and the corresponding expected execution result are formatted and the current control command is generated. The current control command is sent to the target inspection robot so that the current control command triggers the target inspection robot to perform the corresponding inspection action.

[0054] Specifically, for each single-step meta-skill action in the ordered meta-skill action sequence, the execution logic of the sequence is strictly followed. Based on the target inspection robot's robot parameters and the preset communication protocol, the single-step meta-skill action to be executed and its expected execution result are encapsulated in a standardized format to generate the current single-step control command. The encapsulated current single-step control command is then sent to the target inspection robot through the real-time communication link between the computing server and the robot. Upon receiving the command, the target inspection robot immediately triggers the execution of the corresponding single-step inspection action, ensuring the synchronization between action execution and command issuance.

[0055] To illustrate, if no confirmation of receipt is received from the robot within 500ms after the current control command is issued, the system will automatically resend the control command, up to 3 times. If no confirmation is received after resending, a communication abnormality alarm will be triggered. Simultaneously, all control commands are encrypted using the national cryptographic algorithm SM4 to ensure the security of command transmission and prevent malicious tampering that could lead to security incidents.

[0056] In a preferred embodiment of the present invention, for the first step of the ordered meta-skill action sequence: navigation to the operation point in front of the AC incoming line cabinet of section II, according to the communication protocol and motion control parameters of robot No. 1 H1, the single-step meta-skill action and the corresponding expected execution result are: the robot arrives at the target point, the camera faces the control panel, the format is encapsulated, and the current single-step control instruction is generated; then the encapsulated single-step control instruction is sent to robot No. 1 H1, and after receiving the instruction, the robot immediately executes the navigation movement action.

[0057] By implementing this embodiment, a complete closed loop of issuing, executing, verifying, and re-issuing is formed, ensuring operational safety. Instructions are encapsulated based on the target inspection robot's parameters and communication protocol, making it adaptable to inspection robots of different brands and models without requiring the development of dedicated instruction generation logic for different robots. Simultaneously, single-step actions and their corresponding expected execution results are encapsulated and issued together, allowing the robot to perform preliminary execution result verification locally, improving the accuracy of action execution. The single-step control instructions have small data volume, low transmission latency, and strong anti-interference capabilities, making them suitable for the complex industrial environment of substations.

[0058] S5. Receive visual perception data and execution status data in real time during the inspection process of the target inspection robot; Specifically, visual perception data refers to the image or video data of the inspection scene collected in real time by the visual acquisition modules such as high-definition cameras and infrared cameras equipped on the inspection robot. It is the core basis for status recognition and action verification. Execution status data refers to the data such as the completion degree of action execution, motor operating parameters, posture parameters, and equipment operating status reported in real time by the inspection robot during the execution of actions.

[0059] To illustrate, the computing server continuously receives visual perception data captured by the robot during its single-step inspection action, as well as execution status data reported by the robot's motion control system, through a real-time communication link with the target inspection robot, providing a complete data foundation for subsequent action verification.

[0060] S6. Analyze the visual perception data and execution status data in the preset multimodal vision model to verify whether the execution result of the current inspection action matches the corresponding expected execution result. If yes, continue to issue the control command for the next inspection action; otherwise, issue a termination command.

[0061] Specifically, the multimodal visual model, or VLM (Visual Language Model), is a large-scale multimodal model capable of simultaneously processing and understanding visual image information and natural language text information. It enables cross-modal matching and alignment of text labels and visual entities, as well as target state recognition. The fault tolerance threshold is a pre-set maximum deviation between the allowed execution result and the expected execution result, used to determine whether the action execution is qualified, preventing task termination due to minor recognition errors and improving the robustness of the method. The termination command is a control command issued to the target inspection robot to terminate all current inspection actions, used for safety handling in case of action failure, preventing the risk of escalating due to unauthorized operations.

[0062] Indicatively, the YOLO target detection model locates the target state point region from visual perception data, an open-source OCR model identifies text labels within the region, and a VLM multimodal vision model performs cross-modal matching between the text labels and the target state point to identify the real-time operating status of the target state point, such as indicator light on / off status, knob position, and instrument readings. Simultaneously, the execution status data is analyzed to obtain the action completion rate and robot operating parameters. The identified real-time status and action completion rate are compared with the expected execution result corresponding to the current action to determine if the comparison deviation is less than a preset fault tolerance threshold. If the comparison deviation is less than the fault tolerance threshold, the verification is considered successful, and the control command for the next single-step action is generated and issued. If the comparison deviation is greater than or equal to the fault tolerance threshold, the verification is considered unsuccessful, a termination command is immediately issued, all robot actions are terminated, and alarm and safety rollback procedures are triggered simultaneously.

[0063] Preferably, the visual perception data and execution status data are analyzed in a preset multimodal visual model to verify whether the execution result of the current inspection action matches the corresponding expected execution result, including: The target state point is located based on visual perception data, and the corresponding text label information is determined based on the location. The text label information and the target state point are matched and aligned across modally using a preset multimodal visual model. After the cross-modal matching and alignment, the real-time running status of the target state point is identified. The execution status data is analyzed to determine the completion rate of the current inspection action of the target inspection robot and the equipment operating parameters. The real-time operating status of the target status point, the completion rate of the current inspection action, and the equipment operating parameters are used as the execution result of the current inspection action. The execution result is compared with the expected execution result corresponding to the current inspection action to determine whether the deviation between the execution result and the expected execution result is less than the preset fault tolerance threshold.

[0064] Specifically, cross-modal matching alignment means that the semantic features of text labels are mapped and matched with the visual features of target state points in the visual image through the VLM visual language large model, ensuring that the identified visual entity is completely consistent with the object described by the text label and avoiding recognition errors. Consistency comparison means that the actual execution result is compared with the expected execution result in multiple dimensions, including state consistency, numerical consistency, and logical consistency, to determine whether the two meet the preset matching requirements.

[0065] Indicatively, the YOLO target detection model locates the region of the target state point corresponding to the current single-step action from the received visual perception data; the OCR optical character recognition model extracts the text label information within this region; and the preset VLM multimodal vision model performs cross-modal matching and alignment between the semantic information of the text label and the visual information of the target state point, ensuring that the identified target state point is completely consistent with the expected verification object. After matching and alignment, the VLM model identifies the real-time operating status of the target state point, such as indicator light on / off status, knob position, and instrument reading. Then, the received execution status data is analyzed to extract the completion rate of the current single-step inspection action of the target inspection robot, motor operating parameters, body posture data, and other equipment operating parameters, clarifying the robot's action execution status. Simultaneously, the real-time operating status of the target state point, the completion rate of the action, and the equipment operating parameters identified by the VLM model are integrated into a complete actual execution result of the current single-step inspection action. The integrated actual execution result is compared with the expected execution result corresponding to the current single-step inspection action in a multi-dimensional consistency comparison, and the comparison deviation is calculated to determine whether the deviation is less than a preset fault tolerance threshold.

[0066] To illustrate, if the initial verification deviation exceeds the fault tolerance threshold, the system will trigger the robot to re-capture the visual image for secondary verification. This avoids misjudgments caused by shooting angle or lighting interference, thus improving the robustness of the verification. Different fault tolerance thresholds can be set according to the safety risk level of different operation types. The fault tolerance threshold for high-pressure operations is set to 0, while the fault tolerance threshold for normal state inspections can be appropriately relaxed, balancing safety and practicality.

[0067] By implementing this embodiment, visual perception data and robot execution status data are combined to verify the operation from two dimensions: the state changes of the object being operated on and the robot's action execution. The verification results are more comprehensive and reliable. By setting a fault tolerance threshold, misjudgments caused by light interference or slight shooting angle deviations can be avoided, improving the robustness of the method in the complex environment of substations. By verifying the entire process of single-step actions, it is ensured that each step achieves the expected effect before proceeding to the next step, fundamentally avoiding power safety accidents caused by misoperation.

[0068] In a preferred embodiment of the present invention, official documents such as the G power grid 35kV substation operation management standards, electrical operation management details, and on-site work permit specifications are stored in a vector database after text segmentation and vector encoding, thus completing the construction of a power inspection vector database. A full-scene scan and mapping of the D substation 35kV distribution room is performed using lidar, marking the names, labels, and spatial coordinates of 12 devices, including the Section II AC incoming line cabinet; the locations of obstacles such as rodent barriers and distribution cabinets; the standby locations of charging piles; the inspection route; and the accessibility of the distribution room. A many-to-many mapping relationship is established between the Section II AC incoming line equipment and the trip buttons, closing buttons, trip indicator lights, closing indicator lights, local control buttons, and remote control knobs. Simultaneously, the meta-skill sets, communication protocols, and motion parameters of three H1 humanoid robots are entered into the system.

[0069] In a preferred embodiment of the present invention, substation maintenance personnel input voice commands via a mobile app: "Perform a tripping operation on the Section II AC incoming switch." Upon receiving the command, the system converts it into a standardized text command using a speech-to-text model. Then, using RAG technology, it retrieves compliant operation procedures and safety specifications for high-voltage switch tripping in the substation from the power inspection vector library. The system verifies the compliance and safety of the command using an LLM large-scale model. Once verified, a standardized inspection command is generated. The system acquires the real-time location, remaining power, task occupancy status, and operational health status of three H1 robots. It parses the standardized inspection command using the LLM large-scale model, identifying the substation equipment to be inspected as the Section II AC incoming switch, the operation type as a tripping operation, and the requirement as compliance with electrical operation specifications. It then uses a semantic map to obtain the spatial coordinates and location of the Section II AC incoming switch cabinet, and uses an equipment operation mapping association table to obtain the corresponding operation point and status point. Navigate to the operating point in front of the Section II AC incoming line cabinet; then adjust the machine's posture and camera angle so that the camera is facing the control panel; rotate the control knob to the local position; simultaneously press the trip button; rotate the control knob to the remote position; and simultaneously match the expected execution result for each step of the action, such as the expected execution result being that the trip indicator light illuminates and the closing indicator light goes out.

[0070] The spatial coordinates of the operation point of the AC incoming line cabinet in section II are extracted from the action sequence as the target inspection spatial location. The travel distance between the three H1 robots and the target point is calculated. Then, the H1 robot No. 1, which is closest, has a remaining battery level of ≥80%, and is in an idle state, is selected as the target inspection robot for this task. The system encapsulates the meta-skill action of the first step, "navigating to the operation point in front of the AC incoming line cabinet in section II" and the corresponding expected result into a single-step control command according to the communication protocol of the H1 robot, and sends it to H1 robot No. 1. The robot executes the navigation action and moves to the target operation point.

[0071] The system receives real-time visual images and execution status data captured by Robot No. 1. Using a VLM (Virtual Modeling) multimodal model, it identifies whether the robot has reached the target location and whether the camera is facing the control panel. If the verification results meet expectations, the verification is considered successful. After successful verification, the system sequentially issues control commands for subsequent single-step actions. For each completed action, the execution result is verified using the VLM model in a closed-loop manner. If a step fails verification, such as the trip indicator light not illuminating after pressing the trip button, the system immediately issues a termination command, sends an abnormal alarm signal to the maintenance personnel's mobile app, and simultaneously pushes real-time visual images and abnormal information. The robot immediately terminates all operations and autonomously returns to the charging pile standby point, awaiting manual intervention. After the entire sequence of actions is executed and all verifications pass, Robot No. 1 sends task completion feedback to the system, simultaneously pushing the visual records, status data, and execution logs of the entire inspection process, and autonomously returns to the charging pile standby point. The trip inspection task is then completed.

[0072] In a preferred embodiment of the present invention, see [link to previous document]. Figure 2 The user client inputs a natural language command: "Open the AC incoming line switch at end II." The computing server, drawing from the power inspection vector library constructed from the power grid operation duty, electrical operation, and on-site safety management series of standard documents on the left-hand compliance benchmark, inputs a multimodal visual model. This breaks down the unstructured natural language command into an ordered sequence of alternating operations and perceptions: confirming the status of the closing indicator light, switching the control knob to the local position, announcing the opening operation, pressing the opening button, confirming the status of the opening indicator light, and switching the control knob to the remote position. The robot vision presents physical entities such as operation buttons and status indicator lights on the control panel, corresponding to operation points, status points, and target inspection spatial locations. It serves as the physical carrier for robot action execution and visual perception data acquisition, enabling interaction between the user client, the robot, and the computing server.

[0073] By implementing this embodiment, natural language commands can be directly monitored. A pre-set power inspection vector library is used for semantic compliance verification to generate inspection commands that the robot can understand. These commands are then parsed using a pre-set substation scene semantic map and equipment operation mapping table to generate specific ordered meta-skill action sequences, solving the problem of when to trigger them. These generated ordered meta-skill action sequences replace manually issued command buttons, solving the problem of when the robot presses a button. Finally, execution result verification based on a multimodal visual model involves analyzing the execution results using visual perception data and execution status data, comparing them with the expected execution results corresponding to the ordered meta-skill action sequences, and deciding whether to issue the next control command or terminate the process. This allows the robot to complete tasks autonomously without real-time human confirmation. Furthermore, by receiving natural language commands, even non-professional power inspection workers can command the robot without needing to understand the principles of robot motion control or complex operating interfaces, significantly reducing the personnel threshold and training costs. Workers with varying levels of power operation knowledge and robot operation skills do not need to memorize specific operation buttons; they can still generate control commands through the substation scene semantic map and equipment operation mapping table to complete inspection tasks, thus improving the applicability of robot control.

[0074] See Figure 3 This is a schematic diagram of the structure of a power inspection robot control device according to an embodiment of the present invention, comprising: The inspection instruction generation module is used to perform instruction compliance verification on natural language instructions based on a preset power inspection vector library when natural language instructions are detected, and then generate inspection instructions. The ordered meta-skill action sequence generation module is used to parse the inspection instructions based on the same preset meta-skill set corresponding to each robot to be inspected, the preset substation scene semantic map, and the preset equipment operation mapping association table, generate ordered meta-skill action sequences, and determine the expected execution results corresponding to the ordered meta-skill action sequences. The target inspection robot determination module is used to match all target inspection spatial locations in the substation scene semantic map according to the ordered element skill action sequence, and determine the appropriate target inspection robot according to the initial target inspection spatial location and the current state of each robot to be inspected. The current control command generation module is used to generate current control commands based on the ordered meta-skill action sequence and the expected execution result, and send the current control commands to the target inspection robot so that the target inspection robot can perform the corresponding inspection action; The execution data receiving module is used to receive visual perception data and execution status data of the target inspection robot in real time during the inspection process. The execution result judgment module is used to analyze the visual perception data and execution status data in a preset multimodal vision model to verify whether the execution result of the current inspection action matches the corresponding expected execution result. If yes, the control command for the next inspection action is issued; otherwise, a termination command is issued.

[0075] This invention provides a power inspection robot control device. According to the inspection instruction generation module, upon detecting a natural language instruction, the device performs instruction compliance verification on the natural language instruction based on a preset power inspection vector library, and generates an inspection instruction. In the ordered meta-skill action sequence generation module, the device parses the inspection instruction based on the same preset meta-skill set corresponding to each robot to be inspected, a preset substation scene semantic map, and a preset equipment operation mapping association table, generating an ordered meta-skill action sequence and determining the expected execution result corresponding to the ordered meta-skill action sequence. In the target inspection robot determination module, the device matches all target inspection spatial locations in the substation scene semantic map based on the ordered meta-skill action sequence, and determines the initial target inspection spatial location and... The current state of each inspection robot determines the appropriate target inspection robot. In the current control command generation module, the current control command is generated based on the ordered element skill action sequence and the expected execution result, and sent to the target inspection robot to enable the target inspection robot to perform the corresponding inspection action. Then, in the execution data receiving module, the visual perception data and execution status data of the target inspection robot during the execution of the inspection action are received in real time. Finally, in the execution result judgment module, the visual perception data and execution status data are analyzed in a preset multimodal vision model to verify whether the execution result of the current inspection action matches the corresponding expected execution result. If yes, the control command for the next inspection action is issued; otherwise, a termination command is issued.

[0076] By directly monitoring natural language commands and utilizing a pre-defined power inspection vector library for semantic compliance verification, robot-understandable inspection commands are generated. These commands are then parsed using a pre-defined substation scenario semantic map and equipment operation mapping table to generate specific ordered meta-skill action sequences, resolving the issue of triggering commands. These generated sequences replace manual button presses, addressing the question of when the robot should press a button. Finally, execution result verification based on a multimodal visual model involves analyzing visual perception and execution status data, comparing the results with the expected execution results corresponding to the ordered meta-skill action sequences, and determining whether to issue the next control command or terminate the process. This allows the robot to complete tasks autonomously without real-time human confirmation. Furthermore, by receiving natural language commands, even non-professional power inspection workers can command the robot without needing to understand the principles of robot motion control or complex interfaces, significantly reducing the personnel threshold and training costs. Workers with varying levels of power operation knowledge and robot operation skills can generate control commands using the substation scenario semantic map and equipment operation mapping table, thereby achieving inspection tasks and improving the applicability of robot control.

[0077] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0078] Those skilled in the art will understand that, for convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0079] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a power inspection robot control method as described in the above embodiments. The terminal device may be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The terminal device may include, but is not limited to, a processor and a memory.

[0080] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0081] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device or other volatile solid-state storage device.

[0082] Another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the power inspection robot control method described in the above embodiment.

[0083] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0084] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A control method for a power inspection robot, characterized in that, include: Upon detecting natural language commands, the commands are validated for compliance based on a pre-set power inspection vector library, and inspection commands are generated. The inspection instructions are parsed based on the same preset meta-skill set corresponding to each robot to be inspected, the preset substation scene semantic map, and the preset equipment operation mapping association table. An ordered meta-skill action sequence is generated, and the expected execution result corresponding to the ordered meta-skill action sequence is determined. Based on the ordered element skill action sequence, match all target inspection space locations in the substation scene semantic map, and determine the appropriate target inspection robot based on the initial target inspection space location and the current state of each robot to be inspected. The current control command is generated based on the ordered element skill action sequence and the expected execution result, and then sent to the target inspection robot so that the target inspection robot can perform the corresponding inspection action. It receives visual perception data and execution status data in real time during the inspection process of the target inspection robot; Based on visual perception data and execution status data, the system analyzes the data in a preset multimodal vision model to verify whether the execution result of the current inspection action matches the expected execution result. If yes, the system continues to issue control instructions for the next inspection action; otherwise, it issues a termination instruction.

2. The power inspection robot control method as described in claim 1, characterized in that, Natural language instructions include: first inspection instructions in the form of voice input by the user, second inspection instructions in the form of text input by the user, or third inspection instructions generated based on inspection documents; Based on a pre-defined power inspection vector library, the natural language commands are validated for compliance, and inspection commands are generated, including: Determine the type of the natural language instruction. If the natural language instruction is the first inspection instruction, convert the first inspection instruction into a text instruction to obtain the text instruction to be verified. When the natural language instruction is the second inspection instruction, the text content in the second inspection instruction is formatted to obtain the text instruction to be verified. When the natural language instruction is the third inspection instruction, the full text content of the inspection document in the third inspection instruction is extracted, and the inspection requirement information is extracted from the full text content based on the preset power inspection vector library. The text instruction to be verified is generated based on the inspection requirement information. The system retrieves structured inspection requirements that match the text instructions to be verified from the preset power inspection vector library. Based on these structured inspection requirements, it verifies the operational compliance and safety of the text instructions to be verified and generates inspection instructions.

3. The power inspection robot control method as described in claim 1, characterized in that, The preset semantic map of the substation scene includes: substation equipment information, substation obstacle marking information, and inspection and access map; Based on the same preset meta-skill set corresponding to each robot to be inspected, a preset substation scene semantic map, and a preset equipment operation mapping association table, the inspection instructions are parsed to generate an ordered meta-skill action sequence, including: The inspection instructions are semantically parsed to extract the substation equipment to be inspected, the operation type, and the work requirements. Based on the substation equipment to be inspected, the corresponding substation equipment information is matched from the preset substation scene semantic map to obtain the equipment number, equipment coordinates and location information of the substation equipment to be inspected. Based on the equipment coordinates and location information of the substation equipment to be inspected, the corresponding inspection access map and substation obstacle marking information are extracted from the substation scene semantic map. Based on the corresponding inspection access map and substation obstacle marking information, the basic parameters of the navigation path of the robot to be inspected from its current position to the substation equipment to be inspected are determined. Based on the preset equipment operation mapping association table, match the corresponding operation points and status points of the substation equipment to be inspected, and determine the operation objects and operation specifications required to complete the inspection command based on the corresponding operation points and status points. Based on the same preset meta-skill set corresponding to each robot to be inspected, and based on the meta-skill set, the inspection instructions are broken down into single-step meta-skill actions ordered according to the execution logic, based on the operation point, state point, basic parameters of the navigation path, operation object, and operation specifications, generating an ordered meta-skill action sequence.

4. The power inspection robot control method as described in claim 3, characterized in that, Based on the ordered meta-skill action sequence, match all target inspection spatial locations in the substation scene semantic map, including: Extract the substation equipment to be inspected based on the ordered element skill action sequence; Based on semantic map matching of substation scenarios, the spatial coordinates of the operation points corresponding to the substation equipment to be inspected are determined, and the spatial coordinates of the corresponding operation points are used as the target inspection spatial location of the target operation points.

5. The power inspection robot control method as described in claim 4, characterized in that, The current status of the robot to be inspected includes: current location, current battery level, current operational health status, and current task occupancy status; The appropriate target inspection robot is determined based on the initial target inspection spatial location and the current state of each robot to be inspected, including: Based on the initial target inspection space position, the travel distance between each robot to be inspected and the target operation point is calculated according to the current position of the robot to be inspected. The comprehensive adaptation value is calculated based on the travel distance, current location, current battery level, current operating health status, current task occupancy status, and corresponding preset weight coefficients. The robot to be inspected with the highest overall fit value will be selected as the target robot for inspection.

6. The power inspection robot control method as described in claim 1, characterized in that, Based on the ordered meta-skill action sequence and the expected execution result, the current control command is generated and sent to the target inspection robot, including: For each single-step skill action in the ordered meta-skill action sequence, based on the execution logic of the ordered meta-skill action sequence, according to the robot parameters of the target inspection robot and the preset communication protocol, the current single-step skill action and the corresponding expected execution result are formatted and the current control command is generated. The current control command is sent to the target inspection robot so that the current control command triggers the target inspection robot to perform the corresponding inspection action.

7. The power inspection robot control method as described in claim 1, characterized in that, Based on visual perception data and execution status data, analysis is performed in a pre-defined multimodal visual model to verify whether the execution result of the current inspection action matches the corresponding expected execution result, including: The target state point is located based on visual perception data, and the corresponding text label information is determined based on the location. The text label information and the target state point are matched and aligned across modally using a preset multimodal visual model. After the cross-modal matching and alignment, the real-time running status of the target state point is identified. The execution status data is analyzed to determine the completion rate of the current inspection action of the target inspection robot and the equipment operating parameters. The real-time operating status of the target status point, the completion rate of the current inspection action, and the equipment operating parameters are used as the execution result of the current inspection action. The execution result is compared with the expected execution result corresponding to the current inspection action to determine whether the deviation between the execution result and the expected execution result is less than the preset fault tolerance threshold.

8. A control device for a power inspection robot, characterized in that, include: The inspection instruction generation module is used to perform instruction compliance verification on natural language instructions based on a preset power inspection vector library when natural language instructions are detected, and then generate inspection instructions. The ordered meta-skill action sequence generation module is used to parse the inspection instructions based on the same preset meta-skill set corresponding to each robot to be inspected, the preset substation scene semantic map, and the preset equipment operation mapping association table, generate ordered meta-skill action sequences, and determine the expected execution results corresponding to the ordered meta-skill action sequences. The target inspection robot determination module is used to match all target inspection spatial locations in the substation scene semantic map according to the ordered element skill action sequence, and determine the appropriate target inspection robot according to the initial target inspection spatial location and the current state of each robot to be inspected. The current control command generation module is used to generate current control commands based on the ordered meta-skill action sequence and the expected execution result, and send the current control commands to the target inspection robot so that the target inspection robot can perform the corresponding inspection action; The execution data receiving module is used to receive visual perception data and execution status data of the target inspection robot in real time during the inspection process. The execution result judgment module is used to analyze the visual perception data and execution status data in a preset multimodal vision model to verify whether the execution result of the current inspection action matches the corresponding expected execution result. If yes, the control command for the next inspection action is issued; otherwise, a termination command is issued.

9. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a power inspection robot control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a power inspection robot control method as described in any one of claims 1 to 7.

Images