Control method, device, equipment, medium and program product of nuclear power operation robot

Abstract

A control method and device of a nuclear power operation robot, an equipment, a storage medium and a program product are disclosed. The method comprises the following steps: receiving a task instruction of the nuclear power operation robot, and generating a reference control parameter of the nuclear power operation robot according to the task instruction based on a large language model; acquiring multi-modal perception data of the nuclear power operation robot, determining a state estimation result of the nuclear power operation robot based on the multi-modal perception data and a pre-acquired skill graph according to a world model; and controlling the nuclear power operation robot according to the reference control parameter and the state estimation result. The scheme solves the problem of poor control reliability of the nuclear power operation robot under complex working conditions, enables the nuclear power operation robot through the world model and the large model, fully utilizes multi-modal information, and improves the robustness of the nuclear power operation robot in executing tasks under complex working conditions.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence technology, and in particular to a control method, device, electronic equipment, storage medium, and program product for a nuclear power plant operation and maintenance robot. Background Technology

[0002] Nuclear power plant operation and maintenance is characterized by high radiation, high risk, and high precision requirements. Manual operation and maintenance suffers from problems such as high radiation dose to personnel, low work efficiency, and high risk of human error. Nuclear power operation and maintenance robots have become the core equipment to replace manual labor in completing high-risk operations.

[0003] There are three main technical approaches to the control of existing nuclear power plant operation and maintenance robots: The first approach is based on rule bases, finite state machines, and hierarchical PID control schemes. These schemes can operate reasonably well in structured environments with stable lighting and minimal occlusion, but their performance degrades significantly under complex conditions. For example, unstable lighting or strong glare causes visual recognition drift in nuclear power plant operation and maintenance robots; weak or repetitive textures cause pose drift; interference factors such as steam, condensate, and dust cause short-term sensor failures; and radiation environments increase sensor noise and cause electronic drift. The second approach is based on end-to-end policy learning (such as deep reinforcement learning and imitation learning). End-to-end policy learning has strong expressive power, but it lacks an interpretable mapping from semantic tasks to control parameters and is sensitive to non-ideal factors such as noise, time delay, friction, gaps, and compliant contact, making it difficult to meet the safety and auditability requirements of nuclear power plant operation and maintenance. The third type is a hybrid architecture that combines offline planning with online tracking, which separates the global path from the local tracking. However, it relies heavily on heuristic rules or offline verification for hard constraints such as dose upper limit, peak torque, velocity upper limit, and hold time, making it difficult to guarantee online feasibility and real-time solution.

[0004] Existing technologies struggle to integrate cross-modal information, cross-temporal information, and prior process information, making it impossible to perform stable task reasoning in complex environments such as occlusion, low illumination, and high noise, thus hindering the reliable control of nuclear power plant operation and maintenance robots. Summary of the Invention

[0005] This invention provides a control method, device, equipment, storage medium, and program product for nuclear power plant operation and maintenance robots to solve the problem of poor control reliability of nuclear power plant operation and maintenance robots under complex working conditions. By empowering nuclear power plant operation and maintenance robots with world models and large models, it realizes the full utilization of multimodal information and improves the robustness of nuclear power plant operation and maintenance robots in performing tasks under complex working conditions.

[0006] According to one aspect of the present invention, a control method for a nuclear power plant operation and maintenance robot is provided, the method comprising: Receive task instructions from the nuclear power plant operation and maintenance robot, and generate reference control parameters for the nuclear power plant operation and maintenance robot based on the task instructions using a large language model; Acquire multimodal perception data of nuclear power plant operation and maintenance robots, and determine the state estimation results of nuclear power plant operation and maintenance robots based on the world model, multimodal perception data, and pre-acquired skill maps; The nuclear power plant operation and maintenance robot is controlled based on reference control parameters and state estimation results.

[0007] According to another aspect of the present invention, a control device for a nuclear power plant operation and maintenance robot is provided, the device comprising: The reference control parameter generation module is used to receive the task instructions of the nuclear power operation and maintenance robot and generate the reference control parameters of the nuclear power operation and maintenance robot based on the task instructions using a large language model. The state estimation result determination module is used to acquire multimodal perception data of the nuclear power plant operation and maintenance robot, and determine the state estimation result of the nuclear power plant operation and maintenance robot based on the world model, multimodal perception data and pre-acquired skill map. The robot control module is used to control the nuclear power plant operation and maintenance robot based on reference control parameters and state estimation results.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the control method for the nuclear power plant operation and maintenance robot according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the control method of the nuclear power plant operation and maintenance robot according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the control method for a nuclear power plant operation and maintenance robot as described in any embodiment of the present invention.

[0011] The technical solution of this invention involves receiving task instructions from a nuclear power plant maintenance robot and generating reference control parameters for the robot based on a large language model. It also involves acquiring multimodal perception data from the robot and determining the robot's state estimation results based on a world model, the multimodal perception data, and a pre-acquired skill map. Finally, it controls the robot according to the reference control parameters and the state estimation results. This solution addresses the problem of poor control reliability of nuclear power plant maintenance robots under complex operating conditions. By empowering the robot with a world model and a large model, it fully utilizes multimodal information and improves the robustness of the robot in performing tasks under complex conditions.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a control method for a nuclear power plant operation and maintenance robot according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a control method for a nuclear power plant operation and maintenance robot according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a control device for a nuclear power plant operation and maintenance robot according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the control method of the nuclear power plant operation and maintenance robot according to an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0017] Example 1 Figure 1 This is a flowchart illustrating a control method for a nuclear power plant operation and maintenance robot according to Embodiment 1 of the present invention. This embodiment is applicable to nuclear power plant operation and maintenance scenarios, particularly for the control of embodied robots in nuclear power plant operation and maintenance scenarios. The method can be executed by a control device for the nuclear power plant operation and maintenance robot, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110: Receive the task instructions of the nuclear power plant operation and maintenance robot, and generate reference control parameters for the nuclear power plant operation and maintenance robot based on the task instructions using a large language model.

[0018] This solution can be executed by electronic devices such as computers and servers. Specifically, the electronic devices can be nuclear power plant maintenance robots or management platforms that interact with them. Nuclear power plant maintenance robots are robots used for the maintenance of equipment within nuclear power plants. To meet the needs of different maintenance tasks, the end effector of the robotic arm of a nuclear power plant maintenance robot can be equipped with various types of actuators, such as wrenches, grippers, suction devices, torque drivers, samplers, barcode scanners, and marking devices.

[0019] Users can issue task instructions to nuclear power plant maintenance robots. These instructions can be natural language commands used to direct the robots to perform target tasks. Understandably, in a nuclear power plant maintenance scenario, target tasks may include valve opening and closing, torque control, component inspection and testing, sample collection and sealing marking, and handling and detours for temporary obstacles. Torque control tasks may include task information such as torque stroke, holding time, and reset constraints. Component inspection and testing tasks may include instrument reading detection, pipeline leakage detection, condensate temperature detection, abnormal equipment noise detection, surface crack detection, and coating peeling detection. Sample collection may include task information such as collection volume, contact pressure, contact duration, and cleanliness requirements.

[0020] The electronic equipment can receive task instructions from the nuclear power plant operation and maintenance robot, parse the task instructions based on a large language model, and generate reference control parameters for the nuclear power plant operation and maintenance robot. For example, if the task instruction is "open the valve clockwise to 90 degrees, maximum 40 N·m, hold for 3 seconds", the reference control parameters output by the large language model can include process operation elements such as target angular displacement, maximum torque, holding time, and reset strategy.

[0021] S120. Acquire multimodal perception data of the nuclear power plant operation and maintenance robot. Based on the world model, determine the state estimation result of the nuclear power plant operation and maintenance robot according to the multimodal perception data and the pre-acquired skill map.

[0022] To facilitate observation of nuclear power plant operation and maintenance scenarios, nuclear power plant operation and maintenance robots and equipment within nuclear power plants can be equipped with various types of sensing devices, such as cameras, radar, thermal infrared imagers, acoustic sensors, radiation dosimeters, and inertial measurement units. Electronic devices can acquire multimodal sensing data collected by these devices deployed within the nuclear power plant operation and maintenance scenario.

[0023] Understandably, different modal sensing devices have different sampling frequencies. Electronic devices can perform time alignment and parameter calibration on the multimodal sensing data after acquisition to eliminate spatiotemporal deviations and noise interference from different sensing devices. For time alignment, electronic devices can use the Precision Time Protocol (PTP) to unify the clock reference of all sensing devices. In a specific example, the clock deviation correction formula can be expressed as: ,in This represents the original timestamp of the sensing device. This represents the clock skew calculated by PTP. If the multimodal sensing data includes image data and point cloud data, the electronic device can perform registration of the point cloud data and image data based on calibration board feature matching to obtain distortion-free multimodal sensing data.

[0024] Electronic devices can extract features from multimodal sensing data based on a world model, and then fuse the feature extraction results of each modality to obtain multimodal fused features. Based on the skill map of the nuclear power plant operation and maintenance robot, the electronic devices can estimate the state of the robot and obtain state estimation results.

[0025] The skill graph of a nuclear power plant operation and maintenance robot can be a knowledge graph with skills as nodes and dependencies between skills as edges, and can be built based on the standard operating procedures of nuclear power plant operation and maintenance robots. The skills of a nuclear power plant operation and maintenance robot can include action capabilities such as grasping, aligning, moving, holding, and recognizing, and the dependencies between skills can include the execution order of different skills.

[0026] World models can be used to characterize the causal relationship between nuclear power plant operation and maintenance robots and the operation and maintenance environment. The state estimation results of nuclear power plant operation and maintenance robots can include the ontological states that the robot aims to achieve at different time steps, such as pose and kinematic states. The pose state can include the position and orientation of the nuclear power plant operation and maintenance robot and its end effector, while the kinematic state can be described by kinematic indices such as linear velocity, angular velocity, acceleration, and displacement of the nuclear power plant operation and maintenance robot and its end effector.

[0027] S130. Control the nuclear power plant operation and maintenance robot according to the reference control parameters and the state estimation results.

[0028] The electronic equipment can use reference control parameters as a standard to control the nuclear power plant operation and maintenance robot based on the state estimation results at different time steps until the nuclear power plant operation and maintenance robot completes the target task.

[0029] The technical solution of this invention involves receiving task instructions from a nuclear power plant maintenance robot and generating reference control parameters for the robot based on a large language model. It also involves acquiring multimodal perception data from the robot and determining the robot's state estimation results based on a world model, the multimodal perception data, and a pre-acquired skill map. Finally, it controls the robot according to the reference control parameters and the state estimation results. This solution addresses the problem of poor control reliability of nuclear power plant maintenance robots under complex operating conditions. By empowering the robot with a world model and a large model, it fully utilizes multimodal information and improves the robustness of the robot in performing tasks under complex conditions.

[0030] Example 2 Figure 2 This is a flowchart of a control method for a nuclear power plant operation and maintenance robot provided in Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiment. Figure 2 As shown, the method includes: S201. Receive the task instructions from the nuclear power plant operation and maintenance robot, and generate reference control parameters for the nuclear power plant operation and maintenance robot based on the task instructions using a large language model.

[0031] In this scheme, before determining the state estimation results of the nuclear power plant operation and maintenance robot, the method further includes: Obtain the standard operating procedures for nuclear power plant operation and maintenance robots, and generate a skill map for nuclear power plant operation and maintenance robots based on the standard operating procedures using a large language model.

[0032] Understandably, the standard operating procedures (SOPs) for nuclear power plant maintenance robots guide their operations, essentially serving as their instruction manuals. Electronic devices can access these SOPs, perform text parsing based on a large language model, extract the robot's skills and dependencies, and generate a skill graph using these skills as nodes and their dependencies as edges.

[0033] S202. Obtain multimodal perception data of the nuclear power plant operation and maintenance robot at the current time step.

[0034] Electronic devices can acquire multimodal perception data from nuclear power plant operation and maintenance robots according to a preset cycle, such as acquiring multimodal perception data once every 5ms.

[0035] S203. Based on the attention unit in the world model, the multimodal perception data of the current time step is fused to obtain the first fused feature of the current time step.

[0036] The world model can include attention units built on an attention mechanism for cross-modal feature fusion of multimodal sensing data. Specifically, the attention units in the world model can encode features from the multimodal sensing data at the current time step, calculate the correlation weights between each modality based on the attention mechanism, and perform weighted fusion of the features from each modality based on these correlation weights to obtain the multimodal fused feature, i.e., the first fused feature. The attention units in the world model can also output an uncertainty assessment value of the environmental state simultaneously with the first fused feature.

[0037] Specifically, the first fusion feature can be represented as: ,in, Indicates modal index, Indicates the number of modes. Representing modes Feature encoding of perceptual data, Representing modes The attention weights are calculated based on modal feature similarity. The uncertainty assessment value of the environmental state can be obtained by quantifying the first fused feature based on the feature variance, expressed as: , This represents the function for calculating characteristic variance.

[0038] S204. Based on the temporal memory unit in the world model, the first fusion feature of the current time step and the memory feature of the historical time step are fused to obtain the second fusion feature of the current time step; the memory feature of the historical time step is determined based on the first fusion feature or the second fusion feature of the historical time step.

[0039] After obtaining the first fusion feature of the current time step, the electronic device can perform feature fusion with the memory features of the historical time step based on the temporal memory unit in the world model to obtain the second fusion feature of the current time step, thereby realizing cross-temporal feature fusion.

[0040] Specifically, the temporal memory unit can be built based on the Transformer-XL (Transformer with Extra-Long-term memory) network to capture the temporal dependencies between different time steps during the execution of the target task, so as to solve the problem of short-term perception failure under complex conditions such as occlusion and low light.

[0041] A historical time step can include all historical time steps up to the current time step during the execution of the target task, or it can be a subset of all historical time steps, such as the 10 consecutive time steps preceding the current time step. The memory features of a historical time step can be either the first or second fusion feature of the historical time step, or they can be combined with prior information such as action residuals of the historical time step based on the first or second fusion feature of the historical time step.

[0042] In a specific example, the second fusion feature at the current time step can be represented as: ,in, This represents the first fusion feature at the current time step. The second fusion feature represents the historical time step. The second fusion feature represents the current time step. This indicates a Transformer-XL network.

[0043] S205. Based on the second fusion feature of the current time step and the pre-acquired skill map, the state estimation unit in the world model determines the state estimation result of the nuclear power operation and maintenance robot at the current time step.

[0044] After obtaining the second fusion feature of the current time step, the electronic device can output the state estimation result of the charged maintenance robot for the current time step based on the second fusion feature of the current time step and the pre-acquired skill map, using the state estimation unit in the world model. The state estimation unit in the world model can be constructed based on a nonlinear state mapping function.

[0045] In a specific example, the state estimation result at the current time step can be expressed as: ,in, This represents the first fusion feature at the current time step. The second fusion feature represents the current time step. Represents the characteristics of the skill graph. This represents a nonlinear state mapping function.

[0046] S206. Based on the reference control parameters, the pre-acquired skill map, and the pre-determined passable area, and based on the pre-set optimization target, generate the action sequence for the nuclear power plant operation and maintenance robot to perform the target task.

[0047] The electronic device can use the reference control parameters of the nuclear power operation and maintenance robot determined in S201 as constraints, and based on the pre-set optimization objectives, perform motion planning in the pre-determined passable area according to the skill map of the nuclear power operation and maintenance robot, and generate the motion sequence of the nuclear power operation and maintenance robot to perform the target task.

[0048] In simple terms, a passable area can be a region within the nuclear power plant's operation and maintenance (NPO) environment that the NPO robot can traverse, as pre-determined by electronic devices based on sensing data of the NPO environment and the robot's dimensions. For example, obstacles can be identified using point cloud data of the NPO environment, a grid map of the NPO environment can be constructed, and the passable areas can be marked on the grid map based on the robot's dimensions and the safe distance between the robot and the nuclear power equipment. The sequence of actions for performing the target task can include action description parameters for each time step during the NPO robot's execution of the target task.

[0049] In a specific example, the optimization objective can be represented as a Model Predictive Control (MPC) objective function. For instance, minimizing the total cost of the action sequence for a nuclear power plant maintenance robot to perform its objective task can be expressed as: ,in, This represents the cost resulting from the error between the reference trajectory and the actual trajectory of the nuclear power plant operation and maintenance robot. This indicates the cost of controlling energy consumption using nuclear power plant maintenance robots. This indicates the radiation dose cost of nuclear power plant maintenance robots. This refers to the penalty costs incurred when the actions of nuclear power plant maintenance robots violate constraints such as timing and path boundaries.

[0050] S207. Perform feasibility verification on the action sequence of the nuclear power operation and maintenance robot to perform the target task based on the pre-set safety constraints, and update the action sequence of the nuclear power operation and maintenance robot to perform the target task according to the verification results; the safety constraints are control barrier functions set based on multiple constrained control variables.

[0051] Electronic devices can pre-set constraint rules to constrain the actions of nuclear power plant maintenance robots, such as setting limits on control quantities like radiation dose, contact force, torque, speed, and acceleration. The electronic devices can map multiple constrained control quantities to control barrier functions, and based on these control barrier functions, perform feasibility checks on the action sequence of the nuclear power plant maintenance robot to execute the target task, determining whether there are safety risks in the action description parameters at each time step.

[0052] If the verification results of the action sequence of the nuclear power plant maintenance robot performing the target task reveal any infeasible actions, the electronic equipment can issue an alarm for the target task and trigger an emergency stop to ensure the safety of the nuclear power plant maintenance robot. In the event of infeasible actions in the verification results of the action sequence of the nuclear power plant maintenance robot performing the target task, the electronic equipment can also determine, according to the action planning process in S206, whether the action sequence of the nuclear power plant maintenance robot performing the target task can be adjusted. If it can be adjusted, the action sequence of the nuclear power plant maintenance robot performing the target task will be updated according to the verification results to ensure the reliability of the nuclear power plant maintenance robot in performing the target task.

[0053] S208. Based on the action sequence of the nuclear power operation and maintenance robot performing the target task and the state estimation result of the nuclear power operation and maintenance robot at the current time step, determine the action residual of the current time step; the state estimation result of the nuclear power operation and maintenance robot at the current time step includes the action description parameters of the nuclear power operation and maintenance robot at the current time step.

[0054] As is easily understood, the action sequence of a nuclear power plant operation and maintenance robot performing a target task includes action description parameters for each time step during the task's execution. The state estimation result of the current time step can represent the action description parameters of the current time step. Specifically, the action description parameters can include parameters such as the position of the nuclear power plant operation and maintenance robot, the robot's orientation, the position and orientation of the end effector, the linear velocity of each joint, the angular velocity of each joint, the acceleration of each joint, and the displacement of each joint.

[0055] The electronic device can compare the state estimation result of the nuclear power operation and maintenance robot at the current time step with the action description parameters of each time step in the nuclear power operation and maintenance robot's execution of the target task. It can then determine the action description parameters of the time step that match the current time step from the action sequence of the nuclear power operation and maintenance robot's execution of the target task, and use them as a reference standard for the state estimation result of the current time step to calculate the action residual of the current time step.

[0056] S209. Adjust the action sequence of the target time step in the action sequence of the nuclear power plant operation and maintenance robot to perform the target task based on the action residual of the current time step; the target time step is the time step after the current time step.

[0057] After obtaining the action residual at the current time step, the electronic equipment can replan the action sequence of the nuclear power operation and maintenance robot in the action sequence of the target task after the current time step based on the action residual at the current time step, so as to match the actual action execution of the nuclear power operation and maintenance robot.

[0058] S210. Determine the memory features of the current time step based on the action residual of the current time step, and update the next time step to the current time step.

[0059] To continuously reduce action residuals and improve action planning accuracy, the electronic device can fuse the action residual of the current time step with either the first or second fusion feature of the current time step to obtain the memory feature of the current time step. The current time step is then updated to the historical time step, and the next time step is updated to the current time step, returning to execute S204. By adding the action residuals to the memory feature of the historical time step, more reference information can be provided to the state estimation unit of the world model, thereby improving the accuracy of state estimation in subsequent time steps.

[0060] In a preferred embodiment, the electronic device can record information such as the task instructions of the nuclear power plant operation and maintenance robot, the reference control parameters of the nuclear power plant operation and maintenance robot, the state estimation results of each time step during the execution of the target task, the action sequence of the nuclear power plant operation and maintenance robot in executing the target task, and the feasibility verification results of the action sequence, and generate an audit log to facilitate the tracing of problems in the execution of the target task and improve the interpretability of the nuclear power plant operation and maintenance robot control strategy.

[0061] This technical solution solves the problem of "understanding the task" through a large language model, solves the problem of "understanding the nuclear power plant operation and maintenance environment" through a world model, achieves global action optimization of the target task execution by setting optimization goals, and ensures the safety of the nuclear power plant operation and maintenance robot during the target task execution process through safety constraints. It constructs a closed-loop control architecture of "task understanding - state modeling - control optimization - safety constraints - anomaly replanning", which is conducive to improving the reliability and scenario applicability of nuclear power plant operation and maintenance robots in the operation and maintenance process.

[0062] Example 3 Figure 3 This is a schematic diagram of the control device for a nuclear power plant operation and maintenance robot provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: The reference control parameter generation module 310 is used to receive the task instructions of the nuclear power operation and maintenance robot and generate the reference control parameters of the nuclear power operation and maintenance robot based on the task instructions according to the large language model. The state estimation result determination module 320 is used to acquire multimodal perception data of the nuclear power operation and maintenance robot, and determine the state estimation result of the nuclear power operation and maintenance robot based on the world model, multimodal perception data and pre-acquired skill map. The robot control module 330 is used to control the nuclear power plant operation and maintenance robot based on reference control parameters and state estimation results.

[0063] In this solution, the device further includes: The skill map generation module is used to obtain the standard operating procedures of the nuclear power operation and maintenance robot before determining the state estimation results of the nuclear power operation and maintenance robot, and generate the skill map of the nuclear power operation and maintenance robot based on the standard operating procedures according to the large language model.

[0064] In one feasible solution, the state estimation result determination module is specifically used for: Based on the attention unit in the world model, the multimodal perception data of the current time step is fused to obtain the first fused feature of the current time step; Based on the temporal memory unit in the world model, the first fusion feature of the current time step and the memory feature of the historical time step are fused to obtain the second fusion feature of the current time step; the memory feature of the historical time step is determined based on the first fusion feature or the second fusion feature of the historical time step. Based on the state estimation unit in the world model, the state estimation result of the nuclear power operation and maintenance robot at the current time step is determined according to the second fusion feature of the current time step and the pre-acquired skill map.

[0065] Based on the above solution, the robot control module is specifically used for: Based on reference control parameters, pre-acquired skill maps, and pre-determined passable areas, and based on pre-set optimization objectives, a sequence of actions is generated for the nuclear power plant operation and maintenance robot to perform the target task. Based on the action sequence of the nuclear power plant operation and maintenance robot performing the target task and the state estimation result of the nuclear power plant operation and maintenance robot at the current time step, the action residual of the current time step is determined; the state estimation result of the nuclear power plant operation and maintenance robot at the current time step includes the action description parameters of the nuclear power plant operation and maintenance robot at the current time step. The action sequence of the nuclear power plant operation and maintenance robot in the action sequence of the target time step is adjusted based on the action residual of the current time step; the target time step is the time step after the current time step.

[0066] Optionally, the device further includes: The feasibility verification module is used to verify the feasibility of the action sequence of the nuclear power operation and maintenance robot to perform the target task after generating the action sequence of the nuclear power operation and maintenance robot to perform the target task based on the pre-set safety constraints, and update the action sequence of the nuclear power operation and maintenance robot to perform the target task according to the verification results; the safety constraints are control barrier functions set based on multiple constrained control variables.

[0067] In a preferred embodiment, the device further includes: The memory feature generation module is used to determine the memory features of the current time step based on the action residual of the current time step after determining the action residual of the current time step, so as to generate the state estimation result of the next time step.

[0068] The control device for the nuclear power plant operation and maintenance robot provided in this embodiment of the invention can execute the control method for the nuclear power plant operation and maintenance robot provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0069] Example 4 Figure 4 A schematic diagram of an electronic device 410 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0070] like Figure 4As shown, the electronic device 410 includes at least one processor 411 and a memory communicatively connected to the at least one processor 411. The memory may be a read-only memory (ROM) 412, a random access memory (RAM) 413, etc. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0071] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0072] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as the control methods for nuclear power plant operation and maintenance robots.

[0073] In some embodiments, the control method for the nuclear power plant maintenance robot can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the control method for the nuclear power plant maintenance robot described above can be performed. Alternatively, in other embodiments, processor 411 can be configured to execute the control method for the nuclear power plant maintenance robot by any other suitable means (e.g., by means of firmware).

[0074] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0075] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable nuclear power plant operation and maintenance robot control device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0076] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0077] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0078] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0079] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0080] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0081] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A control method for a nuclear power plant operation and maintenance robot, characterized in that, The method includes: Receive task instructions from the nuclear power plant operation and maintenance robot, and generate reference control parameters for the nuclear power plant operation and maintenance robot based on the task instructions using a large language model; Acquire multimodal perception data of nuclear power plant operation and maintenance robots, and determine the state estimation results of nuclear power plant operation and maintenance robots based on the world model, multimodal perception data, and pre-acquired skill maps; The nuclear power plant operation and maintenance robot is controlled based on reference control parameters and state estimation results.

2. The method according to claim 1, characterized in that, Before determining the state estimation results of the nuclear power plant maintenance robot, the method further includes: Obtain the standard operating procedures for nuclear power plant operation and maintenance robots, and generate a skill map for nuclear power plant operation and maintenance robots based on the standard operating procedures using a large language model.

3. The method according to claim 1, characterized in that, The state estimation results of the nuclear power plant operation and maintenance robot, determined based on the world model using multimodal perception data and pre-acquired skill maps, include: Based on the attention unit in the world model, the multimodal perception data of the current time step is fused to obtain the first fused feature of the current time step; Based on the temporal memory unit in the world model, the first fusion feature of the current time step and the memory feature of the historical time step are fused to obtain the second fusion feature of the current time step; the memory feature of the historical time step is determined based on the first fusion feature or the second fusion feature of the historical time step. Based on the state estimation unit in the world model, the state estimation result of the nuclear power operation and maintenance robot at the current time step is determined according to the second fusion feature of the current time step and the pre-acquired skill map.

4. The method according to claim 3, characterized in that, The control of the nuclear power plant operation and maintenance robot based on reference control parameters and state estimation results includes: Based on reference control parameters, pre-acquired skill maps, and pre-determined passable areas, and based on pre-set optimization objectives, a sequence of actions is generated for the nuclear power plant operation and maintenance robot to perform the target task. Based on the action sequence of the nuclear power plant operation and maintenance robot performing the target task and the state estimation result of the nuclear power plant operation and maintenance robot at the current time step, the action residual of the current time step is determined; the state estimation result of the nuclear power plant operation and maintenance robot at the current time step includes the action description parameters of the nuclear power plant operation and maintenance robot at the current time step. The action sequence of the nuclear power plant operation and maintenance robot in the action sequence of the target time step is adjusted based on the action residual of the current time step; the target time step is the time step after the current time step.

5. The method according to claim 4, characterized in that, After generating the action sequence for the nuclear power plant operation and maintenance robot to perform the target task, the method further includes: The feasibility of the action sequence of the nuclear power plant operation and maintenance robot to perform the target task is verified based on the pre-set safety constraints, and the action sequence of the nuclear power plant operation and maintenance robot to perform the target task is updated according to the verification results; the safety constraints are control barrier functions set based on multiple constrained control variables.

6. The method according to claim 4, characterized in that, After determining the action residual at the current time step, the method further includes: The memory features of the current time step are determined based on the action residual of the current time step in order to generate the state estimation result of the next time step.

7. A control device for a nuclear power plant operation and maintenance robot, characterized in that, The device includes: The reference control parameter generation module is used to receive the task instructions of the nuclear power operation and maintenance robot and generate the reference control parameters of the nuclear power operation and maintenance robot based on the task instructions according to the large language model. The state estimation result determination module is used to acquire multimodal perception data of the nuclear power plant operation and maintenance robot, and determine the state estimation result of the nuclear power plant operation and maintenance robot based on the world model, multimodal perception data and pre-acquired skill map. The robot control module is used to control the nuclear power plant operation and maintenance robot based on reference control parameters and state estimation results.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the control method for the nuclear power plant operation and maintenance robot according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the control method of the nuclear power plant operation and maintenance robot according to any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the control method for the nuclear power plant operation and maintenance robot according to any one of claims 1-6.

Images