A nuclear power inspection robot navigation method, system, device and medium

Abstract

The application discloses a kind of nuclear power inspection robot navigation method, system, equipment and medium, comprising: obtaining laser radar data and visual image, constructs the environment state chart including robot state and obstacle topology, and utilizes space-time feature extraction network fusion generation environment feature vector;It inputs the reinforcement learning strategy network, and the novelty internal reward and environmental external reward drive network output original movement action are combined;Safety correction module is constructed, the direction of the resultant force of target gravity and obstacle repulsion is calculated, and based on its and the direction consistency of original action, action probability is weightedly corrected, and safety control action is generated;Optimize path using local trajectory optimization algorithm, and complete instrument identification by combining adaptive positioning and image correction matching.The application significantly improves the autonomy and robustness of inspection task.

Description

Technical Field

[0001] This invention relates to the field of automatic navigation technology for nuclear power plant inspection devices, and in particular to a navigation method, system, equipment and medium for a nuclear power plant inspection robot. Background Technology

[0002] Currently, there are four representative solutions for inspection robot technology, but all of them have significant technical shortcomings. First, traditional manual and semi-automatic technologies rely heavily on preset tracks or simple obstacle avoidance for navigation, making them unable to autonomously explore complex, unstructured areas and lacking intelligent diagnostic capabilities for temporal signals. Second, positioning and navigation technologies based on single or simple fusion methods, such as SINS, RFID, or traditional laser SLAM, generally suffer from error accumulation and drift in GPS-denied environments, lack of continuous pose feedback, or severe interference from dust and lighting. Third, detection technologies based on traditional vision and machine learning suffer from high false negative rates in low-light, high-dust scenarios due to existing YOLO or CNN architectures, and lack a spatiotemporal synchronization fusion mechanism, resulting in low accuracy in identifying minor defects. Fourth, path planning technologies based on traditional sampling algorithms, such as RRT and A* algorithms, only pursue the shortest geometric distance while ignoring the attitude and direction constraints when approaching the target, resulting in "reaching the point but unable to perform the task," and exhibiting slow convergence and tortuous paths in dense obstacles.

[0003] In summary, existing inspection robot systems generally suffer from four core pain points: (1) The contradiction between positioning accuracy and robustness makes it difficult for a single sensor to achieve both high accuracy and high stability; (2) The accuracy of intelligent detection is insufficient, the model has poor generalization ability in complex dynamic environments, and the false alarm rate and false negative rate are high; (3) The path planning lacks engineering practicality and ignores the posture requirements during actual operation, resulting in operation failure; (4) The fusion of multimodal data is lagging behind and lacks a deep feature coupling mechanism based on spatiotemporal alignment, resulting in incomplete environmental perception.

[0004] These problems severely restrict the unmanned and intelligent application of inspection robots in complex environments such as nuclear power plants. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides a navigation method, system, equipment, and medium for nuclear power plant inspection robots to address the shortcomings of existing inspection technologies, such as poor positioning drift and robustness, low detection accuracy in complex environments, lack of operational posture constraints in path planning, and lag in multimodal data fusion, which make it difficult to meet the unmanned and intelligent inspection needs of complex scenarios such as nuclear power plants.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a navigation method for a nuclear power plant inspection robot, comprising: Acquire LiDAR data and visual images to construct an environmental state map that includes the robot's state and obstacle topology. Then, use a spatial feature extraction network and a temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, and fuse them to generate an environmental feature vector. The environmental feature vector is input into the reinforcement learning policy network, and the novelty intrinsic reward of the current state is calculated using the intrinsic reward calculation module. The total reward is obtained by combining the extrinsic reward from the environmental feedback, which drives the policy network to output the robot's original motion action. A safety correction module is constructed and the direction of the resultant force of the attraction and repulsion of the target point and the obstacle is calculated. At the same time, the direction consistency parameter between the resultant force direction and the original motion action is calculated, and the probability distribution of the original motion action is weighted and corrected based on the direction consistency parameter to generate a safety control action. The safety control actions are input to the robot's bottom execution layer, and the local trajectory optimization algorithm is used to optimize the global path locally. During the movement, the adaptive positioning algorithm is used to perform real-time pose estimation, and the visual image is simultaneously distorted and matched with a template to complete the instrument reading recognition.

[0008] As a preferred embodiment of the nuclear power plant inspection robot navigation method of the present invention, the step of extracting the spatial features of the environmental state map and the temporal features of the historical trajectory using a spatial feature extraction network and a temporal feature extraction network respectively includes: A graph convolutional network is used to aggregate the node features and adjacency matrix in the environmental state graph to extract spatial feature vectors that represent the topological relationships of obstacles. The sequence data of the historical trajectory is received using a long short-term memory network, and the cell state is updated through a forget gate, an input gate, and an output gate to extract a temporal feature vector characterizing the robot's motion trend. The robot body is equipped with a robot main control development board and a 3D LiDAR. The neural network processor built into the main control development board accelerates the reasoning process of graph convolutional networks and long short-term memory networks.

[0009] As a preferred embodiment of the nuclear power plant inspection robot navigation method of the present invention, the step of inputting the environmental feature vector into the reinforcement learning policy network and calculating the novelty intrinsic reward of the current state using the intrinsic reward calculation module includes: The environmental feature vector is input into the policy network and value network constructed based on the improved proximal policy optimization algorithm; The novelty intrinsic reward of the current state is quantified by using a random network distillation module as the intrinsic reward calculation module, which calculates the Euclidean distance between the fixed random target network and the trainable predictive network on the output features. The improved near-end strategy optimization algorithm utilizes the multi-threading mechanism of the robot's main control development board for parallel sampling and network updates.

[0010] The beneficial effects of this preferred technical solution are that by distilling the novelty of the state through random network to construct intrinsic rewards, and by combining the improved proximal policy optimization algorithm and the multi-threaded parallel sampling and update mechanism, the problem of insufficient exploration caused by the sparse rewards in the complex environment of nuclear power plants is effectively solved, and the robot's autonomous exploration efficiency, policy learning stability and training convergence speed are significantly improved.

[0011] As a preferred embodiment of the navigation method for nuclear power plant inspection robots described in this invention, the step of calculating the directional consistency parameter between the resultant force direction and the original motion includes: The direction of the resultant force of the attraction at the target point and the repulsion of the obstacle is normalized to the resultant vector of attraction and repulsion. The original motion action is mapped to the expected vector of the motion probability distribution; Calculate the cosine similarity between the resultant vector of the gravitational and repulsive forces and the desired vector, and use the cosine similarity as the direction consistency parameter; The calculation of the resultant force direction is based on an artificial potential field algorithm, in which the gravitational potential field guides the robot to move toward the target point, and the repulsive potential field takes effect when the robot enters the obstacle safety threshold.

[0012] The beneficial effects of this preferred technical solution are that by using cosine similarity to quantify the consistency between the original action of reinforcement learning and the force direction of the artificial potential field, and accordingly making weighted corrections to the action probability distribution, it effectively avoids the risk of blind exploration by a single reinforcement learning strategy in a local complex obstacle environment. While ensuring the robot's real-time obstacle avoidance safety, it significantly improves the smoothness and robustness of motion decision-making.

[0013] As a preferred embodiment of the nuclear power plant inspection robot navigation method of the present invention, the step of weighting and correcting the probability distribution of the original motion action based on the direction consistency parameter to generate safety control actions includes: Construct a monotonically increasing weight function with the cosine similarity as the variable, and calculate the corrected weights of each motion component in the original motion. The original action probability distribution output by the policy network is masked and weighted using the corrected weights to suppress the action probability that the angle between the action and the resultant force direction is greater than a preset angle threshold, thereby generating a safety control action. The safety control action is output as a speed command to the differential drive chassis, driving the robot to perform movements that conform to the approach posture constraints.

[0014] As a preferred embodiment of the nuclear power plant inspection robot navigation method of the present invention, the step of using a local trajectory optimization algorithm to optimize the local trajectory of the global path to drive the robot to move includes: A time-elastic band algorithm is used to construct a band graph model that includes robot kinematic and dynamic constraints; The safety control action is transformed into the external potential force of the graph model, and the energy function of the graph model is minimized by a nonlinear optimization solver to generate a collision-free local optimization trajectory. During the movement, the adaptive Monte Carlo localization algorithm combined with the dynamic particle injection strategy is used for real-time pose estimation, including maintaining the short-term likelihood mean and the long-term likelihood mean. The dynamic particle injection strategy includes determining that the localization is lost and triggering random particle injection when the short-term likelihood mean is much smaller than the long-term likelihood mean.

[0015] As a preferred embodiment of the nuclear power plant inspection robot navigation method of the present invention, the step of using an adaptive positioning algorithm for real-time pose estimation includes: An adaptive Monte Carlo localization algorithm is used to initialize the particle set, and the importance of the particles is sampled based on the observation likelihood of LiDAR data and visual images. The covariance of robot pose estimation is calculated in real time. When the covariance exceeds a preset covariance threshold, a dynamic particle injection strategy is triggered to randomly inject new particles in regions with high likelihood. Simultaneously perform distortion correction and template matching on the visual image to complete the instrument reading recognition, including: The camera intrinsic parameter matrix and distortion coefficients are obtained by combining Zhang Zhengyou calibration method with Brownian distortion model, and the image is then processed to remove distortion. The distortion-free image is subjected to adaptive threshold binarization, and the relative mean correlation between the template and the image is calculated using a correlation coefficient matching algorithm to identify the status of instruments and indicator lights.

[0016] Secondly, the present invention provides a nuclear power plant inspection robot navigation system, comprising: The feature fusion module is used to acquire LiDAR data and visual images, construct an environmental state map that includes the robot's state and obstacle topology, and use a spatial feature extraction network and a temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, and fuse them to generate an environmental feature vector. The reward module is used to input the environmental feature vector into the reinforcement learning policy network, and use the intrinsic reward calculation module to calculate the novelty intrinsic reward of the current state and combine it with the extrinsic reward of the environmental feedback to obtain the total reward, which drives the policy network to output the robot's original motion action. The safety potential field correction module is used to construct a safety correction module and calculate the direction of the resultant force of the attraction of the target point and the repulsion of the obstacle. At the same time, it calculates the direction consistency parameter between the resultant force direction and the original motion action, and performs weighted correction on the probability distribution of the original motion action based on the direction consistency parameter to generate a safety control action. The safety potential field correction module is used to input the safety control actions to the robot's bottom execution layer and use a local trajectory optimization algorithm to optimize the local trajectory of the global path. During the movement, it uses an adaptive positioning algorithm to perform real-time pose estimation and simultaneously performs distortion correction and template matching on the visual image to complete the recognition of instrument readings.

[0017] Thirdly, the present invention provides an electronic device, comprising: Memory, used to store programs; A processor is configured to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the nuclear power plant inspection robot navigation method.

[0018] Fourthly, the present invention provides a computer-readable storage medium, comprising: when the program is executed by a processor, the steps of implementing the nuclear power plant inspection robot navigation method.

[0019] The beneficial effects of this invention are as follows: By employing the Zhang Zhengyou calibration method combined with the Brownian distortion model to obtain the camera intrinsic parameter matrix and distortion coefficients, and performing distortion removal processing and adaptive threshold binarization on the image, this invention achieves high-precision preprocessing of visual images, effectively eliminating the interference of lens distortion on subsequent recognition; by utilizing the correlation coefficient matching algorithm to calculate the relative mean correlation between the template and the image, accurate recognition of instrument readings and indicator light status is achieved in the complex environment of nuclear power plants, significantly improving the accuracy and automation of inspection data; by converting safety control actions into external potential forces of a graphical model and minimizing the energy function using a nonlinear optimization solver, rapid generation of collision-free local optimization trajectories is achieved, significantly enhancing the robot's obstacle avoidance ability and trajectory smoothness in dynamic environments; by maintaining short-term and long-term likelihood means and triggering random particle injection, real-time accurate estimation of robot pose and rapid recovery after positioning loss are achieved during movement, ensuring the continuity and reliability of inspection tasks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a basic flowchart illustrating a navigation method for a nuclear power plant inspection robot according to an embodiment of the present invention. Figure 2 This is a diagram of a reinforcement learning navigation architecture for a nuclear power plant inspection robot navigation method that incorporates artificial potential field safety correction, provided in one embodiment of the present invention. Figure 3 This is a diagram illustrating a reinforcement learning training architecture based on the intrinsic reward of a random network distillation for a nuclear power plant inspection robot navigation method, as provided in one embodiment of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0022] Example 1, referring to Figure 1 As one embodiment of the present invention, a navigation method for a nuclear power plant inspection robot is provided, comprising: S100: Acquire LiDAR data and visual images, construct an environmental state map including robot state and obstacle topology, and use spatial feature extraction network and temporal feature extraction network to extract spatial features of environmental state map and temporal features of historical trajectory respectively, and fuse them to generate environmental feature vector; S200: The environmental feature vector is input into the reinforcement learning policy network, and the novelty intrinsic reward of the current state is calculated using the intrinsic reward calculation module. The total reward is obtained by combining the extrinsic reward from the environmental feedback, which drives the policy network to output the robot's original motion action. S300: Construct a safety correction module and calculate the direction of the resultant force of the attraction and repulsion of the target point and the obstacle. At the same time, calculate the direction consistency parameter between the resultant force direction and the original motion action, and perform weighted correction on the probability distribution of the original motion action based on the direction consistency parameter to generate a safety control action. S400: Inputs safety control actions to the robot's bottom execution layer and uses a local trajectory optimization algorithm to optimize the local trajectory of the global path. During movement, it uses an adaptive positioning algorithm to perform real-time pose estimation and simultaneously performs distortion correction and template matching on the visual image to complete the recognition of instrument readings.

[0023] It should be noted that inspection robots in nuclear power plants operate for extended periods in environments characterized by high radiation, strong electromagnetic interference, and complex, unstructured environments, placing extremely high demands on the robustness of their perception systems and decision-making algorithms. During movement, the robot's visual sensors are susceptible to lens distortion and changes in lighting, leading to difficulties in recognizing instrument readings. Simultaneously, the dense and dynamically unpredictable distribution of obstacles in confined spaces makes a single reinforcement learning strategy prone to blind collisions in the early stages of exploration, lacking robust safety constraints. Furthermore, due to slippery surfaces or missing texture features, the robot is highly susceptible to pose estimation drift or even loss of localization, resulting in inspection mission interruptions. Therefore, achieving high-precision instrument recognition and stable pose estimation while ensuring real-time obstacle avoidance safety is crucial for improving the autonomous inspection capabilities of nuclear power robots.

[0024] Therefore, addressing the shortcomings of existing inspection technologies, such as poor positioning drift and robustness, low detection accuracy in complex environments, lack of operational posture constraints in path planning, and lagging multimodal data fusion, which make it difficult to meet the unmanned and intelligent inspection needs of complex scenarios such as nuclear power plants, the following steps (S100-S400) are implemented. These steps involve introducing Long Short Term Memory (LSTM) and Graph Convolutional Networks (GCN) into the original network structure to enhance the robot's understanding of historical trajectories and environmental topology; introducing Random Network Distillation (RND) to improve exploration efficiency, find better solutions, and significantly accelerate policy convergence; and after generating actions based on the improved PPO method, performing secondary corrections based on an Artificial Potential Field (APF) to ensure the robot's real-time obstacle avoidance capability in complex environments.

[0025] Example 2, refer to Figures 2-3 As one embodiment of the present invention, a navigation method for a nuclear power plant inspection robot is provided based on the previous embodiment, comprising: In this embodiment, the autonomous inspection robot system includes a mobile chassis, a sensing module, a main control module, and a power supply module.

[0026] The mobile chassis adopts a differential drive structure, is equipped with a 5-inch brushless hub motor and 2-inch universal wheels, and has a maximum operating speed of 2.4m / s to meet the needs of rapid movement in complex terrains such as nuclear power plant tunnels.

[0027] The perception module includes the Leishen M10P lidar for 360° scanning and ranging with a measurement radius of up to 30m, and for constructing environmental point clouds; and the BL-DT120 intelligent dual-light thermal imaging camera for simultaneously acquiring visible light and infrared thermal imaging data to achieve multimodal perception.

[0028] The main control module is equipped with a ROS main control development board based on the RK3566 chip, with a built-in independent NPU and a computing power of 1 TOPS, used to run navigation stack, image processing algorithms and deep learning models.

[0029] The power supply module uses a 24V 6100mAh lithium battery, which supports the robot to operate for up to 11 hours under light load, meeting the needs of long-term inspection tasks.

[0030] In this embodiment of the application, step S100 involves acquiring the LiDAR data and visual images of the robot body in the target environment. The aim is to use multimodal sensors to perceive any complex scene in which the robot is located, construct an environmental state map that includes the robot's state and the topology of obstacles, and use spatial feature extraction network and temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, and fuse them to generate an environmental feature vector, so as to achieve accurate perception and navigation of the robot in complex dynamic environments.

[0031] In this embodiment of the application, step S100 uses a spatial feature extraction network and a temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, including: A Graph Convolutional Network (GCN) is used to aggregate node features and adjacency matrices in the environment state graph to extract spatial feature vectors representing the topological relationships of obstacles. The GCN employs a 2-layer graph convolutional structure with a hidden layer dimension of 64 and a ReLU activation function. The Long Short-Term Memory (LSTM) network uses a single-layer structure with a hidden state dimension of 128. In the Random Network Distillation (RND) module, both the fixed random target network and the trainable prediction network use a 3-layer Multilayer Perceptron (MLP) with a hidden layer dimension of 128 and an output feature dimension of 32. The node feature extraction method is as follows: robot node features include their two-dimensional coordinates, linear velocity, and orientation angle; obstacle node features are extracted from the cluster centers of the LiDAR point cloud and include relative distance and relative orientation angle; target node features are the distance and orientation of the target point relative to the robot; the above features are concatenated to form a node feature vector.

[0032] By using a Long Short-Term Memory (LSTM) network to receive sequence data of historical trajectories, and updating cell states through forget gates, input gates, and output gates, a temporal feature vector characterizing the robot's motion trend is extracted. The robot body is equipped with a robot main control development board and a 3D LiDAR. The neural network processor built into the main control development board accelerates the reasoning process of graph convolutional networks and long short-term memory networks.

[0033] In an optional implementation, the spatial feature extraction network in step S100 can also convert the point cloud data collected by the lidar into a two-dimensional grid map, and use a convolutional neural network (CNN) to perform convolution operations on the grid map, thereby extracting feature vectors containing the distribution of environmental obstacles and spatial geometric information.

[0034] In an optional implementation, the spatial feature extraction network in step S100 can also directly acquire the original point cloud data sequence collected by the lidar, and use the self-attention mechanism of the PointNet network or Transformer to encode the features of the point cloud data, thereby extracting feature vectors containing environmental spatial geometric information and long-distance dependencies.

[0035] In this embodiment of the application, in order to effectively solve the complex navigation problem in a nuclear power environment through reinforcement learning, the present invention models the robot navigation task as a Markov Decision Process (MDP), specifically consisting of quadruples. Describe it. Among them, This represents the state space, indicating all possible states in the environment. At each time step, the robot constructs its perceived state based on environmental observations acquired by sensors. Specifically, time Perception state It is composed of the robot's own state and relative information about the environment, and its mathematical expression is:

[0036] in, This represents the robot's position coordinates. For linear velocity, For the orientation angle, Angular velocity; Covering robots and the environment Relative distances of entities (including static walls, dynamic crowds, and navigation targets) and relative direction Environmental robots are based on this perceived state. Make decisions. Represents the state transition probability, that is, in the current state Take action below Then transition to the next state The probability distribution of the robot's position and posture after performing an action is not only affected by its own dynamics, but also closely related to the dynamic changes of the environment and external disturbances.

[0037] In this embodiment of the application, the specific process by which the robot makes decisions through the policy network is as follows: Figure 2 As shown, the graph state is constructed. The environment includes three types of nodes: robots, obstacles, and navigation targets. The node feature matrix... Middle node eigenvectors ,in Identify node types; construct an adjacency matrix. The distance of the robot to other nodes The value is 1 if it is less than the perception threshold, and 0 otherwise. The graph state is fed into a shared GCN layer to obtain the spatial structure feature vector. , will feature and The concatenated vectors are then fed into an MLP layer to obtain feature vectors. Then, timing information is obtained by timing fusion through an LSTM network. Based on state representation The strategy head calculates the probability distribution of actions. And sample the original action instructions The value head of the value network is based on the same state representation. Estimate the value of the current state .

[0038] In this embodiment of the application, step S200 involves inputting the environmental feature vector into the reinforcement learning policy network and calculating the novelty intrinsic reward of the current state using the intrinsic reward calculation module, including: The environmental feature vectors are input into the policy network and value network constructed based on the improved proximal policy optimization algorithm; By using a random network distillation module as an intrinsic reward calculation module, the novelty intrinsic reward of the current state is quantified by calculating the Euclidean distance between the fixed random target network and the trainable predictive network on the output features. The improved near-end strategy optimization algorithm utilizes the multi-threading mechanism of the robot's main control development board for parallel sampling and network updates.

[0039] In action space By definition, policy networks are based on the current state. Output the original action vector Including robot linear velocity and angular velocity To ensure safety, this invention introduces an APF safety correction module. This module calculates the artificial potential field force vector based on the gravitational and repulsive potential fields between the robot and the obstacle and target point. By calculating the cosine similarity between the original action direction and the potential field direction, the action probability is weighted and corrected. This strengthens actions that approach the navigation target and suppresses actions that may lead to collisions, ultimately resulting in safe and efficient control actions. .

[0040] Specifically, the total APF force Gravity generated by the target point Repulsive force generated by obstacles Linear superposition, i.e. .

[0041] Among them, gravity The formula used to guide the robot to move towards the target point is:

[0042] In the formula, Let the target point's position vector be... This is the robot's current position vector. This is the gravitational gain coefficient.

[0043] Among them, repulsive force It is a collection of obstacles The repulsive force generated by each obstacle i is calculated by the repulsive force gain coefficient. The Euclidean distance from the robot to the i-th obstacle Threshold of repulsive force range ,obstacle position vector and the unit vector pointing from obstacle i to the robot Calculated. The repulsive force only acts when the robot enters the area of ​​effect of the obstacle. It takes effect immediately, and the closer the robot is to the obstacle, the stronger the repulsive force, forcing the robot to move away from the obstacle to avoid collision. The calculation formula is as follows:

[0044] The force vector in the artificial potential field is calculated. Then, the system further executes the action probability correction logic. This is done by calculating the original action vector output by the policy network. Force vector in artificial potential field Cosine similarity between Quantify the safety and rationality of the direction of action:

[0045] Using this similarity weight to analyze the original action probability distribution Mask weighting is applied and normalized using the Softmax function to generate a corrected safe action probability distribution. :

[0046] In the formula, This represents the corrected probability distribution of safe actions. This represents the original action probability distribution output by the policy network. Represents the normalization function. Represents a logarithmic function; Indicates the robot at time step The perceived current environmental state; The robot obtains the final safety control action based on the corrected probability distribution sampling. This mechanism ensures that... Within the danger zone, the probability of actions aligned with the direction of repulsion is suppressed, while the probability of actions aligned with the direction of gravity or the safe passage is enhanced, thus ensuring the robot's real-time obstacle avoidance safety at the probabilistic level.

[0047] In terms of learning objectives, the robot's task is to learn an excellent strategy based on the current state. Make decisions to complete navigation. A reward function is designed based on this objective. Includes target rewards and collision rewards Among them, target reward The formula for guiding the robot toward the target is defined as follows:

[0048] in S A positive reward constant, representing the current distance the robot travels to the designated target. Less than or equal to the threshold The reward is given when navigation is successful; otherwise, the reward is given. Defined as the distance from the robot to the target in the previous time step. Distance from the target at the current time step The difference in distance is rewarded positively if the distance decreases, and negatively if the distance increases. Collision Rewards The formula for calculating obstacle avoidance is defined as follows:

[0049] In the formula, C<0 This represents the negative reward given when the robot collides with another object. This indicates the distance between the robot and the nearest obstacle. Indicates the collision threshold. Indicates the safety threshold, when The probability of a collision is high when the value is below the safety threshold, necessitating adjustments to the strategy. <0 indicates that the robot is in danger but a collision has not yet occurred. and The product is used to calculate the penalty value at this point.

[0050] The specific values ​​of the aforementioned thresholds can be set based on the actual physical dimensions and motion performance of the robot. Specifically, the collision threshold... The obstacle avoidance threshold can be set based on the robot's maximum radius and braking distance; for example, it can be set to 0.2m to 0.5m to ensure that the robot can trigger the highest level of obstacle avoidance penalty before physical contact occurs. It needs to be greater than the collision threshold to reserve a safe buffer space for the robot; for example, it can be set to 1.0m. In practical applications, it can be adaptively adjusted according to the specific inspection environment (such as narrow corridors or open halls).

[0051] In this embodiment, training is performed based on an improved Proximal Policy Optimization (PPO) method, with the original policy network and value network sharing LSTM and GCN layer structures. When training using reinforcement learning methods, system initialization is required, including environment initialization and setting... The distribution of obstacles, the target location, and the robot's initial state; initialization of the policy network. parameters Value Network parameters Simultaneously initialize the Random Network Distillation (RND) module, including the RND target network. parameters (This parameter remains constant throughout the learning process) and the RND prediction network. parameters (This parameter needs to be updated through learning).

[0052] In an optional implementation, the reinforcement learning strategy in step S200 can also utilize a deep neural network to construct a value network, input the extracted environmental feature vectors into the network to directly output the Q-value (expected return) of each candidate action, and select the action with the largest Q-value as the robot's original motion action, while iteratively updating the network parameters by minimizing the value loss function of the Q-value prediction error.

[0053] In an optional implementation, the reinforcement learning policy in step S200 can also construct a stochastic policy network that simultaneously outputs the mean and variance of the action probability distribution, and introduce a policy entropy regularization term into the reward function. By maximizing the cumulative reward and policy entropy at the same time, the network parameters are driven to update iteratively, and finally the robot's original motion actions are sampled from the output probability distribution.

[0054] The cyclical process of the robot interacting with the environment in real time to collect experience data is as follows: ① Round initialization, time step Reset environment status Initialize the LSTM hidden state ; ② Each time step The robot obtains raw state data through sensors. Build graph state The original action is obtained through policy network decision-making. Safety actions are obtained through the APF safety correction module. ; ③ The robot performs actions transition to a new state According to the preset reward function External rewards for obtaining environmental feedback Collect experience data Stored in the experience pool; ④ If the round does not end, update the state and LSTM hidden state to proceed to the next time step; if the round ends or the total number of experience steps collected reaches the preset size. (like If the process continues (step 1), then the system enters the network update phase to optimize the robot's strategy.

[0055] In this embodiment of the application, after collecting enough N pieces of experience data, interaction with the environment is paused, and the network is updated using the collected experience data. For example... Figure 3 As shown, the network update process specifically includes the following steps: ① Calculate intrinsic reward using RND: by state As input, the RND module contains the target network. and prediction networks The state Input a fixed target network to obtain target features ; will state Input the prediction network to be trained to obtain the predicted features. The squared Euclidean distance between two feature outputs is calculated as the intrinsic reward. The intrinsic reward scaling factor To balance the magnitudes of intrinsic and extrinsic rewards and prevent excessively large intrinsic rewards from causing oscillations during policy network training, its value can be set from 0.01 to 0.1. The extrinsic reward at each step is added to the scaled intrinsic reward to obtain the total reward used for learning. .

[0056] ② Perform a PPO main network update. Using the experience data collected in the cache, obtain the total reward that combines intrinsic and extrinsic rewards. and the value of collection The advantage value for each action is calculated using generalized advantage estimation (GAE). To calculate the updated gradient, the empirical data needs to be input into the policy network using the latest network parameters. and value network Perform forward propagation again to calculate the new action probability. And new state value .

[0057] Calculate the loss of the PPO strategy:

[0058] Where N is the size of the collected empirical data. Is it a matter of new and old strategies in a state? Select action probability ratio, advantage value It is a state Select action Compared to the average quality of movement, The clipping factor is `clip(·)`, which represents the `clip` function. The `clip` operation clips the size to a specified value. Within the specified range, ensure that updates to the new strategy do not deviate too far from the old strategy, thus guaranteeing stability.

[0059] Calculate the loss of value:

[0060] The update objective is to minimize the prediction of the value network. Compared to the target value calculated based on the true total reward The gap between them.

[0061] Calculate the entropy regularization term:

[0062] in It is the action space. It is in state The probability of choosing action 'a' is given by the entropy value. A higher entropy value can increase the incentive for exploration and prevent the strategy from converging too early.

[0063] Total loss is defined as:

[0064] in For the value function loss weight, The weights are entropy regularization weights. The gradient of the total loss with respect to the parameters of the policy network and value network is calculated through backpropagation, and the network parameters are updated using the optimizer.

[0065] ③ Construct an RND prediction network Update. The goal of the RND module is to reduce the prediction error of the prediction network for explored states. Using N empirical data points in the buffer, the prediction network output is computed. With the target network output The mean square error between the two values ​​yields the prediction loss:

[0066] Predicting network parameters only for RND Perform gradient descent updates to the target network parameters. Keep it fixed. After updating, the predictive network can more accurately predict explored states, reducing intrinsic rewards and thus driving the robot to explore underexplored areas.

[0067] In this embodiment of the application, step S300, calculating the directional consistency parameter between the resultant force direction and the original motion, includes: Normalize the direction of the resultant force of the attraction at the target point and the repulsion of the obstacle into the resultant vector of attraction and repulsion. Map the original motion action to the expected vector of the motion probability distribution; Calculate the cosine similarity between the resultant vector of gravitational repulsion and the desired vector, and use the cosine similarity as a direction consistency parameter; The calculation of the resultant force direction is based on an artificial potential field algorithm, in which the gravitational potential field guides the robot to move toward the target point, and the repulsive potential field takes effect when the robot enters the obstacle safety threshold.

[0068] In this embodiment of the application, step S300 involves weighting and correcting the probability distribution of the original motion action based on the direction consistency parameter to generate a safety control action, including: A monotonically increasing weight function with cosine similarity as the variable is constructed to calculate the correction weight of each motion component in the original motion. The monotonically increasing weight function with cosine similarity as the variable can be a linear increasing function or an exponential increasing function to ensure that when the cosine similarity is higher, the corresponding motion component correction weight is greater, so that the motion direction is more inclined to the direction of the artificial potential field force.

[0069] The original action probability distribution output by the policy network is masked and weighted using modified weights to suppress the probability of actions with an angle greater than a preset angle threshold with respect to the resultant force direction, thereby generating safe control actions. Safety control actions are output as speed commands to the differential drive chassis, driving the robot to perform movements that conform to the approach posture constraints.

[0070] In this embodiment, the preset angle threshold is set according to the obstacle avoidance safety redundancy, and the value range can be set to 90° to 150° (preferably 120°); when the angle between the original motion action and the resultant force direction is greater than the preset angle threshold, the motion probability is suppressed to a minimum or zero by the mask.

[0071] In this embodiment of the application, step S400 utilizes a local trajectory optimization algorithm to perform local trajectory optimization on the global path, including: A time-elastic band algorithm is used to construct a band graph model that includes robot kinematic and dynamic constraints; The safety control actions are transformed into external potential forces of the graph model. The energy function of the graph model is minimized by a nonlinear optimization solver to generate a collision-free local optimization trajectory. During the movement, the adaptive Monte Carlo localization algorithm combined with the dynamic particle injection strategy is used for real-time pose estimation, including maintaining the short-term likelihood mean and the long-term likelihood mean. The dynamic particle injection strategy includes determining location loss and triggering random particle injection when the short-term likelihood mean is much smaller than the long-term likelihood mean. The short-term likelihood mean is calculated using an exponential moving average method with a decay coefficient of 0.2 to reflect recent location quality; the long-term likelihood mean is also calculated using an exponential moving average method with a decay coefficient of 0.01 to reflect the global historical average location quality.

[0072] In an optional implementation, the local trajectory optimization algorithm in step S400 can also discretely sample multiple sets of candidate linear velocities and angular velocities within the kinematic constraints of the robot, predict the short-term trajectory corresponding to each velocity according to the dynamic model, and design an evaluation function that includes target proximity, obstacle distance and velocity magnitude to score each trajectory. Finally, the velocity corresponding to the trajectory with the highest score is selected as the control command.

[0073] In an optional implementation, the local trajectory optimization algorithm in step S400 can also construct a virtual force field environment containing the gravitational field of the target point and the repulsive field of the obstacle, calculate the vector resultant force of the gravitational and repulsive forces acting on the robot at its current position in real time, and iteratively update the robot's motion speed and position along the direction of the resultant force to generate a local obstacle avoidance trajectory.

[0074] In this embodiment of the application, step S400, which uses an adaptive localization algorithm for real-time pose estimation, includes: An adaptive Monte Carlo localization algorithm is used to initialize the particle set, and the importance of the particles is sampled based on the observation likelihood of LiDAR data and visual images. The covariance of robot pose estimation is calculated in real time. When the covariance exceeds the preset covariance threshold, a dynamic particle injection strategy is triggered to randomly inject new particles in the region with high likelihood. Simultaneously perform distortion correction and template matching on the visual image to complete the instrument reading recognition, including: The camera intrinsic parameter matrix and distortion coefficients are obtained by combining Zhang Zhengyou calibration method with Brownian distortion model, and the image is then processed to remove distortion. The distortion-reduced image undergoes adaptive threshold binarization, and a correlation coefficient matching algorithm is used to calculate the relative mean correlation between the template image and the image to identify the status of instruments and indicator lights. The adaptive threshold binarization uses Otsu's method to automatically calculate the globally optimal threshold; the correlation coefficient matching algorithm measures the relative mean correlation by calculating the normalized cross-correlation coefficient of the pixel grayscale values ​​of the distortion-reduced image and the template image, thereby identifying the status of instruments and indicator lights.

[0075] In this embodiment of the application, the preset covariance threshold is set according to the positioning accuracy requirements, specifically the trace or the maximum eigenvalue of the pose estimation covariance matrix; when it exceeds the preset covariance threshold (exemplarily, such as 0.5), positioning divergence is determined, and a dynamic particle injection strategy is triggered.

[0076] In an optional implementation, the image recognition and matching algorithm in step S400 can also use a pre-trained deep learning object detection model to perform forward reasoning on the input image, directly outputting the category label of the target object in the image and its precise bounding box coordinates, thereby completing the localization and recognition of a specific target.

[0077] In an optional implementation, the image recognition and matching algorithm in step S400 can also use a traditional feature extraction algorithm to extract key points and descriptors of the reference image and the target image respectively, calculate the similarity between descriptors through a brute-force matcher to obtain initial matching point pairs, and use a random sampling consensus algorithm to eliminate mismatched points, and calculate the geometric transformation relationship between the two images to complete accurate matching.

[0078] Example 3 is an embodiment of the present invention. This embodiment differs from the first embodiment in that it provides a nuclear power plant inspection robot navigation system.

[0079] It should be noted that the technical solution of the nuclear power plant inspection robot navigation system and the technical solution of the nuclear power plant inspection robot navigation method described above belong to the same concept. For details not described in detail in the technical solution of the nuclear power plant inspection robot navigation system in this embodiment, please refer to the description of the technical solution of the nuclear power plant inspection robot navigation method described above.

[0080] This embodiment provides a nuclear power plant inspection robot navigation system, comprising: The feature fusion module is used to acquire LiDAR data and visual images, construct an environmental state map that includes the robot's state and obstacle topology, and use a spatial feature extraction network and a temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, and fuse them to generate an environmental feature vector. The reward module is used to input the environmental feature vector into the reinforcement learning policy network, and use the intrinsic reward calculation module to calculate the novelty intrinsic reward of the current state and combine it with the extrinsic reward of the environmental feedback to obtain the total reward, which drives the policy network to output the robot's original motion action. The safety potential field correction module is used to construct a safety correction module and calculate the direction of the resultant force of the attraction of the target point and the repulsion of the obstacle. At the same time, it calculates the direction consistency parameter between the resultant force direction and the original motion action, and performs weighted correction on the probability distribution of the original motion action based on the direction consistency parameter to generate a safety control action. The safety potential field correction module is used to input the safety control actions to the robot's bottom execution layer and use a local trajectory optimization algorithm to optimize the local trajectory of the global path. During the movement, it uses an adaptive positioning algorithm to perform real-time pose estimation and simultaneously performs distortion correction and template matching on the visual image to complete the recognition of instrument readings.

[0081] This embodiment also provides an electronic device applicable to a nuclear power plant inspection robot navigation method, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement a navigation method for a nuclear power plant inspection robot as described in the above embodiments.

[0082] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements a navigation method for a nuclear power plant inspection robot as proposed in the above embodiments.

[0083] The storage medium proposed in this embodiment belongs to the same inventive concept as the navigation method for a nuclear power plant inspection robot proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0084] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0085] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A navigation method for a nuclear power plant inspection robot, characterized in that, include: Acquire LiDAR data and visual images to construct an environmental state map that includes the robot's state and obstacle topology. Then, use a spatial feature extraction network and a temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, and fuse them to generate an environmental feature vector. The environmental feature vector is input into the reinforcement learning policy network, and the novelty intrinsic reward of the current state is calculated using the intrinsic reward calculation module. The total reward is obtained by combining the extrinsic reward from the environmental feedback, which drives the policy network to output the robot's original motion action. A safety correction module is constructed and the direction of the resultant force of the attraction and repulsion of the target point and the obstacle is calculated. At the same time, the direction consistency parameter between the resultant force direction and the original motion action is calculated, and the probability distribution of the original motion action is weighted and corrected based on the direction consistency parameter to generate a safety control action. The safety control actions are input to the robot's bottom execution layer, and the local trajectory optimization algorithm is used to optimize the global path locally. During the movement, the adaptive positioning algorithm is used to perform real-time pose estimation, and the visual image is simultaneously distorted and matched with a template to complete the instrument reading recognition.

2. The navigation method for a nuclear power plant inspection robot as described in claim 1, characterized in that: The method of extracting spatial features of the environmental state map and temporal features of historical trajectories using spatial feature extraction networks and temporal feature extraction networks respectively includes: A graph convolutional network is used to aggregate the node features and adjacency matrix in the environmental state graph to extract spatial feature vectors that represent the topological relationships of obstacles. The sequence data of the historical trajectory is received using a long short-term memory network, and the cell state is updated through a forget gate, an input gate, and an output gate to extract a temporal feature vector characterizing the robot's motion trend. The robot body is equipped with a robot main control development board and a 3D LiDAR. The neural network processor built into the main control development board accelerates the reasoning process of graph convolutional networks and long short-term memory networks.

3. The nuclear power plant inspection robot navigation method as described in claim 1 or 2, characterized in that: The step of inputting the environmental feature vector into the reinforcement learning policy network and calculating the novelty intrinsic reward of the current state using the intrinsic reward calculation module includes: The environmental feature vector is input into the policy network and value network constructed based on the improved proximal policy optimization algorithm; The novelty intrinsic reward of the current state is quantified by using a random network distillation module as the intrinsic reward calculation module, which calculates the Euclidean distance between the fixed random target network and the trainable predictive network on the output features. The improved near-end strategy optimization algorithm utilizes the multi-threading mechanism of the robot's main control development board for parallel sampling and network updates.

4. The nuclear power plant inspection robot navigation method as described in claim 3, characterized in that: The calculation of the directional consistency parameter between the resultant force direction and the original motion includes: The direction of the resultant force of the attraction at the target point and the repulsion of the obstacle is normalized to the resultant vector of attraction and repulsion. The original motion action is mapped to the expected vector of the motion probability distribution; Calculate the cosine similarity between the resultant vector of the gravitational and repulsive forces and the desired vector, and use the cosine similarity as the direction consistency parameter; The calculation of the resultant force direction is based on an artificial potential field algorithm, in which the gravitational potential field guides the robot to move toward the target point, and the repulsive potential field takes effect when the robot enters the obstacle safety threshold.

5. The nuclear power plant inspection robot navigation method as described in claim 4, characterized in that: The step of weighting and correcting the probability distribution of the original motion action based on the direction consistency parameter to generate a safety control action includes: Construct a monotonically increasing weight function with the cosine similarity as the variable, and calculate the corrected weights of each motion component in the original motion. The original action probability distribution output by the policy network is masked and weighted using the corrected weights to suppress the action probability that the angle between the action and the resultant force direction is greater than a preset angle threshold, thereby generating a safety control action. The safety control action is output as a speed command to the differential drive chassis, driving the robot to perform movements that conform to the approach posture constraints.

6. The navigation method for a nuclear power plant inspection robot as described in claim 5, characterized in that: The method of using a local trajectory optimization algorithm to optimize the local trajectory of the global path to drive the robot's movement includes: A time-elastic band algorithm is used to construct a band graph model that includes robot kinematic and dynamic constraints; The safety control action is transformed into the external potential force of the graph model, and the energy function of the graph model is minimized by a nonlinear optimization solver to generate a collision-free local optimization trajectory. During the movement, the adaptive Monte Carlo localization algorithm combined with the dynamic particle injection strategy is used for real-time pose estimation, including maintaining the short-term likelihood mean and the long-term likelihood mean. The dynamic particle injection strategy includes determining that the localization is lost and triggering random particle injection in response to the short-term likelihood mean being much smaller than the long-term likelihood mean.

7. The nuclear power plant inspection robot navigation method as described in claim 6, characterized in that: The real-time pose estimation using an adaptive positioning algorithm includes: An adaptive Monte Carlo localization algorithm is used to initialize the particle set, and the importance of the particles is sampled based on the observation likelihood of LiDAR data and visual images. The covariance of the robot pose estimation is calculated in real time. In response to the covariance exceeding a preset covariance threshold, a dynamic particle injection strategy is triggered to randomly inject new particles in regions with high likelihood. Simultaneously perform distortion correction and template matching on the visual image to complete the instrument reading recognition, including: The camera intrinsic parameter matrix and distortion coefficients are obtained by combining Zhang Zhengyou calibration method with Brownian distortion model, and the image is then processed to remove distortion. The distortion-free image is subjected to adaptive threshold binarization, and the relative mean correlation between the template and the image is calculated using a correlation coefficient matching algorithm to identify the status of instruments and indicator lights.

8. A nuclear power plant inspection robot navigation system, using the method described in any one of claims 1-7, characterized in that, include: The feature fusion module is used to acquire LiDAR data and visual images, construct an environmental state map that includes the robot's state and obstacle topology, and use a spatial feature extraction network and a temporal feature extraction network to extract the spatial features of the environmental state map and the temporal features of the historical trajectory, respectively, and fuse them to generate an environmental feature vector. The reward module is used to input the environmental feature vector into the reinforcement learning policy network, and use the intrinsic reward calculation module to calculate the novelty intrinsic reward of the current state and combine it with the extrinsic reward of the environmental feedback to obtain the total reward, which drives the policy network to output the robot's original motion action. The safety potential field correction module is used to construct a safety correction module and calculate the direction of the resultant force of the attraction of the target point and the repulsion of the obstacle. At the same time, it calculates the direction consistency parameter between the resultant force direction and the original motion action, and performs weighted correction on the probability distribution of the original motion action based on the direction consistency parameter to generate a safety control action. The safety potential field correction module is used to input the safety control actions to the robot's bottom execution layer and use a local trajectory optimization algorithm to optimize the local trajectory of the global path. During the movement, it uses an adaptive positioning algorithm to perform real-time pose estimation and simultaneously performs distortion correction and template matching on the visual image to complete the recognition of instrument readings.

9. An electronic device, characterized in that, include: Memory, used to store programs; A processor for loading the program to perform the steps of the method as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images