Inspection robot obstacle avoidance algorithm based on imitation learning

Abstract

The invention discloses an inspection robot obstacle avoidance algorithm based on imitation learning. The algorithm comprises sensor data acquisition, picture depth estimation, obstacle detection, multi-modal data fusion and action sequence generation. In the model generated by the action sequence, task actions are divided by adopting an action partitioning technology; the action blocking technology comprises the steps that the size of an action block is set to be k, every k time steps, an action sequence generation model receives a set of multi-modal data, k actions are generated, and the actions are executed in sequence. According to the invention, through multi-modal input data and a generation model, autonomous inspection and obstacle avoidance of the robot can be completed under the conditions of a complex external environment and a dynamic obstacle.

Description

Technical Field

[0001] The present invention relates to the field of robot algorithms, and in particular to an inspection robot obstacle avoidance algorithm based on imitation learning. Background Art

[0002] With the rapid development of industrial automation and intelligence, inspection robots have become an indispensable part of modern industry, electricity, transportation and other fields. In some dangerous production environments, traditional manual patrol and maintenance have safety risks and low efficiency. Therefore, it is of great significance to develop a robot that can efficiently perceive the surrounding environment and autonomously achieve obstacle avoidance and automatic inspection tasks. At present, most of the research on autonomous inspection robots at home and abroad has single sensor data, and the implementation of obstacle avoidance algorithms is highly dependent on artificial prior knowledge design, with strong environmental premise assumptions, resulting in insufficient generalization of the algorithm, single applicable scenarios, and inability to effectively complete designated tasks in dynamically changing scenarios.

[0003] Therefore, technicians in this field are committed to developing an inspection robot obstacle avoidance algorithm based on imitation learning. Imitation learning can quickly make the model converge in new environment training, and the use of action generation model can better fit the complex and multimodal action sequence data distribution, thereby improving the robustness of the algorithm in different scenarios, especially dynamic scenarios. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is that in practical applications, the existing imitation learning algorithm, due to the difference between the actual scenario and the training data, causes the strategy to continuously accumulate errors during execution, and eventually enters a state space that is completely different from the training data, resulting in reduced robustness of the algorithm.

[0005] To achieve the above object, the present invention provides an inspection robot obstacle avoidance algorithm based on imitation learning, characterized in that the algorithm includes sensor data acquisition, image depth estimation, obstacle detection, multimodal data fusion and action sequence generation;

[0006] In the model for generating the action sequence, the action block technology is used to divide the task actions;

[0007] The action block technology includes: setting the size of the action block to k, and every k time steps, the action sequence generation model receives a set of multimodal data, generates k actions, and executes them sequentially.

[0008] The use of obstacle avoidance action block technology in the present invention can reduce the cumulative error of the robot in actual scenarios, improve the success rate of inspection tasks, and during the inspection process, the smoothness and stability of the robot's obstacle avoidance action are improved.

[0009] In a preferred embodiment of the present invention, the action output is queried through the decoder at every k time steps, and different action blocks overlap each other. At a given time step, there will be multiple predicted actions, and the action output is obtained through the time set algorithm;

[0010] The time aggregation algorithm is used to perform weighted averaging of the action predictions that overlap with each other in the same time step through exponential weighting. The formula is as follows:

[0011]

[0012] w i =exp(-m*i)

[0013] Among them, a output for..., a i is..., m is the merging speed of the action prediction sequence in the control time combination process, and w0 is the earliest action prediction weight value.

[0014] In another preferred embodiment of the present invention, the action sequence generation model simulates the mapping of multimodal fusion data to action sequence generation;

[0015] The action sequence generation model includes: mapping multimodal fusion data to a standard normal Gaussian distribution through an encoder, sampling from the standard normal Gaussian distribution to obtain latent variables, and re-parameterizing them, and finally decoding the re-parameterized latent variable sampling values ​​into action sequence generation by a decoder, and outputting them after completion.

[0016] The present invention adopts the action sequence generation model technology, which can enable the inspection robot to adapt to the environment in real time in a dynamic environment, so that the robot has the ability to autonomously determine the action sequence in a pre-unknown scenario without clear instructions.

[0017] In another preferred embodiment of the present invention, the obstacle detection obtains the result of obstacle avoidance or inspection;

[0018] The inspection adopts an inspection behavior cloning algorithm, completes the collection of example data sets in a small batch domain through remote control technology, and uses the collected example data sets to fine-tune the training of the action sequence generation model.

[0019] The remote control technology described in the present invention includes a hardware part and a software control logic.

[0020] The hardware part includes: (a) camera and laser radar; (b) local wireless network component; (c) inspection robot body.

[0021] The software control logic includes: ROS robot control system; using the development tool chain provided by ROS to write the corresponding data collection program.

[0022] The present invention uses the inspection behavior cloning algorithm and only needs to collect a small amount of demonstration data in actual scenarios to complete the training and fine-tuning of the model while ensuring the convergence of the algorithm, which greatly saves time and money costs.

[0023] The cloning algorithm of the present invention includes: data collection, definition of strategy class, training model and model evaluation.

[0024] In another preferred embodiment of the present invention, the multimodal data fusion includes: first, pre-processing the multimodal data with a pre-trained model, semantically aligning the multimodal data to 512 dimensions using an embedding weight matrix W through a FC mapping layer, and positionally encoding different modal data in the process.

[0025] The multimodal data fusion technology used in the present invention can ensure that the robot maintains the ability to perceive the environment under conditions of environmental changes such as weather and lighting, and provide system redundancy.

[0026] In another preferred embodiment of the present invention, the embedding weight matrix W is used to semantically align the multimodal data to 512 dimensions, specifically including:

[0027] For input data x of different modalities, the embedding weight matrix W is first used to transform the x dimension to the data dimension of the hidden layer. The embedded vector y after transformation is of size 1×512. The process is described by the following mathematical formula: y=Wx.

[0028] In another preferred embodiment of the present invention, the sensor data is collected by a monocular camera, a lidar and a speed sensor carried by the robot to collect color image data, a lidar point cloud and robot speed information in the current local environment.

[0029] In another preferred embodiment of the present invention, the depth estimation algorithm of the image depth estimation includes:

[0030] The algorithm receives two consecutive frames I in the video sequence a and I b As input;

[0031] The deep learning network estimates the depth of the two frames and generates the corresponding depth map D a and D b At the same time, the relative pose P between the two frames is estimated ab ;

[0032] Calculated depth map D a and D b The pixel-level depth inconsistency D diff ;

[0033] Using Ddiff Deriving the geometric consistency loss L G , derive the self-discovery mask Ms.

[0034] In another preferred embodiment of the present invention, the inspection robot obstacle avoidance algorithm uses the Transformer architecture to implement the encoder and decoder of the conditional variational autoencoder;

[0035] The input of the encoder is the current speed data and the target action sequence from the demonstration data set, with a length of k, and the CLS mark output by the collision detection module is added to the input to form an input sequence with a length of k+2;

[0036] After being processed by the Transformer, the features corresponding to the CLS are used to predict the mean and variance of the “hidden variable” z, which is passed as input to the decoder. The decoder receives the current observation and z as input and predicts the next k actions.

[0037] In another preferred embodiment of the present invention, the loss function of the conditional variational autoencoder includes reconstruction loss and KL divergence.

[0038] Technical Effects

[0039] The present invention discloses an inspection robot obstacle avoidance algorithm based on imitation learning and a corresponding system block diagram. First, the robot's built-in monocular camera, laser radar and speed sensor are used to collect color image data, laser radar point cloud and robot speed information in the current local environment. Then, the monocular camera image is estimated through convolutional neural network and frame stack technology to obtain the corresponding depth map information, and the depth map and laser radar data are passed to the obstacle detection module to determine whether the robot performs obstacle avoidance operations. Then, by embedding the fusion layer, the image data, laser radar data and speed information are aligned to a unified dimension, and then sent to the conditional generation model network to generate the robot's next action information.

[0040] The present invention can complete the robot's autonomous inspection and obstacle avoidance under the conditions of complex external environments and dynamic obstacles through multi-modal input data and generation models.

[0041] The inspection robot technology of the present invention provides an efficient and safe solution to the automation and intelligent needs of modern industry, electricity, transportation and other fields. Through the imitation learning algorithm, the robot can perform autonomous obstacle avoidance and inspection tasks in complex dynamic environments, significantly improving the robustness and adaptability of the robot.

[0042] In terms of technical advantages, the present invention adopts obstacle avoidance action block technology, and through the action generation of fixed time steps, it reduces the cumulative error of the robot in actual scenarios and improves the success rate of tasks. At the same time, the application of multimodal data fusion technology ensures the stable perception ability of the robot under different environmental conditions and enhances the redundancy of the system. At the same time, the action generation model can enable the inspection robot to adapt to the environment in real time in a dynamic environment and have the ability to autonomously determine the action sequence in unknown scenarios without clear instructions.

[0043] In terms of performance indicators, the inspection behavior cloning algorithm adopted in the present invention enables the robot to quickly train and converge based on a small amount of example data, greatly reducing the cost and time of data collection.

[0044] In terms of production implementation, the technical solution of the present invention has a high prospect for industrial application. The present invention adopts an end-to-end design concept, and its modular design is easy to integrate into the existing automation system with low maintenance cost. In addition, data collection and model fine-tuning completed by remote control technology provide the possibility for rapid deployment and large-scale production.

[0045] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a diagram of the inspection robot obstacle avoidance algorithm architecture of a preferred embodiment of the present invention;

[0047] Figure 2 is a flowchart of a picture depth estimation algorithm of a preferred embodiment of the present invention;

[0048] Figure 3 is a multimodal data fusion architecture diagram of a preferred embodiment of the present invention;

[0049] Figure 4 is a structural diagram of an action sequence generation model of a preferred embodiment of the present invention;

[0050] Figure 5 It is a schematic diagram of action block calculation of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0052] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0053] like Figure 1 As shown, the present invention provides an inspection robot obstacle avoidance algorithm based on imitation learning, including sensor data acquisition, image depth estimation, obstacle detection, multimodal data fusion and action sequence generation.

[0054] Specifically, the robot's multimodal sensing system is used to achieve efficient and accurate obstacle avoidance operations, including the following steps:

[0055] During the sensor data collection phase, the inspection robot acquires local environmental information and the robot's own motion status in real time through the monocular camera, laser radar and speed sensor it carries. The monocular camera and laser radar are installed at different heights to ensure that the robot can still receive valid environmental data when the environment changes or a sensor fails.

[0056] In the depth estimation stage, the color images collected by the monocular camera are processed by a depth estimation algorithm without relying on the label information or prior knowledge of the target object. A convolutional neural network is used to extract image features and predict the depth map of the target object in each frame. The depth value of each pixel in the depth map is optimized by combining the relative camera pose estimation.

[0057] In the obstacle detection phase, the depth image and lidar data are converted to the robot coordinate system, and data preprocessing is performed to filter out noise and outliers. By extracting feature points and comparing safety thresholds, it is decided whether to execute the obstacle avoidance strategy.

[0058] In the modal data fusion stage, the image data, lidar data, and speed information are processed accordingly and unified in dimension, and then fused into an embedding vector to provide input for the generation model.

[0059] In the action sequence generation stage, the conditional variational autoencoder compresses and encodes the embedded multimodal data to obtain the latent variable information after dimensionality reduction, and outputs the next action data through the decoder, including the robot's angular velocity and linear velocity, to achieve obstacle avoidance operations.

[0060] like Figure 2 As shown, the image depth estimation algorithm provided by the present invention first receives two consecutive frames I in a video sequence. a and I b Then, the deep learning network estimates the depth of the two frames and generates the corresponding depth map D aand D b At the same time, the neural network also needs to estimate the relative pose P between the two frames. ab , which includes rotation and translation information, which is crucial for the consistency of the depth map. On this basis, the algorithm calculates two depth maps D a and D b The pixel-level depth inconsistency D diff . D diff It reflects the inconsistency of depth estimation between consecutive frames and provides a basis for subsequent consistency loss calculation. diff To derive the geometric consistency loss L G , this loss measures the consistency of the depth maps of consecutive frames, aiming to ensure that the depth estimation keeps the same scale in different frames, thereby improving the accuracy of the depth map. diff The self-discovery mask M is derived from S , this mask is used to identify areas in the image that may have dynamic changes or occlusions. S The introduction of enhances the network's ability to handle dynamic scenes and occlusion situations.

[0061] like Figure 3 As shown, the multimodal data fusion of the present invention needs to be aligned and fused before generating a model because there are significant differences between the lidar sensor data and the camera data in representing the surrounding environment.

[0062] For input data x of different modes, an embedding weight matrix W is first used to transform the x dimension to the data dimension of the hidden layer. The process can be described by the following mathematical formula. After the transformation, the embedding vector y is unified to 1×512 size.

[0063] y=Wx

[0064] Furthermore, in order to better preserve the relative positions of objects in the image and the data features of the lidar at different angles, we uniformly add sinusoidal position embedding information to the embedding vector sequence for position encoding.

[0065] like Figure 4 As shown in the figure, the structure diagram of the action sequence generation model in the present invention can use the Transformer architecture to implement the encoder and decoder of the conditional variational auto encoder (cVAE). The input of the encoder is the current speed data and the target action sequence from the demonstration dataset, with a length of k. These inputs are preceded by a CLS tag output by the collision detection module. This forms an input sequence of length k+2 ( Figure 4Left). After being processed by the Transformer, the features corresponding to the CLS are used to predict the mean and variance of the “hidden variable” z, which is then passed as input to the decoder. The CVAE decoder (i.e., the action generation strategy) receives the current observation and z as input and predicts the next k actions ( Figure 4 right).

[0066] In the present invention, the Transformer architecture based on the self-attention mechanism has outstanding performance in predicting and generating sequence information and has achieved great success in the field of natural language processing. Therefore, in the present invention, it is proposed that the Transformer architecture can be used to predict action sequences, and the concept of word vectors in NLP can be replaced by action vectors.

[0067] Positional encoding in multimodal data fusion is the additional information supplement to the input token sequence in each self-attention layer in the Transformer model, including "Absolute Positional Encoding" and "Relative Positional Encoding".

[0068] The encoder and decoder of the conditional variational autoencoder (cVAE) can be implemented as a cascade of four self-attention layers (e.g. Figure 4 As shown in Figure 2, each self-attention layer needs to encode the position of the input sequence and retain the order information.

[0069] Furthermore, the loss function of CVAE consists of two parts: Reconstruction Loss: This part measures the difference between the data generated by the decoder from the latent space and the original input data. In the present invention, in order to more accurately model the action sequence, L1 loss is used instead of the more common mean squared error (MSE) to construct the reconstruction loss. KL divergence (Kullback-Leibler Divergence): This part measures the difference between the distribution of the latent variables output by the encoder and the prior distribution. VAE usually assumes that the prior distribution of the latent variables is a standard normal distribution (i.e., a multivariate normal distribution with a mean of 0 and a variance of 1). The encoder outputs the parameters of the latent variables, usually the mean and the logarithmic variance. KL divergence is used to ensure that the distribution defined by these parameters is as close to the prior distribution as possible.

[0070] The action generation model proposed in the present invention simulates a set of mappings from multimodal data to action sequence generation, namely P(action sequence|multimodal fusion data). However, the aforementioned conditional probability distribution is not known, so it is necessary to map the input data of multimodal fusion to a standard Gaussian distribution through an encoder, and then sample the latent variable z from the standard normal Gaussian distribution and reparameterize it, and finally decode the reparameterized latent variable sample value into an action sequence output by a decoder.

[0071] In order to combat the problem of compounding error in imitation learning, the present invention uses action chunking technology to divide long-distance task actions. Action chunking reduces the length of the action sequence that the model needs to predict, thereby reducing the cumulative error. In the implementation of this patent, the size of the action chunk is fixed to k. That is, every k time steps, the generative model receives a set of multimodal data from the sensor end, generates the next k actions, and executes these actions sequentially. Through action chunking, the algorithm proposed in this patent reduces the effective observation range of the task to 1 / k of the original. Specifically, the modeling formula of the generative model is π θ (ɑ t:t+k |s t ) instead of π θ (ɑ t |s t ).

[0072] like Figure 5 As shown, it is a schematic diagram of the action block calculation of the present invention. The simple implementation of action block may be suboptimal. Considering the actual scenario, suddenly merging a new environmental observation result every k steps may make the robot's action stiff. In order to avoid the robot from switching discontinuously between executing actions and observing the environment, the present invention proposes that the action output can be queried through the decoder at each time step, which will cause different action blocks to overlap with each other, and at a given time step, there will be multiple predicted actions, and then the final action output is obtained through the time set algorithm.

[0073] The time aggregation algorithm uses exponential weighting to perform weighted averaging of overlapping action predictions within the same time step, which can be described by the following formula:

[0074]

[0075] w i =exp(-m*i)

[0076] In the above formula, w0 is the earliest action prediction weight value. The parameter m controls the merging speed of the new action prediction sequence in the time combination process. The smaller the parameter m, the faster the action merging speed.

[0077] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. An inspection robot obstacle avoidance algorithm based on imitation learning, characterized in that: The algorithm includes sensor data acquisition, image depth estimation, obstacle detection, multimodal data fusion and action sequence generation; In the model for generating the action sequence, the action block technology is used to divide the task actions; The action block technology includes: setting the size of the action block to k, and every k time steps, the action sequence generation model receives a set of multimodal data, generates k actions, and executes them sequentially.

2. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The action output is queried through the decoder at every k time steps. Different action blocks overlap with each other. At a given time step, there will be multiple predicted actions. The action output is obtained through the time set algorithm; The time aggregation algorithm is used to perform weighted averaging of the action predictions that overlap with each other in the same time step through exponential weighting. The formula is as follows: w=exp(-m*i) Among them, ai is the action sequence vector at each moment in the action block, and the value range of i is 1 to k; aoutput is the real action vector output at that moment, which is used to control the movement of the robot. i Weighted average of action sequence vectors, with weight coefficient w i , m is the merging speed of the action prediction sequence in the process of controlling time combination, and w0 is the earliest action prediction weight value.

3. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The action sequence generation model simulates the mapping from multimodal fusion data to action sequence generation; The action sequence generation model includes: mapping multimodal fusion data to a standard normal Gaussian distribution through an encoder, sampling from the standard normal Gaussian distribution to obtain latent variables, and re-parameterizing them, and finally decoding the re-parameterized latent variable sampling values ​​into action sequence generation by a decoder, and outputting them after completion.

4. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The obstacle detection obtains the result of obstacle avoidance or inspection; The inspection adopts an inspection behavior cloning algorithm, completes the collection of example data sets in a small batch domain through remote control technology, and uses the collected example data sets to fine-tune the training of the action sequence generation model.

5. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The multimodal data fusion includes: firstly, pre-processing the multimodal data with a pre-trained model, semantically aligning the multimodal data to 512 dimensions by using an embedding weight matrix W through an FC mapping layer, and positionally encoding different modal data in the process.

6. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 5, characterized in that: The embedding weight matrix W is used to semantically align the multimodal data to 512 dimensions, including: For input data x of different modalities, the embedding weight matrix W is first used to transform the x dimension to the data dimension of the hidden layer. The embedded vector y after transformation is of size 1×512. The process is described by the following mathematical formula: y=Wx.

7. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The sensor data acquisition is to collect color image data, lidar point cloud and robot speed information in the current local environment through the monocular camera, lidar and speed sensor carried by the robot.

8. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The depth estimation algorithm of the image depth estimation includes: The algorithm receives two consecutive frames I in the video sequence a and I b As input; The deep learning network estimates the depth of the two frames and generates the corresponding depth map D a and D b At the same time, the relative pose P between the two frames is estimated ab ; Calculated depth map D a and D b The pixel-level depth inconsistency D diff ; Using D diff Deriving the geometric consistency loss L G , derive the self-discovery mask Ms.

9. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 1, characterized in that: The inspection robot obstacle avoidance algorithm uses the Transformer architecture to implement the encoder and decoder of the conditional variational autoencoder; The input of the encoder is the current speed data and the target action sequence from the demonstration data set, with a length of k, and the CLS mark output by the collision detection module is added to the input to form an input sequence with a length of k+2; After processing by Transformer, the features corresponding to CLS are used to predict the mean and variance of the "hidden variable" z, which is passed as input to the decoder. The decoder receives the current observation and z as input and predicts the next k actions.

10. The inspection robot obstacle avoidance algorithm based on imitation learning as claimed in claim 9, characterized in that: The loss function of the conditional variational autoencoder includes reconstruction loss and KL divergence.

Images