Robot visual navigation method based on comparative learning and related device

Abstract

The invention belongs to a robot navigation method, and aims to solve the technical problems of difficulty in establishing a geometric map in a non-structural scene, dependence on tags from laser radar data, easiness in ignoring long-distance features and slow reasoning time in the conventional navigation or exploration method. According to the robot visual navigation method based on comparative learning and the related device, after current observation is obtained, the current observation is input into a trained observation encoder to obtain a corresponding feature vector, and then a pre-selected batch action sequence passes through an action encoder to extract the corresponding feature vector; and calculating the similarity between the current observation and each action sequence through a vector inner product, selecting the action sequence with the highest similarity, and inputting the selected action sequence into a robot controller for controlling the robot to execute corresponding actions. Compared with a method for directly generating the action sequence, the method has the advantages that the difficulty of selecting one action sequence is lower, and the expansibility and generalization are better.

Description

Technical Field

[0001] The present application relates to a robot navigation method, and more specifically to a robot vision navigation method based on contrastive learning and related devices. Background Art

[0002] Vision-based navigation or exploration is a key technology in the field of robotics. Its core goal is to input the images obtained by the robot's camera sensor as observations, output the robot's trajectory or control signals as action sequences, and then achieve collision-free navigation or exploration tasks. Robots operating in real environments will encounter complex situations. Traditional navigation or exploration problems mostly require the perception module to build a local or global map to estimate the drivable area around the robot, and then use a path planning algorithm to plan the trajectory target.

[0003] Existing navigation or exploration methods can be divided into three categories according to the method of building maps: methods based on geometric maps, methods based on feature maps, and methods based on implicit maps. However, methods based on geometric maps find it difficult to use geometric information to build geometric maps in some unstructured scenes; methods based on feature maps usually rely on labels from lidar data, and parameters such as the installation location of related sensors will also affect the quality of prediction. In addition, convolution or attention mechanism calculations also require a lot of computing power, and long-distance features will be directly ignored during calculations; methods based on implicit maps usually find it difficult to meet the real-time requirements in practical applications, resulting in slow reasoning time. Summary of the invention

[0004] This application provides a robot visual navigation method and related devices based on contrastive learning to address the technical problems that existing navigation or exploration methods have difficulty in establishing geometric maps in unstructured scenes, rely on labels from lidar data, easily ignore long-distance features, and have slow reasoning time.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, the present application proposes a robot visual navigation method based on contrastive learning, comprising: Get the current observation; Input the current observation into the trained observation encoder, and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; Use vector inner product to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; Inputting the execution action sequence into the robot controller to control the robot to perform corresponding actions; The training method of the observation encoder and the action encoder includes: Obtain multiple historical observations and historical action sequences; By using the paired relationship between observations and action sequences, features are extracted from each historical observation and each historical action sequence using observation encoder and action encoder respectively, and the historical observation feature vector and the historical action sequence feature vector are obtained correspondingly. Calculate the similarity between every two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; wherein the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

[0006] Furthermore, the observation encoder and the action encoder both use convolutional neural networks; The observation encoder and the action encoder are both connected to a layer of feedforward neural network as a feature projection layer.

[0007] Furthermore, the calculating of the similarity between every two historical action sequences includes: The following formula is used as the action sequence similarity function to calculate the similarity between each two historical action sequences:

[0008] in, Representing action sequence and action sequences The similarity between represents the length of the action sequence, Indicates the maximum length of the action sequence, Indicates the length is Action sequence , Indicates the length is Action sequence .

[0009] Furthermore, the observation-action similarity matrix includes:

[0010] in, represents the observation-action similarity matrix, represents the first historical observation feature vector, represents the second historical observation feature vector, Indicates N The historical observation feature vector, represents the first historical action sequence feature vector, The second historical action sequence feature vector of the hour, Indicates N The feature vector of the historical action sequence.

[0011] Furthermore, when the observation encoder and the action encoder are trained, the loss function used includes:

[0012] in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

[0013] Furthermore, the observation-action loss and the action-observation loss ,include:

[0014]

[0015] in:

[0016] in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express Dimensional identity matrix Middle Line Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Line Column elements, represents the first row element;

[0017] in, middle represents a natural constant, represents the first Line Column elements, Represents the mask matrix Middle Line Column elements.

[0018] In the second aspect, the present application proposes a robot visual navigation system based on contrastive learning, comprising: Observation module, used to obtain current observations; A feature extraction module is used to input the current observation into the trained observation encoder and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; An execution sequence acquisition module is used to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector using vector inner product, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; A control module is used to input the execution action sequence into the robot controller to control the robot to perform the corresponding action; The training method of the observation encoder and the action encoder includes: Obtain multiple historical observations and historical action sequences; By using the paired relationship between observations and action sequences, features are extracted from each historical observation and each historical action sequence using observation encoder and action encoder respectively, and the historical observation feature vector and the historical action sequence feature vector are obtained correspondingly. Calculate the similarity between every two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; wherein the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

[0019] Furthermore, when the observation encoder and the action encoder are trained, the loss function used includes:

[0020] in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

[0021] Furthermore, the observation-action loss and the action-observation loss ,include:

[0022]

[0023] in:

[0024] in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express Dimensional identity matrix Middle Line Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Line Column elements, represents the first row element;

[0025] in, middle represents a natural constant, represents the first Line Column elements, Represents the mask matrix Middle Line Column elements.

[0026] In a third aspect, the present application proposes an electronic device comprising: a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the above-mentioned robot visual navigation method based on contrastive learning.

[0027] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned robot visual navigation method based on contrastive learning are implemented.

[0028] Compared with the prior art, this application has the following beneficial effects: The present application proposes a robot visual navigation method based on contrastive learning. After obtaining the current observation, it is input into the trained observation encoder to obtain the corresponding feature vector, and then the pre-selected batch action sequence is passed through the action encoder to extract the corresponding feature vector. The similarity between the current observation and each action sequence is calculated by the vector inner product, and the action sequence with the highest similarity is selected and input into the robot controller to control the robot to perform the corresponding action. Compared with the method of directly generating an action sequence, it is less difficult to select an action sequence, and has better scalability and generalization. In addition, when training the observation encoder and the action encoder, the present application adopts the contrastive learning method for self-supervision. The features of the observation and action sequences can be aligned in the latent space without manual data annotation, thereby improving the robustness and generalization of the model. The present application can achieve efficient and accurate navigation without relying on labels from lidar data.

[0029] The present application also proposes a robot visual navigation system, electronic device and computer storage medium based on contrastive learning, which possess all the advantages of the above-mentioned robot visual navigation method based on contrastive learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 This is a schematic diagram of the first flow chart of the robot visual navigation method based on contrastive learning in this application; Figure 2 This is a second flow chart of the robot visual navigation method based on contrastive learning in this application; Figure 3 Schematic diagram of the training process of the observation encoder and the action encoder in the embodiment of the present application; Figure 4 A schematic diagram of a flow chart of the reasoning stage in the embodiment of the present application; Figure 5 This is a schematic diagram of the training results of using only the first part of the real scene data set in the embodiment of the present application; Figure 6 This is a schematic diagram of the results of jointly training using the first part and the second part of real scene data in an embodiment of the present application; Figure 7 This is a schematic diagram of a robot visual navigation system based on contrastive learning in this application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0035] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0036] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0037] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0038] Vision-based navigation or exploration is a core area in robotics that enables robots to move autonomously in complex, dynamic, and often unknown environments. This technology requires robots to not only understand and interpret image information acquired from camera sensors, but also to be able to make decisions based on this information and plan and execute safe navigation paths.

[0039] The robot preprocesses the original image captured by the camera, such as denoising, correction, and enhancement, to improve the image quality. The key information in the image, such as object outlines, obstacle locations, and road signs, is identified through feature extraction technology. Based on the above key information, the robot can build or update the local map around it in real time, including obstacle locations, walkable areas, etc. The precise positioning of the robot in the map is the basis for navigation. Then, on the global map, the path planning algorithm is used to plan an optimal path from the starting point to the target point for the robot. In the actual navigation process, the robot may need to make local adjustments to the path according to the real-time perceived environmental changes. Finally, based on the results of path planning, the trajectory that the robot should follow is generated, including control parameters such as speed and direction. The robot executes the trajectory through its motion control system to achieve collision-free navigation.

[0040] Existing vision-based navigation methods can be divided into three categories according to the method of building maps: geometric map-based methods, feature map-based methods and implicit map-based methods.

[0041] Most methods based on geometric maps use the Simultaneous Localization and Mapping (SLAM) algorithm to build a point cloud map and represent the probability of space occupancy or occupancy, and continuously and dynamically maintain the local occupancy map. The difficulty of this process is that in some unstructured scenes, such as wild environments with weeds or sidewalks on the side of the road, it is difficult to use geometric information to build a geometric map.

[0042] The feature map-based method is to learn a strategy from observation to action from the previous driving experience of the robot. With the development of deep learning, people have realized that deep neural networks can be used to help build maps of feature representation. For autonomous driving, an effective way to represent the feature space is the Bird's Eye-View (BEV), which has strong perception capabilities and can fuse data from different sensors into a unified BEV representation. Another method is to use a 3D occupancy network, which also inputs information from multiple cameras and outputs perception results. However, these methods usually rely on labels from lidar data. Parameters such as the installation position of the sensor can also affect the quality of the prediction. Another difficulty is that the model parameters are large. Even in sparse scenes, convolution or attention mechanism calculations require a lot of computing power. In addition, these methods directly ignore long-distance features, which also affects the perception effect.

[0043] Implicit map-based methods do not use displayed maps, but directly obtain predicted action sequences from current observations and learn navigation from past experience. The method of mapping current observations to action sequences is difficult to model multiple possibilities of action sequences. This is unreasonable in the real world. For example, if there is an obstacle in front of the robot, or the robot travels to a T-junction, the output action sequence should have different directions, moving both left and right. The application of diffusion models can solve this problem. It models the action space under observation conditions. However, when the diffusion model generates action sequences from noise, dozens or even hundreds of iterative reasoning are required, which makes real-time requirements difficult to meet and leads to slow reasoning time.

[0044] Another difficulty in robot navigation or exploration is that it is difficult to obtain and label data. Self-supervised learning methods can avoid labeling large-scale data. Contrastive learning has been widely used in unsupervised representation learning. Contrastive learning usually requires positive samples and negative samples to appear together. For single-modal data, contrastive learning usually uses its own data as positive samples and other data as negative samples, and makes positive samples as close as possible in feature space and negative samples as far away as possible in feature space. For paired multimodal data, contrastive learning uses the data pair itself as a positive sample, different data pairs as negative samples, and aligns the sample pairs in the feature space.

[0045] Based on the above situation, the present application proposes a robot visual navigation method and related devices based on contrastive learning. The present application is described in detail below in conjunction with embodiments and drawings.

[0046] like Figure 1As shown, it is a schematic diagram of a robot visual navigation method based on contrastive learning in the present application, which may include: S101, obtain the current observation.

[0047] The robot obtains images or video frames of the current environment as observation data through its visual sensors (such as cameras), providing input for subsequent feature extraction and action selection.

[0048] S102, inputting the current observation into the trained observation encoder, and inputting the pre-selected batch action sequence into the trained action encoder, to obtain the current observation feature vector and the batch pre-selected action feature vector.

[0049] Convert high-dimensional observation and action sequence data into low-dimensional feature vectors to facilitate subsequent calculations and comparisons.

[0050] Both the observation encoder and the action encoder can be a deep learning model. The observation encoder accepts the observation data (such as images or video frames) obtained by the robot from the environment as input and outputs a fixed-length feature vector. This feature vector contains the key information in the observation data for subsequent comparison with the action sequence feature vector. The action encoder accepts the action sequence that the robot may perform as input and outputs the feature vector corresponding to each action sequence. These feature vectors are used to compare with the observation feature vector to select the most appropriate action sequence.

[0051] S103, using vector inner products to respectively calculate feature similarities between the current observation feature vector and each pre-selected action feature vector, and selecting the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence.

[0052] The action sequence corresponding to the pre-selected action feature vector with the highest feature similarity is selected as the execution action sequence. The most appropriate action sequence can be selected according to the current observation to achieve collision-free navigation or exploration tasks.

[0053] S104, inputting the execution action sequence into the robot controller to control the robot to execute corresponding actions.

[0054] The robot controller can control the movement of the robot according to the execution action sequence, including parameters such as speed and direction, so that the robot can navigate or explore the environment according to the selected action sequence.

[0055] The training method of the observation encoder and the action encoder includes: (1) Obtain multiple historical observations and historical action sequences.

[0056] (2) By utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the corresponding historical observation feature vector and historical action sequence feature vector.

[0057] (3) Calculate the similarity between every two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix.

[0058] Among them, the mask matrix can be used to distinguish positive samples (the same observation and action sequence data pairs) from negative samples (different observation and action sequence data pairs).

[0059] (4) Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix.

[0060] (5) Combining the mask matrix and the observation-action similarity matrix, the observation encoder and the action encoder are trained using the unit matrix as the label; wherein the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and to minimize the distance between feature vectors from the same observation and action sequence data pair.

[0061] In practical applications, the unit matrix is ​​used as a label, indicating that the similarity between the feature vectors of positive samples should be 1, and the similarity between the feature vectors of negative samples should be 0. The observation encoder and action encoder are trained using a contrastive learning loss function. The training goal is to maximize the similarity between the feature vectors of positive samples (i.e., minimize the distance between feature vectors) and minimize the similarity between the feature vectors of negative samples (i.e., maximize the distance between feature vectors). Through training, the observation encoder and action encoder can extract feature vectors with discriminability, so that they can more accurately match the current observation and action sequence when selecting an action sequence to execute.

[0062] This application solves the problems that explicit maps are difficult to express geometric information, that direct mapping from observations to action sequences is difficult to model multiple possibilities in the same scenario, and that data acquisition and labeling for real robots are difficult.

[0063] like Figure 2 As shown, it is a second flow chart of the robot visual navigation method based on contrastive learning of the present application, which may include: S201, utilizing the natural paired relationship between observations and action sequences, respectively use observation encoders and action encoders to extract features of historical observations and features of historical action sequences, and obtain their corresponding historical observation feature vectors and historical action sequence feature vectors in the feature space.

[0064] In this embodiment, the training data used simulation data and real scene data respectively. In the simulation data, 3 hours and 46 minutes of autonomous driving data were collected using the CARLA autonomous driving simulator. In the driving of the simulation scene, most of the time it is relatively stable, so this embodiment manually adds noise that conforms to the uniform distribution to the planned trajectory points when collecting data so as to be more in line with the actual situation. The real scene data can be divided into two parts. The first part is 41 minutes of data collected by controlling the robot using a remote control under cloudy weather conditions. The second part is an open source data set selected from about 10 hours of collection by different robots. Both parts of the data set include roads, buildings, pedestrians, vehicles, etc. These data are processed into observation-action sequence data pairs. The observation is the picture of the current frame, and the action sequence is the coordinates of the spatial points at the next 16 moments.

[0065] In this embodiment, observation encoders are used respectively and action encoder Extract features of historical observations and historical action sequences:

[0066]

[0067] in, For observation, For the action sequence, and are the feature vectors corresponding to the observation and action sequences respectively.

[0068] In some embodiments of the present application, a feedforward neural network layer can be connected after each encoder as a feature projection layer to better align the features of the observation and action sequence in the latent space. In addition, the encoder can use a convolutional neural network, and the dimension of the latent space is set to 256 dimensions.

[0069] S202, for a batch of action sequences, the similarity between any two action sequences is calculated according to a defined action sequence similarity function, a similarity matrix of a batch of action sequences is obtained, and a mask matrix is ​​calculated according to the similarity matrix of the batch of action sequences and a threshold coefficient.

[0070] In the training phase, this embodiment calculates the similarity between a batch of action sequences by defining an action sequence similarity function:

[0071] in, Representing action sequence and action sequences The similarity between represents the length of the action sequence, Indicates the maximum length of the action sequence, Indicates the length is Action sequence , Indicates the length is Action sequence . You can get Similarity results, this The similarity results are arranged in order to obtain the action sequence similarity matrix of this batch of data. The more similar the two action sequences are, the smaller the corresponding similarity results are.

[0072] The action sequence similarity function can be designed according to the specific characteristics of the action sequence, such as considering factors such as the type, sequence, and duration of the action. Common similarity measurement methods include edit distance, dynamic time warping (DTW), cosine similarity, etc., which can be selected according to actual conditions.

[0073] In order to improve the stability of the training phase and the generalization of the reasoning phase, this embodiment uses the action sequence similarity matrix result to obtain the mask matrix :

[0074] in, is the set threshold parameter. This formula indicates that in the action sequence similarity matrix, the value of the mask matrix at the position of two similar action sequences is 0, which should be masked, and the value of the remaining positions is 1, which should be retained. Specifically, if the similarity value is greater than or equal to the threshold coefficient, the mask value of the corresponding position is 1, indicating that the two action sequences are regarded as positive sample pairs; otherwise, it is 0, indicating that they are regarded as negative sample pairs. In practical applications, the choice of threshold coefficient can be determined according to the characteristics of specific tasks and datasets, and usually needs to be tuned through experiments.

[0075] S203, using vector inner product to calculate all observation feature vectors and action sequence feature vectors to obtain an observation-action similarity matrix.

[0076] In the training phase of this embodiment, a batch of observation-action sequence data pairs are given, and the feature vector of each pair of observation-action sequences is calculated in step S201. The data can be obtained The observed feature vector values ​​and The action sequence feature vector value is calculated by performing the vector inner product calculation on the observed feature vector and the action sequence feature vector. The larger the value, the more the corresponding observation and action sequence match. The observation-action matching values ​​are arranged in order to obtain the observation-action sequence similarity matrix of this batch of data. :

[0077] The observation-action sequence similarity matrix of The diagonal elements represent the inner products of features from the same observation-action sequence data pair, and the rest The elements represent the inner products of features from different observation-action sequence data pairs, and the goal of the training phase is to maximize the diagonal elements and minimize the elements at other positions.

[0078] S204, using the unit matrix as a label to train the observation encoder and the action encoder, maximize the distance between feature vectors from the same observation-action data pair, and minimize the distance between feature vectors from different observation-action data pairs.

[0079] In this embodiment, in order to make the observation and action sequence share more information, the loss function is set to two parts:

[0080] in, is the observation-action loss, is the action-observation loss, and Denote the corresponding weights respectively. Given the observation-action sequence similarity matrix calculated in step S203 , this example uses the identity matrix as the label and trains the observation encoder and action encoder using the following symmetric cross entropy loss:

[0081] middle, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express Dimensional identity matrix Middle Line Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Line Column elements, represents the first Row element.

[0082]

[0083] in, middle represents a natural constant, represents the first Line Column elements, Represents the mask matrix Middle Line Column elements. The positions where the mask matrix is ​​0 can ensure that similar action sequences are invisible to each other and do not affect each other during the training phase, thereby improving the stability of training. The observation-action loss and action-observation loss are defined as follows:

[0084]

[0085] This embodiment uses the above loss function to transform the problem of generating action sequences into a classification problem, selects the most appropriate action sequence category when given observations, and selects the most appropriate observation category when given action sequences, thereby reducing the difficulty of model training and improving the robustness of the robot during navigation or exploration.

[0086] Figure 3 As shown, it is a schematic diagram of the training process of the observation encoder and the action encoder in the above-mentioned embodiment.

[0087] S205, based on the trained observation encoder and action encoder, input the current robot's observation and a batch of saved action sequences, use vector inner product to calculate the feature similarity between the current observation and each action sequence, and select the action sequence with the highest similarity as the execution action sequence.

[0088] Based on the trained observation encoder and action encoder The observation and action sequence can be encoded into the latent space and their features can be aligned. In the deployment inference phase, this embodiment randomly selects 100 action sequences from the training data as candidate action sequences. At each moment, the action sequence that best matches the current observation in the candidate action sequence is obtained:

[0089] in, It is the final selected execution action sequence. The vector inner product is used to calculate the feature similarity between the current observation and each candidate action sequence. The greater the similarity, the greater the inner product result, and the most similar action sequence in the feature space is finally executed.

[0090] like Figure 4FIG. 1 is a flow chart of the reasoning phase of this embodiment.

[0091] S206, inputting the execution action sequence into the robot controller, outputting the control signal of the robot, and testing the navigation or exploration task performance of the robot.

[0092] In this embodiment, a proportional-differential controller is selected as the controller of the robot, and the execution action sequence obtained in step S205 is input. The controller can calculate the control signal of the robot to control the movement of the robot.

[0093] like Figure 5 As shown in the figure, it is a schematic diagram of the training results using only the first part of the real scene data set. Each column represents a scene, and each picture from left to right is a third-person perspective, a first-person perspective, and an action sequence diagram. Among them, blue is the execution action sequence that ultimately generates a control signal, green is the candidate action sequence with the highest score, and red is the candidate action sequence with the lowest score. It can be seen that despite the changes in weather and environment, even at night, this embodiment can still complete the collision-free exploration task. And it can be seen from Figure 5 As can be seen from the figure, when the robot comes to the T-junction, the action sequences with higher scores include both left turns and right turns, showing the diversity of action sequences.

[0094] like Figure 6 The following is a schematic diagram of the results of training with the first and second parts of real scene data. Figure 6 It can be seen that in an environment that has never been seen before, this embodiment can still understand the drivable area and plan the correct route to complete the collision-free exploration task, demonstrating the robustness of the model.

[0095] like Figure 7 As shown, it is a schematic diagram of a robot visual navigation system based on contrastive learning in the present application, which may include: Observation module, used to obtain current observations; A feature extraction module is used to input the current observation into the trained observation encoder and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; An execution sequence acquisition module is used to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector using vector inner product, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; A control module is used to input the execution action sequence into the robot controller to control the robot to perform the corresponding action; The training method of the observation encoder and the action encoder includes: Obtain multiple historical observations and historical action sequences; By using the paired relationship between observations and action sequences, features are extracted from each historical observation and each historical action sequence using observation encoder and action encoder respectively, and the historical observation feature vector and the historical action sequence feature vector are obtained correspondingly. Calculate the similarity between every two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; wherein the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

[0096] In some embodiments of the robot visual navigation system based on contrastive learning of the present application, the loss function used when training the observation encoder and the action encoder includes:

[0097] in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

[0098] In some embodiments of the robot visual navigation system based on contrastive learning in this application, the observation-action loss and the action-observation loss ,include:

[0099]

[0100] in:

[0101] in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express Dimensional identity matrix Middle Line Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Line Column elements, represents the first row element;

[0102] in, middle represents a natural constant, represents the first Line Column elements, Represents the mask matrix Middle Line Column elements.

[0103] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0104] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] An embodiment of the present application also provides an electronic device, which may include one or more processors, a memory, and a communication interface.

[0106] The memory, the communication interface and the processor are coupled, for example, the memory, the communication interface and the processor may be coupled together via a bus.

[0107] The communication interface is used for data transmission with other devices. The memory stores computer program code. The computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the robot visual navigation method based on contrast learning.

[0108] Among them, the processor can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like. The processor can be used to support electronic devices to execute the method steps provided in the above embodiments.

[0109] The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The above bus may be divided into an address bus, a data bus, a control bus, etc.

[0110] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned robot visual navigation method based on contrastive learning are implemented.

[0111] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0112] For the description of the relevant parts of the robot visual navigation system based on contrastive learning, the electronic device, and the computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the robot visual navigation method based on contrastive learning provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0113] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A robot visual navigation method based on contrastive learning, characterized in that: include: Get the current observation; Input the current observation into the trained observation encoder, and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; Use vector inner product to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; Inputting the execution action sequence into the robot controller to control the robot to perform corresponding actions; The training method of the observation encoder and the action encoder includes: Obtain multiple historical observations and historical action sequences; By using the paired relationship between observations and action sequences, features are extracted from each historical observation and each historical action sequence using observation encoder and action encoder respectively, and the historical observation feature vector and the historical action sequence feature vector are obtained correspondingly. Calculate the similarity between every two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; wherein the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

2. The robot visual navigation method based on contrastive learning according to claim 1, characterized in that: The observation encoder and the action encoder both adopt convolutional neural networks; The observation encoder and the action encoder are both connected to a layer of feedforward neural network as a feature projection layer.

3. The robot visual navigation method based on contrastive learning according to claim 1, characterized in that: The calculating the similarity between each two historical action sequences includes: The following formula is used as the action sequence similarity function to calculate the similarity between each two historical action sequences: in, Representing action sequence and action sequences The similarity between represents the length of the action sequence, Indicates the maximum length of the action sequence, Indicates the length is Action sequence , Indicates the length is Action sequence .

4. The robot visual navigation method based on contrastive learning according to claim 3 is characterized in that: The observation-action similarity matrix includes: in, represents the observation-action similarity matrix, represents the first historical observation feature vector, represents the second historical observation feature vector, Indicates N The historical observation feature vector, represents the first historical action sequence feature vector, The second historical action sequence feature vector of the hour, Indicates N The feature vector of the historical action sequence.

5. The robot visual navigation method based on contrastive learning according to claim 1, characterized in that: When training the observation encoder and the action encoder, the loss functions used include: in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

6. The robot visual navigation method based on contrastive learning according to claim 5, characterized in that: The observation-action loss and the action-observation loss ,include: in: in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express Dimensional identity matrix Middle Line Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Line Column elements, represents the first row element; in, middle represents a natural constant, represents the observation-action similarity matrix Line Column elements, Represents the mask matrix Middle Line Column elements.

7. A robot visual navigation system based on contrastive learning, characterized in that: include: Observation module, used to obtain current observations; A feature extraction module is used to input the current observation into the trained observation encoder and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; An execution sequence acquisition module is used to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector using vector inner product, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; A control module is used to input the execution action sequence into the robot controller to control the robot to perform the corresponding action; The training method of the observation encoder and the action encoder includes: Obtain multiple historical observations and historical action sequences; By using the paired relationship between observations and action sequences, features are extracted from each historical observation and each historical action sequence using observation encoder and action encoder respectively, and the historical observation feature vector and the historical action sequence feature vector are obtained correspondingly. Calculate the similarity between every two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; wherein the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

8. The robot visual navigation system based on contrastive learning according to claim 7, characterized in that: When training the observation encoder and the action encoder, the loss functions used include: in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

9. The robot visual navigation system based on contrastive learning according to claim 8, characterized in that: The observation-action loss and the action-observation loss ,include: in: in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express Dimensional identity matrix Middle Line Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Line Column elements, represents the first row element; in, middle represents a natural constant, represents the first Line Column elements, Represents the mask matrix Middle Line Column elements.

10. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the robot visual navigation method based on contrastive learning as described in any one of claims 1 to 6.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot visual navigation method based on contrastive learning are implemented as claimed in any one of claims 1 to 6.

Images