A robot visual navigation method based on contrastive learning and related devices

Abstract

The present application relates to a robot navigation method. In response to the technical problems of existing navigation or exploration methods, such as difficulty in establishing geometric maps in unstructured scenes, reliance on labels from lidar data, easy neglect of long-distance features, and slow reasoning time, a robot visual navigation method and related devices based on contrastive learning are provided. After obtaining the current observation, the observation is input into the trained observation encoder to obtain the corresponding feature vector. Then, the pre-selected batch action sequence is passed through the action encoder to extract the corresponding feature vector. The similarity between the current observation and each action sequence is calculated by vector inner product. The action sequence with the highest similarity is selected and input into the robot controller to control the robot to perform the corresponding action. Compared with the method of directly generating an action sequence, selecting an action sequence is less difficult and has better scalability and generalization.

Description

Technical Field

[0001] The present application relates to a robot navigation method, and specifically to a robot visual navigation method and related devices based on contrastive learning. Background Art

[0002] Vision-based navigation or exploration is a key technology in robotics. Its core goal is to input images captured by the robot's camera sensor as observations and output the robot's trajectory or control signals as a sequence of actions, thereby achieving collision-free navigation or exploration. Robots operating in real-world environments encounter complex situations. Traditional navigation or exploration problems often require a perception module to build a local or global map to estimate the robot's drivable area. Path planning algorithms are then used to plan a trajectory toward the target.

[0003] Existing navigation or exploration methods can be divided into three categories based on the method used to build maps: geometric map-based methods, feature map-based methods, and implicit map-based methods. However, geometric map-based methods struggle to utilize geometric information to build geometric maps in some unstructured scenarios. Feature map-based methods often rely on labels from LiDAR data, and parameters such as the installation location of the relevant sensors can also affect the quality of predictions. Furthermore, convolution or attention mechanism calculations require significant computational power and often ignore long-range features. Implicit map-based methods often struggle to meet the real-time requirements of practical applications, resulting in slow inference times. Summary of the Invention

[0004] This application addresses the technical problems that existing navigation or exploration methods have difficulty in establishing geometric maps in unstructured scenes, rely on labels from lidar data, easily ignore long-distance features, and have slow reasoning time. It provides a robot visual navigation method and related devices based on contrastive learning.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application proposes a robot visual navigation method based on contrastive learning, comprising:

[0007] Get the current observation;

[0008] Input the current observation into the trained observation encoder, and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector;

[0009] Use vector inner product to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence;

[0010] Input the execution action sequence into the robot controller to control the robot to perform the corresponding actions;

[0011] The training methods of the observation encoder and the action encoder include:

[0012] Obtain multiple historical observations and historical action sequences;

[0013] Utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the historical observation feature vector and the historical action sequence feature vector respectively.

[0014] Calculate the similarity between each two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix;

[0015] Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors to obtain the corresponding observation-action similarity matrix;

[0016] Combining the mask matrix and the observation-action similarity matrix, the observation encoder and the action encoder are trained with the identity matrix as the label; the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

[0017] Furthermore, the observation encoder and the action encoder both adopt convolutional neural networks;

[0018] The observation encoder and the action encoder are both connected to a feedforward neural network as a feature projection layer.

[0019] Furthermore, the calculation of the similarity between each two historical action sequences includes:

[0020] The following formula is used as the action sequence similarity function to calculate the similarity between each two historical action sequences:

[0021]

[0022] in, Represents an action sequence and action sequences The similarity between Indicates the length of the action sequence, Indicates the maximum length of the action sequence, Indicates the length is Action sequence , Indicates the length is Action sequence .

[0023] Furthermore, the observation-action similarity matrix includes:

[0024]

[0025] in, represents the observation-action similarity matrix, represents the first historical observation feature vector, represents the second historical observation feature vector, Indicates the N The historical observation feature vector, represents the first historical action sequence feature vector, The second historical action sequence feature vector of the hour, Indicates the N The feature vector of the historical action sequence.

[0026] Furthermore, when training the observation encoder and the action encoder, the loss function used includes:

[0027]

[0028] in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

[0029] Furthermore, the observation-action loss and the action-observation loss ,include:

[0030]

[0031]

[0032] in:

[0033]

[0034] in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express dimensional identity matrix Middle Rank Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Rank Column elements, represents the first row elements;

[0035]

[0036] in, middle represents a natural constant, represents the first Rank Column elements, Represents the mask matrix Middle Rank Column element.

[0037] In a second aspect, the present application proposes a robot visual navigation system based on contrastive learning, comprising:

[0038] Observation module, used to obtain current observations;

[0039] A feature extraction module is used to input the current observation into the trained observation encoder and the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector;

[0040] An execution sequence acquisition module is used to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector using vector inner product, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence;

[0041] The control module is used to input the execution action sequence into the robot controller to control the robot to perform the corresponding actions;

[0042] The training methods of the observation encoder and the action encoder include:

[0043] Obtain multiple historical observations and historical action sequences;

[0044] Utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the historical observation feature vector and the historical action sequence feature vector respectively.

[0045] Calculate the similarity between each two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix;

[0046] Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors to obtain the corresponding observation-action similarity matrix;

[0047] The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

[0048] Furthermore, when training the observation encoder and the action encoder, the loss function used includes:

[0049]

[0050] in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

[0051] Furthermore, the observation-action loss and the action-observation loss ,include:

[0052]

[0053]

[0054] in:

[0055]

[0056] in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express dimensional identity matrix Middle Rank Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Rank Column elements, represents the first row elements;

[0057]

[0058] in, middle represents a natural constant, represents the first Rank Column elements, Represents the mask matrix Middle Rank Column element.

[0059] In a third aspect, the present application proposes an electronic device comprising: a memory, one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the above-mentioned robot visual navigation method based on contrastive learning.

[0060] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned robot visual navigation method based on contrastive learning are implemented.

[0061] Compared with the prior art, this application has the following beneficial effects:

[0062] The present application proposes a robot visual navigation method based on contrastive learning. After obtaining the current observation, it is input into the trained observation encoder to obtain the corresponding feature vector. Then, the pre-selected batch action sequence is passed through the action encoder to extract the corresponding feature vector. The similarity between the current observation and each action sequence is calculated by the vector inner product, and the action sequence with the highest similarity is selected and input into the robot controller to control the robot to perform the corresponding action. Compared with the method of directly generating an action sequence, it is less difficult to select an action sequence, and has better scalability and generalization. In addition, when training the observation encoder and the action encoder, the present application adopts the contrastive learning method for self-supervision. The features of the observation and action sequences can be aligned in the latent space without manual data labeling, thereby improving the robustness and generalization of the model. The present application can achieve efficient and accurate navigation without relying on labels from lidar data.

[0063] This application also proposes a robot visual navigation system, electronic device and computer storage medium based on contrastive learning, which have all the advantages of the above-mentioned robot visual navigation method based on contrastive learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0065] Figure 1 This is a schematic diagram of the first flow chart of the robot visual navigation method based on contrastive learning in this application;

[0066] Figure 2 This is a second flow chart of the robot visual navigation method based on contrastive learning in this application;

[0067] Figure 3 Schematic diagram of the training process of the observation encoder and the action encoder in the embodiment of the present application;

[0068] Figure 4 A flowchart of the reasoning stage in the embodiment of the present application;

[0069] Figure 5 This is a schematic diagram of the training results using only the first part of the real scene data set in the embodiment of this application;

[0070] Figure 6 This is a schematic diagram of the results of jointly training using the first and second parts of real scene data in an embodiment of the present application;

[0071] Figure 7 This is a schematic diagram of the robot visual navigation system based on contrastive learning in this application. DETAILED DESCRIPTION

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0073] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0074] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0075] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or the orientation or position relationship in which the product of the invention is usually placed when in use. This is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, it should not be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0076] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0077] In the description of the embodiments of this application, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in this application based on specific circumstances.

[0078] Vision-based navigation or exploration is a core area in robotics, enabling robots to move autonomously in complex, dynamic, and often unknown environments. This technology requires robots to not only understand and interpret image information acquired from camera sensors, but also to make decisions based on this information and plan and execute safe navigation paths.

[0079] The robot preprocesses the raw images captured by the camera, performing techniques such as denoising, correction, and enhancement to improve image quality. Feature extraction techniques are used to identify key information in the images, such as object outlines, obstacle locations, and landmarks. Based on this key information, the robot constructs or updates a local map of its surroundings in real time, including obstacle locations and traversable areas. The robot's precise positioning within the map is fundamental to navigation. A path planning algorithm is then used to plan an optimal path from the starting point to the destination on the global map. During actual navigation, the robot may need to make local adjustments to the path based on real-time environmental changes. Finally, based on the path planning results, the robot generates a trajectory, including control parameters such as speed and direction. The robot executes the trajectory through its motion control system, achieving collision-free navigation.

[0080] Existing vision-based navigation methods can be divided into three categories according to the method of building maps: geometric map-based methods, feature map-based methods and implicit map-based methods.

[0081] Most geometric map-based methods use simultaneous localization and mapping (SLAM) algorithms to create a point cloud to represent spatial occupancy or occupancy probabilities, and continuously and dynamically maintain a local occupancy map. This process is difficult to implement in unstructured scenarios, such as overgrown wilderness or sidewalks on the side of a road, where geometric information is difficult to use to create a geometric map.

[0082] Feature map-based approaches learn an observation-to-action strategy from the robot's past driving experience. With the development of deep learning, it has been realized that deep neural networks can be used to help construct maps for feature representation. For autonomous driving, an effective way to represent the feature space is the bird's eye view (BEV), which has strong perception capabilities and can fuse data from different sensors into a unified BEV representation. Another approach is to use a 3D occupancy network, which also inputs information from multiple cameras and outputs perception results. However, these methods often rely on labels from lidar data. Parameters such as the sensor's installation position can also affect the quality of the prediction. Another difficulty lies in the large number of model parameters. Even in sparse scenes, convolution or attention mechanism calculations require a lot of computing power. In addition, these methods directly ignore long-range features, which also affects the perception effect.

[0083] Implicit map-based methods do not use displayed maps, but directly obtain predicted action sequences from current observations and learn navigation from past experience. The method of mapping current observations to action sequences has difficulty in modeling multiple possibilities of action sequences. This is unreasonable in the real world. For example, if there is an obstacle in front of the robot, or the robot travels to a T-junction, the output action sequence should have different directions, moving both left and right. The application of diffusion models can solve this problem. It models the action space under observation conditions. However, when the diffusion model generates action sequences from noise, dozens or even hundreds of iterative inferences are required, which makes real-time requirements difficult to meet and results in slow inference time.

[0084] Another difficulty in robot navigation or exploration is the difficulty in acquiring and labeling data. Self-supervised learning methods can avoid labeling large amounts of data. Contrastive learning has been widely used in unsupervised representation learning. Contrastive learning usually requires the presence of positive and negative samples. For single-modal data, contrastive learning usually uses the data itself as positive samples and other data as negative samples, and makes the positive samples as close as possible in the feature space and the negative samples as far apart as possible in the feature space. For paired multimodal data, contrastive learning uses the data pair itself as positive samples and different data pairs as negative samples, and aligns the sample pairs in the feature space.

[0085] Based on the above situation, the present application proposes a robot visual navigation method and related devices based on contrastive learning. The present application is described in detail below with reference to the embodiments and drawings.

[0086] like Figure 1 FIG. 1 is a schematic diagram of a robot visual navigation method based on contrastive learning in the present application, which may include:

[0087] S101, obtain the current observation.

[0088] The robot obtains images or video frames of the current environment through its visual sensors (such as cameras) as observation data, providing input for subsequent feature extraction and action selection.

[0089] S102, inputting the current observation into the trained observation encoder, and inputting the pre-selected batch action sequence into the trained action encoder, to obtain the current observation feature vector and the batch pre-selected action feature vectors.

[0090] Convert high-dimensional observation and action sequence data into low-dimensional feature vectors to facilitate subsequent calculations and comparisons.

[0091] Both the observation encoder and the action encoder can be deep learning models. The observation encoder accepts observation data (such as images or video frames) acquired by the robot from the environment as input and outputs a fixed-length feature vector. This feature vector contains key information from the observation data, which is then compared with the action sequence feature vector. The action encoder accepts possible action sequences as input and outputs a feature vector corresponding to each action sequence. These feature vectors are used to compare with the observation feature vector to select the most appropriate action sequence.

[0092] S103, using vector inner products to calculate feature similarities between the current observation feature vector and each pre-selected action feature vector, and selecting the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence.

[0093] The action sequence corresponding to the pre-selected action feature vector with the highest feature similarity is selected as the execution action sequence. The most appropriate action sequence can be selected according to the current observation to achieve collision-free navigation or exploration tasks.

[0094] S104: Inputting the execution action sequence into the robot controller to control the robot to execute corresponding actions.

[0095] The robot controller can control the movement of the robot according to the execution action sequence, including parameters such as speed and direction, so that the robot can navigate or explore the environment according to the selected action sequence.

[0096] The training methods of the observation encoder and the action encoder include:

[0097] (1) Obtain multiple historical observations and historical action sequences.

[0098] (2) Utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the corresponding historical observation feature vector and historical action sequence feature vector.

[0099] (3) Calculate the similarity between each two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix.

[0100] Among them, the mask matrix can be used to distinguish positive samples (the same observation and action sequence data pairs) from negative samples (different observation and action sequence data pairs).

[0101] (4) Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors respectively to obtain the corresponding observation-action similarity matrix.

[0102] (5) Combining the mask matrix and the observation-action similarity matrix, the observation encoder and the action encoder are trained with the identity matrix as the label; wherein the training objective is to maximize the distance between the feature vectors from the same observation and action sequence data pair, and minimize the distance between the feature vectors from the same observation and action sequence data pair.

[0103] In practical applications, the identity matrix is ​​used as the label, indicating that the similarity between feature vectors of positive samples should be 1, and the similarity between feature vectors of negative samples should be 0. The observation encoder and action encoder are trained using a contrastive learning loss function. The training objective is to maximize the similarity between feature vectors of positive samples (i.e., minimize the distance between feature vectors) and minimize the similarity between feature vectors of negative samples (i.e., maximize the distance between feature vectors). Through training, the observation encoder and action encoder can extract discriminative feature vectors, thereby more accurately matching the current observation and action sequence when selecting an action sequence to execute.

[0104] This application solves the problems that explicit maps are difficult to express geometric information, direct mapping from observations to action sequences is difficult to model multiple possibilities in the same scenario, and data acquisition and labeling for real robots are difficult.

[0105] like Figure 2 FIG. 1 is a second flow chart of the robot visual navigation method based on contrastive learning of the present application, which may include:

[0106] S201, utilizing the natural paired relationship between observations and action sequences, uses the observation encoder and action encoder to extract the features of historical observations and the features of historical action sequences respectively, and obtains their corresponding historical observation feature vectors and historical action sequence feature vectors in the feature space.

[0107] In this embodiment, the training data used simulation data and real scene data respectively. In the simulation data, 3 hours and 46 minutes of autonomous driving data were collected using the CARLA autonomous driving simulator. In the driving of the simulation scene, most of the time is relatively stable, so this embodiment manually added noise that conforms to the uniform distribution to the planned trajectory points when collecting data to make it more consistent with the actual situation. The real scene data can be divided into two parts. The first part is 41 minutes of data collected by controlling the robot using a remote control under cloudy weather conditions. The second part is an open source data set selected from about 10 hours of collection by different robots. Both parts of the data set include roads, buildings, pedestrians, vehicles, etc. These data are processed into observation-action sequence data pairs. The observation is the picture of the current frame, and the action sequence is the coordinates of the spatial points at the next 16 moments.

[0108] In this embodiment, observation encoders are used respectively and action encoder Extract features of historical observations and historical action sequences:

[0109]

[0110]

[0111] in, For observation, For action sequences, and are the feature vectors corresponding to the observation and action sequences, respectively.

[0112] In some embodiments of the present application, a feedforward neural network layer can be added after each encoder as a feature projection layer to better align the features of the observation and action sequences in the latent space. Alternatively, the encoder can use a convolutional neural network with a latent space dimension of 256.

[0113] S202 , for a batch of action sequences, calculate the similarity between any two action sequences according to a defined action sequence similarity function, obtain a similarity matrix for the batch of action sequences, and calculate a mask matrix according to the similarity matrix for the batch of action sequences and a threshold coefficient.

[0114] In the training phase, this embodiment calculates the similarity between a batch of action sequences by defining an action sequence similarity function:

[0115]

[0116] in, Representing an action sequence and action sequences The similarity between Indicates the length of the action sequence, Indicates the maximum length of the action sequence, Indicates the length is Action sequence , Indicates the length is Action sequence . You can get Similarity results, this The similarity results are arranged in order to obtain the action sequence similarity matrix of this batch of data. The more similar the two action sequences are, the smaller the corresponding similarity result is.

[0117] Action sequence similarity functions can be designed based on the specific characteristics of the action sequence, such as the type, sequence, and duration of the action. Common similarity metrics include edit distance, dynamic time warping (DTW), and cosine similarity, which can be selected based on the actual situation.

[0118] In order to improve the stability of the training phase and the generalization of the reasoning phase, this embodiment uses the action sequence similarity matrix to obtain the mask matrix :

[0119]

[0120] in, is the threshold parameter. This formula indicates that in the action sequence similarity matrix, the mask matrix value at the position between two similar action sequences is 0 and should be masked, while the value at all other positions is 1 and should be retained. Specifically, if the similarity value is greater than or equal to the threshold coefficient, the mask value at the corresponding position is 1, indicating that the two action sequences are considered a positive pair; otherwise, it is 0, indicating that they are considered a negative pair. In practical applications, the choice of threshold coefficient can be determined based on the characteristics of the specific task and dataset, and usually requires tuning through experimentation.

[0121] S203, using vector inner product to calculate all observation feature vectors and action sequence feature vectors to obtain an observation-action similarity matrix.

[0122] In the training phase of this embodiment, a batch of observation-action sequence data pairs are given, and the feature vector of each pair of observation-action sequence is calculated in step S201. The data can be obtained The observed eigenvector values ​​and The action sequence feature vector value is calculated by performing the inner product calculation of the observation feature vector and the action sequence feature vector. The larger the value, the more the observation and action sequence match. The observation-action matching values ​​are arranged in order to obtain the observation-action sequence similarity matrix of this batch of data. :

[0123]

[0124] The observation-action sequence similarity matrix of The diagonal elements represent the inner product of features from the same observation-action sequence data pair, and the rest The elements represent the inner products of features from different observation-action sequence data pairs, and the goal of the training phase is to maximize the diagonal elements and minimize the elements at other positions.

[0125] S204, using the identity matrix as a label to train the observation encoder and the action encoder, maximize the distance between feature vectors from the same observation-action data pair, and minimize the distance between feature vectors from different observation-action data pairs.

[0126] In this embodiment, in order to make the observation and action sequence share more information, the loss function is set to two parts:

[0127]

[0128] in, is the observation-action loss, is the action-observation loss, and Represent the corresponding weights respectively. Given the observation-action sequence similarity matrix calculated in step S203 , this example uses the identity matrix as the label and trains the observation encoder and action encoder using the following symmetric cross entropy loss:

[0129]

[0130] middle, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express dimensional identity matrix Middle Rank Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Rank Column elements, represents the first Row element.

[0131]

[0132] in, middle represents a natural constant, represents the first Rank Column elements, Represents the mask matrix Middle Rank The mask matrix is ​​set to 0 to ensure that similar action sequences are invisible to each other and do not affect each other during the training phase, thereby improving the stability of training. The observation-action loss and action-observation loss are defined as follows:

[0133]

[0134]

[0135] This embodiment uses the above loss function to transform the problem of generating action sequences into a classification problem, selecting the most appropriate action sequence category when given observations, and selecting the most appropriate observation category when given action sequences, thereby reducing the difficulty of model training and improving the robustness of the robot during navigation or exploration.

[0136] Figure 3 FIG. 1 is a schematic diagram of the training process of the observation encoder and the action encoder in the above embodiment.

[0137] S205, based on the trained observation encoder and action encoder, input the current robot observation and a batch of saved action sequences, use the vector inner product to calculate the feature similarity between the current observation and each action sequence, and select the action sequence with the highest similarity as the execution action sequence.

[0138] Based on the trained observation encoder and action encoder Observations and action sequences can be encoded into a latent space and their features can be aligned. In the deployment inference phase, this embodiment randomly selects 100 action sequences from the training data as candidate action sequences. At each moment, the action sequence among the candidate action sequences that best matches the current observation is obtained:

[0139]

[0140] in, It is the final selected execution action sequence. The vector inner product is used to calculate the feature similarity between the current observation and each candidate action sequence. The greater the similarity, the greater the inner product result. Finally, the most similar action sequence in the feature space is executed.

[0141] like Figure 4 FIG. 1 is a flow chart of the inference phase of this embodiment.

[0142] S206: Input the execution action sequence into the robot controller, output the robot control signal, and test the robot's navigation or exploration task performance.

[0143] In this embodiment, a proportional-differential controller is selected as the controller of the robot, and the execution action sequence obtained in step S205 is input. The controller can calculate the control signal of the robot and control the movement of the robot.

[0144] like Figure 5 The figure shows the training results of using only the first part of the real scene dataset. Each column represents a scene, and each picture from left to right is a third-person perspective, a first-person perspective, and an action sequence diagram. The blue color represents the execution action sequence that ultimately generates the control signal, the green color represents the candidate action sequences with the highest scores, and the red color represents the candidate action sequences with the lowest scores. It can be seen that despite the changes in weather conditions and even at night, this embodiment can still complete the collision-free exploration task. And it can be seen from Figure 5 As can be seen from the figure, when the robot comes to the T-junction, the action sequences with higher scores include both left turns and right turns, showing the diversity of action sequences.

[0145] like Figure 6 The following is a diagram showing the results of training with the first and second parts of real scene data. Figure 6 It can be seen that in an environment that has never been seen before, this embodiment can still understand the drivable area and plan the correct route to complete the collision-free exploration task, demonstrating the robustness of the model.

[0146] like Figure 7 FIG. 1 is a schematic diagram of a robot visual navigation system based on contrastive learning in the present application, which may include:

[0147] Observation module, used to obtain current observations;

[0148] A feature extraction module is used to input the current observation into the trained observation encoder and the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector;

[0149] An execution sequence acquisition module is used to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector using vector inner product, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence;

[0150] The control module is used to input the execution action sequence into the robot controller to control the robot to perform the corresponding actions;

[0151] The training methods of the observation encoder and the action encoder include:

[0152] Obtain multiple historical observations and historical action sequences;

[0153] Utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the historical observation feature vector and the historical action sequence feature vector respectively.

[0154] Calculate the similarity between each two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix;

[0155] Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors to obtain the corresponding observation-action similarity matrix;

[0156] The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

[0157] In some embodiments of the robot visual navigation system based on contrastive learning of the present application, the loss function used when training the observation encoder and the action encoder includes:

[0158]

[0159] in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

[0160] In some embodiments of the robot visual navigation system based on contrastive learning in this application, the observation-action loss and the action-observation loss ,include:

[0161]

[0162]

[0163] in:

[0164]

[0165] in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express dimensional identity matrix Middle Rank Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Rank Column elements, represents the first row elements;

[0166]

[0167] in, middle represents a natural constant, represents the first Rank Column elements, Represents the mask matrix Middle Rank Column element.

[0168] It should be noted that in the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The modules described as separate components may or may not be physically separated. The components displayed as modules may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0169] In addition, the modules in the various embodiments of the present invention may be integrated into a single processing unit, each module may exist physically separately, or two or more modules may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0170] An embodiment of the present application also provides an electronic device, which may include one or more processors, memories, and communication interfaces.

[0171] The memory, the communication interface, and the processor are coupled together. For example, the memory, the communication interface, and the processor may be coupled together via a bus.

[0172] The communication interface is used to transmit data with other devices. The memory stores computer program code. The computer program code includes computer instructions that, when executed by the processor, cause the electronic device to perform the steps of the aforementioned robot visual navigation method based on contrastive learning.

[0173] Among them, the processor can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute the various exemplary logic blocks, modules and circuits described in conjunction with the contents of this disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The processor can be used to support electronic devices in executing the method steps provided in the above embodiments.

[0174] The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The above buses may be divided into an address bus, a data bus, a control bus, etc.

[0175] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned robot visual navigation method based on contrastive learning are implemented.

[0176] The computer-readable storage medium involved in this application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0177] For the description of the relevant parts of the robot visual navigation system based on contrastive learning, the electronic device, and the computer-readable storage medium provided in the embodiments of this application, please refer to the detailed description of the corresponding parts in the robot visual navigation method based on contrastive learning provided in the embodiments of this application, and no further description is given here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of this application that are consistent with the corresponding technical solutions in the prior art in the implementation principle are not described in detail to avoid excessive elaboration.

[0178] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A robot visual navigation method based on contrastive learning, characterized in that: include: Get the current observation; Input the current observation into the trained observation encoder, and input the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; Use vector inner product to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; Input the execution action sequence into the robot controller to control the robot to perform the corresponding actions; The training methods of the observation encoder and the action encoder include: Obtain multiple historical observations and historical action sequences; Utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the historical observation feature vector and the historical action sequence feature vector respectively. Calculate the similarity between each two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

2. The robot visual navigation method based on contrastive learning according to claim 1, characterized in that: The observation encoder and the action encoder both adopt convolutional neural networks; The observation encoder and the action encoder are both connected to a feedforward neural network as a feature projection layer.

3. The robot visual navigation method based on contrastive learning according to claim 1, characterized in that: The calculation of the similarity between each two historical action sequences includes: The following formula is used as the action sequence similarity function to calculate the similarity between each two historical action sequences: in, Representing an action sequence and action sequences The similarity between Indicates the length of the action sequence, Indicates the maximum length of the action sequence, Indicates the length is Action sequence , Indicates the length is Action sequence .

4. The robot visual navigation method based on contrastive learning according to claim 3, characterized in that: The observation-action similarity matrix includes: in, represents the observation-action similarity matrix, represents the first historical observation feature vector, represents the second historical observation feature vector, Indicates the N The historical observation feature vector, represents the first historical action sequence feature vector, The second historical action sequence feature vector of the hour, Indicates the N The feature vector of the historical action sequence.

5. The robot visual navigation method based on contrastive learning according to claim 1, characterized in that: When training the observation encoder and the action encoder, the loss functions used include: in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

6. The robot visual navigation method based on contrastive learning according to claim 5, characterized in that: The observation-action loss and the action-observation loss ,include: in: in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express dimensional identity matrix Middle Rank Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Rank Column elements, represents the first row elements; in, middle represents a natural constant, represents the first Rank Column elements, Represents the mask matrix Middle Rank Column element.

7. A robot visual navigation system based on contrastive learning, characterized in that: include: Observation module, used to obtain current observations; A feature extraction module is used to input the current observation into the trained observation encoder and the pre-selected batch action sequence into the trained action encoder to obtain the current observation feature vector and the batch pre-selected action feature vector; An execution sequence acquisition module is used to calculate the feature similarity between the current observation feature vector and each pre-selected action feature vector using vector inner product, and select the action sequence corresponding to the pre-selected action feature vector with the highest feature similarity as the execution action sequence; The control module is used to input the execution action sequence into the robot controller to control the robot to perform the corresponding actions; The training methods of the observation encoder and the action encoder include: Obtain multiple historical observations and historical action sequences; Utilizing the paired relationship between observations and action sequences, we use the observation encoder and action encoder to extract features from each historical observation and each historical action sequence, and obtain the historical observation feature vector and the historical action sequence feature vector respectively. Calculate the similarity between each two historical action sequences to obtain the similarity matrix of the historical action sequences, and combine it with the set threshold coefficient to obtain the corresponding mask matrix; Perform vector inner product calculations on the historical observation feature vectors and the historical action sequence feature vectors to obtain the corresponding observation-action similarity matrix; The observation encoder and action encoder are trained by combining the mask matrix and the observation-action similarity matrix with the identity matrix as the label; the training objective is to maximize the distance between feature vectors from the same observation and action sequence data pair, and minimize the distance between feature vectors from the same observation and action sequence data pair.

8. The robot visual navigation system based on contrastive learning according to claim 7, characterized in that: When training the observation encoder and the action encoder, the loss functions used include: in, represents the loss function, represents the weight of observation-action loss, represents the observation-action loss, represents the weight of the action-observation loss, represents the action-observation loss.

9. The robot visual navigation system based on contrastive learning according to claim 8, characterized in that: The observation-action loss and the action-observation loss ,include: in: in, express, represents the dimension of the identity matrix, represents the row number of the identity matrix, represents the column number of the identity matrix, express dimensional identity matrix Middle Rank Column elements, Each row of the observation-action sequence similarity matrix is ​​first masked by the mask matrix and then the output of the softmax operation is used. represents the first Rank Column elements, represents the first row elements; in, middle represents a natural constant, represents the first Rank Column elements, Represents the mask matrix Middle Rank Column element.

10. An electronic device, characterized in that: include: A memory, one or more processors; the memory is coupled to the processor; wherein a computer program code is stored in the memory, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the robot visual navigation method based on contrastive learning as described in any one of claims 1 to 6.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the robot visual navigation method based on contrastive learning according to any one of claims 1 to 6.

Images