Transformer-based target object navigation method

Abstract

This invention discloses a target object navigation method based on Transformer, comprising: inputting the current environment image and the name of the target object to obtain a local environment feature vector, a global environment feature vector, and a target object feature vector; inputting the three vectors into a target vector Transformer model to obtain the final target vector at the current time; inputting the final target vectors at the current time and the previous time step into a navigation direction vector Transformer model to obtain a navigation direction vector; inputting the navigation direction vector into an A3C reinforcement learning model to obtain the action probability distribution vector, action score vector, and reward value at the current time step, and then selecting the action with the highest probability in the action probability distribution vector for execution; repeating the above process until the robot completes the action, and updating the model weights according to the action score vector and reward value at different times. This invention can improve the navigation effect and generalization ability.

Description

Technical Field

[0001] This invention relates to the technical field of visual target object navigation, and in particular to a target object navigation method based on Transformer. Background Technology

[0002] With the continuous development of mobile robots and smart home concepts, mobile robots are increasingly appearing in daily home life. Indoor navigation is one of the important functions of mobile robots. As the application scenarios of mobile robots become more diverse, they need to adapt to various unknown home environments. However, traditional mapping and navigation methods cannot meet the navigation needs of multiple scenarios. Therefore, with the continuous development of deep learning technology, target navigation methods based on deep reinforcement learning have gradually emerged. However, existing methods need improvement in terms of effectiveness and generalization ability. How to enable mobile robots to understand the semantics, position, and object relationships in first-person perspective images, and to find objects associated with the target object in the image by its name and execute appropriate actions to navigate towards the target object has become an urgent problem to be solved.

[0003] The target object navigation method is a deep reinforcement learning-based visual navigation system for mobile robots in unknown environments. It associates target objects with objects in an image, enabling the mobile robot to adapt to unknown home environments and complete visual navigation tasks. For example, given a target object, a lamp, the mobile robot needs to find the lamp or an object associated with it in the visual input, and finally navigate to within one meter in front of the lamp and execute the completion command.

[0004] Previous target object navigation technologies have the following main drawbacks: during the navigation process, it is impossible to find the relationship between objects in the image and the target object, and the target object does not play a guiding role in the navigation of the mobile robot, resulting in poor navigation effect and generalization ability. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a target object navigation method based on Transformer. The Transformer model is used to extract semantic, positional and object relationship information in the current image through an encoder and decoder structure. The target object feature vector is used multiple times as a query vector to discover objects associated with the target object in the current image and historical images, thereby enhancing the Transformer model's cognitive generalization ability to the current environment and navigation process, thus improving the navigation effect and generalization ability.

[0006] To achieve the above objectives, the technical solution provided by this invention is: a target object navigation method based on Transformer, comprising the following steps:

[0007] 1) Preprocessing of input data, including: using the DETR model to extract local environmental feature vectors from the current time-input environmental image; using the ResNet-18 model to extract global environmental feature vectors from the current time-input environmental image; and using a word embedding model to embed the target object name to obtain the target object feature vector.

[0008] 2) Input the local environment feature vector, global environment feature vector, and target object feature vector into the target vector Transformer model to obtain the final target vector at the current time step;

[0009] 3) Input the final target vector at the current moment into the navigation direction vector Transformer model to obtain the navigation direction vector;

[0010] 4) Input the navigation direction vector into the A3C reinforcement learning model to obtain the action probability distribution vector, action score vector and reward value at the current time, and then select the action with the highest probability in the action probability distribution vector to execute;

[0011] 5) Repeat steps 1) to 4) until the robot completes the action. At this point, a target object navigation event is completed. Update the model weights based on the action score vector and reward value at different times.

[0012] Furthermore, in step 1), the current environmental image is input into the DETR model to obtain the local environmental feature vector L. i,j ∈R 100×256 Where 100×256 represents a feature layer of size 100×256, with a size of 100 in the i-direction and 256 in the j-direction, R 100×256 Represents a set of real numbers of size 100×256; input the current environmental image into the ResNet-18 model to obtain the global environmental feature vector G. i,j ∈R 49×256 Where 49×256 represents a feature layer of size 49×256, with a size of 49 in the i-direction and 256 in the j-direction, R 49×256 Represents a set of real numbers of size 49×256; the target object feature vector T is obtained by embedding the target object name using a word embedding model. i,j ∈R 1×256 Where 1×256 represents a feature layer of size 1×256, with a size of 1 in the i-direction and a size of 256 in the j-direction, R 1×256 Let represent the set of real numbers, which is 1 × 256.

[0013] Further, in step 2), the target vector Transformer model includes a visual encoder, a visual decoder, and a target object decoder, which performs the following operations: the visual encoder encodes the local environment feature vector to obtain an enhanced local environment feature vector; the global environment feature vector is added with position encoding to obtain an enhanced global environment feature vector; the enhanced local environment feature vector and the enhanced global environment feature vector are input into the visual decoder to obtain an environment feature vector; the environment feature vector and the target object feature vector are input into the target object decoder to obtain the preliminary target vector at the current moment; the preliminary target vector at the current moment is dimensionality reduced by a fully connected layer; the action probability distribution vector at the previous moment is dimensionality increased by a fully connected layer; the dimensionality increased action probability distribution vector and the dimensionality reduced preliminary target vector at the current moment are concatenated and flattened to obtain the final target vector at the current moment.

[0014] Furthermore, the target vector Transformer model uses imitation learning for pre-training, enabling the model to obtain good initial weights, which helps to accelerate the convergence speed during formal training. The model uses multiple teaching trajectories of the robot navigating in the environment as the pre-training dataset P = {(s1,a1),(s2,a2),...,(s...}. n ,a n )}, where (s n ,a n ) represents the parameter of the nth teaching trajectory, s n The robot state for the nth taught trajectory includes local environment feature vectors, global environment feature vectors, and target object feature vectors, a. n Let be the probability distribution vector of the action for the nth teaching trajectory, i.e., the probability distribution of walking forward, turning left, turning right, looking up, looking down, and completing the action; during the pre-training process of imitation learning, the nth teaching trajectory, input s n The Transformer model outputs the final target vector at the current time step. This final target vector is then input into a fully connected layer to generate an action probability distribution vector. This vector is then used to calculate the cross-entropy with the action probability distribution vector of the nth taught trajectory. The objective function τ is expressed as:

[0015]

[0016] In the formula, n represents the nth teaching trajectory, P(a n Q(a) represents the probability distribution of the actions on the nth taught trajectory, i.e., the true distribution; n ) represents the action probability distribution of the model pre-training, i.e., the prediction distribution; λ is the hyperparameter of imitation learning; the smaller the cross-entropy between the true distribution and the prediction distribution, the closer their distributions are. The purpose of learning each teaching trajectory by minimizing the objective function τ is to achieve the purpose of initializing the weights of the pre-trained model.

[0017] Furthermore, step 2) includes the following steps:

[0018] 2.1) Using local environment feature vector L i,j ∈R 100×256 The query vector, key vector, and value vector of the multi-head self-attention mechanism MHA() are input to the visual encoder VisionEncoder(L i,j ,L i,j ,L i,j The enhanced local environment feature vector L' is obtained. i,j ∈R 100×256 The visual encoder is defined as follows:

[0019] L' i,j =VisionEncoder(L i,j ,L i,j ,L i,j )=Add&Norm(Linear(Add&Norm(MHA(L i,j ,L i,j ,L i,j ))))

[0020] In the formula, Add&Norm() is the residual connection and layer normalization, Linear() is the fully connected layer, and MHA() is the multi-head self-attention mechanism;

[0021] 2.2) Using the global environment feature vector G i,j Adding positional encoding yields an enhanced global environment feature vector G' i,j This will enhance the global environmental feature vector G' i,j and enhance the local environmental feature vector L' i,j Input visual decoder VisionDecoder(G' i,j ,L' i,j ,L' i,j Obtain the environmental feature vector E at the current moment. i,j ∈R 49×256 The visual encoder is defined as follows:

[0022] E i,j =VisionDecoder(Pos(G i,j ),L' i,j ,L' i,j =MHA(Add&Norm(MHA(G')) i,j ,G' i,j ,G' i,j )),L' i,j ,L' i,j )

[0023] In the formula, Pos() is the position code;

[0024] The feature vector T of the target object i,j ∈R 1×256 Repeated stacking and adding positional encoding yields the enhanced target object feature vector T' i,j ∈R 49×256 This will enhance the feature vector T' of the target object. i,j ∈R 49×256 and E i,j ∈R 49×256 Input target object decoder TargetDecoder(T' i,j E i,j E i,j The initial target vector Lt at the current time is obtained from ) i,j ∈R 49×256 The target object decoder is defined as follows:

[0025] Lt i,j =TargetDecoder(T' i,j E i,j E i,j =Add&Norm(MHA(T') i,j E i,j E i,j ))

[0026] The action probability distribution vector a from the previous moment t ∈R 1×6 t represents the vector a' obtained by upsizing through a fully connected layer at time t. t ∈R 1×64 , where R 1×64 Represent the set of real numbers of 1×64, and then use the initial target vector Lt at the current time. i,j ∈R 49 ×256 The vector Lt' is obtained by dimensionality reduction through a fully connected layer. i,j ∈R 49×64 R 49×64 Represent the set of real numbers, which is 49 × 64, and then use vector a' t Sum vector Lt' i,j Connect and flatten to obtain the final target vector Lt at the current time step. i,j ∈R 1×3200 R 1×3200 The set of real numbers, representing 1 × 3200, is expressed as:

[0027] Lt” i,j =Flatten(Concate(LinearUp(a t),LinearDown(Lt i,j )))

[0028] In the formula, Flatten() flattens, Concate() connects, LinearUp() increases the dimensionality of the fully connected layer, and LinearDown() decreases the dimensionality of the fully connected layer.

[0029] Furthermore, in step 3), the navigation direction vector Transformer model includes a visual encoder and a target object decoder, which performs the following operations: stacking the final target vectors of the current time step and the previous time step, adding position encoding, and then inputting them into the visual encoder to obtain the hidden layer vector; inputting the hidden layer vector and the target object feature vector into the target object decoder to obtain the navigation direction vector.

[0030] Furthermore, step 3) includes the following steps:

[0031] The final target vector Lt obtained from the Transformer model of the target vectors at the current and previous time steps is calculated. i,j ∈R 1×3200 Stacking them together yields the vector Lt” i,j ∈R 2×3200 R 2×3200 The set of real numbers representing 2×3200, after being encoded with position, is input into the navigation direction vector of the Transformer model's visual encoder, VisionEncoder(Pos(Lt)). i，j ),Pos(Lt” i，j ),Pos(Lt” i，j The hidden layer vector H is obtained. i,j ∈R 2×3200 The hidden layer vector is input into the fully connected layer to reduce the dimension to 256, resulting in vector H'. i,j ∈R 2×256 R 2×256 Let H' represent the set of real numbers, which is 2 × 256. i,j ∈R 2×256 The target object feature vector, along with the navigation direction vector, is input into the target object decoder (TargetDecoder(T)) of the Transformer model. i,j ,H' i,j ,H' i,j ) Obtain the navigation direction vector GT t ∈R 1×256 .

[0032] Further, in step 4), the A3C reinforcement learning model includes an action generation network, an action scoring network, and a reward / penalty function, which performs the following operations: taking the navigation direction vector as input to the action generation network to obtain the action probability distribution vector at the current time, and then selecting and executing the action with the highest probability in the action probability distribution vector; taking the navigation direction vector as input to the action scoring network to obtain the action score vector at the current time, representing the score of the action with the highest probability in the action probability distribution vector; and generating the reward value at the current time according to the reward / penalty function.

[0033] Furthermore, the action generation network consists of a single fully connected layer, with the input navigation direction vector GT. t Generate an action probability distribution vector a t ∈R 1×6 The action scoring network consists of two fully connected layers, with the input navigation direction vector GT. t Generate an action rating vector v t ∈R 1×1 R 1×1 Let represent a 1×1 set of real numbers; the reward / penalty function consists of two parts: the first part is a reward of 5.0 if the robot reaches within one meter of the target object and completes the action; the second part is a penalty of -0.01 if the robot does not reach the target object; after each step, if the robot does not reach the vicinity of the target object, a penalty reward of -0.01 is given, the purpose of which is to enable the A3C reinforcement learning model to reduce the number of steps; conversely, if navigation is successful, a large positive reward is given. Maximizing the reward will enable the A3C reinforcement learning model to complete the navigation task with the fewest possible steps.

[0034] Furthermore, step 5) includes the following steps:

[0035] Initially, given the target object name and the current location's environmental image, the robot preprocesses the input data. Then, using the target vector Transformer model and the navigation direction vector Transformer model, it outputs the corresponding target vector and navigation direction vector, respectively. Next, the A3C reinforcement learning model outputs an action, which the robot executes to reach the next state. It then obtains the target object name and the current location's environmental image again, repeating the above process until the A3C reinforcement learning model completes the action or reaches its maximum number of executions and automatically completes the action. At this point, a target object navigation event is completed. The model weights are updated based on the action score vector and reward value at different times.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] 1. This invention uses the Transformer model and the feature vector of the target object to better help mobile robots understand the semantics, location, object relationships, and association with the target object in the image.

[0038] 2. This invention uses the Transformer model to extract the historical context of the entire navigation process and uses the target object feature query again, which can better perceive the feature information of the target object during the navigation process and generate a navigation direction vector.

[0039] 3. This invention enables robots to navigate based on current environmental images and target object names. Experiments show that the proposed method has excellent performance. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the logic flow of the present invention.

[0041] Figure 2 This is a schematic diagram illustrating the process of extracting local environment feature vectors, global environment feature vectors, and target object feature vectors in this invention.

[0042] Figure 3 This is a schematic diagram of the process for generating the initial target vector in this invention.

[0043] Figure 4 This is a schematic diagram of the process for generating navigation direction vectors in this invention.

[0044] Figure 5 This is a schematic diagram illustrating the process of generating action probability distribution vectors and updating model weights in this invention. Detailed Implementation

[0045] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0046] like Figures 1 to 5 As shown, this embodiment discloses a target object navigation method based on Transformer, including the following steps:

[0047] 1) Preprocessing of the input data, specifically as follows:

[0048] Input the current environment image into the DETR model to obtain the local environment feature vector L. i,j ∈R 100×256 Where 100×256 represents a feature layer of size 100×256, with a size of 100 in the i-direction and 256 in the j-direction, R 100×256 Represents a set of real numbers of size 100×256; input the current environmental image into the ResNet-18 model to obtain the global environmental feature vector G.i,j ∈R 49×256 Where 49×256 represents a feature layer of size 49×256, with a size of 49 in the i-direction and 256 in the j-direction, R 49×256 Represents a set of real numbers of size 49×256; the target object feature vector T is obtained by embedding the target object name using a word embedding model. i,j ∈R 1×256 Where 1×256 represents a feature layer of size 1×256, with a size of 1 in the i-direction and a size of 256 in the j-direction, R 1×256 Let represent the set of real numbers, which is 1 × 256.

[0049] 2) Input the local environment feature vector, global environment feature vector, and target object feature vector into the target vector Transformer model. The target vector Transformer model includes a visual encoder, a visual decoder, and a target object decoder. It performs the following operations: encode the local environment feature vector through the visual encoder to obtain an enhanced local environment feature vector; add position encoding to the global environment feature vector to obtain an enhanced global environment feature vector; input the enhanced local environment feature vector and the enhanced global environment feature vector into the visual decoder to obtain an environment feature vector; input the environment feature vector and the target object feature vector into the target object decoder to obtain the preliminary target vector at the current time; reduce the dimensionality of the preliminary target vector at the current time through a fully connected layer; increase the dimensionality of the action probability distribution vector at the previous time through a fully connected layer; concatenate the increased dimensionality action probability distribution vector and the reduced dimensionality preliminary target vector at the current time and flatten them to obtain the final target vector at the current time; including the following steps:

[0050] 2.1) Using local environment feature vector L i,j ∈R 100×256 The query vector, key vector, and value vector of the multi-head self-attention mechanism MHA() are input to the visual encoder VisionEncoder(L i,j ,L i,j ,L i,j The enhanced local environment feature vector L' is obtained. i,j ∈R 100×256 The visual encoder is defined as follows:

[0051] L' i,j =VisionEncoder(L i,j ,L i,j ,L i,j )=Add&Norm(Linear(Add&Norm(MHA(L i,j ,L i,j ,L i,j ))))

[0052] In the formula, Add&Norm() is the residual connection and layer normalization, Linear() is the fully connected layer, and MHA() is the multi-head self-attention mechanism;

[0053] 2.2) Using the global environment feature vector G i,j Adding positional encoding yields an enhanced global environment feature vector G' i,j This will enhance the global environmental feature vector G' i,j and enhance the local environmental feature vector L' i,j Input visual decoder VisionDecoder(G' i,j ,L' i,j ,L' i,j Obtain the environmental feature vector E at the current moment. i,j ∈R 49×256 The visual encoder is defined as follows:

[0054] E i,j =VisionDecoder(Pos(G i,j ),L' i,j ,L' i,j =MHA(Add&Norm(MHA(G')) i,j ,G' i,j ,G' i,j )),L' i,j ,L' i,j )

[0055] In the formula, Pos() is the position code;

[0056] The feature vector T of the target object i,j ∈R 1×256 Repeated stacking and adding positional encoding yields the enhanced target object feature vector T' i,j ∈R 49×256 This will enhance the feature vector T' of the target object. i,j ∈R 49×256 and E i,j ∈R 49×256 Input target object decoder TargetDecoder(T' i,j E i,j E i,j The initial target vector Lt at the current time is obtained from ) i,j ∈R 49×256 The target object decoder is defined as follows:

[0057] Lt i,j =TargetDecoder(T' i,j E i,j E i,j=Add&Norm(MHA(T') i,j E i,j E i,j ))

[0058] The action probability distribution vector a from the previous moment t ∈R 1×6 t represents the vector a' obtained by upsizing through a fully connected layer at time t. t ∈R 1×64 , where R 1×64 Represent the set of real numbers of 1×64, and then use the initial target vector Lt at the current time. i,j ∈R 49 ×256 The vector Lt' is obtained by dimensionality reduction through a fully connected layer. i,j ∈R 49×64 R 49×64 Represent the set of real numbers, which is 49 × 64, and then use vector a' t Sum vector Lt' i,j Connect and flatten to obtain the final target vector Lt at the current time step. i,j ∈R 1×3200 R 1×3200 The set of real numbers, representing 1 × 3200, is expressed as:

[0059] Lt” i,j =Flatten(Concate(LinearUp(a t ),LinearDown(Lt i,j )))

[0060] In the formula, Flatten() flattens, Concate() connects, LinearUp() increases the dimensionality of the fully connected layer, and LinearDown() decreases the dimensionality of the fully connected layer.

[0061] The target vector Transformer model uses imitation learning for pre-training, enabling the model to obtain good initial weights, which helps to accelerate the convergence speed during formal training. The model uses multiple teaching trajectories of the robot navigating in the environment as the pre-training dataset P = {(s1,a1),(s2,a2),...,(s...}. n ,a n )}, where (s n ,a n ) represents the parameter of the nth teaching trajectory, s n The robot state for the nth taught trajectory includes local environment feature vectors, global environment feature vectors, and target object feature vectors, a. nLet be the probability distribution vector of the actions for the nth teaching trajectory, i.e., the probability distribution of walking forward, turning left, turning right, looking up, looking down, and completing the action; in the pre-training process of imitation learning, such as for the nth teaching trajectory, the input s n The Transformer model outputs the final target vector at the current time step. This final target vector is then input into a fully connected layer to generate an action probability distribution vector. This vector is then used to calculate the cross-entropy with the action probability distribution vector of the nth taught trajectory. The objective function τ is expressed as:

[0062]

[0063] In the formula, n represents the nth teaching trajectory, P(a n Q(a) represents the probability distribution of the actions on the nth taught trajectory, i.e., the true distribution; n ) represents the action probability distribution of the model pre-training, i.e., the prediction distribution; λ is the hyperparameter of imitation learning; the smaller the cross-entropy between the true distribution and the prediction distribution, the closer their distributions are. The purpose of learning each teaching trajectory by minimizing the objective function τ is to achieve the purpose of initializing the weights of the pre-trained model.

[0064] 3) Input the final target vector at the current time step into the navigation direction vector Transformer model. The navigation direction vector Transformer model includes a visual encoder and a target object decoder, which performs the following operations: stacking the final target vectors at the current time step and the previous time step, adding position encoding, and then inputting them into the visual encoder to obtain the hidden layer vector; inputting the hidden layer vector and the target object feature vector into the target object decoder to obtain the navigation direction vector; including the following steps:

[0065] The final target vector Lt obtained from the Transformer model of the target vectors at the current and previous time steps is calculated. i,j ∈R 1×3200 Stacking them together yields the vector Lt” i，j ∈R 2×3200 R 2×3200 The set of real numbers representing 2×3200, after being encoded with position, is input into the navigation direction vector of the Transformer model's visual encoder, VisionEncoder(Pos(Lt)). i，j ),Pos(Lt” i，j ),Pos(Lt” i，j The hidden layer vector H is obtained. i,j ∈R 2×3200 The hidden layer vector is input into the fully connected layer to reduce the dimension to 256, resulting in vector H'. i,j ∈R 2×256 R 2×256Let H' represent the set of real numbers, which is 2 × 256. i,j ∈R 2×256 The target object feature vector, along with the navigation direction vector, is input into the target object decoder (TargetDecoder(T)) of the Transformer model. i,j ,H' i,j ,H' i,j ) Obtain the navigation direction vector GT t ∈R 1×256 .

[0066] 4) Input the navigation direction vector into the A3C reinforcement learning model, which includes an action generation network, an action scoring network, and a reward / penalty function, and performs the following operations: Use the navigation direction vector as input to the action generation network to obtain the action probability distribution vector at the current time, then select and execute the action with the highest probability in the action probability distribution vector; Use the navigation direction vector as input to the action scoring network to obtain the action score vector at the current time, representing the score of the action with the highest probability in the action probability distribution vector; Generate the reward value at the current time according to the reward / penalty function.

[0067] The action generation network consists of a single fully connected layer, and the input navigation direction vector GT is... t Generate an action probability distribution vector a t ∈R 1×6 The action scoring network consists of two fully connected layers, with the input navigation direction vector GT. t Generate an action rating vector v t ∈R 1×1 R 1×1 Let represent a 1×1 set of real numbers; the reward / penalty function consists of two parts: the first part is a reward of 5.0 if the robot reaches within one meter of the target object and completes the action; the second part is a penalty of -0.01 if the robot does not reach the target object; after each step, if the robot does not reach the vicinity of the target object, a penalty reward of -0.01 is given, the purpose of which is to enable the A3C reinforcement learning model to reduce the number of steps; conversely, if navigation is successful, a large positive reward is given. Maximizing the reward will enable the A3C reinforcement learning model to complete the navigation task with the fewest possible steps.

[0068] 5) Repeat steps 1) to 4) until the robot completes the action. At this point, a target object navigation event is completed. Update the model weights based on the action score vector and reward value at different times, including the following steps:

[0069] Initially, given the target object name and the current location's environmental image, the robot preprocesses the input data. Then, using the target vector Transformer model and the navigation direction vector Transformer model, it outputs the corresponding target vector and navigation direction vector, respectively. Next, the A3C reinforcement learning model outputs an action, which the robot executes to reach the next state. It then obtains the target object name and the current location's environmental image again, repeating the above process until the A3C reinforcement learning model completes the action or reaches its maximum number of executions and automatically completes the action. At this point, a target object navigation event is completed. The model weights are updated based on the action score vector and reward value at different times.

[0070] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A target object navigation method based on Transformer, characterized in that, Includes the following steps: 1) Preprocessing of input data, including: using the DETR model to extract local environmental feature vectors from the current time-input environmental image; using the ResNet-18 model to extract global environmental feature vectors from the current time-input environmental image; and using a word embedding model to embed the target object name to obtain the target object feature vector. Input the current environment image into the DETR model to obtain the local environment feature vector. ,in, Indicates size is Feature layer, Dimensions in direction , Dimensions in direction , express The set of real numbers; input the current environment image into the ResNet-18 model to obtain the global environment feature vector. ,in, Indicates size is Feature layer, Dimensions in direction , Dimensions in direction , express The set of real numbers; using a word embedding model to embed the target object name to obtain the target object feature vector. ,in, Indicates size is Feature layer, Dimensions in direction , Dimensions in direction , express The set of real numbers; 2) Input the local environment feature vector, global environment feature vector, and target object feature vector into the target vector Transformer model to obtain the final target vector at the current time step; The target vector Transformer model includes a visual encoder, a visual decoder, and a target object decoder. It performs the following operations: the visual encoder encodes the local environment feature vector to obtain an enhanced local environment feature vector; the global environment feature vector is added with position encoding to obtain an enhanced global environment feature vector; the enhanced local environment feature vector and the enhanced global environment feature vector are input into the visual decoder to obtain an environment feature vector; the environment feature vector and the target object feature vector are input into the target object decoder to obtain the preliminary target vector at the current moment; the preliminary target vector at the current moment is reduced in dimension by a fully connected layer; the action probability distribution vector at the previous moment is increased in dimension by a fully connected layer; the increased action probability distribution vector and the reduced preliminary target vector at the current moment are concatenated and flattened to obtain the final target vector at the current moment. The target vector Transformer model uses imitation learning for pre-training, enabling the model to obtain good initial weights, which helps to accelerate the convergence speed during formal training. The model uses multiple teaching trajectories of the robot navigating in the environment as the pre-training dataset. ,in Indicates the first The parameters of the taught trajectory, For the first The robot's state for the taught trajectory includes local environment feature vectors, global environment feature vectors, and target object feature vectors. For the first The input vector represents the probability distribution of actions on each taught trajectory, specifically the probability distribution of moving forward, turning left, turning right, looking up, looking down, and completing the action. During the pre-training process of imitation learning, the nth taught trajectory is input... The Transformer model outputs the final target vector at the current time step. This final target vector is then input into a fully connected layer to generate an action probability distribution vector. This vector is then used to calculate the cross-entropy with the action probability distribution vector of the nth taught trajectory. The objective function used is... Represented as: ； In the formula, n represents the nth teaching trajectory. This represents the probability distribution of actions along the nth teaching trajectory, i.e., the true distribution. This represents the action probability distribution during model pre-training, i.e., the prediction distribution; These are hyperparameters in imitation learning; the smaller the cross-entropy between the true and predicted distributions, the closer their distributions are, and this is achieved by minimizing the objective function. This is used to learn each teaching trajectory to achieve the purpose of initializing the weights in the pre-trained model; Using local environment feature vectors As a multi-head self-attention mechanism The query vector, key vector, and value vector are input into the visual encoder. Obtain enhanced local environment feature vectors The visual encoder is defined as follows: ； In the formula, It involves residual connections and layer normalization. It is a fully connected layer. It is a multi-head self-attention mechanism; Using global environment feature vectors Adding positional encoding yields an enhanced global environment feature vector. This will enhance the global environmental feature vector. and enhance local environmental feature vectors Input visual decoder Obtain the environmental feature vector at the current time. The visual encoder is defined as follows: ； In the formula, It is a positional encoding; The feature vector of the target object Repeated stacking and adding positional encoding yields an enhanced target object feature vector. This will enhance the feature vector of the target object. and Input target object decoder The initial target vector at the current time is obtained from this. The target object decoder is defined as follows: ； The action probability distribution vector from the previous moment t represents the vector obtained by upsizing through a fully connected layer at time t. ,in express The set of real numbers, and then the initial target vector at the current moment. The vector is obtained by dimensionality reduction using a fully connected layer. , express The set of real numbers, then the vector sum vector Connect and flatten to obtain the final target vector at the current time step. , express The set of real numbers is represented as: ； In the formula, It is flattened. It's a link. It's a dimensional upgrade using fully connected layers. It is a dimensionality reduction using fully connected layers; 3) Input the final target vector at the current moment into the navigation direction vector Transformer model to obtain the navigation direction vector; The navigation direction vector Transformer model includes a visual encoder and a target object decoder, which performs the following operations: stacking the final target vectors of the current time step and the previous time step, adding position encoding, and then inputting them into the visual encoder to obtain the hidden layer vector; inputting the hidden layer vector and the target object feature vector into the target object decoder to obtain the navigation direction vector. The final target vector obtained from the Transformer model using the target vectors of the current time step and the previous time step. Stacking them together yields a vector. , express The set of real numbers, plus the position encoding, is input into the visual encoder of the Transformer model as the navigation direction vector. Obtain the hidden layer vector The hidden layer vector is input into the fully connected layer to reduce the dimensionality to 256 to obtain the vector. , express The set of real numbers, and then the vector The target object feature vector is input along with the navigation direction vector into the target object decoder of the Transformer model. Obtain the navigation direction vector ; 4) Input the navigation direction vector into the A3C reinforcement learning model to obtain the action probability distribution vector, action score vector and reward value at the current time, and then select the action with the highest probability in the action probability distribution vector to execute; The A3C reinforcement learning model includes an action generation network, an action scoring network, and a reward / penalty function, which performs the following operations: The navigation direction vector is used as input to the action generation network to obtain the action probability distribution vector at the current time, and then the action with the highest probability in the action probability distribution vector is selected and executed; the navigation direction vector is used as input to the action scoring network to obtain the action score vector at the current time, representing the score of the action with the highest probability in the action probability distribution vector; and the reward value at the current time is generated according to the reward / penalty function. 5) Repeat steps 1) to 4) until the robot completes the action. At this point, a target object navigation event is completed. Update the model weights based on the action score vector and reward value at different times.

2. The target object navigation method based on Transformer according to claim 1, characterized in that: The action generation network consists of a single fully connected layer, which takes a navigation direction vector as input. Generate an action probability distribution vector The action scoring network consists of two fully connected layers, with the input navigation direction vector. Generate an action score vector , express The set of real numbers; the reward and punishment function consists of two parts: the first part is a reward value of 5.0 when the robot reaches within one meter of the target object and completes the action; the second part is a punishment reward value of -0.01 when the robot does not reach the target object; after each step the robot takes, if it does not reach the vicinity of the target object, it is given a punishment reward value of -0.01, the purpose of which is to enable the A3C reinforcement learning model to reduce the number of steps; conversely, if the navigation is successful, a large positive reward is given. Maximizing the reward will enable the A3C reinforcement learning model to complete the navigation task with the fewest possible steps.

3. The target object navigation method based on Transformer according to claim 2, characterized in that: Step 5) includes the following steps: Initially, given the target object name and the current location's environmental image, the robot preprocesses the input data. Then, using the target vector Transformer model and the navigation direction vector Transformer model, it outputs the corresponding target vector and navigation direction vector, respectively. Next, the A3C reinforcement learning model outputs an action, which the robot executes to reach the next state. It then obtains the target object name and the current location's environmental image again, repeating the above process until the A3C reinforcement learning model completes the action or reaches its maximum number of executions and automatically completes the action. At this point, a target object navigation event is completed. The model weights are updated based on the action score vector and reward value at different times.

Images