Audio-visual navigation method for carrying out audio-visual mode fusion based on converter model

Abstract

The invention discloses an audiovisual navigation method for carrying out audiovisual mode fusion based on a converter model. The method comprises the following steps: collecting visual information and auditory information; the expected discount return and reward are maximized; two encoders are respectively used for carrying out feature extraction on audio-visual sense organs; performing feature mapping by using a spatial audio encoder; fusing the information of the visual mode and the auditory mode; weighting a strategy for successfully measuring the task that the sound source is a mobile sound source by using the dynamic path length; and training the network by using a near-end strategy optimization algorithm. According to the method, visual and auditory input can be better coordinated, so that audio and visual modes can be better fused, and the accuracy and precision of navigation are improved. The converter model is used for replacing a traditional gating circulation unit, and the high-difficulty navigation requirement in a noisy environment and a dynamic sound source task can be met. The near-end strategy optimization algorithm is used for training, manpower is saved, and economic benefits are improved.

Description

Technical Field

[0001] The present invention relates to an embodied audio-visual navigation method, in particular to an embodied audio-visual navigation method for audio-visual modality fusion based on a converter model. Background Art

[0002] With the development of science and technology, intelligent agents, such as various robots, can help humans perform more and more tasks, but most intelligent agents still passively perform preset tasks and cannot consciously find tasks to perform. Therefore, whether it can capture signals such as sound and reach the target point autonomously is an important criterion for judging the ability of robot intelligent agents. At this time, someone thought of using audio to "tell" the intelligent agent that there is a task to be processed, and at the same time, visual assistance can be used to move the intelligent agent to the target point, that is, auditory indication of the target point and visual assistance for path finding. Traditional non-reinforcement learning speech recognition algorithms can only achieve accurate recognition in a relatively quiet environment, and are almost unusable in a noisy environment. The existing latest audio-visual navigation model uses reinforcement learning methods and uses gated recurrent units for modal fusion. However, the structure of the gated recurrent unit is too simple and the performance is relatively insufficient. There is a lot of room for improvement in processing complex tasks such as embodied audio-visual navigation; and the gated recurrent unit is particularly sensitive to the setting of hyperparameters, and training takes a lot of time and energy, and is difficult to tune.

[0003] The transformer model is a neural network model based on the self-attention mechanism. The core feature of this model is that its entire main structure is a self-attention mechanism, which enables it to process all position information in the input sequence in parallel, thereby greatly improving the processing speed and efficiency. The transformer model is mainly used for natural language processing tasks and can also be used in the visual field, but no one has applied it to the field of embodied audio-visual navigation. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention proposes an embodied audio-visual navigation method based on a transformer model for audio-visual modality fusion, which has more powerful performance and is not overly dependent on hyperparameter settings.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: an embodied audio-visual navigation method for audio-visual modality fusion based on a converter model comprises the following steps:

[0006] A. Collecting visual and auditory information

[0007] A microphone array is used to collect binaural sounds to obtain a binaural spectrogram; a depth camera is used to collect RGB and depth images.

[0008] B. Maximizing Expected Discounted Returns and Rewards

[0009] At each time step t, the agent receives a RGB image v t and the depth image d t The current observation value o t , and binaural spectrogram b t Given the current observation and the agent’s previous state s t-1 , the agent follows the strategy π(a t |o t ,s t-1 ) generates the next action a t , where the actions include moving forward, rotating left, rotating right, or stopping. The goal of the agent is to maximize the expected discounted return, using It is expressed as follows:

[0010]

[0011] Where T is the total number of time steps, γ is the discount factor, r(s t-1 ,a t ) is a reward.

[0012] If the agent successfully stops at the target point, it will receive a large reward R1; if the shortest distance from the agent to the sound source decreases, it will receive a small reward R2; if the shortest distance from the agent to the sound source increases, it will receive a small penalty R3; and each time step of the agent increases, it will receive a tiny penalty R4.

[0013] C. Use two encoders to extract features from the audiovisual senses

[0014] C1. Use a deep encoder to extract features from the depth map. The deep encoder consists of three convolutional layers and a fully connected layer with 512 neurons. The first convolutional layer uses an 8×8 convolution kernel with a step size of 4; the second convolutional layer uses a 4×4 convolution kernel with a step size of 2; and the third convolutional layer uses a 3×3 convolution kernel with a step size of 2.

[0015] C2. Use an audio encoder to extract features from binaural sound spectrograms. The audio encoder consists of three convolutional layers and a fully connected layer with 512 neurons. The first convolutional layer uses an 8×8 convolution kernel with a step size of 4; the second convolutional layer uses a 4×4 convolution kernel with a step size of 2; and the third convolutional layer uses a 3×3 convolution kernel with a step size of 1.

[0016] D. Feature Mapping Using Spatial Audio Encoder

[0017] The binaural spectrograms are mapped into the spatial feature space using a spatial audio encoder, which consists of two deconvolution layers. The first layer uses an 8×8 convolution kernel with a stride of 3; the second layer uses a 4×4 convolution kernel with a stride of 1.

[0018] E. Fusion of information from both visual and auditory modalities

[0019] The converter model is used to fuse the visual and auditory information output by the deep encoder and audio encoder.

[0020] E1. Features are extracted by the deep encoder and audio encoder respectively to create visual and audio indicators, and the length of the input is normalized to allow batch processing by padding and using attention masks.

[0021] E2. Concatenate the visual and audio indicators into an input indicator and pass the input indicator into the converter model. At the same time, remove the position encoding of the converter model.

[0022] F. Strategy for successfully weighing tasks where the sound source is moving using dynamic path length weighting

[0023] Assuming a simple goal-directed behavior for the target, the source starts with a random pose on the map and draws a target point from the traversable grid other than the agent's current position. It is ensured that there is a traversable path from the start point to the target point. The source then moves to the next node with probability P along the shortest path to the target.

[0024] Dynamic path length weighted success is used to measure the closeness of the agent to the optimal strategy. Let i be the current event number, N be the total number of events, S i Indicates whether the event is successful, g i is the shortest geodesic distance between the agent’s starting position and the closest position where the agent can capture the sound source, p i is the length of the path taken by the agent, then the dynamic path length weighted success DSPL is expressed as follows:

[0025]

[0026] G. Use the proximal policy optimization algorithm to train the network

[0027] G1. Initialize the actor network, actor 2 network and critic network respectively. The actor network is used to select actions, with the input being the state and the output being the action probability in the discrete action space; the actor 2 network is used to copy the parameters of the actor network in the first step of each iteration. The actor network has the same structure as the actor 2 network. The critic network is used to predict the quality of the state, with the input being the state and the output being the value of the state.

[0028] G2, use the contrastive language-image pre-training algorithm to update the network, contrastive language-image pre-training loss L CLIPThe process of gradient descent (θ) is expressed as follows:

[0029]

[0030] in, represents the empirical average of a finite batch of samples, θ is the strategy parameter vector updated in each round, is the estimator of the advantage function at time step t, the advantage function is the advantage function in reinforcement learning, ò is the clipping threshold, which is a hyperparameter. t (θ) represents the probability ratio, which is defined as follows:

[0031]

[0032] Where π is the current policy, θ old is the strategy parameter vector before update, a t is the next action generated by the agent from the strategy π, s t is the current state of the agent.

[0033] Furthermore, in step B, the large reward R1 takes a value between +8 and +12; the small reward R2 takes a value between +0.2 and +0.3; the small penalty R3 takes a value between -0.2 and -0.3; and the micro penalty R4 takes a value between -0.01 and -0.02.

[0034] Furthermore, the probability P in step F takes a value between 25% and 35%.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The present invention constructs a reinforcement learning process. First, three encoders, namely, a deep encoder, an auditory encoder and a spatial audio encoder, are used to perform feature extraction and encoding processing on the input audiovisual information, and then a converter model is used for multi-modal fusion. The above measures better coordinate visual and auditory inputs, so that the audiovisual modalities can be better integrated to improve the accuracy and precision of navigation.

[0037] 2. The present invention uses a converter model to replace the traditional gated recurrent unit, which improves the performance of the model and also has high robustness to meet the high-difficulty navigation requirements in noisy environments and dynamic sound source tasks. The present invention uses a proximal strategy optimization algorithm for training, which has good stability and reliability, is relatively simple to implement, and does not require manual data annotation, saving manpower and improving economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention has attached Figure 8 Zhang, among which:

[0039] Figure 1 It is the overall architecture diagram of the present invention.

[0040] Figure 2 It is a diagram of the deep encoder network structure.

[0041] Figure 3 This is a diagram of the audio encoder network structure.

[0042] Figure 4 This is a diagram of the spatial audio encoder network structure.

[0043] Figure 5 It is the network structure diagram of the audio-visual modality fusion module of the converter.

[0044] Figure 6 It is the structure diagram of the actor and actor-two network in the proximal strategy optimization algorithm.

[0045] Figure 7 This is the structure diagram of the judge network in the proximal strategy optimization algorithm.

[0046] Figure 8 It is a flowchart of updating the network by contrasting the language-image pre-training algorithm. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings.

[0048] like Figure 1 As shown, an embodied audio-visual navigation method for audio-visual modality fusion based on a transformer model comprises the following steps:

[0049] The agent first uses a depth camera to shoot the environment according to step A of the invention, selects continuous frames of images, and obtains RGB and depth images; at the same time, a microphone array is used to obtain audio and convert the obtained audio into a binaural spectrum. According to step B of the invention, the agent derives the next strategy based on the input modality. Then, according to step C of the invention, an encoder is used to extract a feature from the input audiovisual modality information. The encoder network structure is as follows: Figure 2 , 3 At the same time, according to step D of the invention, a spatial audio encoder is used to perform spatial mapping on the auditory senses. The network structure is as follows Figure 4 Then, according to step E of the invention, the audio-visual modalities are fused using the converter model, and the network structure is as follows: Figure 5 , and use the average weighted success to measure the closeness of the agent to the optimal strategy according to step F of the invention. Finally, use the proximal strategy optimization algorithm to train the network according to step G of the invention, such as Figure 6-7 As shown; use the contrastive language-image pre-training algorithm to perform gradient descent to update the network, such as Figure 8 shown.

[0050] The present invention is not limited to this embodiment, and any equivalent concepts or changes within the technical scope disclosed by the present invention are included in the protection scope of the present invention.

Claims

1. An embodied audio-visual navigation method for audio-visual modality fusion based on a transformer model, characterized in that: The following steps are involved: A. Collecting visual and auditory information Using a microphone array to collect binaural sounds to obtain a binaural spectrogram; Use a depth camera to collect RGB and depth images; B. Maximizing Expected Discounted Returns and Rewards At each time step t, the agent receives a RGB image v t and the depth image d t The current observation value o t , and binaural spectrogram b t ; Given the current observation and the agent’s previous state s t-1 , the agent follows the strategy π(a t |o t ,s t-1 ) generates the next action a t , the actions include moving forward, rotating left, rotating right, or stopping; the agent's goal is to maximize the expected discounted return, using It is expressed as follows: Where T is the total number of time steps, γ is the discount factor, r(s t-1 ,a t ) is a reward; If the agent successfully stops at the target point, it will receive a large reward R1; if the shortest distance from the agent to the sound source decreases, it will receive a small reward R2; if the shortest distance from the agent to the sound source increases, it will receive a small penalty R3; each time step of the agent increases, it will receive a small penalty R4; C. Use two encoders to extract features from the audiovisual senses C1. Use a deep encoder to extract features from the depth map. The deep encoder consists of three convolutional layers and a fully connected layer with 512 neurons. The first convolutional layer uses an 8×8 convolution kernel with a step size of 4; the second convolutional layer uses a 4×4 convolution kernel with a step size of 2; the third convolutional layer uses a 3×3 convolution kernel with a step size of 2. C2. Use an audio encoder to extract features from the binaural sound spectrogram. The audio encoder consists of three convolutional layers and a fully connected layer with 512 neurons. The first convolutional layer uses an 8×8 convolution kernel with a step size of 4; the second convolutional layer uses a 4×4 convolution kernel with a step size of 2; and the third convolutional layer uses a 3×3 convolution kernel with a step size of 1. D. Feature Mapping Using Spatial Audio Encoder A spatial audio encoder is used to map the binaural spectrogram into a spatial feature space. The spatial audio encoder consists of two deconvolution layers. The first layer uses an 8×8 convolution kernel with a stride of 3. The second layer uses a 4×4 convolution kernel with a stride of 1; E. Fusion of information from both visual and auditory modalities Use the converter model to fuse the visual and auditory information output by the deep encoder and audio encoder. E1. The deep encoder and audio encoder extract features to create visual and audio indicators respectively, and normalize the length of the input to allow batch processing by padding and using attention masks; E2, concatenate the visual and audio indicators into an input indicator, and pass the input indicator into the converter model; at the same time, remove the position code of the converter model; F. Strategy for successfully weighing tasks where the sound source is moving using dynamic path length weighting Assuming a simple goal-directed behavior, the source starts with a random pose on the map and draws a goal point from the traversable grid other than the agent's current position; Ensure that there is a traversable path from the starting point to the target point; then, the sound source moves to the next node with probability P along the shortest path to the target; Dynamic path length weighted success is used to measure the closeness of the agent to the optimal strategy. Let i be the current event number, N be the total number of events, S i Indicates whether the event is successful, g i is the shortest geodesic distance between the agent’s starting position and the closest position where the agent can capture the sound source, p i is the length of the path taken by the agent, then the dynamic path length weighted success DSPL is expressed as follows: G. Use the proximal policy optimization algorithm to train the network G1, respectively initialize the actor network, the actor second network and the critic network; the actor network is used to select actions, the input is the state, and the output is the action probability in the discrete action space; the actor second network is used to copy the parameters of the actor network in the first step of each iteration; the actor network has the same structure as the actor second network; the critic network is used to predict the quality of the state, the input is the state, and the output is the value of the state; G2, use the contrastive language-image pre-training algorithm to update the network, contrastive language-image pre-training loss L CLIP The process of gradient descent (θ) is expressed as follows: in, represents the empirical average of a finite batch of samples, θ is the strategy parameter vector updated in each round, is the estimator of the advantage function at time step t, the advantage function is the advantage function in reinforcement learning, ò is the clipping threshold, which is a hyperparameter; r t (θ) represents the probability ratio, which is defined as follows: Where π is the current policy, θ old is the strategy parameter vector before update, a t is the next action generated by the agent from the strategy π, s t is the current state of the agent.

2. The embodied audio-visual navigation method for audio-visual modality fusion based on a transformer model according to claim 1, characterized in that: In step B, the large reward R1 takes a value between +8 and +12; the small reward R2 takes a value between +0.2 and +0.3; the small penalty R3 takes a value between -0.2 and -0.3; and the micro penalty R4 takes a value between -0.01 and -0.

02.

3. The embodied audio-visual navigation method for audio-visual modality fusion based on a transformer model according to claim 1, characterized in that: The probability P in step F takes a value between 25% and 35%.

Images