Multimodal characterization method and system based on reinforcement learning

Abstract

The invention provides a multi-modal representation method and system based on reinforcement learning, and the method comprises the steps: carrying out the modal feature coding of each modal data in multi-modal original data, and obtaining a multi-modal primary representation matrix; inputting the multi-modal primary representation matrix into a strategy network, and generating a reward signal of reinforcement learning according to an output result; performing adaptive adjustment on weight distribution of each modal feature in the strategy network through the reward signal to obtain a dynamic fusion weight matrix; performing adaptive fusion operation on the multi-modal primary representation matrix according to the dynamic fusion weight matrix, and determining a task performance index based on the actual output of the obtained multi-modal joint representation vector on the target task; and when the task performance index is lower than a preset threshold value, re-triggering the strategy updating process of the strategy network. By adopting the scheme of the invention, adaptive fusion and dynamic representation of the multi-modal data can be realized in a multi-source sensing environment so as to improve the semantic relevance of multi-modal task output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal representation, more particularly, the present application relates to a multi-modal representation method and system based on reinforcement learning. BACKGROUND

[0002] Multi-modal representation learning is one of the core technologies in the field of artificial intelligence, which integrates data information from different sources (such as vision, speech, text, etc.) to construct a unified feature representation, and is widely used in complex scenarios such as intelligent unmanned aerial vehicles, autonomous navigation, human-computer interaction, etc.

[0003] Multi-modal representation based on reinforcement learning refers to using the framework and ideas of reinforcement learning to learn or optimize joint representation or aligned representation from different modal data; traditional multi-modal fusion methods mainly use fixed weight weighted average, static fusion based on attention mechanism or early / late fusion strategies, etc. These methods, although to some extent, realize the integration of multi-modal information, but their fusion strategies are usually based on preset rules or static models, and cannot dynamically adjust the contribution of each modality according to task requirements and environmental changes, however, in practical applications, the importance of different modal information will change dynamically with the task context, environmental state and data quality, and the static fusion strategy lacks the ability to adapt to the task relevance, and cannot prioritize the most discriminative modal features in complex environments; due to the fixed weight fusion method, it is difficult to handle the asymmetric semantic association between modalities, resulting in low information fusion efficiency; when facing modal missing, noise interference or distribution deviation, the traditional method lacks a dynamic adjustment mechanism, causing a significant decline in system performance; therefore, how to realize the adaptive fusion and dynamic representation of multi-modal data in a multi-source perception environment to improve the semantic relevance of multi-modal task output has become a difficult problem in the industry. SUMMARY

[0004] The present application provides a multi-modal representation method and system based on reinforcement learning, which can realize adaptive fusion and dynamic representation of multi-modal data in a multi-source perception environment to improve the semantic relevance of multi-modal task output.

[0005] In a first aspect, the present application provides a multi-modal representation method based on reinforcement learning, comprising the following steps: Obtaining multi-modal raw data of a target object in a multi-source perception environment; Synchronously calibrating the time and resolution of the multi-modal raw data, and then modality feature encoding each modality data in the multi-modal raw data obtained by synchronous calibration to obtain a multi-modal primary representation matrix; inputting the multi-modal primary representation matrix into the trained policy network, wherein a current multi-modal fusion state is taken as a state input, an operation of selecting and weightedly fusing features of each modality is taken as an action output, and then a reward signal of reinforcement learning is generated according to performance feedback of the output result on a target task; In the weighted fusion process, the weight distribution of the features of each modality in the policy network is adaptively adjusted through the generated reward signal, and a dynamic fusion weight matrix reflecting multi-modal semantic association and task correlation is obtained. According to the dynamic fusion weight matrix, an adaptive fusion operation driven by reinforcement learning is performed on the multi-modal primary representation matrix, a multi-modal joint representation vector is obtained, and a task performance index is determined based on the actual output of the multi-modal joint representation vector on the target task. When the task performance index is lower than a preset threshold, the policy updating process of the policy network is retriggered.

[0006] In some embodiments, the modal feature encoding is respectively performed on each modality data in the multi-modal raw data obtained through synchronous calibration, and the multi-modal primary representation matrix is obtained, which specifically includes: The spatial features of the visual image data, the spectral features of the speech signal data, and the semantic features of the text semantic data are extracted from the multi-modal raw data obtained through synchronous calibration. The extracted spatial features, spectral features, and semantic features are respectively encoded to obtain a visual feature vector, a speech feature vector, and a text feature vector. The multi-modal primary representation matrix is generated from the visual feature vector, the speech feature vector, and the text feature vector.

[0007] In some embodiments, the reward signal of reinforcement learning is generated according to the performance feedback of the output result on the target task, which specifically includes: The multi-modal primary representation matrix is weightedly fused based on the output result to obtain a multi-modal fusion feature vector. The multi-modal fusion feature vector is input into a pre-trained task prediction model, and a current task performance index is output. The performance change amount of the current task performance index relative to a historical performance benchmark is determined. The reward signal of reinforcement learning is generated based on the performance change amount.

[0008] In some embodiments, in the weighted fusion process, the weight distribution of the features of each modality in the policy network is adaptively adjusted through the generated reward signal, and a dynamic fusion weight matrix reflecting multi-modal semantic association and task correlation is obtained, which specifically includes: The advantage function value of the policy network is determined through the generated reward signal. The weight distribution of each modality feature in the policy network is updated based on the dominance function value; New action vectors are generated using the updated policy network; The new action vector is converted into a diagonal matrix form to obtain a dynamic fusion weight matrix that reflects multimodal semantic association and task relevance.

[0009] In some embodiments, performing a reinforcement learning-driven adaptive fusion operation on the multimodal primary representation matrix based on the dynamic fusion weight matrix to obtain a multimodal joint representation vector specifically includes: The dynamic fusion weight matrix is ​​adaptively fused with the multimodal primary representation matrix to obtain the feature matrix; Feature aggregation is performed by summing the columns of the feature matrix; The aggregated feature vectors are dimension-normalized to obtain a multimodal joint representation vector.

[0010] In some embodiments, determining the task performance index based on the actual output of the multimodal joint representation vector on the target task specifically includes: The multimodal joint representation vector is input into the task prediction model to obtain the actual output of the task prediction model on the target task. The basic performance indicators are determined by the actual output. The basic performance indicators are averaged using a sliding window method to obtain stable task performance indicators.

[0011] In some embodiments, the multimodal raw data includes visual image data, speech signal data, and text semantic data.

[0012] Secondly, this application provides a multimodal representation system based on reinforcement learning, comprising: The acquisition module is used to acquire the multimodal raw data of the target object in a multi-source sensing environment; The processing module is used to perform time and resolution synchronization calibration on the multimodal raw data, and then perform modal feature encoding on each modal data in the synchronously calibrated multimodal raw data to obtain a multimodal primary characterization matrix. The processing module is further configured to input the multimodal primary representation matrix into the trained policy network, wherein the current multimodal fusion state is used as the state input, and the operation of selecting and weighting the features of each modality is used as the action output, and then a reward signal for reinforcement learning is generated based on the performance feedback of the output result on the target task. The processing module is also used to adaptively adjust the weight distribution of each modal feature in the policy network through the generated reward signal during the weighted fusion process, so as to obtain a dynamic fusion weight matrix that reflects the multimodal semantic association and task relevance. The processing module is further configured to perform an adaptive fusion operation driven by reinforcement learning on the multimodal primary representation matrix according to the dynamic fusion weight matrix to obtain a multimodal joint representation vector, and determine the task performance index based on the actual output of the multimodal joint representation vector on the target task. The execution module is used to re-trigger the policy update process of the policy network when the performance index of the task is lower than a preset threshold.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device executes the above-described multimodal representation method based on reinforcement learning.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned multimodal representation method based on reinforcement learning.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: In this application, multimodal raw data of the target object in a multi-source perception environment is acquired; the multimodal raw data is synchronously calibrated in terms of time and resolution, and then modal feature encoding is performed on each modal data in the synchronously calibrated multimodal raw data to obtain a multimodal primary representation matrix; the multimodal primary representation matrix is ​​input into a trained policy network, wherein the current multimodal fusion state is used as the state input, and the operation of selecting and weighting the fusion of each modal feature is used as the action output, and a reward signal for reinforcement learning is generated based on the performance feedback of the output result on the target task; during the weighted fusion process, the weight distribution of each modal feature in the policy network is adaptively adjusted by the generated reward signal to obtain a dynamic fusion weight matrix that reflects the multimodal semantic association and task relevance; the multimodal primary representation matrix is ​​subjected to an adaptive fusion operation driven by reinforcement learning based on the dynamic fusion weight matrix to obtain a multimodal joint representation vector, and the task performance index is determined based on the actual output of the multimodal joint representation vector on the target task; when the task performance index is lower than a preset threshold, the policy update process of the policy network is retried.

[0016] Therefore, in this application, firstly, modal feature encoding is performed on each modal data in the multimodal raw data obtained from synchronous calibration to obtain a multimodal primary representation matrix. This transforms the original modal information into a primary representation matrix with structural consistency and semantic expressiveness, enabling each modality to form a unified feature representation space before entering the fusion stage. Secondly, during the weighted fusion process, the weight distribution of each modality feature in the policy network is adaptively adjusted using the generated reward signal to obtain a dynamic fusion weight matrix that reflects the semantic association and task relevance of the multimodal features. This allows each modality to obtain different weights in different contexts, enabling explicit learning of the semantic association and task discriminability between modalities, achieving self-evolution of the fusion strategy, and improving the sensitivity and responsiveness of the multimodal representation to task differences. Then, reinforcement learning-driven adaptive fusion is performed on the multimodal primary representation matrix based on the dynamic fusion weight matrix. The fusion operation yields a multimodal joint representation vector, enabling optimal coupling and information compensation of cross-modal information at the semantic level. This results in a final representation with task interpretability and semantic enhancement capabilities, improving feature discrimination quality and semantic integrity in complex task environments. Finally, based on the actual output of the multimodal joint representation vector on the target task, task performance indicators are determined. A closed-loop self-supervised optimization mechanism can be constructed, giving the fusion strategy continuous self-correction capabilities. Furthermore, using the task performance indicators as a performance evaluation criterion allows for real-time monitoring of changes in representation capabilities, ensuring continuous matching between feature representation and environmental states. This enables multimodal representations to possess self-recovery, self-optimization, and long-term performance stability, maintaining high semantic discriminative power and robust fusion performance in complex and changing scenarios. In summary, this scheme can achieve adaptive fusion and dynamic representation of multimodal data in multi-source perception environments to improve the semantic relevance of multimodal task outputs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an exemplary flowchart of a multimodal representation method based on reinforcement learning, according to some embodiments of this application; Figure 2 This is an exemplary flowchart illustrating the determination of a multimodal primary characterization matrix according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the determination of a dynamic fusion weight matrix according to some embodiments of this application; Figure 4This is a schematic diagram of the structure of a multimodal representation system based on reinforcement learning, as shown in some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device implementing a multimodal representation method based on reinforcement learning, according to some embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] refer to Figure 1 The figure is an exemplary flowchart of a reinforcement learning-based multimodal representation method according to some embodiments of this application. The reinforcement learning-based multimodal representation method mainly includes the following steps: In step 101, the multimodal raw data of the target object in a multi-source sensing environment is obtained.

[0021] In specific implementation, the acquisition of multimodal raw data of the target object in a multi-source sensing environment can be achieved in the following way: multi-sensor data of the target object can be collected synchronously through multiple sensors mounted in the multi-source sensing environment of the target object, including visual image data, voice signal data, and text semantic data. Specifically, a video stream of the area where the target object is located can be acquired by a visible light camera as visual image data, an ambient sound in the area where the target object is located can be acquired by a directional microphone array as voice signal data, and the received control command text can be parsed by an onboard computing unit as text semantic data, thereby obtaining the multimodal raw data of the target object in a multi-source sensing environment. In addition, as a preferred embodiment, the synchronous acquisition of multiple sensors can use a unified clock source based on Global Positioning System (GPS) timing to timestamp each modal data. In other embodiments, strict synchronization of the acquisition of each sensor can also be achieved by hardware triggering signals, which is not specifically limited here.

[0022] It should be noted that the target object in this application refers to the core entity being observed, identified, and analyzed; the multi-source sensing environment in this application refers to an operational scenario in which multiple heterogeneous sensors are deployed, which is used to provide collaborative data acquisition conditions for sensing the target object from different physical dimensions; the multimodal raw data in this application refers to the set of physical signals collected by multiple sensors in the multi-source sensing environment, which is used to record data information of the target object in multiple modalities such as vision, hearing, and semantics. The visual image data refers to the two-dimensional light intensity information of the appearance, shape, texture, and spatial relationship information of the target object and its environment; the speech signal data refers to the continuous temporal signal containing the speech content, acoustic features, and event information of the sound waves generated by vibration in the target object and its environment; and the text semantic data refers to the discrete character sequence that accurately expresses instructions, descriptions, and conceptual information related to the target object or task in a symbolic and structured form.

[0023] In step 102, the multimodal raw data is subjected to time and resolution synchronization calibration, and then each modal data in the synchronously calibrated multimodal raw data is encoded with modal features to obtain a multimodal primary characterization matrix.

[0024] In specific implementation, the time and resolution synchronization calibration of the multimodal raw data can be achieved in the following way: the modal data in the multimodal raw data are aligned. For example, the frame rate of visual image data can be uniformly resampled to 30Hz, and its spatial resolution can be normalized to 640×480 pixels through bilinear interpolation; the speech signal data can be uniformly resampled to 16kHz and normalized to a mono 16-bit deep format; the text semantic data can be associated and marked with the audio and video data in the corresponding time window. In a preferred embodiment, the resampling process can use a linear interpolation algorithm to ensure temporal continuity. In other embodiments, spline interpolation or phase compensation methods can also be used for signal reconstruction, which is not limited here.

[0025] It should be noted that the time and resolution synchronization calibration in this application refers to the process of eliminating temporal misalignment and feature dimension inconsistency caused by differences in acquisition equipment and sampling mechanisms among multimodal raw data, and establishing a standardized data foundation that allows features of each modality to be correlated and fused under a unified spatiotemporal reference.

[0026] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart of determining the multimodal primary characterization matrix in some embodiments of this application. In this embodiment, modal feature encoding is performed on each modal data in the multimodal raw data obtained by synchronous calibration to obtain the multimodal primary characterization matrix, which can be achieved by the following steps: First, in step 1021, spatial features of visual image data, spectral features of speech signal data, and semantic features of text semantic data are extracted from the multimodal raw data obtained by synchronous calibration. Secondly, in step 1022, the extracted spatial features, spectral features and semantic features are encoded respectively to obtain visual feature vectors, speech feature vectors and text feature vectors; Finally, in step 1023, a multimodal primary representation matrix is ​​generated from the visual feature vector, the speech feature vector, and the text feature vector.

[0027] In specific implementation, the extraction of spatial features of visual image data, spectral features of speech signal data, and semantic features of text semantic data from the multimodal raw data obtained through synchronous calibration can be achieved in the following ways: Each modal feature in the multimodal raw data obtained through synchronous calibration can be extracted separately through parallel processing channels. For example, for visual image data, target detection algorithms, such as the backbone network of the YOLOv5 model (YOLOv5 You Only Look Once version 5), can be used to extract multi-scale feature maps containing target spatial location and contour features as spatial features of the visual image data; for speech signal data, the Librosa toolkit can be used to extract Mel spectrograms, and phase spectrum features can be obtained through short-time Fourier transform as spectral features of the speech signal data; for text semantic data, pre-trained language models, such as the attention mechanism of the BERT model (Bidirectional Encoder Representations from Transformers), can be used to capture context-related semantic units as semantic features of the text semantic data. Other methods can also be used in other embodiments, which are not limited here.

[0028] In specific implementation, the extracted spatial features, spectral features, and semantic features are encoded separately to obtain visual feature vectors, speech feature vectors, and text feature vectors. This can be achieved in the following way: the extracted spatial features, spectral features, and semantic features can be input into a dedicated encoding network. For example, the spatial features corresponding to vision can be compressed into a 512-dimensional vector through a global max pooling layer, and then reduced to 256 dimensions through a two-layer fully connected network as the visual feature vector; the spectral features corresponding to speech can be input into a lightweight CNN-LSTM hybrid network, and after temporal pooling, 256 dimensions are generated through attention weighting as the speech feature vector; the semantic features corresponding to text are taken from the 768-dimensional output corresponding to the BERT model label, and projected to 256 dimensions through a learnable weight matrix as the text feature vector; in addition, all generated feature vectors retain timestamp identifiers to ensure temporal alignment with the input requirements of the subsequent reinforcement learning strategy network; preferably, each encoding network can use a LayerNorm layer to stabilize the training process, and in other embodiments, adversarial training can also be used to enhance feature robustness.

[0029] In specific implementation, the multimodal primary representation matrix generated from the visual feature vector, the speech feature vector, and the text feature vector can be achieved in the following way: the three 256-dimensional feature vectors—the visual feature vector, the speech feature vector, and the text feature vector—can be concatenated into a 768-dimensional joint vector according to the modal dimension. Subsequently, it is mapped to a standardized multimodal primary representation matrix through a learnable linear transformation layer. As a preferred embodiment, the matrix dimension can be normalized to a 3×256 two-dimensional tensor, where rows correspond to modal categories and columns correspond to feature dimensions. In other embodiments, modal weighted concatenation or tensor decomposition and compression techniques can also be used, which are not limited here.

[0030] It should be noted that, in this application, the spatial features of visual image data refer to the distributed representation used to describe the geometric layout, contour shape, relative position, and scale relationship of target objects in a visual scene; the spectral features of speech signal data refer to time-varying physical parameters used to characterize the energy distribution characteristics, formant structure, and time-frequency variation law of speech signals in the frequency dimension; the semantic features of text semantic data refer to the abstract symbolic representation used to carry the conceptual connotations, contextual relationships, and intent information contained in the vocabulary, phrases, and syntactic structures of the text content; and the visual feature vector in this application refers to a fixed-dimensional numerical vector formed after compressing and abstracting the original spatial features, which is used to efficiently represent visual features. The core discriminative information of the sensory content; the speech feature vector in this application refers to a fixed-dimensional numerical vector formed by temporal modeling and selective enhancement of the original spectral features, which is used to centrally express the acoustic pattern features most relevant to the task; the text feature vector in this application refers to a fixed-dimensional numerical vector formed by deep encoding and dimensional alignment of the original semantic features, which is used to standardize the semantic connotation of the text and its distribution relationship in the feature space; the multimodal primary representation matrix in this application refers to a two-dimensional data container used to structure and store the feature vectors of each modality and establish the initial correspondence between different modal features in a unified vector space, which is used to provide a benchmark input for cross-modal fusion.

[0031] In step 103, the multimodal primary representation matrix is ​​input into the trained policy network, wherein the current multimodal fusion state is used as the state input, and the operation of selecting and weighting the features of each modality is used as the action output. Then, a reward signal for reinforcement learning is generated based on the performance feedback of the output result on the target task.

[0032] In specific implementation, the multimodal primary representation matrix is ​​input into the trained policy network. The current multimodal fusion state serves as the state input, and the operation of selecting and weighting the features of each modality as the action output can be implemented as follows: This process can be achieved by constructing an Actor-Critic network architecture based on proximal policy optimization. The state input consists of concatenating the current multimodal fusion state (i.e., the current multimodal primary representation matrix) with historical fusion feature vectors from several past time steps. This concatenated vector is encoded by a fully connected layer and then input into the Actor network and the Critic network, respectively. The Actor network is connected to the Softmatrix network via two fully connected layers. The x activation function outputs an action vector that conforms to a probability distribution, i.e., the output result. Each dimension of this action vector corresponds to a normalized fusion weight of a modality (e.g., visual, speech, text), which is used to select and weight modal features. The Critic network evaluates the long-term benefit of the current state through a state value function for the calculation of the advantage function. In a preferred embodiment, the historical fusion feature vector can be taken from the fusion results of the last 5 time steps. The historical fusion feature vector is temporally encoded into a fixed-dimensional representation through a gated recurrent unit. In other embodiments, an attention mechanism can be used to dynamically weight the historical state, or a deep deterministic policy gradient algorithm can be used to process the continuous action space. There are no limitations here.

[0033] It should be noted that the policy network in this application refers to an intelligent agent that makes autonomous decisions by learning the state-action mapping relationship. It is used to dynamically generate the optimal action policy based on the environmental state and to self-optimize the decision-making process by continuously interacting with the environment and obtaining reward signals, so as to achieve the goal of maximizing long-term cumulative benefits. In specific implementation, the pre-training of the policy network can be achieved by imitation learning. For example, firstly, expert demonstration data is collected, which consists of a multimodal primary representation matrix and manually labeled optimal fusion weights. Then, the behavior cloning method is used, with mean squared error as the loss function, and the Actor network is initially trained through supervised learning to approximate the expert policy. At the same time, the Critic network predicts the state value through temporal difference learning. As a preferred embodiment, the Adam optimizer can be used to minimize the loss function iteratively. In other embodiments, generative adversarial imitation learning or inverse reinforcement learning can also be used to obtain the initial policy parameters.

[0034] It should be noted that after inputting the multimodal primary representation matrix into the trained policy network, the output is an action vector representing the selection of each modality feature and the weighted fusion weight. Each dimension of the action vector corresponds to the normalized fusion weight of a modality, which is used to indicate the relative importance of each modality feature in the subsequent fusion operation.

[0035] In some embodiments, generating a reward signal for reinforcement learning based on the performance feedback of the output on the target task can be achieved through the following steps: The multimodal primary representation matrix is ​​weighted and fused based on the output results to obtain the multimodal fused feature vector; The multimodal fusion feature vector is input into a pre-trained task prediction model, which outputs the current task performance index. Determine the performance change of the current task's performance metrics relative to historical performance benchmarks; Reward signals for reinforcement learning are generated based on the performance change mapping.

[0036] In specific implementation, the multimodal primary representation matrix is ​​weighted and fused based on the action vector corresponding to the output result to obtain the multimodal fused feature vector. This can be achieved in the following way: obtain the action vector corresponding to the output result, and use the normalized fusion weight of each dimension of the action vector corresponding to a modality as the weight coefficient of the corresponding modality feature. Then, multiply it element-wise with the corresponding modality feature vector in the multimodal primary representation matrix. Subsequently, sum all the weighted modality feature vectors element-wise to generate a fixed-dimensional multimodal fused feature vector. In a preferred embodiment, the weighted fusion process can use a broadcast mechanism to achieve automatic alignment of vectors of different dimensions. In other embodiments, higher-order interaction information can also be retained through outer product operation, which is not limited here.

[0037] In specific implementation, the multimodal fusion feature vector is input into a pre-trained task prediction model, and the current task performance index is output in the following way: the multimodal fusion feature vector is input into a pre-trained task prediction model composed of a fully connected layer and a Softmax output layer to obtain the predicted probability distribution of each category, and then the current task performance index is calculated by comparing the prediction results with the true labels. In a preferred embodiment, the current task performance index can be the F1 score to balance precision and recall. In other embodiments, precision, area under the curve (AUC value), or other domain-specific indicators can also be used, which are not limited here.

[0038] In specific implementation, the performance change of the current task performance index relative to the historical performance benchmark can be determined in the following way: First, maintain a historical performance index buffer of fixed length, calculate the moving average of the performance index of the most recent N time steps in the buffer as the historical performance benchmark, and then take the difference between the current task performance index and the historical performance benchmark as the performance change. In a preferred embodiment, the buffer length can be set to 10 time steps. In other embodiments, exponential weighted moving average or time decay-based weighted average methods can also be used, which are not limited here.

[0039] In specific implementation, the reward signal for reinforcement learning can be generated based on the performance change. This can be achieved by inputting the performance change into a preset reward function, which then maps and generates the reward signal for reinforcement learning. The reward function can be designed as a piecewise linear mapping. For example, when the performance change exceeds a positive threshold, a fixed positive reward is output; when it falls below a negative threshold, a fixed negative reward is output; and within the threshold range, the reward is mapped linearly to the corresponding reward value. In a preferred embodiment, the positive and negative thresholds can be set to +5% and -5% respectively, and the corresponding fixed reward values ​​can be +1 and -1. That is, when the performance change is within the range of [-5%, +5%], it can be mapped linearly to the range of [-1, +1]. In other embodiments, an S-shaped function can be used for smooth mapping, or a dynamic adjustment mechanism based on the changing trend can be introduced; this is not limited here.

[0040] It should be noted that the task prediction model in this application refers to an optimized parameter model specifically designed to perform a particular target task, such as classification or regression, and to map input features to the task output space. In specific implementation, the pre-training of the task prediction model can be achieved using supervised learning. For example, first, a training dataset containing multimodal joint representation vectors and their corresponding real labels is collected. Then, the cross-entropy loss function and the Adam optimizer are used to iteratively update the model parameters by minimizing the difference between the predicted output and the real label. In a preferred embodiment, early stopping can be used to prevent overfitting. Other implementations can also use label smoothing or data augmentation techniques to improve the model's generalization ability, which will not be elaborated here.

[0041] Additionally, it should be noted that the multimodal fusion feature vector in this application refers to the feature information of multiple modalities used to integrate and condense the feature information of multiple modalities to form a unified vector representation that can comprehensively reflect the cross-modal characteristics of the target object; the current task performance index in this application refers to a numerical measure used to quantitatively evaluate the instantaneous performance level of the task prediction model at the current moment; the historical performance benchmark in this application is a measure that characterizes the average performance level of the task prediction model in recent historical time periods, and is used as a reference standard for evaluating current performance changes; the performance change amount in this application refers to a quantitative difference value used to reflect the degree and direction of deviation of the current performance from the historical performance benchmark; and the reward signal in this application is a standardized feedback signal that transmits the degree of merit of the current action policy to the policy network, and is used to guide the optimization direction of the policy network.

[0042] In step 104, during the weighted fusion process, the weight distribution of each modal feature in the policy network is adaptively adjusted using the generated reward signal to obtain a dynamic fusion weight matrix that reflects multimodal semantic association and task relevance.

[0043] In some embodiments, referenceFigure 3 As shown, this figure is an exemplary flowchart of determining the dynamic fusion weight matrix in some embodiments of this application. In the weighted fusion process, the weight distribution of each modal feature in the policy network is adaptively adjusted by the generated reward signal to obtain the dynamic fusion weight matrix that reflects the multimodal semantic association and task relevance. This can be achieved by the following steps: The advantage function value of the policy network is determined by the generated reward signal; The weight distribution of each modality feature in the policy network is updated based on the dominance function value; New action vectors are generated using the updated policy network; The new action vector is converted into a diagonal matrix form to obtain a dynamic fusion weight matrix that reflects multimodal semantic association and task relevance.

[0044] In specific implementation, determining the dominance function value of the policy network through the generated reward signal can be achieved in the following way: during the weighted fusion process, a generalized dominance estimation algorithm can be used to combine the generated reward signal with the state value function output by the Critic network, and the dominance function value of the policy network can be calculated by weighted sum of temporal difference errors; wherein, as a preferred embodiment, the discount factor of the generalized dominance estimation can be set to 0.99, and the trade-off parameter can be set to 0.95. In other embodiments, the Monte Carlo method or the n-step temporal difference algorithm can also be used for dominance estimation, which is not limited here.

[0045] In specific implementation, updating the weight distribution of each modality feature in the policy network based on the advantage function value can be achieved in the following way: the pruning objective function of the near-end policy optimization algorithm can be used to update the parameters of the Actor network through backpropagation based on the advantage function value, while the parameters of the Critic network are updated using the mean squared error loss function, thereby directly adjusting the mapping function inside the policy network used to generate the weight distribution of each modality feature; wherein, as a preferred embodiment, the pruning range can be set to [0.8, 1.2], and the learning rate can be set to 3e-4. In other embodiments, trust region policy optimization or natural policy gradient method can also be used for policy update, which is not limited here.

[0046] In specific implementation, the generation of new action vectors through the updated policy network can be achieved in the following way: the current multimodal fusion state can be input into the Actor network of the updated policy network, and after forward propagation, a new action vector conforming to the current policy can be generated through the Softmax output layer. This vector is the new weight distribution of each modality feature after adaptive adjustment. In a preferred embodiment, the forward propagation process includes two fully connected layers and a ReLU activation function. In other embodiments, Gaussian policies or deterministic policies can also be used to generate action vectors, which is not limited here.

[0047] In specific implementation, converting the new action vector into a diagonal matrix form to obtain a dynamic fusion weight matrix reflecting multimodal semantic association and task relevance can be achieved in the following way: placing each element of the new action vector on the main diagonal of the matrix and setting the off-diagonal elements to zero, constructing a square matrix corresponding to the number of modalities. Finally, the obtained square matrix is ​​used as the dynamic fusion weight matrix reflecting multimodal semantic association and task relevance, and its diagonal elements are the adjusted weights of each modality feature. In a preferred embodiment, the conversion process can be implemented through tensor operations to ensure that the matrix dimension is aligned with the modality dimension of the multimodal primary representation matrix. In other embodiments, a block diagonal matrix form can also be used to retain the intermodal association information, which is not limited here.

[0048] It should be noted that the advantage function value in this application refers to a scalar value used to quantify the relative superiority or inferiority of taking a specific action (i.e., the current weight distribution) relative to the average level in the current state, which provides directional guidance for policy updates; the weight distribution of each modality feature in this application refers to the probability allocation used to characterize the relative importance of different modality features in the multimodal fusion process; the new action vector in this application refers to the latest weight allocation decision output by the policy network for the current state after parameter updates, which is a direct numerical representation of the optimized fusion policy; the dynamic fusion weight matrix in this application refers to the matrix representing the optimal weight allocation of each modality feature, which is adaptively generated by the policy network according to the environmental state. It is used to reflect the fusion representation of the current multimodal semantic association and task relevance, and it is a standard mathematical form that can be directly used for matrix operations with the multimodal primary representation matrix to achieve weighted fusion operations.

[0049] In step 105, an adaptive fusion operation driven by reinforcement learning is performed on the multimodal primary representation matrix according to the dynamic fusion weight matrix to obtain a multimodal joint representation vector, and the task performance index is determined based on the actual output of the multimodal joint representation vector on the target task.

[0050] In some embodiments, performing a reinforcement learning-driven adaptive fusion operation on the multimodal primary representation matrix based on the dynamic fusion weight matrix to obtain the multimodal joint representation vector can be achieved through the following steps: The dynamic fusion weight matrix is ​​adaptively fused with the multimodal primary representation matrix to obtain the feature matrix; Feature aggregation is performed by summing the columns of the feature matrix; The aggregated feature vectors are dimension-normalized to obtain a multimodal joint representation vector.

[0051] In specific implementation, the dynamic fusion weight matrix and the multimodal primary representation matrix are adaptively fused to obtain the feature matrix. This can be achieved by multiplying the dynamic fusion weight matrix and the multimodal primary representation matrix using matrix multiplication, where the dynamic fusion weight matrix is ​​used as the left multiplication matrix and the multimodal primary representation matrix is ​​used as the right multiplication matrix. This achieves a weighted combination of each modality feature, ultimately yielding the feature matrix. In a preferred embodiment, the matrix multiplication can be implemented using the torch.mm function in the PyTorch framework to ensure that the dimensions of the two matrices satisfy the matrix multiplication rules. In other implementations, the Einstein summation convention or block matrix multiplication can also be used to improve computational efficiency, which is not limited here.

[0052] In a specific implementation, feature aggregation by summing the feature matrix by column can be achieved in the following way: the weighted feature matrix can be summed along the column direction (i.e., the feature dimension) to compress the two-dimensional matrix into a one-dimensional feature vector; wherein, as a preferred embodiment, the summation operation can be implemented by a summation function along dimension 1 to retain the weighted contribution of each feature channel. In other embodiments, average pooling or max pooling can also be used as alternative aggregation methods, which are not limited here.

[0053] In specific implementation, the multimodal joint representation vector can be obtained by dimension normalization of the aggregated feature vectors in the following way: the aggregated feature vectors can be input into a fully connected layer, mapped to a preset target dimension through linear transformation, and then normalized by a LayerNorm layer to finally obtain the multimodal joint representation vector. In a preferred embodiment, the target dimension can be set to 512 dimensions, the fully connected layer can be initialized using Xavier, and the LayerNorm layer can be configured with default parameters. In other implementations, batch normalization or instance normalization can also be used, which is not limited here.

[0054] It should be noted that the adaptive fusion operation in this application refers to the dynamic weight allocation scheme used to execute the policy network output, which is a process of dynamically weighting and combining the features of each modality according to their relevance to the task; the feature matrix in this application refers to the intermediate data organization form used to temporarily store the modal features after dynamic weighting and to maintain the spatial structure and correlation between features; the feature aggregation in this application refers to the operation process used to compress and condense the distributed feature information contained in the feature matrix, transforming it into a more compact low-dimensional vector representation containing all weighted feature information; the aggregated feature vector in this application refers to the transitional data carrier used to carry the comprehensive information extracted from the weighted features of all modalities and to serve as an intermediate bridge connecting feature aggregation and final representation generation; the multimodal joint representation vector in this application refers to the final cross-modal feature representation used to integrate and uniformly express the multimodal semantic information after reinforcement learning optimization and fusion, and to serve as a direct input to the downstream task model.

[0055] In some embodiments, determining the task performance index based on the actual output of the multimodal joint representation vector on the target task can be achieved by the following steps: The multimodal joint representation vector is input into the task prediction model to obtain the actual output of the task prediction model on the target task. The basic performance indicators are determined by the actual output. The basic performance indicators are averaged using a sliding window method to obtain stable task performance indicators.

[0056] In specific implementation, the multimodal joint representation vector is input into the task prediction model to obtain the actual output of the task prediction model on the target task. This can be achieved in the following way: the multimodal joint representation vector can be input into the task prediction model composed of two fully connected layers and a Softmax output layer. The first fully connected layer compresses the dimension of the multimodal joint representation vector from 512 to 256, the second fully connected layer further compresses it to the number of task categories, and finally the Softmax layer outputs the probability distribution of each category to obtain the actual output of the task prediction model on the target task. As a preferred embodiment, for example in a drone scenario, the task prediction model can output the probability distribution of three categories: "normal flight", "target loss", and "emergency obstacle avoidance". In other embodiments, the network structure and output dimension can be adjusted according to the specific task requirements, which is not limited here.

[0057] In specific implementation, the basic performance index can be determined by the actual output in the following way: the category with the highest probability in the Softmax output can be taken as the prediction result, compared with the true label, and the classification accuracy can be calculated as the basic performance index by the ratio of the number of correctly predicted samples to the total number of samples. In a preferred embodiment, the true label can be from the validation dataset manually labeled during the drone flight. In other embodiments, the F1 score or AUC value can also be used as the basic performance index, which is not limited here.

[0058] In specific implementation, the stable task performance index can be obtained by performing sliding window averaging on the basic performance index. This can be achieved by maintaining a circular buffer of fixed length 5 to store the basic performance index of the most recent 5 time steps. Each time the calculation is performed, the arithmetic mean of all values ​​in the buffer is taken as the final stable task performance index. In a preferred embodiment, the buffer can adopt a first-in-first-out update strategy. In other embodiments, weighted moving average or exponential smoothing methods can also be used for stability processing, which is not limited here.

[0059] It should be noted that the actual output on the target task in this application refers to the specific inference result generated by the task prediction model based on the input multimodal joint representation vector, which is the model's final judgment on the current environmental state or target object; the basic performance index in this application refers to the accuracy and reliability of the task prediction model in a single inference process; the task performance index in this application refers to the overall performance level of the task prediction model over a continuous time period, which is used to provide a stable and reliable performance evaluation benchmark for decision-making.

[0060] In step 106, when the task performance index is lower than a preset threshold, the policy update process of the policy network is retried.

[0061] It should be noted that the preset threshold in this application is used to determine whether the current performance meets the basic operating requirements and serves as a benchmark value for the decision boundary that triggers the retraining of the policy network. In specific implementation, the preset threshold can be set based on task requirements and historical performance data. That is, firstly, the benchmark performance level of the target task on the validation set is analyzed, and combined with the safety margin requirements of system operation, the threshold is set to 90%-95% of the benchmark performance. For example, in the UAV target tracking task, when the benchmark classification accuracy is 0.9, the preset threshold can be set to 0.85, which ensures that the system can adjust in time when the performance drops slightly, and avoids unnecessary frequent retraining due to an excessively high threshold. Other implementations can also adopt a dynamic threshold adjustment strategy to adaptively adjust the threshold size according to the environmental complexity, which is not specifically limited here.

[0062] In specific implementation, when the task performance index is lower than a preset threshold, the policy update process of the policy network can be re-triggered in the following way: when the task performance index is lower than the preset threshold, the policy update process of the policy network is re-triggered. The policy update process is as follows: first, save the parameter state of the current policy network; then, re-initialize the momentum state of the Adam optimizer; then, start collecting new multimodal raw data from the current moment to construct new training samples; finally, execute a complete policy network training cycle based on the new sample data. In a preferred embodiment, the policy update process can adopt an incremental learning method, using new experience replay buffer data for training while retaining existing knowledge. In other embodiments, a complete retraining or transfer learning method can also be used to adjust the policy network, which is not limited here.

[0063] In practice, when the task performance index is not lower than the preset threshold, the parameters of the current policy network can be kept unchanged, and the normal feature fusion and task prediction process can continue to be executed, while the changing trend of the task performance index is continuously monitored. In this operating state, the system will skip the policy update stage and directly enter the next cycle of multimodal data acquisition and processing, thereby avoiding unnecessary consumption of computing resources while ensuring the stability of system performance.

[0064] Furthermore, in another aspect of this application, in some embodiments, this application provides a multimodal representation system based on reinforcement learning, with reference to... Figure 4 The figure is a schematic diagram of the structure of a multimodal representation system based on reinforcement learning according to some embodiments of this application. The multimodal representation system 400 based on reinforcement learning includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire the multimodal raw data of the target object in a multi-source sensing environment; Processing module 402, in this application, is mainly used to perform time and resolution synchronization calibration on the multimodal raw data, and then perform modal feature encoding on each modal data in the synchronously calibrated multimodal raw data to obtain a multimodal primary characterization matrix. The processing module 402 described in this application is further used to input the multimodal primary representation matrix into the trained policy network, wherein the current multimodal fusion state is used as the state input, and the operation of selecting and weighting the features of each modality is used as the action output, and then a reward signal for reinforcement learning is generated based on the performance feedback of the output result on the target task. The processing module 402 described in this application is also used to adaptively adjust the weight distribution of each modal feature in the policy network through the generated reward signal during the weighted fusion process, so as to obtain a dynamic fusion weight matrix that reflects the multimodal semantic association and task relevance. The processing module 402 described in this application is further configured to perform an adaptive fusion operation driven by reinforcement learning on the multimodal primary representation matrix according to the dynamic fusion weight matrix to obtain a multimodal joint representation vector, and determine the task performance index based on the actual output of the multimodal joint representation vector on the target task. The execution module 403 in this application is mainly used to re-trigger the policy update process of the policy network when the performance index of the task is lower than a preset threshold.

[0065] The foregoing has detailed examples of the multimodal representation method and system based on reinforcement learning provided in the embodiments of this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0066] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described reinforcement learning-based multimodal representation method.

[0067] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a schematic diagram of the structure of a computer device implementing the reinforcement learning-based multimodal representation method of this application. The reinforcement learning-based multimodal representation method in the above embodiments can be... Figure 5 The computer device 500 shown is used to implement this, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 may be a terminal device, a server or a chip.

[0068] The processor 501 can be a general-purpose processor or a special-purpose processor. For example, the processor 501 can be a central processing unit (CPU). The CPU can be used to control the computer device 500, execute software programs, and process data from the software programs. The computer device 500 may also include a communication unit 505 for inputting (receiving) and outputting (transmitting) signals.

[0069] For example, computer device 500 may be a chip, communication unit 505 may be the input and / or output circuit of the chip, or communication unit 505 may be the communication interface of the chip, and the chip may be a component of terminal device, network device or other device.

[0070] For example, computer device 500 may be a terminal device or a server, and communication unit 505 may be a transceiver of the terminal device or the server, or communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0071] The computer device 500 may include one or more memories 502 storing a program 504. The program 504 can be executed by a processor 501 to generate instructions 503, causing the processor 501 to perform the methods described in the above method embodiments according to the instructions 503. Optionally, the memory 502 may also store data (such as a target audit model). Optionally, the processor 501 may also read data stored in the memory 502, which may be stored at the same storage address as the program 504, or the data may be stored at a different storage address than the program 504.

[0072] The processor 501 and memory 502 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0073] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0074] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described reinforcement learning-based multimodal representation method.

[0076] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0077] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A multimodal representation method based on reinforcement learning, characterized in that, Includes the following steps: Acquire multimodal raw data of the target object in a multi-source sensing environment; The multimodal raw data is subjected to time and resolution synchronization calibration, and then each modal data in the synchronously calibrated multimodal raw data is encoded with modal features to obtain a multimodal primary characterization matrix. The multimodal primary representation matrix is ​​input into the trained policy network, wherein the current multimodal fusion state is used as the state input, and the operation of selecting and weighting the features of each modality is used as the action output. Then, a reward signal for reinforcement learning is generated based on the performance feedback of the output result on the target task. During the weighted fusion process, the weight distribution of each modal feature in the policy network is adaptively adjusted by the generated reward signal to obtain a dynamic fusion weight matrix that reflects multimodal semantic association and task relevance. Based on the dynamic fusion weight matrix, an reinforcement learning-driven adaptive fusion operation is performed on the multimodal primary representation matrix to obtain a multimodal joint representation vector, and the task performance index is determined based on the actual output of the multimodal joint representation vector on the target task. When the task performance index falls below a preset threshold, the policy update process of the policy network is retried.

2. The method as described in claim 1, characterized in that, Modal feature encoding is performed on each modal data in the multimodal raw data obtained from synchronous calibration to obtain the multimodal primary characterization matrix, which specifically includes: Spatial features of visual image data, spectral features of speech signal data, and semantic features of text semantic data are extracted from the multimodal raw data obtained from synchronous calibration. The extracted spatial features, spectral features, and semantic features are encoded to obtain visual feature vectors, speech feature vectors, and text feature vectors; A multimodal primary representation matrix is ​​generated from the visual feature vector, the speech feature vector, and the text feature vector.

3. The method as described in claim 1, characterized in that, The generation of reward signals for reinforcement learning based on the performance feedback of the output on the target task specifically includes: The multimodal primary representation matrix is ​​weighted and fused based on the output results to obtain the multimodal fused feature vector; The multimodal fusion feature vector is input into a pre-trained task prediction model, which outputs the current task performance index. Determine the performance change of the current task's performance metrics relative to historical performance benchmarks; Reward signals for reinforcement learning are generated based on the performance change mapping.

4. The method as described in claim 1, characterized in that, In the weighted fusion process, the weight distribution of each modal feature in the policy network is adaptively adjusted using the generated reward signal to obtain a dynamic fusion weight matrix that reflects multimodal semantic association and task relevance. Specifically, this includes: The advantage function value of the policy network is determined by the generated reward signal; The weight distribution of each modality feature in the policy network is updated based on the dominance function value; New action vectors are generated using the updated policy network; The new action vector is converted into a diagonal matrix form to obtain a dynamic fusion weight matrix that reflects multimodal semantic association and task relevance.

5. The method as described in claim 1, characterized in that, Based on the dynamic fusion weight matrix, an reinforcement learning-driven adaptive fusion operation is performed on the multimodal primary representation matrix to obtain the multimodal joint representation vector, specifically including: The dynamic fusion weight matrix is ​​adaptively fused with the multimodal primary representation matrix to obtain the feature matrix; Feature aggregation is performed by summing the columns of the feature matrix; The aggregated feature vectors are dimension-normalized to obtain a multimodal joint representation vector.

6. The method as described in claim 1, characterized in that, Determining task performance metrics based on the actual output of the multimodal joint representation vector on the target task specifically includes: The multimodal joint representation vector is input into the task prediction model to obtain the actual output of the task prediction model on the target task. The basic performance indicators are determined by the actual output. The basic performance indicators are averaged using a sliding window method to obtain stable task performance indicators.

7. The method as described in claim 1, characterized in that, The multimodal raw data includes visual image data, speech signal data, and text semantic data.

8. A multimodal representation system based on reinforcement learning, characterized in that, include: The acquisition module is used to acquire the multimodal raw data of the target object in a multi-source sensing environment; The processing module is used to perform time and resolution synchronization calibration on the multimodal raw data, and then perform modal feature encoding on each modal data in the synchronously calibrated multimodal raw data to obtain a multimodal primary characterization matrix. The processing module is further configured to input the multimodal primary representation matrix into the trained policy network, wherein the current multimodal fusion state is used as the state input, and the operation of selecting and weighting the features of each modality is used as the action output, and then a reward signal for reinforcement learning is generated based on the performance feedback of the output result on the target task. The processing module is also used to adaptively adjust the weight distribution of each modal feature in the policy network through the generated reward signal during the weighted fusion process, so as to obtain a dynamic fusion weight matrix that reflects the multimodal semantic association and task relevance. The processing module is further configured to perform an adaptive fusion operation driven by reinforcement learning on the multimodal primary representation matrix according to the dynamic fusion weight matrix to obtain a multimodal joint representation vector, and determine the task performance index based on the actual output of the multimodal joint representation vector on the target task. The execution module is used to re-trigger the policy update process of the policy network when the performance index of the task is lower than a preset threshold.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory being used to store a computer program, and the processor being used to call and run the computer program from the memory, causing the computer device to perform the multimodal representation method based on reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the reinforcement learning-based multimodal representation method as described in any one of claims 1 to 7.

Images