Model training method, action sequence determination method, device, and storage medium

Abstract

This application provides a model training method, an action sequence determination method, a device, and a storage medium, relating to the field of artificial intelligence model training. The method includes: inputting training samples into a preset model for forward propagation to obtain the hidden state tensor of the preset model, where the preset model is a visual language action model; determining the distribution distance between the hidden state tensor and the real action sequence; determining a target loss function based on the distribution distance, wherein a larger distribution distance results in a larger loss weight in the target loss function corresponding to the training sample; and updating the parameters of the preset model based on the target loss function to obtain a target model, which is used to determine the action sequence of a target device. This method can balance the differences in sample difficulty and the quality of intermediate representations, improve the model's learning ability for complex tasks, and enhance the accuracy of action sequence generation, training convergence efficiency, and deployment stability.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence model training, and in particular to a model training method, an action sequence determination method, a device, and a storage medium. Background Technology

[0002] In training Vision-Language-Action (VLA) models, a Transformer-based multimodal fusion architecture is typically used, combined with generative methods such as Flow Matching to model continuous action sequences.

[0003] However, existing solutions mostly optimize the action generation process with uniform loss weights, focusing mainly on the final action output. They lack effective differentiation of the difficulty differences between training samples and time steps, and have insufficient constraints on the correlation between intermediate features of the model and the distribution of target actions. This limits the training efficiency, action prediction accuracy, and adaptability to distribution drift under complex multimodal tasks.

[0004] Therefore, how to balance the differences in sample difficulty and the alignment of intermediate representations during the training of VLA models in order to improve action prediction accuracy, training efficiency and robustness has become a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a model training method, an action sequence determination method, a device, and a storage medium to solve the aforementioned technical problems. This method, oriented towards the training process of visual language action models, comprehensively considers the distribution differences of training samples in action generation tasks and the matching relationship between intermediate model representations and target actions. It adaptively adjusts the training optimization process, thereby balancing sample difficulty differences and representation alignment requirements during model learning, improving the relevance and effectiveness of action sequence modeling.

[0006] In a first aspect, embodiments of this application provide a model training method, including:

[0007] The training samples are input into the preset model for forward propagation to obtain the hidden state tensor of the preset model, which is a visual language action model.

[0008] Determine the distribution distance between the hidden state tensor and the real action sequence;

[0009] The target loss function is determined based on the distribution distance, where the larger the distribution distance, the larger the loss weight corresponding to the training sample in the target loss function;

[0010] The parameters of the preset model are updated according to the target loss function to obtain the target model, which is used to determine the action sequence of the target device.

[0011] In one possible implementation, determining the distribution distance between the hidden state tensor and the real action sequence includes:

[0012] Project the hidden state tensor onto the dimensional space of the real action sequence to obtain the projected hidden state;

[0013] Calculate the distribution difference between the projected hidden state and the actual action sequence to obtain the distribution distance.

[0014] In one possible implementation, calculating the distribution difference between the projected hidden state and the real action sequence includes: calculating the maximum mean difference between the projected hidden state and the real action sequence using a kernel function, and using the maximum mean difference as the distribution difference.

[0015] In one possible implementation, calculating the distribution difference between the projected hidden state and the true action sequence includes:

[0016] In terms of temporal sequence, the projected hidden state and the real action sequence are respectively fitted to Gaussian distributions;

[0017] The KL divergence between two Gaussian distributions is defined as the distribution difference.

[0018] In one possible implementation, calculating the distribution difference between the projected hidden state and the real action sequence includes: calculating the minimum transfer cost required to transfer the distribution of the projected hidden state to the distribution of the real action sequence, and using the minimum transfer cost as the distribution difference.

[0019] In one possible implementation, determining the target loss function based on the distribution distance includes:

[0020] Determine the loss weight based on the distribution distance;

[0021] Determine the target loss function based on the loss weights and the initial loss function.

[0022] In one possible implementation, determining the loss weight based on the distribution distance includes:

[0023] Multiply the distribution distance by the preset first hyperparameter to obtain the first product result;

[0024] The first product result is added to a preset value to obtain the loss weight.

[0025] In one possible implementation, determining the loss weight based on the distribution distance includes:

[0026] Multiply the distribution distance by the preset second hyperparameter to obtain the second product result;

[0027] The loss weights are obtained by calculating the function value with the natural constant as the base and the result of the second product as the exponent.

[0028] In one possible implementation, determining the loss weight based on the distribution distance includes:

[0029] The range of values ​​for the distribution distance is divided into at least two intervals by using at least one grading threshold.

[0030] Determine the target interval to which the distribution distance belongs;

[0031] The loss weight is determined based on the preset weight value corresponding to the target interval.

[0032] In one possible implementation, determining the target loss function includes:

[0033] The loss term is obtained by multiplying the loss weights by the initial loss function.

[0034] The regularization term is determined based on the distribution distance, and the regularization term is positively correlated with the distribution distance;

[0035] The sum of the loss term and the regularization term is used to determine the target loss function.

[0036] In one possible implementation, updating the parameters of a preset model according to the target loss function includes:

[0037] Backpropagation is performed based on the target loss function to obtain the gradient information of at least one parameter of the preset model;

[0038] Update the parameters of the preset model based on gradient information and the preset learning rate.

[0039] In one possible implementation, projecting the hidden state tensor onto the dimensional space of the real action sequence includes:

[0040] The linear projection matrix is ​​determined by the total number of features of the hidden state tensor and the number of columns by the total number of features of the real action sequence.

[0041] The hidden state tensor is flattened and multiplied by the transpose of the linear projection matrix to obtain the projected hidden state.

[0042] In one possible implementation, the method further includes:

[0043] Construct a pre-defined model, which includes a visual encoder, a language encoder, a multimodal fusion module, and an action decoder;

[0044] Acquire training samples, which include image data, language instruction data, and corresponding real action sequences.

[0045] In one possible implementation, obtaining the hidden state tensor of a preset model includes:

[0046] Visual features are obtained by processing image data through a visual encoder.

[0047] Language features are obtained by processing language instruction data through a language encoder.

[0048] Visual and linguistic features are processed by a multimodal fusion module to obtain fused features;

[0049] The fused features are processed by an action decoder, and the hidden state tensor is extracted after the target layer in the action decoder.

[0050] Secondly, embodiments of this application provide a method for determining an action sequence, including:

[0051] Acquire the data to be processed, which includes image data and language command data;

[0052] The target model is used to process the data to be processed to obtain an action sequence. The target model is obtained by training a preset model using the model training method in the first aspect. The action sequence is used to control the target device to perform the target action.

[0053] Thirdly, embodiments of this application provide a model training apparatus, comprising:

[0054] The first acquisition module is used to input training samples into a preset model for forward propagation and acquire the hidden state tensor of the preset model, which is a visual language action model.

[0055] The first determining module is used to determine the distribution distance between the hidden state tensor and the real action sequence;

[0056] The second determination module is used to determine the target loss function based on the distribution distance, wherein the larger the distribution distance, the larger the loss weight corresponding to the training sample in the target loss function;

[0057] The first processing module is used to update the parameters of the preset model according to the target loss function to obtain the target model, which is used to determine the action sequence of the target device.

[0058] Fourthly, embodiments of this application provide an action sequence determination device, comprising:

[0059] The second acquisition module is used to acquire data to be processed, which includes image data and language instruction data.

[0060] The second processing module is used to process the data to be processed through the target model to obtain the action sequence. The target model is obtained by training a preset model through a model training method, and the action sequence is used to control the target device to perform the target action.

[0061] Fifthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0062] The memory stores the instructions that the computer executes;

[0063] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above, or causing the processor to perform the second aspect and / or various possible implementations of the second aspect as described above.

[0064] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect, or, when executed by a processor, are used to implement the second aspect and / or various possible implementations of the second aspect.

[0065] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect, or when executed by a processor, implements the second aspect and / or various possible implementations of the second aspect.

[0066] The model training method, action sequence determination method, device, and storage medium provided in this application embodiment obtain hidden state tensors by inputting training samples into a preset model for forward propagation, and determine the target loss function based on the distribution distance between the hidden state tensor and the real action sequence. Furthermore, the training samples with larger distribution distances correspond to larger loss weights, which enables the model to pay more attention to samples with large deviations between intermediate representations and target action distributions during parameter update, thereby improving the training efficiency, action prediction accuracy, and adaptability to distribution drift of the visual language action model. Attached Figure Description

[0067] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0068] Figure 1 Flowchart of the model training method provided in this application Figure 1 ;

[0069] Figure 2Flowchart of the model training method provided in this application Figure 2 ;

[0070] Figure 3 A flowchart illustrating the action sequence determination method provided in this application;

[0071] Figure 4 A schematic diagram of the structure of the model training device provided in this application;

[0072] Figure 5 A schematic diagram of the action sequence determination device provided in this application;

[0073] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.

[0074] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0075] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0076] This technology relates to the training of visual-language-action models and is applicable to embodied intelligence scenarios that require the generation of continuous action sequences based on visual observations and verbal commands. In typical applications, the system usually includes a visual information processing module, a language information processing module, a multimodal fusion module, and an action generation module, used to drive robotic arms, warehouse robots, or service robots to complete tasks such as grasping, handling, assembly, and sorting.

[0077] Existing visual language action model training schemes typically rely on Transformer for multimodal feature fusion and combine generative methods such as Flow Matching to model continuous action sequences. During training, images, text commands, and real action sequences from training samples are input into the model. Intermediate feature representations are obtained through forward propagation, and the model parameters are updated based on the action generation error, enabling the model to gradually learn the mapping relationship between semantic understanding and visual perception to action output.

[0078] However, most of these solutions use uniform loss weights to process all training samples and time steps, making it difficult to distinguish the differences in learning difficulty between simple and complex action samples, and also difficult to identify the different importance of key action phases and ordinary action phases in training. For tasks with occlusion, lighting changes, multiple path solutions, or fine-grained operation requirements, the model is prone to overfitting to easy-to-learn samples, while underfitting to learn the difficult samples that truly determine the quality of task completion.

[0079] Meanwhile, existing training processes primarily focus on optimizing the final action output, neglecting to effectively utilize the distribution relationship between the hidden states formed during model forward propagation and the actual action sequence. This results in insufficient correspondence between intermediate representations and the target action. When the task scenario experiences distribution drift or increased environmental noise, the model struggles to adjust its training focus in a timely manner, thereby affecting action prediction accuracy, training convergence efficiency, and post-deployment stability.

[0080] Therefore, balancing the differences in sample difficulty with the degree of matching between intermediate representations and real actions during the training of visual language action models has become a technical problem that needs to be solved. To address this issue, a model training method is provided. This method involves inputting training samples into the visual language action model for forward propagation to obtain a hidden state tensor. The distribution distance between the hidden state tensor and the real action sequence is then determined, and a target loss function is established based on this distance. Training samples with larger distribution distances receive larger loss weights. The model parameters are then updated according to the target loss function, thereby improving the model's ability to determine the action sequences of the target device.

[0081] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0082] It should be noted that the execution entity in each embodiment of this application can be a server, processor, microprocessor, etc. The specific execution entity in each embodiment of this application is not limited, and it can be selected and set according to actual needs. In the following embodiments, a server is used as an example of the execution entity, which does not constitute a limitation on the actual execution entity.

[0083] Figure 1 Flowchart of the model training method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0084] S101. Input the training samples into the preset model for forward propagation to obtain the hidden state tensor of the preset model.

[0085] The preset model is a visual language action model.

[0086] In this step, the preset model receives training samples and performs forward propagation, outputting a hidden state tensor. Subsequently, the parameters are updated according to the target loss function to obtain the target model. In the context of this application, the preset model is a visual-language-action model. The hidden state tensor is an intermediate feature representation obtained after forward propagation of the preset model, used to characterize the internal expression corresponding to the current training sample. The training samples include visual information, linguistic information, and corresponding action information. Visual information can be a single-frame image, multiple consecutive frames, or video clips; linguistic information can be natural language task instructions; and action information can be a sequence of real actions corresponding to the visual-language input, such as the angle values, angular velocity values, gripper opening / closing amounts, or end effector pose sequences of each joint of a robotic arm over consecutive time steps.

[0087] The hidden state tensor can be represented as a three-dimensional tensor, or as a higher-dimensional tensor when multiple intermediate representations exist. It should be understood that the above examples are for demonstration purposes only and are not limiting. The hidden state tensor can be an intermediate output of the preset model during the forward propagation process, as long as the tensor can represent the intermediate state of the preset model and be used for subsequent distribution difference characterization.

[0088] S102. Determine the distribution distance between the hidden state tensor and the real action sequence.

[0089] In this step, the real action sequence serves as the supervised target in the training samples, used to compare with the hidden state tensor and calculate the distribution distance. The real action sequence can originate from manual teaching records, operation acquisition results, or historical execution logs, and its content consists of continuous action labels of the target device when performing the corresponding task. The distribution distance is used to measure the difference between the hidden state tensor and the real action sequence, and is the basis for subsequently determining the loss weights and the target loss function.

[0090] In this application, the hidden state tensor and the real action sequence can first be converted into statistically comparable representations, and then the differences between the two can be quantified. This distributional difference can reflect both the degree to which the model's internal representation fits the evolution of real actions and the degree to which a particular sample has learned in the current training phase.

[0091] In practice, the distribution distance between the hidden state tensor and the real action sequence can be calculated at the sample level, the time step level, or the overall level. For temporal tasks, the distance at each time step can be calculated separately along the time dimension, and then all valid time steps can be aggregated to generate the final distribution distance of the training sample.

[0092] In one exemplary implementation, the hidden state tensor is Where T is the length of the real action sequence, D is the hidden dimension, and the representation of the real action sequence is as follows: Where A is the motion space dimension, such as joint angles, end effector pose, etc., and the distribution distance is denoted as d, which can be expressed as... , where f represents a preset distance calculation function. A larger value for this function indicates a greater difference in distribution between the hidden state tensor and the actual action sequence.

[0093] To ensure that the distance values ​​are comparable within a batch, normalization can be performed on d. For example, linear normalization can be performed using the minimum and maximum values ​​of the current batch, or normalization can be performed based on the historical moving average and standard deviation to obtain normalized distance values.

[0094] This step establishes an explicit distribution relationship between the hidden state tensor and the real action sequence, incorporating intermediate representations of the model into the training adjustment criteria, rather than being limited to the final output error. For training samples with occlusion, lighting variations, multiple path solutions, or fine manipulation requirements, when the preset model can provide superficially similar local action outputs, but its intermediate states have not yet formed a stable correspondence with the real action patterns, the distribution distance will still maintain a large value, thus providing an effective basis for subsequent loss weighting.

[0095] S103. Determine the target loss function based on the distribution distance.

[0096] The greater the distribution distance, the greater the loss weight corresponding to the training sample in the target loss function.

[0097] In this step, the target loss function is used to assign different loss weights based on the distribution distance and serves as the optimization objective for updating the preset model parameters. The loss weights are determined by the distribution distance, and the larger the distribution distance, the larger the weight, which is used to adjust the influence of the corresponding training sample in the target loss function. Based on this, this application introduces sample-level weights driven by distribution distance, enabling the model to apply higher optimization intensity to training samples whose current intermediate representation deviates significantly from the true action distribution in each training round.

[0098] In practice, the loss weights can be determined first based on the distribution distance, and then combined with the loss corresponding to the sample to obtain the target loss function. When a batch contains multiple training samples, the target losses corresponding to all samples can be aggregated to form the total target loss function for that batch.

[0099] This step integrates the sample learning difficulty and the degree of matching of intermediate representations into the target loss function by coupling the distribution distance with the loss weights. Thus, within the same training batch, simple samples, fully learned samples, and samples with high alignment between intermediate states and real actions receive smaller weights, while complex samples, under-learned samples, and samples with large deviations in intermediate states receive larger weights. The training gradient is therefore directed towards samples that require more correction. This approach corresponds to the dynamic allocation of training resources, enabling the pre-defined model to undergo more thorough parameter adjustments on key action phases and complex scene samples.

[0100] S104. Update the parameters of the preset model according to the target loss function to obtain the target model.

[0101] The target model is used to determine the action sequence of the target device.

[0102] In this step, the target model is the result obtained after updating the parameters using the target loss function, and is used to determine the action sequence of the target device. The target device is the object that the target model ultimately controls or serves; in this application, it can be a robotic arm, a warehouse robot, a service robot, or other devices with action execution capabilities. Updating the parameters of the preset model according to the target loss function specifically involves optimizing the parameters of the preset model based on the target loss function, iteratively updating the model parameters in the direction of reducing the target loss function until the preset number of training rounds, validation set convergence conditions, or performance threshold requirements are met.

[0103] In practice, after calculating the target loss function, the parameters of the preset model can be updated based on the target loss function. After each parameter update, the next batch of training samples can be read and S101 to S104 can be repeated until a trained target model is formed. A validation phase can also be set during training, where validation samples not used in training are input into the current model to evaluate metrics such as action prediction error, sequence trajectory deviation, or task completion rate. Based on this, the parameter version with the best performance is selected as the final target model. During deployment, the target model receives visual observations and linguistic information from the target device's operating environment and outputs the target device's action sequence through a forward structure consistent with the training phase. This action sequence can be represented as joint control commands, end-effector pose increments, gripper control signals, or chassis speed commands on a continuous time step, and is sent to the target device for execution via the control interface.

[0104] This step goes beyond simply updating regular parameters. Instead, it continuously corrects the learning bias of the preset model for difficult samples and key action phases under the constraint of the target loss function weighted by distribution distance. Since the update basis includes both the basic action error and the distribution matching information between the hidden state and the real action, the resulting target model can still generate the action sequence of the target device based on a more stable intermediate representation when facing tasks with enhanced environmental noise, scene distribution drift, or multiple operation paths.

[0105] Based on the above analysis, this application provides a model training method, which includes inputting training samples into a visual-language-action model for forward propagation to obtain a hidden state tensor, determining the distribution distance between the hidden state tensor and the real action sequence, constructing a target loss function corresponding to the sample difficulty based on the distribution distance, and updating the model parameters according to the target loss function to obtain the target model. In this application, by explicitly introducing the distribution difference between intermediate hidden states and the real action sequence into the training optimization process, training samples with larger distribution distances receive higher weights in the loss function, thereby enabling the preset model to learn more fully from complex samples, key action stages, and cases with large intermediate representation biases during the training phase. Thus, the target model's ability to determine the target device's action sequence corresponds to the allocation of priorities during the training convergence process, enabling it to output continuous action sequences that are more consistent with visual observation and linguistic information in embodied intelligence tasks. It should be understood that the above examples are merely illustrative and not limiting. Without departing from the core ideas of this application, the extraction position of the hidden state tensor, the calculation method of the distribution distance, the mapping function of the loss weights, and the parameter update strategy can all be adjusted accordingly.

[0106] Optionally, before inputting the training samples into the preset model for forward propagation, the method further includes a process of determining the training samples and the preset model. The model training method provided in this application is described below with reference to the accompanying drawings.

[0107] Figure 2 Flowchart of the model training method provided in this application Figure 2 ,like Figure 2 As shown, the method includes:

[0108] S201. Construct a preset model.

[0109] The preset model includes a visual encoder, a speech encoder, a multimodal fusion module, and an action decoder.

[0110] S202. Obtain training samples.

[0111] The training samples include image data, language instruction data, and corresponding real action sequences.

[0112] Training samples can consist of offline collected task data, with image data, language command data, and real action sequences corresponding one-to-one. These real action sequences can be obtained from a teaching robotic arm, operating equipment, or manually labeled trajectories, and can include continuous positions, postures, gripper opening / closing amounts, or joint control quantities. By inputting these samples into the training model, the model simultaneously learns scene visual information, task semantic information, and the relationship between action evolution during forward propagation, and subsequently updates its parameters under a loss constraint related to the distance of the hidden state tensor distribution.

[0113] During this training process, the visual encoder converts the raw image into a visual representation that can be computed by the subsequent network, the language encoder converts natural language instructions into semantic vectors, the multimodal fusion module performs interactive fusion of the two types of representations, and the action decoder maps the fusion result into an intermediate representation corresponding to the real action sequence. The setting of training samples enables the model to establish a mapping relationship between image data, language instruction data, and real action sequences, and to develop stable action prediction capabilities in tasks involving occlusion, complex paths, or fine manipulation.

[0114] With the above structure, the training model can jointly model visual and linguistic information, and continuously correct the action output using samples containing real action sequences, making the correspondence between the hidden state representation and the target action clearer, thereby improving the model's learning accuracy and training convergence stability for the target device's action sequence.

[0115] S203. Input the training samples into the preset model for forward propagation to obtain the hidden state tensor of the preset model.

[0116] For example, obtaining the hidden state tensor of a preset model includes: processing image data through a visual encoder to obtain visual features; processing language instruction data through a language encoder to obtain language features; processing the visual features and language features through a multimodal fusion module to obtain fused features; processing the fused features through an action decoder; and extracting the hidden state tensor after the target layer in the action decoder.

[0117] During operation, image data and language command data are extracted by the visual encoder and language encoder, respectively, and then fed into the multimodal fusion module for semantic alignment. The action decoder further integrates multi-source information in the cross-attention layer to form a hidden state tensor related to action sequence prediction. The target layer can be any layer in the action decoder, such as after the last cross-attention layer and before being input to the Flow Matching transform network. This tensor serves as the input for subsequent distribution distance calculations, enabling model training to not only rely on the final action output but also consider the correspondence between intermediate representations and the real action sequence.

[0118] Through the above processing methods, the model can obtain more semantically consistent intermediate representations during forward propagation and provide stable input for subsequent loss based on distribution distance, thereby improving the quality of action representations in different sample scenarios and the stability of training convergence.

[0119] S204. Project the hidden state tensor onto the dimensional space of the real action sequence to obtain the projected hidden state.

[0120] For example, projecting the hidden state tensor onto the dimensional space of the real action sequence includes: determining a linear projection matrix, wherein the number of rows of the linear projection matrix is ​​determined by the total number of features of the hidden state tensor and the number of columns is determined by the total number of features of the real action sequence; and multiplying the flattened hidden state tensor by the transpose of the linear projection matrix to obtain the projected hidden state.

[0121] In this scheme, the hidden state tensor is typically derived from an intermediate representation resulting from the fusion of visual, linguistic, and multimodal information. Its flattening process eliminates permutation differences in the temporal, channel, or spatial dimensions of the original tensor, facilitating its integration into the action sequence's dimensional space via a unified linear mapping. The linear projection matrix is ​​a trainable matrix that can be updated along with other parameters in the model during training. The projected hidden state, obtained after matrix multiplication, can be directly used to calculate the distribution differences between the hidden state and the actual action sequence, allowing for comparison within the same feature space and providing fundamental data for determining the loss weights.

[0122] When training samples of size B are input into a pre-defined model, the hidden state tensor H has a shape of [B,T,D], and the real action sequence... The shape is [B,T,A], and the projected linear matrix is... Hidden state after projection The shape is [B,T,A].

[0123] Through the mapping method described above, the hidden state tensor can be transformed into a representation with the same dimensions as the real action sequence, reducing comparison bias caused by inconsistent feature dimensions and enabling the explicit utilization of the relationship between intermediate representations and the target action. This results in more stable metric results in subsequent distribution distance calculations, further enhancing the model's ability to characterize the learning focus of action sequences, thereby improving training convergence and action prediction accuracy.

[0124] S205. Calculate the distribution difference between the hidden state after projection and the real action sequence to obtain the distribution distance.

[0125] In one possible implementation, the maximum mean discrepancy (MMD) between the projected hidden state and the true action sequence can be calculated using a kernel function, such as a radial basis function kernel (RBF), and the maximum mean discrepancy can be used as the distribution discrepancy.

[0126] In the specific implementation, the kernel matrix is ​​calculated based on the projected hidden state sample set and the real action sequence sample set, respectively. The kernel matrix is ​​then used to statistically analyze the mean embedding difference between the two sets of samples, yielding the corresponding maximum mean difference value. This value can be directly used as the distribution difference input to the subsequent loss weight calculation module, allowing the loss weights corresponding to the samples to be adjusted according to the degree of distribution deviation during training. The bandwidth parameter of the kernel function can be set according to the range of action amplitude variation, the frequency of task stage switching, and the dispersion of samples in the training batch. When the bandwidth is small, the model is more sensitive to local differences; when the bandwidth is large, the model's representation of the overall distribution shift is more stable. In practical applications, this parameter can also be dynamically adjusted according to the training task, but this application does not impose any limitations on this.

[0127] This implementation obtains the projected hidden state through forward propagation, maps it to a unified metric space with the real action sequence through a kernel function, and then quantifies the distribution relationship between the two by the maximum mean difference. This allows the subsequent target loss function to directly reflect the degree of matching between the intermediate representation of the model and the target action based on the distribution difference.

[0128] In this way, the distribution difference no longer depends solely on the final action error, but is determined by combining the statistical deviation results measured by the kernel function. This allows for the assignment of higher weights to samples whose intermediate representations are inconsistent with the actual action sequences during training, thereby enhancing the model's learning strength for difficult samples and key action stages, and improving the stability and convergence of action sequence modeling.

[0129] In one possible implementation, the distribution difference between the projected hidden state and the real action sequence is calculated by: fitting the projected hidden state and the real action sequence to Gaussian distributions in time; and determining the distribution difference as the KL (Kullback-Leibler) divergence between the two Gaussian distributions.

[0130] In practical processing, after the hidden state tensor is mapped to the same dimension as the real action sequence, the corresponding vector is extracted time-by-time along the action time axis and mapped one-to-one with the action vector at the same time position in the real action sequence. For each time position, the system calculates the mean and variance of the projected hidden state vector and the real action vector. If the action sequence is long, samples can be accumulated within a fixed-length time window, and the mean vector and covariance parameters within the window can be estimated to form a Gaussian distribution representation for that time segment. The mean can be represented as the expected value of the samples in each dimension at that time position, and the variance or covariance is used to characterize the dispersion and correlation between action dimensions, thus giving both the hidden state distribution and the real action distribution a computable probability model. Subsequently, the KL divergence is calculated based on the mean, variance, or covariance parameters of the two Gaussian distributions, and the obtained divergence value is output as the distribution difference of the sample in the time sequence for subsequent loss weight adjustment.

[0131] In this process, KL divergence, based on temporal distribution, directly reflects the degree of matching between the projected hidden state and the statistical characteristics of the real action sequence. When the mean deviation or variance difference between the hidden state distribution and the real action distribution is significant, the calculated KL divergence increases, indicating a high distribution bias for that sample. Conversely, when the statistical characteristics of both at various temporal positions tend to be consistent, the KL divergence decreases, indicating a lower distribution difference. Based on this divergence result, the training process can determine the distribution distance and further provide weighting criteria for the target loss function.

[0132] With this approach, the distribution difference is no longer indirectly represented by the final action error, but is directly measured by Gaussian fitting and KL divergence over time to measure the statistical consistency between the intermediate representation and the real action. This allows the training process to more accurately identify the degree of deviation between the hidden state and the action sequence, and to form differentiated loss weights accordingly, thereby improving the model's ability to match continuous action sequences and the targeting of parameter updates.

[0133] In one possible implementation, calculating the distribution difference between the projected hidden state and the true action sequence includes: calculating the minimum migration cost (Earth Mover's Distance) required to shift the distribution of the projected hidden state to the distribution of the true action sequence, and using the minimum migration cost as the distribution difference. Mathematically, the minimum migration cost corresponds to the Wasserstein distance.

[0134] In practical implementation, the projected hidden state and the real action sequence can first be sampled or discretized, so that they correspond to several weighted sample points. Then, a transport cost matrix is ​​constructed based on the distance metric between the sample points, where the distance metric can be Euclidean distance, Mahalanobis distance, or a weighted distance consistent with the action dimension. Subsequently, the optimal transport problem is solved under the marginal constraints of the source and target distributions to obtain the matching scheme that minimizes the overall transport cost. This minimum total cost is then used as the distribution difference input into the subsequent loss weight determination process. The cost can be calculated by using linear programming, entropy-normalized optimal transport, or other solution methods that satisfy the minimum transport constraint. In practical applications, other solution methods can also be selected, and this application does not limit them.

[0135] Based on the above processing, the model can explicitly quantify the overall distribution deviation between the hidden state representation and the real action sequence as a minimum transfer cost, thereby reflecting the degree to which the projected hidden state fits the target action distribution. Since this cost directly comes from the global matching results between samples, it can stably represent distribution differences in scenarios with large differences in action patterns, strong local perturbations, or complex multimodal alignment.

[0136] With this approach, the distribution difference no longer depends solely on local statistics, but is uniformly characterized by the minimum transfer cost. This allows the training process to more accurately distinguish the degree of matching between the hidden state and the real action sequence, and provides a more discriminative basis for subsequent loss weight allocation, thereby improving the response accuracy of the model parameters to the distribution of difficult samples and key actions.

[0137] In one possible implementation, the mean square error between the projected hidden state and the real action sequence after passing through a learnable linear projection layer can be calculated and determined as the distribution distance.

[0138] S206. Determine the loss weight based on the distribution distance.

[0139] In practical implementation, the distribution distance can be input into the weight mapping module, which outputs the loss weights corresponding to that distribution distance according to a preset mapping rule. The mapping rule can be linear, piecewise, or exponential. During training, when the distribution distance between the hidden state tensor and the real action sequence is large, the system generates larger loss weights and accordingly amplifies the proportion of the initial loss function in the target loss function, making model parameter updates more focused on difficult-to-learn samples. When the distribution distance is small, the loss weights are reduced accordingly, decreasing the magnitude of model updates for easy-to-learn samples.

[0140] In the specific implementation, the calculated distribution distance d is mapped to the loss weight through a monotonically increasing weight function w=f(d). This function satisfies the condition that the larger the distribution distance, the higher the weight. Furthermore, to prevent the loss weight from being too large, w can be pruned and limited to the range [0.5, 5].

[0141] In one possible implementation, determining the loss weight based on the distribution distance includes: multiplying the distribution distance by a preset first hyperparameter to obtain a first product result; and adding the first product result to a preset value to obtain the loss weight.

[0142] The loss weight w can be expressed as .in, This is the first hyperparameter.

[0143] The first hyperparameter α controls the sensitivity of the loss weights to changes in distribution distance. The larger the value of the first hyperparameter α, the more sensitive the loss weights are to changes in distribution distance, meaning that a small increase in distribution distance will lead to a large increase in the loss weights; the smaller the value of the first hyperparameter α, the less sensitive the loss weights are to changes in distribution distance.

[0144] The default value is one, which means that when the distribution distance is zero, that is, when the distribution of the hidden state tensor is completely aligned with the distribution of the real action sequence, the loss weight is one, and no weighting adjustment is made to the basic loss function at this time.

[0145] The specific value of the first hyperparameter α can be adjusted according to the actual application scenario and the characteristics of the training data. Generally, when the proportion of difficult samples in the training data is high, a larger hyperparameter value can be selected to enhance the model's focus on difficult samples; when the proportion of difficult samples in the training data is low, a smaller hyperparameter value can be selected to avoid excessively amplifying the interference of noisy samples on the training process. Those skilled in the art can determine the optimal hyperparameter value through a small number of experiments, based on the convergence curve of the loss function during training and the performance on the validation set. An example value of 5 for the first hyperparameter is used.

[0146] By employing the aforementioned linear mapping method, the loss weights can monotonically change with the distribution distance, and a preset value provides the basic bias. This allows for higher weighting of samples with significant deviations from the actual action sequences during training, while retaining the basic constraints on samples with low distribution distances. Consequently, in the alignment training between visual observation, linguistic information, and continuous action sequences, the model can more stably focus on sample regions with poor matching, improving the sensitivity of the target loss function to key difficult samples and enhancing the targeting of parameter updates. This, in turn, improves the accuracy of action sequence prediction and training convergence efficiency.

[0147] In a possible implementation, determining the loss weight according to the distribution distance includes: multiplying the distribution distance by a preset second hyperparameter to obtain a second product result; calculating the function value with the natural constant as the base and the second product result as the exponent to obtain the loss weight.

[0148] The loss weight w can be expressed as . where is the second hyperparameter.

[0149] The second hyperparameter β is used to control the growth rate of the loss weight as the distribution distance changes. The larger the value of the second hyperparameter β, the faster the loss weight grows as the distribution distance increases; the smaller the value of the second hyperparameter β, the more gradual the growth rate of the loss weight as the distribution distance increases. The value principle of the second hyperparameter is the same as that of the first hyperparameter, and details are not repeated here.

[0150] After adopting the above implementation, the loss weight grows exponentially when the distribution distance is large, which can strengthen the learning intensity of the model for deviated samples, make the training target more concentrated on the data with large differences between the intermediate representation and the real action sequence, thereby improving the pertinence of parameter update, and improving the fitting effect and training convergence consistency of the vision-language action model for complex action sequences.

[0151] In a possible implementation, determining the loss weight according to the distribution distance includes: dividing the value range of the distribution distance into at least two intervals through at least one binning threshold; determining the target interval to which the distribution distance belongs; determining the loss weight according to the preset weight value corresponding to the target interval.

[0152] In specific implementation, the binning threshold can be determined according to the statistical distribution of the distribution distances of all training samples in the current batch. Specifically, the distribution distance of each training sample in the current batch is acquired; the preset quantile of each distribution distance is calculated, and the preset quantile is taken as the binning threshold. For example, the K1-th quantile and the K2-th quantile of the distribution distance can be used as binning thresholds (0<K1<K2<100), to divide the value range of the distribution distance into three intervals: an interval less than or equal to the K1-th quantile, an interval greater than the K1-th quantile and less than or equal to the K2-th quantile, and an interval greater than the K2-th quantile. Different intervals correspond to different preset weight values respectively.

[0153] Through the above interval mapping method, the loss weight no longer depends on continuous function calculation, but is directly determined according to the interval to which the distribution distance belongs. The weight configuration is more convenient to adjust according to the task difficulty, and it is also convenient to stably distinguish the optimization priority of different samples during the training process, so that the model maintains higher attention to samples with large distribution deviation, and improves the pertinence of the target loss function for training update.

[0154] S207. Determine the target loss function based on the loss weights and the initial loss function.

[0155] The loss weights are applied to the loss term of the initial loss function to form a weighted target loss function. This can be achieved by multiplying the entire initial loss function by the weight coefficients, or by weighting the local loss terms corresponding to the action prediction error. This strengthens the optimization constraints on samples with large distribution deviations during training. The initial loss function can be the mean squared error loss, cross-entropy loss, or regression loss used for continuous action generation. Other basic loss forms can be chosen in practical applications, and this application does not limit their selection.

[0156] In this way, the objective loss function can dynamically adjust the training weights according to the distribution distance, so that the model can allocate higher optimization intensity to samples with large distribution deviations, and enhance the matching ability between hidden states and real action sequences in continuous action sequence training, thereby improving the targeting of training convergence and the stability of action prediction.

[0157] In one possible implementation, determining the target loss function includes: calculating the product of the loss weight and the initial loss function to obtain the loss term; determining the regularization term based on the distribution distance, wherein the regularization term is positively correlated with the distribution distance; and determining the sum of the loss term and the regularization term as the target loss function.

[0158] The initial loss function can be the standard Flow Matching loss. The loss term can be expressed as .

[0159] The regular term can be represented as .in, λ is the regularization coefficient. The regularization coefficient controls the relative importance of the regularization term in the target loss function. A larger value of the regularization coefficient results in a stronger direct penalty to distribution distance during training, leading to a higher degree of distribution alignment between the hidden state tensor and the real action sequence. Conversely, a smaller value of the regularization coefficient λ weakens the impact of the regularization term on the training process, and the model focuses more on minimizing the basic loss function itself. An example regularization value is 0.1.

[0160] The target loss function can be expressed as .

[0161] With this configuration, the objective loss function can simultaneously reflect the differences in sample difficulty and the distribution matching constraint, enabling higher optimization intensity to be assigned to difficult samples and samples with large deviations during training, and prompting the model's hidden representation to gradually approach the real action distribution, thereby improving the convergence stability and action prediction accuracy of the visual language action model.

[0162] S208. Update the parameters of the preset model according to the target loss function to obtain the target model.

[0163] For example, updating the parameters of a preset model according to a target loss function includes: performing backpropagation according to the target loss function to obtain gradient information of at least one parameter of the preset model; and updating the parameters of the preset model according to the gradient information and a preset learning rate.

[0164] In actual training, once the target loss function is determined, the training framework inputs it into the automatic differentiation module to perform backpropagation. The automatic differentiation module backtracks through each computation node layer by layer according to the computation graph and outputs the gradient value corresponding to at least one parameter in the preset model. Subsequently, the optimizer receives the gradient information and the preset learning rate, and corrects the model parameters according to the update relationship of parameter changes along the direction of reducing the loss function, thereby obtaining the updated parameter set. The learning rate is, for example, 0.0001. For cases using stochastic gradient descent, momentum update, or adaptive optimization strategies, the optimizer can complete the parameter update based on the gradient information and the learning rate. In practical applications, other optimizer models can also be selected, and this application does not limit this.

[0165] In this implementation, backpropagation establishes a calculable gradient correspondence between the target loss function and the model parameters. The learning rate, after scaling the gradient magnitude, drives parameter iteration, causing the model to converge towards a smaller target loss after each training iteration. Since the target loss function already incorporates weight information corresponding to the distribution distance, the parameter update process can continuously focus training on samples with a low degree of matching with the real action sequences, thereby enabling the model to gradually acquire a more stable action prediction ability and improving training convergence efficiency and output consistency after deployment.

[0166] The model training method provided in this application will be introduced below with a specific example.

[0167] The training samples were obtained by using a publicly available robot operation dataset. Each sample contained: an RGB image sequence, a language command (such as picking up the red cube and placing it on the left tray), and the corresponding joint angle action sequence (length T=16 steps).

[0168] Taking a batch as an example, the training samples of the batch (size B=32) are input into the preset model to obtain the hidden state tensor H, with shape [B,T,D]=[32,16,512]. Simultaneously, the real action sequence is obtained. The shape is [32,16,7].

[0169] Furthermore, MMD is used to determine the distribution distance. This is calculated independently for each batch of samples, and H and... These are flattened into vectors of 16×512=8192 and 16×7=112 dimensions respectively. Due to the dimension mismatch, a trainable linear projection matrix is ​​initially used. Projecting H onto the action space dimension: Shape [B,16,7].

[0170] calculate and The MMD distance for each sample within a batch. The RBF kernel function is selected. ,in Let d be the median distance between sample pairs. The final distance vector d has shape [B].

[0171] Constructing loss weights includes: using a linear mapping In this example, α = 5.0. To prevent the weights from being too large, w is clipped and limited to the range [0.5, 5.0]. The resulting weight vector w has a shape of [B].

[0172] The calculation of the target loss function includes: first, calculating the standard Flow Matching loss. For each sample, uniform sampling is performed on [0,1] at time step t. The MSE loss between the model's predicted vector field and the theoretical vector field is then calculated to obtain the scalar. .

[0173] Calculate the loss term:

[0174]

[0175] Add regular expression:

[0176]

[0177] The regularity coefficient λ = 0.1.

[0178] The target loss function is: .

[0179] Backpropagation and optimization include: using the AdamW optimizer with a learning rate of 0.0001, and executing... Backpropagation updates all parameters of the model (including the linear projection matrix).

[0180] Furthermore, after obtaining the target model through the model training method, this application also provides a method for using the model, which will be described below with reference to the accompanying drawings.

[0181] Figure 3 A flowchart illustrating the action sequence determination method provided in this application. See also... Figure 3 ,include:

[0182] S301. Obtain the data to be processed, which includes image data and language command data.

[0183] S302. The target model is used to process the data to be processed to obtain the action sequence.

[0184] The target model is obtained by training a preset model using the methods provided above, and the action sequence is used to control the target device to perform the target action.

[0185] By acquiring unprocessed data that simultaneously contains image data and language command data, the target model can utilize environmental visual observations and task semantic constraints concurrently during action generation, thereby more accurately understanding the target's location, spatial relationships, and operational requirements. Since the target model is optimized and trained to match differences in sample difficulty and hidden state distribution, it exhibits stronger adaptability to complex scenes, critical action stages, and noise interference. This results in a higher consistency between the output action sequence and real task requirements, thereby improving the accuracy, stability, and efficiency of robotic arms, warehouse robots, or service robots in performing target actions such as grasping, handling, assembly, and sorting. Therefore, it can improve the model's generalization performance in scenarios with distribution drift.

[0186] Figure 4 A schematic diagram of the structure of the model training device provided in this application is shown below. Figure 4 As shown, the model training device 10 provided in this embodiment includes:

[0187] The first acquisition module 11 is used to input training samples into a preset model for forward propagation and acquire the hidden state tensor of the preset model, wherein the preset model is a visual language action model.

[0188] The first determining module 12 is used to determine the distribution distance between the hidden state tensor and the real action sequence;

[0189] The second determining module 13 is used to determine the target loss function based on the distribution distance, wherein the greater the distribution distance, the greater the loss weight corresponding to the training sample in the target loss function;

[0190] The first processing module 14 is used to update the parameters of the preset model according to the target loss function to obtain the target model, which is used to determine the action sequence of the target device.

[0191] In one possible implementation, the first determining module 12 is specifically used for:

[0192] Project the hidden state tensor onto the dimensional space of the real action sequence to obtain the projected hidden state;

[0193] Calculate the distribution difference between the projected hidden state and the actual action sequence to obtain the distribution distance.

[0194] In one possible implementation, the first determining module 12 is further configured to: calculate the maximum mean difference between the projected hidden state and the real action sequence using a kernel function, and use the maximum mean difference as the distribution difference.

[0195] In one possible implementation, the first determining module 12 is further configured to:

[0196] In terms of temporal sequence, the projected hidden state and the real action sequence are respectively fitted to Gaussian distributions;

[0197] The KL divergence between two Gaussian distributions is defined as the distribution difference.

[0198] In one possible implementation, the first determining module 12 is further configured to: calculate the minimum transfer cost required to transfer the distribution of the projected hidden state to the distribution of the real action sequence, and use the minimum transfer cost as the distribution difference.

[0199] In one possible implementation, the second determining module 13 is specifically used for:

[0200] Determine the loss weight based on the distribution distance;

[0201] Determine the target loss function based on the loss weights and the initial loss function.

[0202] In one possible implementation, the second determining module 13 is further configured to:

[0203] Multiply the distribution distance by the preset first hyperparameter to obtain the first product result;

[0204] The first product result is added to a preset value to obtain the loss weight.

[0205] In one possible implementation, the second determining module 13 is further configured to:

[0206] Multiply the distribution distance by the preset second hyperparameter to obtain the second product result;

[0207] The loss weights are obtained by calculating the function value with the natural constant as the base and the result of the second product as the exponent.

[0208] In one possible implementation, the second determining module 13 is further configured to:

[0209] The range of values ​​for the distribution distance is divided into at least two intervals by using at least one grading threshold.

[0210] Determine the target interval to which the distribution distance belongs;

[0211] The loss weight is determined based on the preset weight value corresponding to the target interval.

[0212] In one possible implementation, the second determining module 13 is further configured to include:

[0213] The loss term is obtained by multiplying the loss weights by the initial loss function.

[0214] The regularization term is determined based on the distribution distance, and the regularization term is positively correlated with the distribution distance;

[0215] The sum of the loss term and the regularization term is used to determine the target loss function.

[0216] In one possible implementation, the first processing module 14 is specifically used for:

[0217] Backpropagation is performed based on the target loss function to obtain the gradient information of at least one parameter of the preset model;

[0218] Update the parameters of the preset model based on gradient information and the preset learning rate.

[0219] In one possible implementation, the first determining module 12 is further configured to:

[0220] The linear projection matrix is ​​determined by the total number of features of the hidden state tensor and the number of columns by the total number of features of the real action sequence.

[0221] The hidden state tensor is flattened and multiplied by the transpose of the linear projection matrix to obtain the projected hidden state.

[0222] In one possible implementation, the model training device 10 is further used for:

[0223] Construct a pre-defined model, which includes a visual encoder, a language encoder, a multimodal fusion module, and an action decoder;

[0224] Acquire training samples, which include image data, language instruction data, and corresponding real action sequences.

[0225] In one possible implementation, the first acquisition module 11 is specifically used for:

[0226] Visual features are obtained by processing image data through a visual encoder.

[0227] Language features are obtained by processing language instruction data through a language encoder.

[0228] Visual and linguistic features are processed by a multimodal fusion module to obtain fused features;

[0229] The fused features are processed by an action decoder, and the hidden state tensor is extracted after the target layer in the action decoder.

[0230] The model training device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0231] Figure 5 A schematic diagram of the action sequence determination device provided in this application is shown below. Figure 5 As shown, the action sequence determination device 20 provided in this embodiment includes:

[0232] The second acquisition module 21 is used to acquire data to be processed, which includes image data and language instruction data.

[0233] The second processing module 22 is used to process the data to be processed through the target model to obtain an action sequence. The target model is obtained by training a preset model through a model training method, and the action sequence is used to control the target device to perform the target action.

[0234] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 30 provided in this embodiment includes at least one processor 31 and a memory 32. Optionally, the device 30 also includes a communication component 33. The processor 31, memory 32, and communication component 33 are connected via a bus.

[0235] In the specific implementation process, at least one processor 31 executes computer execution instructions stored in memory 32, causing at least one processor 31 to execute the above-mentioned model training method or action sequence determination method.

[0236] The specific implementation process of processor 31 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0237] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0238] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0239] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0240] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0241] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0242] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0243] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0244] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0245] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0246] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0247] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0248] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0249] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A model training method, characterized in that, include: The training samples are input into a preset model for forward propagation to obtain the hidden state tensor of the preset model, which is a visual language action model. Determine the distribution distance between the hidden state tensor and the actual action sequence; The target loss function is determined based on the distribution distance, wherein the larger the distribution distance, the larger the loss weight in the target loss function corresponding to the training sample; The parameters of the preset model are updated according to the target loss function to obtain the target model, which is used to determine the action sequence of the target device.

2. The method according to claim 1, characterized in that, Determining the distribution distance between the hidden state tensor and the real action sequence includes: Project the hidden state tensor onto the dimensional space of the real action sequence to obtain the projected hidden state; The distribution difference between the projected hidden state and the actual action sequence is calculated to obtain the distribution distance.

3. The method according to claim 2, characterized in that, The step of calculating the distribution difference between the projected hidden state and the real action sequence includes: calculating the maximum mean difference between the projected hidden state and the real action sequence using a kernel function, and using the maximum mean difference as the distribution difference.

4. The method according to claim 2, characterized in that, The calculation of the distribution difference between the projected hidden state and the actual action sequence includes: In terms of temporal sequence, the projected hidden state and the real action sequence are respectively fitted to Gaussian distributions; The KL divergence between the two Gaussian distributions is defined as the distribution difference.

5. The method according to claim 2, characterized in that, The calculation of the distribution difference between the projected hidden state and the real action sequence includes: calculating the minimum migration cost required to move the distribution of the projected hidden state to the distribution of the real action sequence, and using the minimum migration cost as the distribution difference.

6. The method according to claim 1, characterized in that, Determining the target loss function based on the distribution distance includes: The loss weight is determined based on the distribution distance; The target loss function is determined based on the loss weights and the initial loss function.

7. The method according to claim 6, characterized in that, Determining the loss weight based on the distribution distance includes: Multiply the distribution distance by a preset first hyperparameter to obtain the first product result; The loss weight is obtained by adding the first product result to a preset value.

8. The method according to claim 6, characterized in that, Determining the loss weight based on the distribution distance includes: Multiply the distribution distance by a preset second hyperparameter to obtain the second product result; The loss weight is obtained by calculating the function value with the natural constant as the base and the result of the second product as the exponent.

9. The method according to claim 6, characterized in that, Determining the loss weight based on the distribution distance includes: The range of values ​​for the distribution distance is divided into at least two intervals by using at least one grading threshold. Determine the target interval to which the distribution distance belongs; The loss weight is determined based on the preset weight value corresponding to the target interval.

10. The method according to claim 6, characterized in that, Determining the target loss function includes: The loss term is obtained by multiplying the loss weight by the initial loss function. A regularization term is determined based on the distribution distance, and the regularization term is positively correlated with the distribution distance; The sum of the loss term and the regularization term is determined as the target loss function.

11. The method according to claim 1, characterized in that, The step of updating the parameters of the preset model according to the target loss function includes: Backpropagation is performed based on the target loss function to obtain the gradient information of at least one parameter of the preset model; The parameters of the preset model are updated based on the gradient information and the preset learning rate.

12. The method according to claim 2, characterized in that, The step of projecting the hidden state tensor onto the dimensional space of the real action sequence includes: A linear projection matrix is ​​determined, wherein the number of rows of the linear projection matrix is ​​determined by the total number of features of the hidden state tensor, and the number of columns is determined by the total number of features of the real action sequence; The hidden state tensor is flattened and multiplied by the transpose of the linear projection matrix to obtain the projected hidden state.

13. The method according to claim 1, characterized in that, The method further includes: The preset model is constructed, which includes a visual encoder, a language encoder, a multimodal fusion module, and an action decoder; The training samples are obtained, which include image data, language instruction data, and corresponding real action sequences.

14. The method according to claim 13, characterized in that, The step of obtaining the hidden state tensor of the preset model includes: The image data is processed by the visual encoder to obtain visual features; The language instruction data is processed by the language encoder to obtain language features; The visual features and the language features are processed by the multimodal fusion module to obtain fused features; The fused features are processed by the action decoder, and the hidden state tensor is extracted after the target layer in the action decoder.

15. A method for determining an action sequence, characterized in that, include: Acquire data to be processed, which includes image data and language instruction data; The data to be processed is processed by the target model to obtain an action sequence, wherein the target model is obtained by training a preset model using the model training method described in any one of claims 1-14, and the action sequence is used to control the target device to perform the target action.

16. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-14, or causing the processor to perform the method as described in claim 15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-14, or, when executed by a processor, are used to implement the method as described in claim 15.

18. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method of any one of claims 1-14, or the computer program that, when executed by a processor, implements the method of claim 15.

Images