A training method and an inference method of a flow matching generation model and related devices

Abstract

Embodiments of the present application provide a training method and an inference method of a flow matching generation model and related devices, to improve the accuracy of the action sequence predicted by the trained flow matching generation model. The method of the embodiments of the present application comprises: obtaining noise, time t and environment features of a first action, wherein the environment features of the first action at least include observation values of the first action; inputting the noise, time t and environment features into an initialized flow matching generation model to obtain an output predicted velocity field vector of a conditional probability path of time t+1; calculating a loss between the predicted velocity field vector and a real velocity field vector by using a preset loss function, wherein the preset loss function includes at least one of a first loss function and a second loss function, and a third loss function; and training the initialized flow matching generation model by using the loss and a back propagation algorithm until the flow matching generation model converges, to obtain a trained flow matching generation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal data generation, in particular to a training method and an inference method of a flow matching generation model and related devices. BACKGROUND

[0002] In the field of embodied intelligence, existing generative strategies (visuomotor policy or vision-language-action policy) are all in the imitation learning paradigm. Under this paradigm, the strategy mainly predicts continuous actions based on observations. The core action generation module of the generative strategy is mainly transferred from image generation field algorithms.

[0003] However, action sequences are significantly different from ordinary pictures. High-frequency information in pictures is relatively important, so low-pass filters are often used, that is, only high-frequency information or full-frequency information is retained. However, in action sequences, the frequency component distribution within each action block changes dynamically during task execution.

[0004] The root of this change lies in the fact that a robot operation sequence is usually composed of alternating still and non-stationary motion stages, which generally include a low dynamic motion stage and a high dynamic motion stage. In the low dynamic motion stage, only part of the action dimension shows significant changes, and the remaining dimensions remain relatively smooth. In the high dynamic motion stage, high-frequency changes are more significant and information-rich. However, existing generative strategies either select actions in the low dynamic motion stage or select actions in the high dynamic motion stage, making it difficult to take into account all action information. SUMMARY

[0005] Embodiments of the present application provide a training method and an inference method of a flow matching generation model and related devices, which are used to adaptively select action states of different motion stages for learning during the training process of the flow matching generation model, thereby improving the accuracy of the predicted action sequence of the trained flow matching generation.

[0006] A first aspect of an embodiment of the present application provides a training method of a flow matching generation model, which comprises:

[0007] obtaining noise, a time t, and an environment feature of the first action, wherein the environment feature of the first action at least includes an observation value of the first action;

[0008] inputting the noise, the time t, and the environment feature into an initialized flow matching generation model to obtain a predicted velocity field vector of a conditional probability path at time t+1 output by the initialized flow matching generation model;

[0009] calculating a loss between the predicted velocity field vector and a real velocity field vector by using a preset loss function, wherein the preset loss function comprises at least one of a first loss function, a second loss function and a third loss function, the first loss function is used to adaptively constrain a difference between predicted velocity field vectors at any two time points between the time point t and the time point t+1 to be minimized, the second loss function is used to adaptively constrain predicted actions at a future u1 time point of actions starting from different time points between the time point t and the time point t+1 to converge to the same position, and the third loss function is used to constrain the loss between the predicted velocity field and the real velocity field to be minimized.

[0010] training the initialized flow matching generation model by using the loss and a back propagation algorithm until the flow matching generation model converges, so as to obtain a trained flow matching generation model.

[0011] As an optional embodiment, the method further comprises:

[0012] determining s1 and r1 in the any two time points;

[0013] obtaining a first predicted velocity field vector at the s1 time point;

[0014] obtaining a second predicted velocity field vector at the r1 time point;

[0015] The first loss function is specifically used to:

[0016] According to the difference between the first predicted velocity field vector and the second predicted velocity field vector, an adaptive weighting strategy is used to calculate the loss between the first predicted velocity field vector and the second predicted velocity field vector.

[0017] As an optional embodiment, the method further comprises:

[0018] determining s1 and r1 in the any two time points;

[0019] obtaining a first predicted action from the s1 time point to the u1 time point;

[0020] obtaining a second predicted action from the r1 time point to the u1 time point;

[0021] The second loss function is specifically used to:

[0022] According to a difference between the first predicted action and the second predicted action, a loss between the first predicted action and the second predicted action is calculated by using an adaptive weighting strategy.

[0023] As an optional embodiment, after obtaining the predicted velocity field, the method further comprises:

[0024] projecting the predicted velocity field vector into a frequency domain space to obtain a spectrum coefficient of the predicted velocity field vector;

[0025] The first loss function comprises a fourth loss function, and the fourth loss function is used for:

[0026] According to a difference between the predicted velocity field vectors of any two time points between the time point t and the time point t+1 in the frequency domain space, a difference between spectrum coefficients of the predicted velocity field vectors of the any two time points is adaptively constrained to be minimized;

[0027] The second loss function comprises a fifth loss function, and the fifth loss function is used for:

[0028] According to a difference between the predicted actions of the actions starting from different time points between the time point t and the time point t+1 in the future u1 time point in the frequency domain space, a consistency of spectrum coefficients of the predicted actions of the actions starting from the different time points in the future u1 time point is adaptively constrained.

[0029] As an optional embodiment, the method further comprises:

[0030] determining s1 time point and r1 time point in the any two time points;

[0031] obtaining a first predicted velocity field vector of the s1 time point;

[0032] obtaining a second predicted velocity field vector of the r1 time point;

[0033] converting the first predicted velocity field vector from a time domain to a frequency domain to obtain a first spectrum coefficient of the s1 time point;

[0034] converting the second predicted velocity field vector from the time domain to the frequency domain to obtain a second spectrum coefficient of the r1 time point;

[0035] The fourth loss function is specifically used for:

[0036] According to a difference between the first spectrum coefficient and the second spectrum coefficient, a loss between the first spectrum coefficient and the second spectrum coefficient is calculated by using an adaptive weighting strategy.

[0037] As an optional embodiment, the method further comprises:

[0038] determining any time s1 and r1 in the different time instants;

[0039] obtaining a first predicted action from the s1 time instant to the u1 time instant;

[0040] obtaining a second predicted action from the r time instant to the u1 time instant;

[0041] converting the first predicted action from the time domain to the frequency domain to obtain third spectral coefficients of the first predicted action from the s1 time instant to the u1 time instant;

[0042] converting the second predicted action from the time domain to the frequency domain to obtain fourth spectral coefficients of the second predicted action from the r1 time instant to the u1 time instant;

[0043] The fifth loss function is specifically used for:

[0044] According to the difference between the third spectral coefficients and the fourth spectral coefficients, an adaptive weighting strategy is used to calculate the loss between the third spectral coefficients and the fourth spectral coefficients.

[0045] As an optional embodiment, the initialized flow matching generation model includes an initialized flow matching one-step generation model.

[0046] The second aspect of the embodiments of the present application provides an inference method based on a flow matching generation model, and the method comprises:

[0047] obtaining an environment feature of noise, a time instant t and a first action, wherein the environment feature of the first action at least includes an observation value of the first action;

[0048] inputting the noise, the time instant t and the environment feature into the trained flow matching generation model to obtain a second action predicted by a conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector.

[0049] As an optional embodiment, if the trained flow matching generation model includes a trained flow matching one-step generation model;

[0050] The inputting the noise, the time instant t and the environment feature of the first action into the trained flow matching generation model to obtain the second action obtained by the trained flow matching generation model with the conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector, comprises:

[0051] input the noise, the time t and the environment feature of the first action to the trained flow matching one-step generation model to obtain a second action generated by the trained flow matching one-step generation model in a preset speed field vector prediction condition probability path one-step.

[0052] The third aspect of the embodiment of the application provides a flow matching generation model training device, and the device comprises:

[0053] An acquisition unit is configured to acquire noise, a time t and environment features of the first action, wherein the environment features of the first action at least comprise observation values of the action;

[0054] An input unit is configured to input the noise, the time t and the environment features to an initialized flow matching generation model to obtain a predicted speed field vector of a conditional probability path at a t+1 time output by the initialized flow matching generation model;

[0055] A calculation unit is configured to calculate a loss between the predicted speed field vector and a real speed field vector by using a preset loss function, wherein the preset loss function comprises at least one of a first loss function and a second loss function, and a third loss function, the first loss function is used to adaptively constrain a difference between predicted speed field vectors of any two times between the time t and the t+1 time to be minimized, the second loss function is used to adaptively constrain predicted actions of actions starting from different times between the time t and the t+1 time to converge to the same position at a future u1 time, and the third loss function is used to constrain the loss between the predicted speed field and the real speed field to be minimized;

[0056] A training unit is configured to train the initialized flow matching generation model by using the loss and a back propagation algorithm until the flow matching generation model converges to obtain a trained flow matching generation model.

[0057] The fourth aspect of the embodiment of the application provides a flow matching generation model-based inference device, and the device comprises:

[0058] An acquisition unit is configured to acquire noise, a time t and environment features of the first action, wherein the environment features of the first action at least comprise observation values of the first action;

[0059] a prediction unit configured to input the noise, the time t, and the environmental feature into the trained flow matching generation model provided in the first aspect of the present application to obtain a second action predicted by a conditional probability path predicted by the trained flow matching generation model at a preset speed field vector.

[0060] The fifth aspect of the present application provides a computer device, comprising a processor configured to implement the training method of the flow matching generation model provided in the first aspect of the present application or the inference method based on the flow matching generation model provided in the second aspect of the present application when executing a computer program stored in a memory.

[0061] The sixth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is configured to implement the training method of the flow matching generation model provided in the first aspect of the present application or the inference method based on the flow matching generation model provided in the second aspect of the present application when executed by a processor.

[0062] The seventh aspect of the present application provides a computer program product having a computer program stored thereon, wherein the computer program is configured to implement the training method of the flow matching generation model provided in the first aspect of the present application or the inference method based on the flow matching generation model provided in the second aspect of the present application when executed by a processor.

[0063] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

[0064] In the training process of the initialized flow matching generation model in the embodiments of the present application, not only the third loss function is used to calculate the loss between the predicted speed field and the real speed field, but also the first loss function is used to adaptively constrain the difference between the predicted speed field vectors at any two time points between time t and time t+1 to be minimized, and / or the second loss function is used to adaptively constrain the modal objects starting from different time points between time t and time t+1 to converge to the same position at future time u1 according to the difference between the predicted actions at future time u1 of the actions starting from different time points between time t and time t+1, so that the initialized flow matching generation model can adaptively learn the action state of different motion stages in the backward denoising training process, thereby improving the accuracy of the action sequence predicted by the initialized flow matching generation model. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 An embodiment schematic diagram of the training method of the flow matching generation model in the embodiments of the present application;

[0066] Figure 2 A schematic diagram of action state values in different motion stages in an embodiment of the present application;

[0067] Figure 3 An embodiment schematic diagram of a process of calculating loss by using a first loss function in an embodiment of the present application;

[0068] Figure 4 An embodiment schematic diagram of a process of calculating loss by using a second loss function in an embodiment of the present application;

[0069] Figure 5 Another embodiment schematic diagram of a training method of a flow matching generation model in an embodiment of the present application;

[0070] Figure 6 An embodiment schematic diagram of a process of calculating loss by using a fourth loss function in an embodiment of the present application;

[0071] Figure 7 An embodiment schematic diagram of a process of calculating loss by using a fifth loss function in an embodiment of the present application;

[0072] Figure 8 An embodiment schematic diagram of an inference method based on a flow matching generation model in an embodiment of the present application;

[0073] Figure 9 An embodiment schematic diagram of a training device of a flow matching generation model in an embodiment of the present application;

[0074] Figure 10 An embodiment schematic diagram of an inference device of a flow matching generation model in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The embodiments of the present application provide a training method and an inference method of a flow matching generation model and related devices, which are used for adaptively selecting action states in different motion stages for learning in a training process of the flow matching generation model, so as to improve the accuracy of an action sequence predicted by the trained flow matching generation.

[0076] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the protection scope of the present application.

[0077] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of the present application and above-described drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of such terms is interchangeable under appropriate circumstances such that the embodiments described herein are capable of operation in other sequences than described or illustrated herein. In addition, the terms "comprise", "comprising", "including", "having" and their conjugates, merit the inclusion of items that do not exclude the presence of other items. For example, the process, method, article, or apparatus that "comprises", "comprising", "including", "having" one step can comprise, among other steps, additional steps.

[0078] For the convenience of understanding, the training method of the flow matching generation model in the present application is described below. Please refer to Figure 1 An embodiment of the training method of the flow matching generation model in the embodiments of the present application comprises:

[0079] 101. Obtain noise, time t, and environment features of the first action, wherein the environment features of the first action at least include observation values of the first action;

[0080] In the existing generative learning strategy, especially in the training process of the backward inference of the flow matching generation model, either actions in the low dynamic motion stage or actions in the high dynamic motion stage are selected for learning, so that it is difficult to take into account all the action information.

[0081] To solve this problem, the present application adaptively learns actions in different motion stages to improve the accuracy of the predicted actions in the prediction stage.

[0082] Specifically, the training process of the flow matching generation model generally includes a forward noise adding process and a backward inference process. The forward noise adding process is used to gradually (i.e. multi-step) add noise to the real modal object until the real modal object evolves into a noise object satisfying a Gaussian distribution. The backward inference process is to sample from a noise object satisfying a Gaussian distribution, and then gradually denoise the sampled object to obtain the real modal object.

[0083] The forward noise adding process in the present application is similar to the forward noise adding process in the prior art, which will not be described here. The difference from the prior art is that the backward denoising process in the present application uses a loss function different from the prior art to improve the accuracy of the predicted object generated by the trained flow matching generation model of the present application.

[0084] Further, in the backward inference training process of the initialized flow matching generation model, noise is first obtained, and the environment feature of the first action is collected from the noise at time t. Here, time t can be regarded as the sampling time, and the adjacent two times are regarded as one step of denoising. For example, from time t to time t+1, it is regarded as the first step of denoising, and from time t+1 to time t+2, it is regarded as the second step of denoising. Here, the noise refers to a noise object obeying a Gaussian distribution, and the environment feature of the first action generally includes the action observation value at time t and the environment feature of the action observation value. The environment feature is generally related to the predicted action of the initialized flow matching generation model. For example, the environment feature can be a process video of the robot performing the action, or a process picture of the robot performing the action, etc. As long as the environment feature can help the robot improve the prediction accuracy of the action, it is within the protection scope of the present application.

[0085] 102. inputting the noise, the time t and the environment feature into the initialized flow matching generation model to obtain a predicted velocity field vector of a conditional probability path at time t+1 output by the initialized flow matching generation model;

[0086] After obtaining the noise, the time t and the environment feature of the first action, the noise, the time t and the environment feature of the first action are input into the initialized flow matching generation model to obtain a predicted velocity field of a conditional probability path at time t+1 output by the initialized flow matching generation model.

[0087] Here, the predicted velocity field refers to the predicted motion velocity (including the predicted velocity size and velocity direction) of the conditional probability path from time t to time t+1. Because in the backward denoising training process of the flow matching generation model, it is often expected that the flow matching generation model can recover from the noise object to the predicted action with straight-line motion, therefore in the present application, it is hoped that the predicted velocity field is a constant. Because only when the predicted velocity field is a constant, it can be said that the conditional probability path is a straight line.

[0088] 103. calculate a loss between the predicted velocity field vector and a real velocity field vector using a preset loss function, wherein the preset loss function comprises at least one of a first loss function, a second loss function and a third loss function, the first loss function is used to adaptively constrain a difference between predicted velocity field vectors of any two time points between the time point t and the time point t+1 to be minimized, the second loss function is used to adaptively constrain predicted actions of future u1 time points of actions starting from different time points between the time point t and the time point t+1 to converge to the same position, and the third loss function is used to constrain the loss between the predicted velocity field and the real velocity field to be minimized;

[0089] In the training process of backward denoising of the flow matching generation model, after obtaining the predicted velocity field vector output by the flow matching generation model, the loss between the predicted velocity field vector and the real velocity field vector is calculated, wherein the real velocity field is the difference between the first action at time t and the predicted second action at time t+1.

[0090] Different from the prior art, in addition to using the third loss function similar to the prior art to constrain the loss between the predicted velocity field vector and the real velocity field vector to be minimized, the present application also sets at least one of the first loss function and the second loss function to constrain the real velocity field, wherein the first loss function is used to adaptively constrain the difference between the predicted velocity field vectors of any two time points between the time point t and the time point t+1 to be minimized, and the second loss function is used to adaptively constrain the predicted actions of future u1 time points of actions starting from different time points between the time point t and the time point t+1 to converge to the same position.

[0091] For the convenience of understanding, Figure 2 The action state values of different motion stages are given, wherein the action state 6 belongs to the high dynamic motion stage, and the action states 0-5 belong to the low dynamic motion stage, and by setting at least one of the first loss function and the second loss function, the initialized flow matching generation model can adaptively learn the action states of different dynamic motion stages, thereby improving the accuracy of the action sequence predicted by the trained flow matching generation model.

[0092] Because the first loss function in the present application is to adaptively constrain the difference between the predicted velocity field vectors of any two time points to be minimized according to the difference between the predicted velocity field vectors of any two time points when constraining the predicted velocity field vectors of any two time points, that is, the user does not need to manually add a filter to select the motion state of different time points (here, different time points generally include high dynamic motion stage and low dynamic motion stage), thereby improving the accuracy of the predicted motion sequence, and the second loss function is to adaptively constrain the predicted motion of the action starting from different time points between time t and time t+1 to converge to the same position at future u1 time according to the difference between the predicted motions of the action starting from different time points at future u1 time, and the user does not need to add a filter according to the motion stage to select different motion states, further improving the accuracy of the predicted motion sequence.

[0093] For the process of how to adaptively constrain in the embodiments of the present application, it will be described in the following embodiments, which will not be repeated here.

[0094] 104, using the loss and back propagation algorithm, training the initialized flow matching generation model until the flow matching generation model converges to obtain the trained flow matching generation model.

[0095] After obtaining the loss calculated by the first loss function and / or the loss calculated by the second loss function and the loss calculated by the third loss function in step 103, the loss and back propagation algorithm is used to train the initialized flow matching generation model until the flow matching generation model converges to obtain the trained flow matching generation model.

[0096] In the training process of the initialized flow matching generation model in the embodiments of the present application, not only the third loss function is used to calculate the loss between the predicted velocity field and the real velocity field, but also the first loss function is used to adaptively constrain the difference between the predicted velocity field vectors of any two time points between time t and t+1 to be minimized according to the difference between the predicted velocity field vectors of any two time points, and / or the second loss function is used to adaptively constrain the predicted motion of the modal object starting from different time points between time t and t+1 to converge to the same position at future u1 time according to the difference between the predicted motions of the action starting from different time points at future u1 time, so that the initialized flow matching generation model can adaptively learn the motion state of different motion stages in the backward denoising training process, thereby improving the accuracy of the predicted motion sequence of the initialized flow matching generation model.

[0097] Based on Figure 1The embodiments described above, the following process for calculating the loss using the first loss function, please refer to Figure 3 :

[0098] 301, determine the s1 time and r1 time in the two time points;

[0099] Specifically, the present application needs to determine any two time points between t time and t+1 time, assuming s1 time and r1 time are any two time points between t time and t+1 time.

[0100] 302, obtain the first predicted velocity field vector of s1 time;

[0101] After determining s1 time, the first predicted velocity field vector of s1 time is obtained, wherein when obtaining the first predicted velocity field vector of s time, the action state generated at s time, s time and the environment features of s time action state are input into the initialized flow matching generation model, and then the first predicted velocity field vector of s time output by the flow matching generation model can be obtained.

[0102] 303, obtain the second predicted velocity field vector of r1 time;

[0103] Similarly, the present application can also obtain the second predicted velocity field vector of r1 time.

[0104] 304, according to the difference between the first predicted velocity field vector and the second predicted velocity field vector, the adaptive weighting strategy is used to calculate the loss between the first predicted velocity field vector and the second predicted velocity field vector.

[0105] After obtaining the first predicted velocity field vector and the second predicted velocity field vector, the present application then calculates the loss between the first predicted velocity field vector and the second predicted velocity field vector according to the difference between the first predicted velocity field vector and the second predicted velocity field vector, and uses the adaptive weighting strategy, wherein when calculating the loss between the first predicted velocity field vector and the second predicted velocity field vector, L1 loss function can be used for calculation, or L2 loss function can be used for calculation, etc., which is not limited here.

[0106] The following formula 1 and formula 2 give an example of the first loss function, wherein the first loss function calculates the expected value of L2 loss between the first predicted velocity field vector and the second predicted velocity field vector.

[0107]

[0108] In formula 1, r, s represent any two time points collected from normalized time, a ra(r, D) represents the action collected from the noise D at the r moment s a(s, D) represents the action collected from the noise D at the s moment, and v θ (s, a s ) represents the first predicted velocity field vector at the s moment θ (r, a r ) represents the second predicted velocity field vector at the r moment, and W k represents the weighting coefficient of the loss between the first predicted velocity field vector of each dimension and the second predicted velocity field vector of each dimension.

[0109] In formula 2, j belongs to the dimension of the predicted velocity field vector, assuming that the predicted velocity field vector is a one-row five-column vector, then the value of j is 5.

[0110] In the embodiments of the present application, the calculation process of the first loss function is described in detail, and the first loss function is designed to predict the velocity field between the s1 moment and the r1 moment between the t moment and the t+1 moment, so that the loss between the predicted velocity field vectors of any two moments is adaptively calculated according to the difference between the predicted velocity field vectors of any two moments, so that the initialized flow matching generation model can learn the predicted velocity field vector of the action state of different motion stages, thereby improving the accuracy of the action sequence predicted by the trained flow matching generation model.

[0111] Next, the process of calculating the loss by the second loss function in the embodiments of the present application is described, please refer to Figure 4 :

[0112] 401, determining any moment s1 and r1 in different moments;

[0113] In the embodiments of the present application, different moments need to be determined between the t moment and the t+1 moment, for the convenience of description, the s1 moment and the r1 moment between the t moment and the t+1 moment are also determined in the embodiments of the present application.

[0114] 402, obtaining the first predicted action from the s1 moment to the u1 moment;

[0115] After determining the s1 moment, the u1 moment lagging behind the s1 moment is determined between the t moment and the t+1 moment, in order to obtain the first predicted action generated by the initialized flow matching generation model from the s1 moment to the u1 moment, the modal object generated at the s1 moment, the environment features of the s1 moment and the s1 moment action are input into the initialized flow matching generation model, then the first predicted action of the s1 moment output by the flow matching generation model can be obtained.

[0116] 403、obtain a second predicted action from r1 time to u1 time;

[0117] Similar to step 302, the application can also obtain a second predicted action from r1 time to u1 time.

[0118] 404、According to the difference between the first predicted action and the second predicted action, an adaptive weighting strategy is used to calculate the loss between the first predicted action and the second predicted action.

[0119] In the above steps, after obtaining the first predicted action and the second predicted action, the difference between the first predicted action and the second predicted action is further used to calculate the loss between the first predicted action and the second predicted action using an adaptive weighting strategy. In calculating the loss, an L1 loss function or an L2 loss function can also be used. The type of the second loss function for calculating the loss is not limited here.

[0120] For convenience of understanding, the following gives an example of the second loss function in formula 3 and formula 4, wherein the second loss function calculates the expected value of the L2 loss between the first predicted action and the second predicted action.

[0121]

[0122] In formula 3, r1, s1, and u1 represent three time points randomly collected from normalized time points, a r1 represents the action collected from noise D at r1 time, a s1 represents the action collected from noise D at s1 time, a u1 represents the action collected from noise D at u1 time, and v θ (s1, a s1 ) represents the first predicted velocity field vector at s1 time, v θ (s1, a r1 ) represents the second predicted velocity field vector at r1 time, and (a s1 +(u1-s1)v θ (s1, a s1 ) represents the first predicted action vector, and (a r1 +(u1-r1)v θ (s1, a r1 ) represents the second predicted action vector.

[0123] In formula 4, j belongs to the dimension of the predicted velocity field vector. Assuming that the predicted velocity field vector is a one-row five-column vector, the value of j is 5.

[0124] In this embodiment, the calculation process of the second loss function is described in detail. The second loss function calculates the loss between the first predicted action vector and the second predicted action vector based on the difference between any time from time t and time t+1 to the future time u1 using an adaptive weighting strategy. This allows the initialized flow matching generation model to learn the action states at different motion stages, thereby improving the accuracy of the action sequences predicted by the trained flow matching generation model.

[0125] The above embodiments, when predicting actions, do not consider the characteristics of the actions in the frequency domain. To address this issue, this application embodiment can further perform the following steps to adaptively constrain the first and second predicted velocity field vectors in the frequency domain. Please refer to [link to relevant documentation]. Figure 5 :

[0126] 501. Obtain noise, time t, and environmental features of the first action, wherein the environmental features of the first action include at least the observed values ​​of the action;

[0127] 502. Input the noise, the time t, and the environmental features into the initialized flow matching generation model to obtain the predicted velocity field vector of the conditional probability path at time t+1 output by the initialized flow matching generation model.

[0128] Steps 501 to 502 in the embodiments of this application are... Figure 1 The steps 101 to 102 in the embodiments are described similarly and will not be repeated here.

[0129] 503. Project the predicted velocity field vector into the frequency domain space to obtain the spectral coefficients of the predicted velocity field vector;

[0130] To improve the temporal consistency of the predicted actions, this embodiment further projects the predicted velocity field to the frequency domain, that is, transforms the predicted velocity field generated in step 502 from the time domain to the frequency domain. In the process of transformation, Fourier transform formula or discrete cosine transform (DCT) can be used. Here, there are no specific restrictions on the process of transforming from the time domain to the frequency domain.

[0131] For ease of understanding, this application uses Discrete Cosine Transform (DCT) as an example to describe the process of converting the predicted velocity field from the time domain to the frequency domain using Equation 5:

[0132]

[0133] In Equation 5, k represents the dimension of the predicted velocity field vector.

[0134] 504、calculating a loss between the predicted velocity field vector and the real velocity field vector by using a preset loss function, wherein the preset loss function comprises at least one of a fourth loss function and a fifth loss function, and a third loss function, the fourth loss function is used to adaptively constrain the difference between the spectral coefficients of the predicted velocity field vectors at any two time points between time t and time t+1 to be minimized according to the difference between the predicted velocity field vectors at the any two time points in the frequency domain space, the fifth loss function is used to adaptively constrain the consistency of the spectral coefficients of the predicted actions at future time u1 of the actions starting from different time points between time t and time t+1 according to the difference between the predicted actions at future time u1 of the actions starting from the different time points in the frequency domain space, and the third loss function is used to constrain the loss between the predicted velocity field and the real velocity field to be minimized;

[0135] After the predicted velocity field vector is projected into the frequency domain space, the third loss function in the present application still functions to constrain the loss between the predicted velocity field and the real velocity field to be minimized, and the first loss function corresponds to the fourth loss function in the present application, and the second loss function corresponds to the fifth loss function in the present application, wherein the fourth loss function is used to adaptively constrain the difference between the spectral coefficients of the predicted velocity field vectors at any two time points between time t and time t+1 to be minimized according to the difference between the predicted velocity field vectors at the any two time points in the frequency domain space, and the fifth loss function is used to adaptively constrain the consistency of the spectral coefficients of the predicted actions at future time u1 of the actions starting from different time points between time t and time t+1 according to the difference between the predicted actions at future time u1 of the actions starting from the different time points in the frequency domain space.

[0136] The calculation process of the fourth loss function and the fifth loss function will be described in the following embodiments, and will not be described here.

[0137] 505、using the loss and a back propagation algorithm to train the initialized flow matching generation model until the flow matching generation model converges, so as to obtain a trained flow matching generation model.

[0138] After the loss is calculated in step 504 according to the fourth loss function, the fifth loss function and the third loss function, the initialized flow matching generation model is trained by using the loss and a back propagation algorithm until the flow matching generation model converges, so as to obtain a trained flow matching generation model.

[0139] The fourth loss function is used to calculate the loss, the velocity field vectors at any two time points are converted from the time domain to the frequency domain to obtain the frequency spectrum coefficients corresponding to the velocity field vectors at the any two time points, and then the difference between the frequency spectrum coefficients at the any two time points is used to adaptively constrain the minimization of the difference between the frequency spectrum coefficients of the predicted velocity field vectors at the any two time points, so that the initialized flow matching generation model can learn the frequency spectrum coefficients of the predicted velocity field of the action state in different motion stages, and the fifth loss function in the application is used to calculate the loss, the predicted action of the action starting from different time points at the future u1 time point is converted from the time domain to the frequency domain, and the consistency of the frequency spectrum coefficients of the predicted action of the action starting from different time points at the future u1 time point is adaptively constrained according to the difference between the frequency spectrum coefficients of the predicted action of the action starting from different time points at the future u1 time point, so that the initialized flow matching generation model can learn the frequency spectrum coefficients of the action state in different motion stages, thereby further improving the consistency of the generated predicted action sequence in time sequence on the basis of improving the accuracy of the predicted action sequence of the trained flow matching generation model.

[0140] Based on the embodiments of the application, the process of calculating the loss by using the fourth loss function is described below, please refer to Figure 5 Figure 6

[0141] 601, determining s1 time and r1 time in the any two time points;

[0142] 602, obtaining a first predicted velocity field vector at the s1 time;

[0143] 603, obtaining a second predicted velocity field vector at the r1 time;

[0144] It should be noted that the steps 601 to 603 in the embodiments of the application are similar to the description of the steps 301 to 303 in the embodiments of the application, and will not be described here. Figure 3

[0145] 604, converting the first predicted velocity field vector from the time domain to the frequency domain to obtain the first frequency spectrum coefficient at the s1 time;

[0146] In order to further improve the continuity of the predicted action in time and improve the consistency of the generated action sequence in time sequence, the first predicted velocity field vector is converted from the time domain to the frequency domain to obtain the first frequency spectrum coefficient at the s1 time, wherein the conversion process of the first frequency spectrum coefficient can refer to formula 5, and for the convenience of description, the first frequency spectrum coefficient at the s1 time is F(v s1 ).

[0147] ​​​605、convert the second predicted velocity field vector from the time domain to the frequency domain to obtain the second spectral coefficient at r1;

[0148] Similarly, the second predicted velocity field vector is also converted from the time domain to the frequency domain to obtain the second spectral coefficient at r in the embodiment of the application. For the convenience of description, the second spectral coefficient at r1is also recorded as F(v r1 ).

[0149] 606、According to the difference between the first spectral coefficient and the second spectral coefficient, an adaptive weighting strategy is used to calculate the loss between the first spectral coefficient and the second spectral coefficient.

[0150] After obtaining the first spectral coefficient and the second spectral coefficient, the embodiment of the application further calculates the loss between the first spectral coefficient and the second spectral coefficient according to the difference between the first spectral coefficient and the second spectral coefficient by using an adaptive weighting strategy.

[0151] For the convenience of understanding, the process of calculating the loss between the first spectral coefficient and the second spectral coefficient by using an adaptive weighting strategy is described below by taking Formula 6 and Formula 7 as examples:

[0152]

[0153] Sim1(v r1 ,v s1 )=Sim1(v θ (s1,a s1 ),v θ (r1,a r1 ))=W k ·||F(v r1 )-F(v s1 )| 2 (Formula 7)

[0154] In Formula 7, the calculation formula of W k can be referred to Formula 4, which will not be described here.

[0155] In the embodiment of the application, the calculation process of the fourth loss function is described in detail, and the fourth loss function adaptively constrains the spectral coefficients of the predicted velocity fields at any two time points s and t by calculating the spectral coefficients of the predicted velocity fields at any two time points s and t and according to the difference between the spectral coefficients of the predicted velocity fields at any two time points s and t, so that the initialized flow matching generation model can adaptively learn the frequency domain information of the velocity field vectors of the predicted actions in different motion stages, thereby further improving the consistency of the generated predicted action sequence in time sequence on the basis of improving the accuracy of the predicted action sequence predicted by the trained flow matching generation model.

[0156] Based on the embodiments of Figure 5 , the process of calculating the loss by using the fifth loss function is described as follows, please refer to Figure 7 :

[0157] 701、Determine any time s1 and r1 in the different time;

[0158] 702、Obtain the first predicted action from the s1 time to the u1 time;

[0159] 703、Obtain the second predicted action from the r time to the u1 time;

[0160] It should be noted that the steps 701 to 703 in the embodiments of the present application are similar to the description of the steps 401 to 403 in the embodiments of the present application, and will not be described here. Figure 4

[0161] 704、Convert the first predicted action from the time domain to the frequency domain to obtain the third spectral coefficient of the first predicted action from the s1 time to the u1 time;

[0162] Here, the process of converting the first predicted action from the time domain to the frequency domain can refer to formula 5, for the convenience of description, the third spectral coefficient from the s1 time to the u1 time is recorded as F(v(a s1 +(u1-s1)v θ (s1,a s1 ))),wherein a s1 represents the action at s1 time, a s1 +(u1-s1)v θ (s1,a s1 ) represents the predicted action at u1 time from the s1 time to the u1 time.

[0163] 705、Convert the second predicted action from the time domain to the frequency domain to obtain the fourth spectral coefficient of the second predicted action from the r1 time to the u1 time;

[0164] Similarly, the embodiments of the present application can also convert the second predicted action from the time domain to the frequency domain to obtain the fourth spectral coefficient of the second predicted action from the r1 time to the u1 time, for the convenience of description, the third spectral coefficient from the r1 time to the u1 time is recorded as F(v(a r1 +(u1-r1)v θ (r1,a r1 ))),wherein a r1 represents the action at r1 time, a r1 +(u1-r1)v​θ (r1,a r1 ) from r1 to u1, the predicted action at u1.

[0165] 706、According to the difference between the third spectral coefficient and the fourth spectral coefficient, an adaptive weighting strategy is used to calculate the loss between the third spectral coefficient and the fourth spectral coefficient.

[0166] After obtaining the third spectral coefficient and the fourth spectral coefficient, then further according to the third spectral coefficient and the fourth spectral coefficient, an adaptive weighting strategy is used to calculate the loss between the third spectral coefficient and the fourth spectral coefficient, wherein the calculation of the loss can be L1 function or L2 function, and here the calculation process of the loss is not limited.

[0167] For the convenience of understanding, the following takes formula 8 and formula 9 as an example to describe the process of calculating the loss between the third spectral coefficient and the fourth spectral coefficient:

[0168]

[0169] Sim1(v(a s1 +(u1-s1)v θ (s1,a s1 ),v(a r1 +(u1-r1)v θ (r1,a r1 ))=

[0170] W k ·||F(v(a s1 +(u1-s1)v θ (s1,a s1 ))-F(v(a r1 +(u1-r1)v θ (r1,a r1 ))|| 2 (formula 9)

[0171] W k The calculation process can refer to formula 4, and will not be described here.

[0172] In the embodiments of the present application, the calculation process of the fifth loss function is described in detail, and the fifth loss function adaptively constrains the consistency of the third spectral coefficient and the fourth spectral coefficient from the arbitrary time to the u1 moment according to the difference between the first predicted action and the second predicted action from the arbitrary time to the u1 moment, so as to ensure that the initialized flow matching generation model can adaptively learn the frequency domain information of the action state in different motion stages, thereby further improving the consistency of the generated predicted action sequence in time sequence on the basis of improving the accuracy of the predicted action sequence of the trained flow matching generation model.

[0173] Based on Figures 1 to 7 In the embodiments, because the action state in different motion stages (such as low dynamic motion stage and high dynamic motion stage) is adaptively learned in the backward denoising training process, the initialized flow matching generation model in the embodiments is generally trained by using a multi-step method, and in order to further improve the training speed of the initialized flow matching generation model, the initialized flow matching generation model can be improved to an initialized flow matching one-step generation model, so that the real action state can be recovered from the noise in one step in the backward denoising training process, thereby further improving the efficiency of obtaining the real action state.

[0174] The training process of the flow matching generation model in the embodiments of the present application is described above, and the inference process of the flow matching generation model in the embodiments of the present application is described as follows. Figure 8 One embodiment of the inference method of the flow matching generation model in the embodiments of the present application comprises:

[0175] 801, obtain noise, time t and environment characteristics of a first action, wherein the environment characteristics of the first action at least include observation values of the first action;

[0176] Specifically, step 801 is similar to step 101 described above, and will not be described here.

[0177] 802, input the noise, the time t and the environment characteristics into the trained flow matching generation model to obtain a second action predicted by a conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector.

[0178] After obtaining the noise, the time t and the environment characteristics of the first action in step 801, the noise, the time t and the environment characteristics of the first action are input into the flow matching generation model trained according to the above method embodiments to obtain a second action predicted by a conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector.

[0179] After obtaining the trained flow matching generation model, the noise, the time t, and the environment feature of the first action are input into the trained flow matching generation model, and then the second action predicted by the conditional probability path predicted by the flow matching generation model according to the preset prediction speed field vector can be obtained. Furthermore, because the flow matching generation model is initialized and adaptively learns the action state of different movement stages in the training process, the trained flow matching generation model in the embodiment of the present application can improve the accuracy of the predicted action sequence.

[0180] Based on Figure 8 When the flow matching generation model is a one-step flow matching generation model, the noise, the time t, and the environment feature of the first action are input into the trained one-step flow matching generation model, and then the second action predicted by the trained one-step flow matching generation model can be obtained.

[0181] The embodiment of the present application also provides a computer program product, which stores a computer program. When the computer program is executed by a processor, the computer program is used to implement the training method of the flow matching generation model provided by the method embodiments of the present application, or is used to implement the inference method based on the flow matching generation model provided by the method embodiments of the present application.

[0182] The training process of the flow matching generation model and the inference process based on the flow matching generation model in the embodiment of the present application are described in detail above. Next, the training device of the flow matching generation model in the embodiment of the present application is described. Please refer to Figure 9 An embodiment of the training device of the flow matching generation model in the embodiment of the present application comprises:

[0183] The acquisition unit 901 is configured to acquire noise, a time t, and an environment feature of a first action, wherein the environment feature of the first action at least comprises an observation value of the action.

[0184] The input unit 902 is configured to input the noise, the time t, and the environment feature into an initialized flow matching generation model, so as to obtain a prediction speed field vector of a conditional probability path at a time t+1 output by the initialized flow matching generation model.

[0185] The calculation unit 903 is configured to calculate a loss between the predicted velocity field vector and the real velocity field vector by using a preset loss function, wherein the preset loss function comprises at least one of a first loss function, a second loss function and a third loss function, the first loss function is used to adaptively constrain a difference between predicted velocity field vectors at any two time points between the time point t and the time point t+1 to be minimized, the second loss function is used to adaptively constrain predicted actions at a future u1 time point of actions starting from different time points between the time point t and the time point t+1 to converge to the same position, and the third loss function is used to constrain the loss between the predicted velocity field and the real velocity field to be minimized.

[0186] The training unit 904 is configured to train the initialized flow matching generation model by using the loss and a back propagation algorithm until the flow matching generation model converges, so as to obtain a trained flow matching generation model.

[0187] As an optional embodiment, the acquisition unit 901 is further configured to:

[0188] determine s1 and r1 in the any two time points;

[0189] acquire a first predicted velocity field vector at the s1 time point;

[0190] acquire a second predicted velocity field vector at the r1 time point;

[0191] The calculation unit 903 is specifically configured to:

[0192] adopt an adaptive weighting strategy to calculate a loss between the first predicted velocity field vector and the second predicted velocity field vector according to a difference between the first predicted velocity field vector and the second predicted velocity field vector.

[0193] As an optional embodiment, the acquisition unit 901 is further configured to:

[0194] determine s1 and r1 in the any two time points;

[0195] acquire a first predicted action when transferring from the s1 time point to the u1 time point;

[0196] acquire a second predicted action when transferring from the r1 time point to the u1 time point;

[0197] The calculation unit 903 is specifically configured to:

[0198] According to a difference between the first predicted action and the second predicted action, a loss between the first predicted action and the second predicted action is calculated by using an adaptive weighting strategy.

[0199] As an optional embodiment, the calculation unit 903 is further configured to:

[0200] projecting the predicted velocity field vector into a frequency domain space to obtain a spectrum coefficient of the predicted velocity field vector;

[0201] The first loss function comprises a fourth loss function, and the fourth loss function is used for:

[0202] According to a difference between the predicted velocity field vectors of any two time points between the time point t and the time point t+1 in the frequency domain space, a difference between the spectrum coefficients of the predicted velocity field vectors of the any two time points is adaptively constrained to be minimized;

[0203] The second loss function comprises a fifth loss function, and the fifth loss function is used for:

[0204] According to a difference between the predicted actions of the actions starting from different time points between the time point t and the time point t+1 in the future u1 time point in the frequency domain space, a consistency of the spectrum coefficients of the predicted actions of the actions starting from the different time points in the future u1 time point is adaptively constrained.

[0205] As an optional embodiment, the acquisition unit 901 is further configured to:

[0206] determining s1 time and r1 time in the any two time points;

[0207] acquiring a first predicted velocity field vector of the s1 time;

[0208] acquiring a second predicted velocity field vector of the r1 time;

[0209] The calculation unit 903 is further configured to:

[0210] convert the first predicted velocity field vector from a time domain to a frequency domain to obtain a first spectrum coefficient of the s1 time;

[0211] convert the second predicted velocity field vector from the time domain to the frequency domain to obtain a second spectrum coefficient of the r1 time;

[0212] According to a difference between the first spectrum coefficient and the second spectrum coefficient, a loss between the first spectrum coefficient and the second spectrum coefficient is calculated by using an adaptive weighting strategy.

[0213] As an optional embodiment, the acquisition unit 901 is further configured to:

[0214] determining any time s1 and r1 in the different time instants;

[0215] obtaining a first predicted action from the s1 time instant to the u1 time instant;

[0216] obtaining a second predicted action from the r time instant to the u1 time instant;

[0217] The calculation unit 903 is further configured to:

[0218] convert the first predicted action from the time domain to the frequency domain to obtain third spectral coefficients of the first predicted action from the s1 time instant to the u1 time instant;

[0219] convert the second predicted action from the time domain to the frequency domain to obtain fourth spectral coefficients of the second predicted action from the r1 time instant to the u1 time instant;

[0220] According to the difference between the third spectral coefficients and the fourth spectral coefficients, an adaptive weighting strategy is used to calculate the loss between the third spectral coefficients and the fourth spectral coefficients.

[0221] As an optional embodiment, the initialized flow matching generation model includes an initialized flow matching one-step generation model.

[0222] It should be noted that the functions of the above-mentioned units are similar to those described in the corresponding method embodiments, which will not be described here.

[0223] In the training process of the initialized flow matching generation model, in the training process of the initialized flow matching generation model, not only the calculation unit 903 calculates the loss between the predicted velocity field and the real velocity field by using the existing third loss function, but also the first loss function is used to adaptively constrain the difference between the predicted velocity field vectors of any two time instants between time instants t and t+1 to be minimized, and / or the second loss function is used to adaptively constrain the modal objects starting from different time instants between time instants t and t+1 to converge to the same position at future u1 time instant according to the difference between the predicted actions of the actions starting from different time instants between time instants t and t+1 at future u1 time instant, so that the initialized flow matching generation model can adaptively learn the action state of different motion stages in the backward denoising training process, thereby improving the accuracy of the action sequence predicted by the initialized flow matching generation model.

[0224] Next, the inference device based on the flow matching generation model in the embodiments of the present application will be described. Please refer to Figure 10 An embodiment of the inference device based on the flow matching generation model in the embodiments of the present application comprises:

[0225] an obtaining unit 1001, configured to obtain noise, a time t, and an environment feature of a first action, wherein the environment feature of the first action at least comprises an observation value of the first action;

[0226] a prediction unit 1002, configured to input the noise, the time t, and the environment feature into a trained flow matching generation model, to obtain a second action predicted by a conditional probability path predicted by the trained flow matching generation model at a preset speed field vector.

[0227] As an optional embodiment, if the trained flow matching generation model comprises a trained flow matching one-step generation model;

[0228] the prediction unit 1002 is specifically configured to:

[0229] input the noise, the time t, and the environment feature of the first action into the trained flow matching one-step generation model, to obtain the second action generated by a conditional probability one-step path predicted by the trained flow matching one-step generation model at a preset speed field vector.

[0230] It should be noted that the functions of the above units are similar to those described in the corresponding method embodiments, and will not be described here.

[0231] After obtaining the trained flow matching generation model, the noise, the time t, and the environment feature of the first action are input into the trained flow matching generation model, and then the second action predicted by the conditional probability path predicted by the flow matching generation model at a preset prediction speed field vector can be obtained by the prediction unit 1002. Furthermore, because the flow matching generation model initialized in the training process learns the action states of different movement stages adaptively, the trained flow matching generation model in the embodiment can improve the accuracy of the predicted action sequence.

[0232] The above describes the embodiment of the present application from the perspective of modular functional entities. The computer device in the embodiment of the present application is described from the perspective of hardware processing as follows:

[0233] The computer device is configured to implement the functions of the training device side of the flow matching generation model. One embodiment of the computer device in the embodiment of the present application comprises:

[0234] a processor and a memory;

[0235] The memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that each step in the above training method of the flow matching generation model can be implemented.

[0236] The computer device is also used to implement the function of the inference device side based on the flow matching generation model. In an embodiment of the present application, the computer device includes:

[0237] a processor and a memory;

[0238] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that each step of the inference method based on the flow matching generation model can be implemented.

[0239] It can be understood that when the processor in the computer device executes the computer program, the functions of each unit in the corresponding device embodiment described above can also be implemented, which will not be described here. For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments that can complete a specific function, which are used to describe the execution process of the computer program in the flow matching generation model training device / inference device based on the flow matching generation model. For example, the computer program can be divided into each unit in the flow matching generation model training device, and each unit can implement the specific functions as described above in the corresponding flow matching generation model training device.

[0240] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing devices. The computer device can include but is not limited to a processor and a memory. Those skilled in the art can understand that the processor and the memory are only examples of the computer device, and do not constitute a limitation on the computer device, and can include more or fewer components, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0241] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like, which is a control center of the computer device and connects all parts of the computer device through various interfaces and lines.

[0242] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer device by running or executing the computer program and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, and the like; and the data storage area can store data created according to the use of the terminal and the like. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0243] The application further provides a computer readable storage medium for implementing the function of the training device of the flow matching generation model, and the computer readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the processor can be used to implement each step in the training method of the flow matching generation model.

[0244] The application further provides another computer readable storage medium for implementing the function of the inference device based on the flow matching generation model, and the computer readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the processor can be used to implement each step in the inference method based on the flow matching generation model.

[0245] It can be understood that the integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a corresponding computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned corresponding embodiment methods can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms. The computer readable medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0246] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, another division mode can be used. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0247] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0248] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0249] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced by equivalent replacements; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A training method for a flow matching generation model, characterized in that, The method includes: Acquire noise, time t, and environmental features of the first action, wherein the environmental features of the first action include at least the observations of the first action; The noise, the time t, and the environmental features are input into the initialized flow matching generation model to obtain the predicted velocity field vector of the conditional probability path at time t+1 output by the initialized flow matching generation model. The loss between the predicted velocity field vector and the true velocity field vector is calculated using a preset loss function. The preset loss function includes at least one of a first loss function and a second loss function, and a third loss function. The first loss function is used to adaptively constrain the difference between the predicted velocity field vectors at any two times between time t and time t+1 to minimize the difference. The second loss function is used to adaptively constrain the predicted actions at the future time u1 to converge to the same position based on the difference between the predicted actions at different times between time t and time t+1. The third loss function is used to constrain the loss between the predicted velocity field and the true velocity field to minimize the difference. The initial flow matching generation model is trained using the loss and backpropagation algorithm until it converges, thus obtaining the trained flow matching generation model.

2. The training method according to claim 1, characterized in that, The method further includes: Determine time s1 and time r1 from any two time points; Obtain the first predicted velocity field vector at time s1; Obtain the second predicted velocity field vector at time r1; The first loss function is specifically used for: Based on the difference between the first predicted velocity field vector and the second predicted velocity field vector, an adaptive weighting strategy is used to calculate the loss between the first predicted velocity field vector and the second predicted velocity field vector.

3. The training method according to claim 1, characterized in that, The method further includes: Determine any two times, s1 and r1, among the different times; Obtain the first predicted action when transitioning from time s1 to time u1; Obtain the second predicted action when transitioning from time r1 to time u1; The second loss function is specifically used for: Based on the difference between the first prediction action and the second prediction action, an adaptive weighting strategy is used to calculate the loss between the first prediction action and the second prediction action.

4. The training method according to claim 1, characterized in that, After obtaining the predicted velocity field, the method further includes: The predicted velocity field vector is projected into the frequency domain to obtain the spectral coefficients of the predicted velocity field vector. The first loss function includes a fourth loss function, which is used for: Based on the difference between the predicted velocity field vectors at any two times between time t and time t+1 in the frequency domain space, the difference in the spectral coefficients between the predicted velocity field vectors at any two times is adaptively constrained to be minimized. The second loss function includes a fifth loss function, which is used for: Based on the differences in the frequency domain space between actions starting from different times between time t and time t+1, the consistency of the spectral coefficients of the predicted actions at future time u1 is adaptively constrained.

5. The method according to claim 4, characterized in that, The method further includes: Determine time s1 and time r1 from any two time points; Obtain the first predicted velocity field vector at time s1; Obtain the second predicted velocity field vector at time r1; The first predicted velocity field vector is converted from the time domain to the frequency domain to obtain the first spectral coefficients at time s1. The second predicted velocity field vector is converted from the time domain to the frequency domain to obtain the second spectral coefficients at time r1; The fourth loss function is specifically used for: Based on the difference between the first spectral coefficient and the second spectral coefficient, an adaptive weighting strategy is used to calculate the loss between the first spectral coefficient and the second spectral coefficient.

6. The method according to claim 4, characterized in that, The method further includes: Determine any two times, s1 and r1, among the different times; Obtain the first predicted action that transitions from time s1 to time u1; Obtain the second predicted action when transitioning from time r1 to time u1; The first prediction action is transformed from the time domain to the frequency domain to obtain the third spectral coefficients of the first prediction action when it is transferred from time s1 to time u1; The second prediction action is transformed from the time domain to the frequency domain to obtain the fourth spectral coefficient of the second prediction action when it transitions from time r1 to time u1; The fifth loss function is specifically used for: Based on the difference between the third spectral coefficient and the fourth spectral coefficient, an adaptive weighting strategy is used to calculate the loss between the third spectral coefficient and the fourth spectral coefficient.

7. The method according to claim 1, characterized in that, The initialized flow matching generation model includes an initialized one-step flow matching generation model.

8. A reasoning method based on a flow matching generation model, characterized in that, The method includes: Acquire noise, time t, and environmental features of the first action, wherein the environmental features of the first action include at least the observations of the first action; The noise, the time t, and the environmental features are input into the trained flow matching generation model as described in any one of claims 1 to 7 to obtain the second action predicted by the conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector.

9. The reasoning method according to claim 8, characterized in that, If the trained flow matching generation model includes a trained one-step flow matching generation model; The step of inputting the noise, the time t, and the environmental features of the first action into the trained flow matching generation model to obtain the second action obtained by the conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector includes: The noise, the time t, and the environmental features of the first action are input into the trained flow matching one-step generation model to obtain the second action generated by the conditional probability path predicted by the trained flow matching one-step generation model with a preset velocity field vector.

10. A training apparatus for a stream matching generation model, characterized in that, The device includes: An acquisition unit is used to acquire noise, time t, and environmental features of the first action, wherein the environmental features of the first action include at least the observation value of the first action; The input unit is used to input the noise, the time t, and the environmental features into the initialized flow matching generation model to obtain the predicted velocity field vector of the conditional probability path at time t+1 output by the initialized flow matching generation model. A calculation unit is configured to calculate the loss between the predicted velocity field vector and the true velocity field vector using a preset loss function. The preset loss function includes at least one of a first loss function and a second loss function, and a third loss function. The first loss function adaptively constrains the difference between the predicted velocity field vectors at any two times between time t and time t+1 to minimize the difference. The second loss function adaptively constrains the predicted actions at future time u1 to converge to the same position based on the difference between actions originating from different times between time t and time t+1 and the predicted actions at future time u1. The third loss function constrains the loss between the predicted velocity field and the true velocity field to minimize the difference. The training unit is used to train the initialized flow matching generation model using the loss and backpropagation algorithm until the flow matching generation model converges, so as to obtain the trained flow matching generation model.

11. An inference device based on a stream matching generation model, characterized in that, The device includes: An acquisition unit is used to acquire noise, time t, and environmental features of the first action, wherein the environmental features of the first action include at least the observation value of the first action; The prediction unit is used to input the noise, the time t, and the environmental features into the trained flow matching generation model as described in any one of claims 1 to 7, so as to obtain the second action predicted by the conditional probability path predicted by the trained flow matching generation model with a preset velocity field vector.

12. A computer device comprising a processor, characterized in that, When the processor executes a computer program stored in the memory, it is used to implement the training method of the stream matching generation model as described in any one of claims 1 to 7, or the inference method based on the stream matching generation model as described in any one of claims 8 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it is used to implement the training method of the stream matching generation model as described in any one of claims 1 to 7, or the inference method based on the stream matching generation model as described in any one of claims 8 to 9.

14. A computer program product having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it is used to implement the training method of the stream matching generation model as described in any one of claims 1 to 7, or the inference method based on the stream matching generation model as described in any one of claims 8 to 9.

Images