Behavior clone model training method and device, equipment and medium

Abstract

The invention relates to the technical field of intelligent decision making, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a behavior cloning model training method, device, equipment and medium, and the method comprises the steps: obtaining multi-modal input, carrying out the feature extraction and splicing of the multi-modal input, and obtaining an input representation; utilizing a preset decoder to extract a potential representation of the input representation, and utilizing a preset strategy head to decode the potential representation into action data; calculating a potential representation mutual information loss function value by using an information bottleneck principle, and calculating a mean square error loss value of the action data and preset real action data; iteratively updating parameters of a preset decoder and a preset strategy head according to the mutual information loss function value and the mean square error loss value to obtain an updated model; detecting whether a joint loss function value in the updating model is smaller than a preset threshold value or not in real time; and when the joint loss function value is smaller than a preset threshold value, stopping parameter iteration updating to obtain an optimization model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent decision-making technology, and in particular to a behavior cloning model training method, device, equipment and medium. Background Art

[0002] Behavior cloning is an imitation learning method widely used in robotics. It trains a robot to replicate the expert's actions by using state-action pairs demonstrated by an expert. This method uses supervised learning to enable the robot to learn the mapping between observed states and executed actions, thereby imitating the expert's behavior.

[0003] In the field of healthcare, behavioral cloning can be used to train surgical robots. Doctors first complete a series of complex surgical steps by operating surgical instruments. These operations are accurately recorded. Then the surgical robot learns these operation patterns through the behavioral cloning algorithm. After a large amount of data training, the robot can accurately reproduce the doctor's operations in similar surgical scenarios and assist the doctor in completing the operation.

[0004] In the field of financial technology, behavioral cloning is applied to intelligent investment advisory systems. The process of senior financial investment experts making investment decisions based on market conditions, financial data, etc. is recorded and analyzed. By learning these decision-making behavior patterns, the intelligent investment advisory system can provide personalized investment advice to ordinary investors. According to factors such as investors' risk preferences and investment goals, it analyzes market dynamics like an expert and recommends suitable investment portfolios to help investors better allocate assets, reduce investment risks, increase investment returns, and promote the intelligent and inclusive development of financial technology services.

[0005] In summary, in real business scenarios, existing behavior cloning methods have limitations, including failure to address the redundancy problem in the latent representation, resulting in the inclusion of irrelevant information, reduced policy performance and generalization ability, lack of a rigorous theoretical framework, reliance on experience in design, and difficulty in optimizing the balance between complexity and predictive ability.

[0006] Therefore, the stability and accuracy issues of behavioral cloning in existing technologies need to be urgently addressed. Summary of the Invention

[0007] The present invention provides a behavior cloning model training method, device, equipment and medium to solve the accuracy and stability problems of behavior cloning in complex scenarios, thereby obtaining an optimization model with high stability, accurate decision-making and high efficiency.

[0008] In a first aspect, a behavior cloning model training method is provided, comprising: Obtaining multimodal input, performing feature extraction and splicing on the multimodal input to obtain an input representation; extracting a latent representation of the input representation using a preset decoder, and decoding the latent representation into action data using a preset strategy header; Calculating the mutual information loss function value of the potential representation using the information bottleneck principle, and calculating the mean square error loss value between the action data and the preset real action data; Iteratively updating parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; Detecting in real time whether the joint loss function value in the updated model is less than a preset threshold; When the value of the joint loss function is less than a preset threshold, the parameter iterative update is stopped to obtain an optimized model.

[0009] In a second aspect, a behavior cloning model training device is provided, comprising: An extraction module, configured to obtain multimodal input, perform feature extraction and concatenation on the multimodal input, and obtain an input representation; a decoding module, configured to extract a latent representation of the input representation using a preset decoder, and decode the latent representation into action data using a preset strategy header; a calculation module, configured to calculate the mutual information loss function value of the potential representation by using the information bottleneck principle, and calculate the mean square error loss value between the action data and the preset real action data; An updating module, configured to iteratively update parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; A judgment module, used to detect in real time whether the joint loss function value in the updated model is less than a preset threshold; The generation module is used to stop the iterative update of parameters and obtain the optimized model when the value of the joint loss function is less than a preset threshold.

[0010] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned behavior cloning model training method when executing the computer program.

[0011] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned behavior cloning model training method are implemented.

[0012] In the scheme implemented by the above-mentioned behavior cloning model training method, device, equipment and medium, by obtaining multimodal input and performing feature extraction and splicing, it is possible to integrate information from multiple sources, fully utilize the complementarity of different modal data, and improve the model's ability to understand complex information; use the preset decoder and strategy head to convert the input into action data, realizing the complete process from information processing to actual decision output; construct the loss function based on the information bottleneck principle, combining the mean square error of action data and real data, which not only ensures the model's extraction of key information, but also ensures the accuracy of the output action; through parameter iterative update, the model parameters can be continuously optimized to make the model better adapt to task requirements; using the joint loss function value as the condition for stopping iteration, it is ensured that the model stops training when a certain performance standard is reached, effectively avoiding overfitting, thereby obtaining an optimization model with high stability, accurate decision-making and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 1 is a schematic diagram of an application environment of a behavior cloning model training method according to an embodiment of the present invention; Figure 2 This is a flow chart of a behavior cloning model training method according to one embodiment of the present invention; Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S1; Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S3; Figure 5 1 is a training diagram of a behavior cloning model training method according to an embodiment of the present invention; Figure 6 1 is a structural diagram of a behavior cloning model training device according to an embodiment of the present invention; Figure 7 is a structural diagram of a computer device in one embodiment of the present invention; Figure 8 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0016] The embodiment of the present invention provides a behavior cloning model training method, which can be applied in Figure 1 In an application environment, the client communicates with the server through a network. The server can obtain multimodal input, perform feature extraction and splicing on the multimodal input to obtain an input representation; use a preset decoder to extract a potential representation of the input representation, and use a preset strategy head to decode the potential representation into action data; use the information bottleneck principle to calculate the mutual information loss function value of the potential representation, and calculate the mean square error loss value between the action data and the preset real action data; iteratively update the parameters of the preset decoder and the preset strategy head according to the mutual information loss function value and the mean square error loss value to obtain an updated model; detect in real time whether the joint loss function value in the updated model is less than a preset threshold; when the joint loss function value is less than the preset threshold, stop the iterative update of the parameters to obtain an optimized model. The present invention provides a behavior cloning model training device. For target result business, by obtaining multimodal input and extracting splicing features, it can fuse multi-source information and use modal complementarity to improve the model's understanding ability; with the help of a preset decoder and a strategy head, it can realize the process from information processing to decision output; based on the information bottleneck principle, it constructs a loss function and combines it with the mean square error to ensure the accuracy of key information extraction and output actions; through parameter iteration optimization model, the value of the joint loss function is used as the stopping condition to avoid overfitting, and obtain a stable, accurate and efficient optimization model. Among them, the client can be but is not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0017] See also Figure 2 As shown, Figure 2 A flowchart of a behavior cloning model training method provided by an embodiment of the present invention includes the following steps: S1. Obtain multimodal input, perform feature extraction and splicing on the multimodal input, and obtain input representation.

[0018] In the examples of the present invention, the multimodal input is a comprehensive data input method that integrates multiple types of information from different sensory channels or sensors, such as vision (images, videos), hearing (voice, audio), text, and proprioceptive states, etc. Each modality provides key information about the same phenomenon or task from a unique perspective.

[0019] In the present invention, see Figure 3 As shown, the step of obtaining multimodal input, performing feature extraction and concatenation on the multimodal input, and obtaining input representation includes: S11, using a preset visual transformer to encode the image in the multimodal input to obtain a first feature code; S12, using a preset multi-layer perceptron to convert the proprioceptive state in the multimodal input into an embedding vector; S13, using a preset encoder to encode the language instruction in the multimodal input to obtain a second feature code; S14: Concatenate the first feature code, the embedding vector, and the second feature code to obtain an input representation.

[0020] In an example of the present invention, encoding the image in the multimodal input using a preset visual transformer to obtain a first feature code includes: Segmenting the image in the multimodal input into image blocks; Linearly mapping the image block into an embedding vector; Performing position encoding on the embedding vector to obtain an initial feature sequence; Performing deep feature extraction on the initial feature sequence to obtain a first feature code.

[0021] In the example of the present invention, dividing the image in the multimodal input into image blocks refers to dividing the entire input image into multiple non-overlapping rectangular areas according to a predetermined size, each area being an image block. This operation is usually implemented using a simple spatial division method to ensure that each image block has a uniform size.

[0022] In this example of the present invention, the linear mapping of the image block into an embedding vector refers to flattening each image block obtained by segmentation (converting a two-dimensional pixel matrix into a one-dimensional vector), and then mapping it to a higher-dimensional feature space through a linear transformation (matrix multiplication) with a preset weight matrix to generate an embedding vector with a fixed dimension.

[0023] In the example of the present invention, the position encoding of the embedding vector to obtain the initial feature sequence refers to adding the embedding vector of each image block to a corresponding position encoding vector, wherein the position encoding vector is generated by a predefined function (such as sine and cosine functions) to reflect the relative position of the image block in the original image, thereby injecting spatial information into the embedding vector, and finally arranging all the position-encoded embedding vectors in sequence to form an initial feature sequence containing position information.

[0024] In an example of the present invention, performing deep feature extraction on the initial feature sequence to obtain a first feature code includes: Linearly projecting the initial feature sequence into a triple vector; Performing weighted summation on the triple vectors to obtain a weighted result, and adding the weighted result to the initial feature sequence to obtain a first feature sequence; Perform a linear transformation on the first feature sequence to generate a first feature code.

[0025] In the example of the present invention, the linear projection of the initial feature sequence into a triple vector is performed by using three independent and learnable weight matrices to perform matrix multiplication operations with the initial feature sequence respectively. During the operation, the initial feature sequence is weightedly combined, and the data is transformed from different angles and mapped to three different feature spaces, thereby generating three groups of vectors: query, key, and value.

[0026] In the example of the present invention, the weighted summation of the triple vectors is performed to obtain a weighted result, and the weighted result and the initial feature sequence are added to obtain the first feature sequence. The query vector and the key vector are first used to calculate the similarity between the two, and weights that can reflect the degree of association between the sequence elements are obtained. These weights reflect the importance of each element in the overall sequence. Then, the value vector is weightedly aggregated using the weight, so that the information in the value vector that is closely related to the current element is strengthened to obtain a weighted result. In order to retain the original information of the initial feature sequence, the weighted result is added to the elements at the corresponding position of the initial feature sequence. This not only integrates the key information extracted by the attention mechanism, but also retains the original data features, thereby forming a first feature sequence containing richer semantic and structural information.

[0027] In the examples of the present invention, the linear transformation of the first feature sequence to generate the first feature code is performed by inputting the first feature sequence into a linear transformation layer consisting of a learnable weight matrix and a bias vector. During the linear transformation process, the weight matrix recombines and maps the features of each dimension of the first feature sequence, and the bias vector performs a translation adjustment on the transformation result, thereby completing a linear transformation of the features. This process can uncover the potential linear relationships between features, reorganize and enhance the features. After the linear transformation is completed, the result obtained is the first feature code.

[0028] In the example of the present invention, the multilayer perceptron is a feedforward neural network composed of multiple neuron layers. It inputs the input data into each layer in turn, first performs linear transformation by matrix multiplication with the weight matrix of the layer, maps the data to the new feature space, and then introduces nonlinearity with a nonlinear activation function, and the activation result is used as the input of the next layer. This is repeated, and each layer uses a different weight matrix to extract higher-level features. After multi-layer processing, it finally outputs an embedded vector of fixed length that integrates the key input features. The proprioceptive perception state is the perception and recording of the robot's own state in the form of data when performing a task, covering key information data such as joint angles and gripper states.

[0029] In the examples of the present invention, the use of a preset multilayer perceptron to convert the ontological perception state in the multimodal input into an embedding vector is to input the ontological perception data into the first layer of the multilayer perceptron and perform matrix multiplication with the weight matrix of this layer to achieve a linear transformation. This step maps the input data to a new feature space, and then applies a nonlinear activation function (such as the ReLU function) to the result of the linear transformation, introducing nonlinear characteristics by retaining positive numbers and setting negative numbers to zero. The result of the nonlinear activation is then used as the input of the next layer, and the linear transformation and nonlinear activation function processing are repeated. Each layer uses a different weight matrix to gradually extract higher-level features until the last layer outputs a fixed-length vector. This vector integrates the key features of the input data, namely the embedding vector.

[0030] In the example of the present invention, the language instructions in the multimodal input are encoded using a preset encoder to obtain a second feature code, which is to split the language instruction text into sub-word units and generate corresponding word vectors, and then add position codes to retain word order information. Next, multiple rounds of iterative processing are performed, and the attention weights between word vectors are calculated and weighted aggregation is performed in each round of iteration. At the same time, linear transformation and nonlinear activation are applied to enhance feature expression. Finally, the updated word vectors obtained in each round of iteration are integrated to generate a fixed-dimensional feature vector set that integrates the semantics and contextual information of the language instruction, i.e., the second feature code.

[0031] In the example of the present invention, the vector splicing of the first feature code, the embedding vector and the second feature code to obtain the input representation means arranging the three feature vectors from different sources in order, and then connecting them in dimension to form a longer vector. This new vector integrates multimodal information from images, proprioceptive states and language instructions, and contains comprehensive features required for the task. The final spliced ​​vector is the input representation.

[0032] In the examples of the present invention, the benefit of obtaining multimodal input and performing feature extraction and splicing on the multimodal input to obtain input representation is that it can integrate rich information from different data sources, thereby providing a more comprehensive and detailed data analysis basis.

[0033] For example, in the field of financial technology, multimodal feature splicing can be used for intelligent fraud detection. By integrating transaction text information (such as merchant descriptions and transaction notes), user behavior images (such as operation interface heat maps) and ontological perception data (such as geographic location and network environment parameters collected by device sensors), a multi-dimensional feature vector is constructed to accurately identify abnormal transaction patterns.

[0034] In the field of healthcare, integrating multimodal inputs such as patients' medical images, physiological monitoring data, electronic health records, and genetic information can help doctors obtain more comprehensive information about the patient's condition, which not only enhances the understanding of the patient's health status, but also helps improve the overall quality and efficiency of medical services.

[0035] S2. Use a preset decoder to extract a latent representation of the input representation, and use a preset strategy header to decode the latent representation into action data.

[0036] In this example, the preset decoder is a model component used to extract a latent representation from an input representation. It transforms and abstracts the input data, removing redundant information and extracting a latent representation containing key information. The preset strategy header is a model module that decodes the latent representation into action data. Based on the core features contained in the latent representation, it generates specific action data through mapping and calculation.

[0037] In this example, the latent representation is a high-dimensional and compact data representation that captures the key information and inherent structure of the input data. The action data refers to specific instructions or signals used to guide specific operations or decisions. These data are directly related to the actions that a system or device should perform, such as movement instructions for a robot.

[0038] In the field of FinTech, latent representations are high-dimensional, compact feature vectors extracted from massive amounts of financial transaction data. These vectors encompass key information and inherent structure across multiple dimensions, including user transaction behavior, credit history, and asset status. Banks use these features to assess customer credit risk and generate action data for loan approval.

[0039] In the field of healthcare, latent representations are feature vectors extracted from multi-source data such as electronic medical records, medical images, and physiological monitoring data, which contain key information such as patient health status and disease characteristics.

[0040] In an embodiment of the present invention, the step of extracting a latent representation of the input representation by using a preset decoder includes: Performing a nonlinear transformation on the input representation to obtain preliminary features; Performing dimensionality reduction processing on the preliminary features to obtain key features; Semantic understanding is performed on the key features to obtain potential representations.

[0041] In the example of the present invention, the nonlinear transformation of the input representation to obtain preliminary features is to first perform nonlinear processing on the input representation (such as an activation function), apply a nonlinear function to each element, that is, set negative numbers to zero and retain positive numbers, thereby breaking the linear structure and extracting nonlinear patterns in the data. The set of elements after nonlinear processing constitutes the preliminary features.

[0042] In an example of the present invention, the preliminary features are subjected to dimensionality reduction processing to obtain key features, the preliminary features are organized into a matrix, and the correlation data between the features is obtained by calculating the covariance matrix of the matrix; then, the covariance matrix is ​​subjected to eigenvalue decomposition to obtain a series of eigenvalues ​​and corresponding eigenvectors, wherein the eigenvalue size reflects the degree of discreteness of the data in the corresponding direction, and the larger the eigenvalue, the more information contained in the direction; then, according to a pre-set threshold (such as retaining the dimension that can explain more than 80% of the data changes) or directly specifying the number of retained dimensions, the first k eigenvectors with larger eigenvalues ​​are selected; finally, the preliminary feature matrix is ​​subjected to matrix multiplication operation with the selected k eigenvectors, and the preliminary features are projected into a new low-dimensional space, thereby converting the original high-dimensional preliminary features into streamlined low-dimensional key features that retain key information.

[0043] In the examples of the present invention, the semantic understanding of the key features to obtain a latent representation is to input the key features into a semantic parsing module. This module is usually composed of a multi-layer neural network, and each layer performs weighted summation and nonlinear activation operations on the input key features. In this process, the key features are first passed to the first layer of the neural network, which performs a preliminary weighted summation on the input features, and then introduces nonlinearity through a nonlinear activation function (such as ReLU), so that the model can capture the complex relationships in the data. Subsequently, these nonlinearly transformed features are passed to the next layer of the neural network, and the above-mentioned weighted summation and nonlinear activation operations are repeated. This process is carried out layer by layer in the multi-layer neural network, and each layer gradually extracts and abstracts higher-level semantic information. Ultimately, after multi-layer processing, the obtained features are aggregated into a compact latent representation that can more accurately reflect the semantic meaning of the input data.

[0044] In an example of the present invention, decoding the potential representation into action data using a preset strategy header includes: Performing linear mapping on the potential representation to obtain a mapping vector; Performing nonlinear constraints on the mapping vector to obtain a constraint vector; Performing affine transformation on the constraint vector to obtain action data.

[0045] In the example of the present invention, the linear mapping of the latent representation to obtain the mapping vector is to multiply each element in the latent representation by a corresponding weight, then add all the products, and finally add a bias value. The purpose of this transformation is to map the data from the original feature space to a new feature space to obtain a mapping vector.

[0046] In the examples of the present invention, the nonlinear constraints applied to the mapping vector to obtain the constraint vector involve using an activation function to adjust each value to ensure it meets the needs of the actual application. For example, using the ReLU function converts all negative numbers to 0, while positive numbers remain unchanged. Using the tanh function compresses all values ​​to a range between -1 and 1, making it suitable for applications requiring a limited range, such as motor control. By applying reasonable constraints to the data, the output is ensured to be both logical and within the device's processing range, ultimately resulting in a constraint vector that is both safe and practical.

[0047] In this embodiment of the present invention, an affine transformation is performed on the constraint vector to obtain motion data. Each element of the constraint vector is multiplied by a specific scaling factor (i.e., scaling) to adjust the amplitude of the motion. An offset (i.e., translation) is then added to the scaled result to determine the specific starting point of the motion. For example, if the constraint vector represents the ideal position of a robotic arm, the affine transformation will adjust these position values ​​based on the actual working range of the arm to ensure that the motion data falls within the range within which the robotic arm can safely operate, ultimately generating the motion data. The affine transformation is a mathematical operation that includes linear transformations (such as scaling and rotation) and translations on the data, thereby mapping points in one space to another space.

[0048] In this example, a preset decoder is used to extract the latent representation of the input representation, which is then decoded into action data using a preset strategy header, offering significant advantages. The latent representation efficiently captures the key information and intrinsic structure of the input data, providing a solid foundation for subsequent decoding. The combination of the preset decoder and the strategy header enables efficient conversion from abstract latent representations to concrete executable actions, ensuring that the generated action data is both task-compliant and highly adaptable. This step-by-step approach enhances the model's interpretability and scalability, facilitating targeted optimization and adjustment of each link.

[0049] S3. Calculate the mutual information loss function value of the potential representation using the information bottleneck principle, and calculate the mean square error loss value between the action data and the preset real action data.

[0050] In this example, the information bottleneck principle compresses data to retain important information related to the target variable while discarding redundant information irrelevant to the target. Specifically, suppose there is a random variable (X) and an observed variable (Y). If we want to predict (Y) using the compressed variable (T), we need to find an optimal balance between accuracy and complexity.

[0051] In the present invention, see Figure 4 As shown, the mutual information loss function value of the potential representation is calculated using the information bottleneck principle, including: S31. Regularizing the potential representation to obtain minimized mutual information; S32, using a back-propagation algorithm to enhance the correlation information in the latent representation to obtain maximized mutual information; S33. Calculate a mutual information loss function value according to the minimized mutual information and the maximized mutual information.

[0052] In the examples of the present invention, the minimization of mutual information and the maximization of mutual information are combined to be referred to as mutual information. Mutual information is an important concept in information theory used to measure the degree of mutual dependence between two random variables. It quantitatively describes the amount of information contained in one random variable about another random variable, reflecting the amount of information shared by the two. It can be understood as the degree to which the uncertainty of one variable is reduced by knowing the information of the other variable.

[0053] In the examples of the present invention, the regularization of the latent representation to obtain minimized mutual information is to introduce a prior distribution (such as a standard normal distribution), requiring the distribution of the latent representation to be as close as possible to the prior, so that the encoder filters out redundant features such as noise and background, and adjusts the encoder parameters through backpropagation so that the latent representation only retains key information (such as object position) that is useful for tasks such as predicting actions. Finally, the mutual information between the original input and the latent representation is minimized through regularization constraints, thereby effectively eliminating redundant information.

[0054] In this embodiment of the present invention, the backpropagation algorithm is used to enhance the relevant information in the latent representation and maximize mutual information. This algorithm calculates the error between the predicted information output by the decoder and the actual task information (for example, the deviation between the predicted grasping angle and the actual required angle) and feeds this error back to the network. Based on this error, the encoder and decoder parameters are adjusted to increase the weight of key task-related features in the latent representation (such as object outlines and dimensions) and decrease the weight of irrelevant features, ultimately maximizing the mutual information between the latent representation and the task information.

[0055] In the example of the present invention, the mutual information loss function value is calculated based on the minimized mutual information and the maximized mutual information. A loss function is constructed based on the minimized mutual information I(X;Z) between the original input data X and the potential representation Z and the maximized mutual information I(Z;A) between the potential representation Z and the action data A. The mutual information loss function value is calculated using the following mutual information loss function formula:

[0056] in, represents the mutual information loss function value, Represents the Lagrange multiplier, which is a fixed value , represents the minimized mutual information between the input data and the latent representation, represents the maximum mutual information between the latent representation and the action data, Represents input data, Represents potential representation, Represents action data.

[0057] In the example of the present invention, the calculation of the mean square error loss value between the action data and the preset real action data is to calculate the square of the difference between the action data and the real action data point at each corresponding action data point. This step is to quantify the deviation between each predicted action and the actual action. The use of squares ensures that all deviations are positive numbers, and larger deviations will have a greater contribution. Afterwards, all these square differences are added up to obtain the total sum of square differences. Finally, in order to obtain the average error of each data point, the total sum of square differences is divided by the total number of action data points. The mean square error loss value calculated in this way provides a quantitative indicator to evaluate the overall consistency between the action data generated by the system and the real action data. The smaller the mean square error loss value, the closer the predicted action data is to the real action data, and the better the prediction performance of the model.

[0058] In this example, the information bottleneck principle is used to calculate the mutual information loss function of the latent representation and the mean squared error loss between the action data and the pre-set real action data. This method can bring multiple benefits. By minimizing the mutual information between the input data and the latent representation, it effectively compresses redundant information in the data, thereby improving the model's generalization ability and computational efficiency. By maximizing the mutual information between the latent representation and the action data, it ensures that the model captures the most useful information for predicting actions, thereby improving the accuracy of predictions.

[0059] For example, in the training of surgical robot motion control models in the healthcare sector, the raw input includes multiple sources of information, such as the patient's anatomy. Utilizing the information bottleneck principle to calculate the mutual information loss function removes redundant information, improves model generalization, and focuses on key surgical information. By combining the mean squared error loss between surgical and standard movements, we can accurately measure deviations and optimize parameters, making the surgical robot's movements more precise and stable, thereby improving surgical success rates and safety.

[0060] S4. Iteratively update the parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model.

[0061] In the examples of the present invention, the updated model refers to a new model version obtained by adjusting the model parameters according to the mutual information loss function value and the mean square error loss value through an iterative optimization process, aiming to improve the prediction accuracy and generalization ability of the model, while optimizing information compression and retention of task-related features.

[0062] In an example of the present invention, iteratively updating the parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model includes: Adding the mutual information loss function value and the mean square error loss value to obtain a joint loss function value; Adjusting the parameters in the preset decoder and the preset strategy header using a gradient descent algorithm according to the joint loss function value to obtain a preliminary updated model; Calculate the updated mutual information loss function value and the updated mean square error loss value in the preliminary updated model, use the updated mutual information loss function value and the updated mean square error loss value as the new mutual information loss function value and the new mean square error loss value, and return to the step of adding the mutual information loss function value and the mean square error loss value to obtain the updated model.

[0063] In the examples of the present invention, the method of summing the mutual information loss function value and the mean squared error loss value to obtain a joint loss function value is to directly add the calculated mutual information loss function value and the mean squared error loss value to obtain a comprehensive numerical value as the joint loss function value. This joint loss function value integrates the requirements of information screening and numerical accuracy, ensuring that during the model training process, it not only meets the optimization goal of the potential representation due to the information bottleneck, but also ensures that the numerical output is as close to the actual situation as possible.

[0064] In the example of the present invention, the parameters in the preset decoder and the preset strategy header are adjusted according to the joint loss function value using the gradient descent algorithm to obtain a preliminary updated model, which is to calculate the gradient of the joint loss function relative to the model parameters. These gradients indicate the direction in which the loss function grows fastest in the parameter space; then, a suitable learning rate is selected, which determines the step size of the parameter update; then, according to the rule of gradient descent, each parameter is updated, that is, the parameter value is adjusted in the opposite direction of the gradient, and the specific operation is to subtract the product of the learning rate and the corresponding gradient from the parameter; this update operation is applied to all parameters that need to be optimized in the model, including the weights and biases in the preset decoder and the preset strategy header; after completing these updates, the model is preliminarily updated.

[0065] In the example of the present invention, the updated mutual information loss function value and the updated mean square error loss value in the preliminary updated model are calculated, the updated mutual information loss function value and the updated mean square error loss value are used as the new mutual information loss function value and the new mean square error loss value, and the step of adding the mutual information loss function value and the mean square error loss value is returned to obtain the updated model, which is to recalculate the mutual information loss function value and the mean square error loss value using the updated model parameters. This includes evaluating the performance of the model in information screening and numerical prediction. Then, these two values ​​are added to obtain a new joint loss function value. If the value does not meet the preset convergence standard, this new joint loss function value and mean square error are fed back to the parameter update step, and the parameters are adjusted again using the gradient descent algorithm. Repeat the above process and continuously iterate and update.

[0066] In the examples of the present invention, the parameters of the preset decoder and the preset strategy head are iteratively updated according to the mutual information loss function value and the mean square error loss value, which can optimize the model from the dual dimensions of information processing and numerical fitting. The mutual information loss function value calculated by the information bottleneck principle can eliminate data redundancy, guide the model to focus on key features, and improve the quality of potential representation and task relevance; the mean square error loss value accurately measures the numerical deviation between the predicted action and the real action, ensuring that the model output is close to reality. The two work together, and through iterative updates, the model is prompted to continuously correct the prediction error while retaining the core information, so that the decoder can better generate effective action data, and the strategy head can more accurately output the prediction results, ultimately allowing the updated model to significantly improve information screening, action prediction accuracy and generalization ability.

[0067] S5. Detect in real time whether the joint loss function value in the updated model is less than a preset threshold.

[0068] In the example of the present invention, the preset threshold is a performance indicator value determined in advance, which is used to evaluate whether the value of the joint loss function in the updated model is low enough, so as to decide whether the parameter iterative update process should be stopped.

[0069] In healthcare, the motion control model of surgical robots also requires optimization using preset thresholds. The robot builds a motion prediction model by learning from a large amount of surgical data. A joint loss function comprehensively evaluates the effectiveness of the information filtering of the surgical data and the error in actual motion execution. When the joint loss function value of the updated model is less than the preset threshold, the robot can accurately perform surgical actions, assisting doctors in completing surgeries in a stable and reliable manner, and improving the accuracy and safety of surgical operations.

[0070] In the examples of the present invention, the real-time detection of whether the joint loss function value in the updated model is less than a preset threshold means that during the model training phase, after each parameter update, the system immediately initiates the loss calculation process. The mutual information loss and mean squared error loss are first calculated separately, and then the two are added together according to pre-configured weights to obtain the joint loss function value. Next, a conditional judgment is performed, and the joint loss function value is numerically compared with a pre-set threshold to determine whether the joint loss function value is less than the preset threshold.

[0071] In this example, real-time detection of the joint loss function of the updated model is performed to see if it is less than a preset threshold. This allows for dynamic monitoring of the model training status and prompt termination of training when the loss value meets the required accuracy. This prevents overfitting caused by excessive iterations and conserves computing resources. By setting a reasonable threshold, updates can be stopped when the model converges, balancing training efficiency and model performance.

[0072] S6. When the value of the joint loss function is less than a preset threshold, stop the iterative update of the parameters to obtain an optimized model.

[0073] In the example of the present invention, the optimization model refers to a model that stops training and saves the model by iteratively updating parameters and training optimization of the initial model when the joint loss function value of the model is less than a preset threshold.

[0074] In the example of the present invention, when the value of the joint loss function is less than a preset threshold, the iterative parameter update is stopped to obtain the optimized model. During the model training process, after each parameter update, the system calculates the joint loss function value of the current model and compares it with the preset threshold. If the joint loss function value is less than the preset threshold, the program triggers the termination mechanism through the set stop condition, immediately interrupting the subsequent parameter iterative update process to avoid overfitting during continued model training. At this time, the system will save all the parameters of the current model, including the weights and biases of each layer of the neural network, and ultimately form an optimized model with performance that meets the requirements.

[0075] Specifically, when the joint loss function value is not less than the preset threshold, the model continues to execute the parameter iterative update process, that is, the gradient is calculated based on the current mutual information loss and mean square error loss, the parameters of the decoder and strategy head are adjusted through the optimization algorithm, and then the new joint loss function value is calculated again and compared with the threshold. This process is repeated until the loss value meets the condition of being less than the threshold, ensuring that the model is continuously optimized during training until the preset accuracy requirements are met.

[0076] In this embodiment of the present invention, when the value of the joint loss function falls below a preset threshold, the iterative parameter update is stopped. The benefits of obtaining an optimized model include effectively saving computing resources, improving training efficiency, avoiding overfitting, and ensuring that the model has good generalization capabilities on new data. For example, in the field of financial technology, such as credit risk assessment models, the optimized model can more accurately assess customer credit risk and improve the risk management efficiency of financial institutions.

[0077] In the present invention, see Figure 5 As shown, Figure 5 A training diagram of a behavior cloning model training method provided by an embodiment of the present invention is used to implement multimodal input based on text, proprioception, vision, etc., output actions through encoding, decoding, and strategy head, and build a loss optimization model by combining mutual information neural network and information bottleneck principle. Among them, "Put corn in the red bowl" is used as a language instruction, and the second feature code Enc is obtained by the preset encoder (text encoder). l ; The proprioceptive state is converted into an embedding vector Enc through a preset multi-layer perceptron s ; The scene image is used as the image in the multimodal input, and the first feature code Enc is generated by the preset visual transformer encoding o ; Then, the first feature code, the embedding vector, and the second feature code are concatenated to obtain the input representation X t , input the input representation into the Transformer decoder to decode the potential representation Z t , output action data A through the strategy head t , right side mutual information neural network calculation [X t ,Z t ] and [Z t ,A t ], and construct the loss for model optimization through the following formula:

[0078] in, represents the mutual information loss function value, Represents the Lagrange multiplier, which is a fixed value , represents the minimized mutual information between the input data and the latent representation, represents the maximum mutual information between the latent representation and the action data, Represents input data, Represents potential representation, Represents action data.

[0079] It can be seen that in the above scheme, for the target result business, by obtaining multimodal input and performing feature extraction and splicing, it is possible to integrate information from multiple sources, make full use of the complementarity of different modal data, and improve the model's ability to understand complex information; use the preset decoder and strategy head to convert the input into action data, realizing the complete process from information processing to actual decision output; constructing the loss function based on the information bottleneck principle, combining the mean square error of action data and real data, which not only ensures the model's extraction of key information, but also ensures the accuracy of the output action; through parameter iterative update, the model parameters can be continuously optimized to make the model better adapt to task requirements; using the joint loss function value as the condition for stopping iteration, it ensures that the model stops training when a certain performance standard is reached, effectively avoiding overfitting, and thus obtaining an optimization model with high stability, accurate decision-making and high efficiency.

[0080] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0081] In one embodiment, a behavior cloning model training device is provided, which corresponds to a behavior cloning model training method in the above embodiment. Figure 6 As shown, the behavior cloning model training device includes an extraction module 101, a decoding module 102, a calculation module 103, an update module 104, a judgment module 105 and a generation module 106. The functional modules are described in detail as follows: Extraction module 101, used to obtain multimodal input, perform feature extraction and splicing on the multimodal input, and obtain input representation; A decoding module 102 is configured to extract a latent representation of the input representation using a preset decoder, and decode the latent representation into action data using a preset strategy header; A calculation module 103 is configured to calculate a mutual information loss function value of the potential representation using the information bottleneck principle, and calculate a mean square error loss value between the action data and preset real action data; An updating module 104 is configured to iteratively update parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; A judgment module 105 is used to detect in real time whether the value of the joint loss function in the updated model is less than a preset threshold; The generation module 106 is used to stop the iterative update of parameters and obtain the optimized model when the value of the joint loss function is less than a preset threshold.

[0082] In one embodiment, the extraction module 101 obtains multimodal input, performs feature extraction and concatenation on the multimodal input, and obtains an input representation for: Encoding the image in the multimodal input using a preset visual transformer to obtain a first feature code; Using a preset multi-layer perceptron to convert the proprioceptive state in the multimodal input into an embedding vector; Encoding the language instruction in the multimodal input using a preset encoder to obtain a second feature code; The first feature code, the embedding vector, and the second feature code are vector-concatenated to obtain an input representation.

[0083] In one embodiment, the extraction module 101, after obtaining multimodal input, performs feature extraction and concatenation on the multimodal input to obtain an input representation, and is further configured to: Segmenting the image in the multimodal input into image blocks; Linearly mapping the image block into an embedding vector; Performing position encoding on the embedding vector to obtain an initial feature sequence; Performing deep feature extraction on the initial feature sequence to obtain a first feature code.

[0084] In one embodiment, the extraction module 101, after obtaining multimodal input, performs feature extraction and concatenation on the multimodal input to obtain an input representation, specifically for: Linearly projecting the initial feature sequence into a triple vector; Performing weighted summation on the triple vectors to obtain a weighted result, and adding the weighted result to the initial feature sequence to obtain a first feature sequence; Perform a linear transformation on the first feature sequence to generate a first feature code.

[0085] In one embodiment, the decoding module 102 extracts a latent representation of the input representation using a preset decoder, and decodes the latent representation into action data using a preset strategy header, for: Performing a nonlinear transformation on the input representation to obtain preliminary features; Performing dimensionality reduction processing on the preliminary features to obtain key features; Semantic understanding is performed on the key features to obtain potential representations.

[0086] In one embodiment, the decoding module 102 extracts a latent representation of the input representation using a preset decoder and decodes the latent representation into action data using a preset strategy header, specifically for: Performing linear mapping on the potential representation to obtain a mapping vector; Performing nonlinear constraints on the mapping vector to obtain a constraint vector; Performing affine transformation on the constraint vector to obtain action data.

[0087] In one embodiment, the calculation module 103 calculates the mutual information loss function value of the potential representation using the information bottleneck principle and calculates the mean square error loss value between the action data and the preset real action data, for: Regularizing the latent representation to minimize mutual information; Using a back-propagation algorithm to enhance the correlation information in the latent representation to maximize mutual information; A mutual information loss function value is calculated according to the minimized mutual information and the maximized mutual information.

[0088] In one embodiment, the updating module 104 iteratively updates the parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model for: Adding the mutual information loss function value and the mean square error loss value to obtain a joint loss function value; Adjusting the parameters in the preset decoder and the preset strategy header using a gradient descent algorithm according to the joint loss function value to obtain a preliminary updated model; Calculate the updated mutual information loss function value and the updated mean square error loss value in the preliminary updated model, use the updated mutual information loss function value and the updated mean square error loss value as the new mutual information loss function value and the new mean square error loss value, and return to the step of adding the mutual information loss function value and the mean square error loss value to obtain the updated model.

[0089] The present invention provides a behavior cloning model training device. For target result business, by obtaining multimodal input and performing feature extraction and splicing, it can fuse information from multiple sources, fully utilize the complementarity of different modal data, and improve the model's ability to understand complex information; using a preset decoder and strategy head to convert input into action data, it realizes a complete process from information processing to actual decision output; based on the information bottleneck principle, a loss function is constructed, combining the mean square error of action data and real data, which not only ensures the model's extraction of key information, but also ensures the accuracy of output actions; through parameter iterative updating, the model parameters can be continuously optimized to make the model better adapt to task requirements; using the joint loss function value as the condition for stopping iteration, it ensures that the model stops training when a certain performance standard is reached, effectively avoiding overfitting, thereby obtaining an optimization model with high stability, accurate decision-making and high efficiency.

[0090] The specific definition of a behavior cloning model training device can be found in the definition of a behavior cloning model training method above and will not be repeated here. The various modules in the above-mentioned behavior cloning model training device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the above modules.

[0091] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a behavior cloning model training method.

[0092] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the client side of a behavior cloning model training method. In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed: Obtaining multimodal input, performing feature extraction and splicing on the multimodal input to obtain an input representation; extracting a latent representation of the input representation using a preset decoder, and decoding the latent representation into action data using a preset strategy header; Calculating the mutual information loss function value of the potential representation using the information bottleneck principle, and calculating the mean square error loss value between the action data and the preset real action data; Iteratively updating parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; Detecting in real time whether the joint loss function value in the updated model is less than a preset threshold; When the value of the joint loss function is less than a preset threshold, the parameter iterative update is stopped to obtain an optimized model.

[0093] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: Obtaining multimodal input, performing feature extraction and splicing on the multimodal input to obtain an input representation; extracting a latent representation of the input representation using a preset decoder, and decoding the latent representation into action data using a preset strategy header; Calculating the mutual information loss function value of the potential representation using the information bottleneck principle, and calculating the mean square error loss value between the action data and the preset real action data; Iteratively updating parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; Detecting in real time whether the joint loss function value in the updated model is less than a preset threshold; When the value of the joint loss function is less than a preset threshold, the parameter iterative update is stopped to obtain an optimized model.

[0094] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0095] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0096] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0097] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

[0098] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A behavior cloning model training method, characterized in that: include: Obtaining multimodal input, performing feature extraction and splicing on the multimodal input to obtain an input representation; extracting a latent representation of the input representation using a preset decoder, and decoding the latent representation into action data using a preset strategy header; Calculating the mutual information loss function value of the potential representation using the information bottleneck principle, and calculating the mean square error loss value between the action data and the preset real action data; Iteratively updating parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; Detecting in real time whether the joint loss function value in the updated model is less than a preset threshold; When the value of the joint loss function is less than a preset threshold, the parameter iterative update is stopped to obtain an optimized model.

2. The behavioral cloning model training method according to claim 1, characterized in that: The acquiring of multimodal input, performing feature extraction and concatenation on the multimodal input, and obtaining input representation includes: Encoding the image in the multimodal input using a preset visual transformer to obtain a first feature code; Using a preset multi-layer perceptron to convert the proprioceptive state in the multimodal input into an embedding vector; Encoding the language instruction in the multimodal input using a preset encoder to obtain a second feature code; The first feature code, the embedding vector, and the second feature code are vector-concatenated to obtain an input representation.

3. The behavioral cloning model training method according to claim 2, characterized in that: The method of encoding the image in the multimodal input using a preset visual transformer to obtain a first feature code includes: Segmenting the image in the multimodal input into image blocks; Linearly mapping the image block into an embedding vector; Performing position encoding on the embedding vector to obtain an initial feature sequence; Performing deep feature extraction on the initial feature sequence to obtain a first feature code.

4. The behavioral cloning model training method according to claim 1, characterized in that: The extracting the potential representation of the input representation by using a preset decoder includes: Performing a nonlinear transformation on the input representation to obtain preliminary features; Performing dimensionality reduction processing on the preliminary features to obtain key features; Semantic understanding is performed on the key features to obtain potential representations.

5. The behavioral cloning model training method according to claim 1, characterized in that: The decoding of the potential representation into action data using a preset strategy header includes: Performing linear mapping on the potential representation to obtain a mapping vector; Performing nonlinear constraints on the mapping vector to obtain a constraint vector; Performing affine transformation on the constraint vector to obtain action data.

6. The behavioral cloning model training method according to claim 1, characterized in that: The calculating of the mutual information loss function value of the potential representation by using the information bottleneck principle includes: Regularizing the latent representation to minimize mutual information; Using a back-propagation algorithm to enhance the correlation information in the latent representation to maximize mutual information; A mutual information loss function value is calculated according to the minimized mutual information and the maximized mutual information.

7. The behavioral cloning model training method according to claim 1, characterized in that: The iteratively updating the parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model includes: Adding the mutual information loss function value and the mean square error loss value to obtain a joint loss function value; Adjusting the parameters in the preset decoder and the preset strategy header using a gradient descent algorithm according to the joint loss function value to obtain a preliminary updated model; Calculate the updated mutual information loss function value and the updated mean square error loss value in the preliminary updated model, use the updated mutual information loss function value and the updated mean square error loss value as the new mutual information loss function value and the new mean square error loss value, and return to the step of adding the mutual information loss function value and the mean square error loss value to obtain the updated model.

8. A behavioral cloning model training device, characterized in that: include: An extraction module, configured to obtain multimodal input, perform feature extraction and concatenation on the multimodal input, and obtain an input representation; a decoding module, configured to extract a latent representation of the input representation using a preset decoder, and decode the latent representation into action data using a preset strategy header; a calculation module, configured to calculate the mutual information loss function value of the potential representation by using the information bottleneck principle, and calculate the mean square error loss value between the action data and the preset real action data; An updating module, configured to iteratively update parameters of the preset decoder and the preset strategy header according to the mutual information loss function value and the mean square error loss value to obtain an updated model; A judgment module, used to detect in real time whether the joint loss function value in the updated model is less than a preset threshold; The generation module is used to stop the iterative update of parameters and obtain the optimized model when the value of the joint loss function is less than a preset threshold.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the behavior cloning model training method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the behavior cloning model training method according to any one of claims 1 to 7 is implemented.

Images