Method for training virtual fitting model, method for generating fitting image and related device

Abstract

This application relates to the field of virtual try-on, and more particularly to a method for training a virtual try-on model, a method for generating try-on images, and related apparatus. The method includes constructing a virtual try-on model using a clothing distortion network and a try-on generation network. Human body keypoint images and clothing images are input into the clothing distortion network to obtain distorted clothing images. Real try-on images, distorted clothing images, and model identity images are input into the try-on generation network. By fusing semantic features from the distorted clothing feature image, the model identity feature image, and clothing attribute information, a reconstructed feature image is generated. The reconstructed feature image is decoded by a decoder to generate a virtual try-on image. The clothing distortion network and the try-on generation network are iteratively trained using a loss function until the virtual try-on model converges, resulting in the virtual try-on model. This method effectively improves the realism and quality of virtual try-on, providing better service quality and user experience for online try-on.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for training a virtual fitting model, a method for generating fitting images, and related apparatus. Background Technology

[0002] With the continuous advancement of modern technology, online shopping has gradually become one of the main ways for people to purchase goods. When buying products such as clothing, consumers usually want to see the real-life effect of the product to ensure the right style, size, and fit. Therefore, the demand for virtual try-on technology is gradually increasing, becoming a research focus in the field of computer vision.

[0003] Currently, there are some virtual try-on solutions. However, existing clothing deformation algorithms mainly use TPS and optical flow-based deformation algorithms, but their generated models lack self-correction capabilities. For example, they perform poorly when dealing with details such as necklines and sleeves, resulting in a lack of realism in the try-on effect and obvious texture artifacts. Summary of the Invention

[0004] This application provides a method for training a virtual fitting model, a method for generating fitting images, and related apparatus. This method improves the quality of the fitting effect by introducing the reconstruction and decoding of low-dimensional features during the fitting process, making it closer to the real wearing effect.

[0005] In a first aspect, this application provides a method for training a virtual fitting model, characterized in that the virtual fitting model includes a clothing distortion network and a fitting generation network, and the method includes:

[0006] Obtain a training set, wherein the training set includes multiple training data, the training data including clothing images and real fitting images, wherein in the real fitting images a model is wearing the clothing corresponding to the clothing image;

[0007] The human body key point image and the clothing image are input into the clothing distortion network to obtain the deformed clothing image;

[0008] The real fitting image, the deformed clothing image, and the model identity image are input into the fitting generation network. The real fitting image, the deformed clothing image, and the model identity image are encoded in the fitting generation network to obtain deformed clothing feature images, model identity feature images, and real fitting image feature images. The semantic features of the deformed clothing feature images, model identity feature images, and clothing attribute information are fused to obtain reconstructed feature images. The reconstructed feature images are decoded to generate virtual fitting images.

[0009] Based on the reconstructed feature image and the real fitting feature image, the loss is calculated using a loss function, and the clothing distortion network and the fitting generation network are iteratively trained according to the loss until the virtual fitting model converges, thus obtaining the virtual fitting model.

[0010] In some embodiments, the clothing distortion network includes a first encoding module, a regression network, and a deformation structure;

[0011] The step of inputting the human body key point image and the clothing image into the clothing distortion network to obtain the deformed clothing image includes:

[0012] The human body key point image is input into the first encoding module for encoding to obtain the first feature map;

[0013] The clothing image is input into the first encoding module for encoding to obtain a second feature map;

[0014] By fusing the first feature map and the second feature map, a correlation layer for a single tensor is obtained;

[0015] The correlation layer is input into the regression network for prediction to obtain spatial transformation parameters;

[0016] The deformation structure is used to distort and deform the clothing image according to the spatial transformation parameters to obtain a deformed clothing image.

[0017] In some embodiments, the clothing distortion network includes a second encoding module, a text encoding module, a feature reconstruction network, and a decoding module;

[0018] The process involves inputting the real fitting image, the deformed clothing image, and the model identity image into the fitting generation network. The fitting generation network encodes the real fitting image, the deformed clothing image, and the model identity image respectively, obtaining deformed clothing feature images, model identity feature images, and real fitting image feature images. Semantic features from the deformed clothing feature image, model identity feature image, real fitting image, and clothing attribute information are then fused to obtain a reconstructed feature image. Finally, the reconstructed feature image is decoded to generate a virtual fitting image. This process includes:

[0019] The real fitting image is input into the second encoding module for encoding to obtain the real fitting feature image;

[0020] The deformed clothing image is input into the second encoding module for encoding to obtain the deformed clothing feature image;

[0021] The model's identity image is input into the second encoding module for encoding to obtain the model's identity feature image;

[0022] The clothing attribute information is input into the text encoding module to obtain the semantic features, wherein the clothing attribute information includes at least one of clothing type, collar type, sleeve length and size.

[0023] The deformed clothing feature image, the model identity feature image, and the semantic features are input into the feature reconstruction network to perform convolution operations on the deformed clothing feature image and the model identity feature image to obtain the fitting feature image. The fitting feature image and the semantic features are then fused to obtain the reconstructed feature image.

[0024] The reconstructed feature image is input into the decoding module to obtain a virtual fitting image.

[0025] In some embodiments, the text encoding module includes a text processing unit and a vector encoding unit;

[0026] The step of inputting the clothing attribute information into the text encoding module to obtain semantic features includes:

[0027] The text description of the clothing attribute information is input into the text processing unit to obtain a semantic feature vector;

[0028] The semantic feature vector is input into the vector encoding unit to obtain the semantic features.

[0029] In some embodiments, the text processing unit uses a fixed-length sequential forgetting encoding method to encode the text description of the clothing attribute information. The formula for the fixed-length sequential forgetting encoding is as follows:

[0030] ;

[0031] in, Indicates the current character position in the encoding. Indicates the forgetting factor, Represents the character currently encoded;

[0032] The vector encoding unit uses a bidirectional long short-term memory network to encode the semantic feature vector. The encoding formula of the bidirectional long short-term memory network is as follows:

[0033] ;

[0034] in, This represents a bidirectional long short-term memory network. This represents the set of semantic feature vectors output by the fixed-length sequential forgetting encoding.

[0035] In some embodiments, the fusion formula used to fuse the fitting feature image and the semantic features is as follows:

[0036] ;

[0037] in, Represents the weight matrix. This indicates the semantic feature. Indicates the deviation value. Represents the hyperbolic tangent function. This represents the feature image of the fitting room.

[0038] In some embodiments, the loss function is:

[0039] ;

[0040] in, The loss function is defined as the low-dimensional cross-loss between the reconstructed feature image and the real fitting feature image. Indicates the first Zhang's reconstructed feature image Indicates the first The image shows the actual features of the clothing being tried on.

[0041] Secondly, this application provides a method for generating fitting room images, characterized by comprising:

[0042] Obtain images of the clothing to be tried on and the user's image;

[0043] The user's human body key point image and the image of the clothing to be tried on are input into the clothing distortion network of the virtual fitting model to obtain the deformed image of the clothing to be tried on.

[0044] The image of the deformable clothing to be tried on and the user's identity image are input into the virtual clothing model's clothing generation network. The deformable clothing to be tried on and the user's identity image are encoded in the clothing generation network to obtain the deformable clothing feature image and the user's identity feature image, respectively. The semantic features of the deformable clothing feature image, the user's identity feature image and the attribute information of the clothing to be tried on are fused to obtain the user clothing reconstruction feature image. The user clothing reconstruction feature image is decoded to generate the user clothing image.

[0045] The virtual fitting model is trained using the method described in any one of the first aspects above.

[0046] Thirdly, this application provides an electronic device, characterized in that it comprises:

[0047] At least one processor, and

[0048] The memory communicatively connected to the at least one processor, wherein,

[0049] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0050] Fourthly, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions for causing an electronic device to perform the method described in the first aspect above.

[0051] The beneficial effects of this invention are as follows: Unlike existing technologies, this application provides a method for training a virtual try-on model, a method for generating try-on images, and related apparatus. The method includes constructing a virtual try-on model using a clothing distortion network and a try-on generation network. Human body keypoint images and clothing images are input into the clothing distortion network to obtain deformed clothing images. Real try-on images, deformed clothing images, and model identity images are input into the try-on generation network. By fusing semantic features from the deformed clothing feature image, model identity feature image, and clothing attribute information, a reconstructed feature image is generated. The reconstructed feature image is decoded using a decoder to generate a virtual try-on image. The clothing distortion network and try-on generation network are iteratively trained using a loss function until the virtual try-on model converges, thus obtaining the virtual try-on model. Because this invention calculates the loss based on the reconstructed feature image and the real try-on feature image—that is, based on low-dimensional feature reconstruction loss rather than relying on pixel-level loss as a model learning indicator—the model trained based on this loss has a greater generative capability. This method effectively improves the realism and quality of virtual try-on, providing better service quality and user experience for online try-on. Attached Figure Description

[0052] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0053] Figure 1 A flowchart illustrating a process for training a virtual fitting model, provided as an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the structure of the clothing twisting network provided in an embodiment of this application;

[0055] Figure 3 This is a schematic diagram of the structure of the fitting room generation network provided in an embodiment of this application;

[0056] Figure 4A schematic diagram illustrating a process for generating a fitting image, provided as an embodiment of this application;

[0057] Figure 5 A schematic diagram illustrating a process for generating a fitting image, provided as an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] It should be noted that, unless otherwise specified, the various features in the embodiments of this invention can be combined with each other, all within the scope of protection of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with substantially the same function and effect.

[0062] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0063] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0064] Before providing a detailed description of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0065] (1) A neural network, also known as a neural network (NNs) or a connection model, is an algorithmic mathematical model that imitates the behavioral characteristics of animal neural networks to perform distributed parallel information processing. Neural networks rely on the complexity of the system to adjust the interconnections between a large number of internal nodes, thereby achieving the purpose of processing information. Specifically, a neural network can be composed of neural units, which can be understood as a neural network with an input layer, hidden layers, and an output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the intermediate layers are hidden layers. A neural network with many hidden layers is called a deep neural network (DNN). The work of each layer in a neural network can be described by the mathematical expression y=a(W·x+b). From a physical perspective, the work of each layer in a neural network can be understood as completing the transformation from the input space to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space (the set of input vectors). These five operations include: 1. Dimensional increase / decrease; 2. Magnification / reduction; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are performed by "W·x", operation 4 by "+b", and operation 5 by "a()". The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within that class. W is the weight matrix of each layer in the neural network, where each value represents the weight of a neuron in that layer. This matrix W determines the spatial transformation from the input space to the output space, meaning that the W of each layer in the neural network controls how the space is transformed. The purpose of training the neural network is to ultimately obtain the weight matrices of all layers in the trained neural network. Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0066] It should be noted that, in the embodiments of this application, the models used for machine learning tasks are essentially neural networks.

[0067] Common components in neural networks include convolutional layers, activation function layers, and batch normalization layers. By assembling these common components in neural networks, a model is designed. When the model parameters (weight matrices of each layer) are determined so that the model error meets the preset conditions or the number of model parameters is adjusted to reach the preset threshold, the model converges.

[0068] The convolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform convolution operations on the image. The purpose of convolution is to extract different features from the input image. The first convolutional layer may only extract some low-level features such as edges, lines, and corners, while deeper convolutional layers can iteratively extract more complex features from low-level features.

[0069] Activation function layers are used to allow each neuron in a neural network to accept the output value of the previous layer as its input value and pass the processing result to the next layer. Commonly used activation functions include, but are not limited to, the Rectified Linear Unit (ReLU) function, the Swish function, and the Parametric Rectified Linear Unit (PReLU) function.

[0070] Batch Normalization (BN) layers are used to standardize the features of a certain layer in a network. Their purpose is to solve the problem of numerical instability in deep neural networks. Specifically, as the number of network layers increases, parameter updates during training can easily cause drastic changes in the feature outputs near the output layer, which is not conducive to training an effective neural network.

[0071] (2) The loss function refers to a function that maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. The loss function is a non-negative real number function used to quantify the difference between the predicted label and the true label. In applications, the loss function is often used as a learning criterion in relation to optimization problems, i.e., the model is solved and evaluated by minimizing the loss function. For example, it is used for parametric estimation in statistics and machine learning. During the training of a neural network, because we want the output of the neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the target value we really want, and then update the weight matrix of each layer of the neural network according to the difference between the two (however, there is usually an initialization process before the first update, i.e., pre-configuring the parameters of each layer in the neural network). For example, if the network's predicted value is too high, we adjust the weight matrix to make it predict lower, and continue to adjust until the neural network can predict the target value we really want. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value," which is the loss function or objective function. These are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, a higher output value (loss) of the loss function indicates a greater difference, so training the neural network becomes the process of minimizing this loss as much as possible.

[0072] (3) Low-dimensional features refer to feature vectors or feature representations with lower dimensions in the feature representation or feature space. In fields such as computer vision and pattern recognition, features are often used to represent key information in data for classification, detection, recognition, or other related tasks. The purpose of low-dimensional features is to reduce the dimensionality of high-dimensional data while retaining the main feature information by transforming it into a more compact representation. By reducing the dimensionality of the data, computational and storage overhead can be reduced, and the complexity of the model can be simplified, while also helping to avoid the curse of dimensionality.

[0073] Low-dimensional feature extraction can be achieved through various methods, such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Autoencoders. These methods can transform raw high-dimensional data into lower-dimensional feature representations, facilitating subsequent data analysis and processing. In the virtual try-on method provided in this application, the reconstruction and decoding of low-dimensional features can help improve the realism and texture of the try-on effect. By extracting and reconstructing low-dimensional features, the shape, texture, and wearing effect of clothing can be expressed more accurately, thereby enhancing the realism of virtual try-on and the user experience.

[0074] Before introducing the embodiments of this application, a brief introduction to the virtual try-on technology known to the inventors of this application will be given to facilitate the understanding of the embodiments of this application.

[0075] This invention relates to a virtual try-on technology, and more particularly to a method and system for improving the effect of virtual try-on. With the continuous advancement of modern technology, online shopping has gradually become one of the main ways people acquire goods. When purchasing clothing and other products, consumers usually want to see the real effect of the product to ensure a suitable style, size, and wearing effect. Therefore, the demand for virtual try-on technology is gradually increasing, becoming a research focus in the field of computer vision.

[0076] Currently, several virtual try-on solutions exist, including offline smart try-on and online virtual try-on. Offline smart try-on typically uses interactive mirror devices to provide a complete outfit display and interactive try-on experience based on the user's basic characteristics and selected clothing styles. Online virtual try-on, on the other hand, uses computer vision technology to automatically replace target clothing with images of the user to achieve an online try-on effect. However, existing virtual try-on technologies have some limitations. First, 3D modeling-based methods require acquiring three-dimensional information of the human body and clothing, which is cumbersome and costly. The resulting try-on effect may lack naturalness and fail to realistically reflect the user's body shape and the fit. Second, while 2D image-based methods are less expensive, they have limitations in terms of clothing deformation and try-on effect. Existing clothing deformation algorithms mainly use TPS and optical flow-based deformation algorithms, but their generated models lack self-correction capabilities. For example, they perform poorly in handling details such as necklines and sleeves, resulting in a lack of realism and noticeable texture artifacts in the try-on effect.

[0077] Therefore, this invention aims to provide a method and system for improving the virtual try-on effect, addressing the problems of unrealistic try-on results and obvious texture artifacts in existing technologies. By introducing low-dimensional feature reconstruction and decoding during the try-on generation process, this invention enhances the texture of the try-on effect, making it closer to the real wearing experience. Unlike existing methods, this invention does not rely solely on pixel-level loss as a learning metric, but instead trains the model using low-dimensional spatial feature reconstruction loss, and then uses a decoder to perform pixel-level feature decoding on the low-dimensional features, thereby improving the realism of the try-on effect.

[0078] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0079] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for training a virtual fitting model provided in an embodiment of this application. Specifically, method S100 may include the following steps:

[0080] S101: Obtain the training set.

[0081] The training set includes multiple training data sets, which include clothing images and real fitting images, in which a model is wearing the clothing corresponding to the clothing image.

[0082] To enable the trained virtual try-on model to output high-resolution try-on images, the clothing images in the training data and the real try-on images also have correspondingly high resolution. In some embodiments, the resolution of the try-on images can be 1024. 768.

[0083] It is understood that the training data consists of paired data consisting of clothing images and real-life fitting images. This training data can be collected in advance by those skilled in the art, for example, by storing and downloading clothing images and corresponding model images of the clothing on some clothing sales websites. In some embodiments, the amount of training data is in the tens of thousands, for example, 20,000, which is beneficial for training an accurate general-purpose model. Those skilled in the art can determine the amount of training data according to the actual situation.

[0084] S102: Input the human body key point image and the clothing image into the clothing distortion network to obtain the deformed clothing image.

[0085] Before step S102, the method further includes image processing of the real fitting image to obtain a human body analysis image, a model identity image, and a human body key point image.

[0086] Specifically, human body analysis refers to segmenting a person captured in a real fitting image into multiple semantically consistent regions, such as body parts and clothing, or subcategories of body parts and clothing. In other words, it involves pixel-level recognition of the input image and labeling each pixel in the image with its corresponding object category.

[0087] In some embodiments, the human body image can be obtained using existing human body analysis algorithms. Optionally, this embodiment uses the Graphiconomy algorithm, which segments the image into 20 categories, distinguished by different colors, to classify different body parts. In some embodiments, the aforementioned 20 categories can also be classified using labels 0-19, for example, 0 represents background, 1 represents hat, 2 represents hair, 3 represents gloves, 4 represents sunglasses, 5 represents top, 6 represents dress, 7 represents coat, 8 represents socks, 9 represents pants, 10 represents torso skin, 11 represents scarf, 12 represents skirt, 13 represents face, 14 represents left arm, 15 represents right arm, 16 represents left leg, 17 represents right leg, 18 represents left shoe, and 19 represents right shoe. From the human body image, the category to which each body part belongs in the image can be determined.

[0088] To ensure that the model's identity information remains unchanged during the virtual fitting process, the pixels that represent human body characteristics, such as the head, arms, and feet, need to be categorized as 1, while the rest are categorized as 0, thus obtaining the model's identity image.

[0089] Human key point detection algorithms are used to detect human key points in real fitting images. This can locate human key point information (i.e., several key points on the human body). Key points can be the coordinates of the nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles.

[0090] In some embodiments, the existing OpenPose human skeleton keypoint detection algorithm can be used to detect human keypoints in real fitting images, locate human keypoint information, and obtain human keypoint images. The OpenPose algorithm can detect 15, 18, or 25 human keypoints, and those skilled in the art can set the required number of keypoints to be detected according to the actual situation.

[0091] Understandably, the aforementioned techniques for processing real fitting images and extracting the model's body features serve two purposes: first, they reduce unnecessary interference factors during virtual fitting model training, improving the convergence speed and accuracy of the virtual fitting model; second, by preserving the model's body features, they ensure that the model's body remains realistic after changing into the clothes to be tried on, resulting in a more authentic fitting effect.

[0092] In some embodiments, an existing Gaussian Mixture Model (GMM) network structure can be used to construct a clothing distortion network. The GMM network structure is a commonly used machine learning model for modeling and representing complex probability distributions. In computer vision, GMM networks are frequently used for image processing tasks such as image segmentation, object detection, and face recognition. By aligning and deforming clothing images and human keypoint images using the GMM network structure, that is, by extracting and combining features from clothing and human keypoints, the GMM network can learn the deformation and distortion patterns of the clothing and generate clothing images with deformed effects, helping to improve the realism and lifelikeness of the fitting model.

[0093] In some embodiments, please refer to Figure 2 The clothing distortion network constructed in this embodiment includes a first encoding module, a regression network, and a deformation structure.

[0094] The first encoding module is constructed based on the Mobilinet network. Mobilinet is a lightweight convolutional neural network (CNN) designed to provide efficient image classification and feature extraction capabilities. The first encoding module is used to extract features from clothing images and human body key point images.

[0095] A regression network is constructed. A regression network is a type of neural network that consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers. The design of these layers is usually optimized according to the specific application scenario and task. The regression network is used to predict the spatial transformation parameter θ, which describes the degree of deformation of the clothing.

[0096] Construct a TPS (Thin Plate Spline) deformation structure. TPS deformation is a nonlinear deformation method based on control points, which can deform the image according to the given control points and corresponding displacements.

[0097] Specifically, step S102 includes steps S1021-S1025:

[0098] S1021: Input the human body key point image into the first encoding module for encoding to obtain the first feature map.

[0099] S1022: Input the clothing image into the first encoding module for encoding to obtain the second feature map.

[0100] S1023: Fuse the first feature map and the second feature map to obtain the correlation layer of a single tensor.

[0101] Specifically, the first and second feature maps are fused to generate a single tensor correlation layer. Fusion methods can include concatenation, addition, or weighted summation.

[0102] S1024: Input the correlation layer into the regression network for prediction to obtain spatial transformation parameters.

[0103] Specifically, the relevant layers are input into the regression network, and after a series of convolution and pooling operations, features related to the spatial transformation parameters are extracted. In the last layer or multiple fully connected layers of the regression network, the extracted features are input into one or more nodes (neurons). These nodes learn weights and biases to perform linear combinations and nonlinear transformations on the input features. Finally, the outputs of these nodes are interpreted as the predicted spatial transformation parameters θ.

[0104] S1025: The clothing image is distorted and deformed using the deformation structure according to the spatial transformation parameters to obtain a deformed clothing image.

[0105] Specifically, based on the spatial transformation parameters, the positions and corresponding displacements of the control points used for TPS deformation are determined. These control points are typically located in key areas of the clothing, such as the collar and cuffs. Using the TPS deformation method, the clothing image is distorted based on the control points and displacements. The TPS method maps each pixel position of the original clothing image to its deformed position by calculating the pixel displacement values, thereby achieving image deformation and ultimately obtaining a deformed clothing image.

[0106] S103: Input the real fitting image, the deformed clothing image, and the model identity image into the fitting generation network, so that the real fitting image, the deformed clothing image, and the model identity image are encoded in the fitting generation network to obtain deformed clothing feature images, model identity feature images, and real fitting image features, respectively. Then, the semantic features of the deformed clothing feature images, model identity feature images, and clothing attribute information are fused to obtain reconstructed feature images. The reconstructed feature images are decoded to generate virtual fitting images.

[0107] In some embodiments, please refer to Figure 3 The virtual try-on generation network constructed in this embodiment includes a second encoding module, a text encoding module, a feature reconstruction network, and a decoding module.

[0108] A second encoding module is constructed to extract low-dimensional features from real fitting images, deformed clothing images, and model identity images. The second encoding module can comprehensively consider factors such as task requirements, data characteristics, and computing resources to select an encoder type suitable for specific tasks and data types, such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM). In some embodiments, a pre-trained network model can be used as the second encoding module, that is, the parameters of the second encoding module are known, and the second encoding module does not participate in model training, so as to reduce the complexity of virtual fitting model training, reduce training time, and improve model performance.

[0109] A text encoding module is constructed based on FOFE (Fixed-size Ordinally Encoded) series encoding and Bidirectional Long Short-Term Memory (Bi-LSTM) units. This module is used to guide feature reconstruction from clothing attribute information (such as collar type, sleeve length, size, etc.) to obtain semantic feature vectors. FOFE series encoding is a fixed-length encoding method used to encode variable-length sequences into fixed-length vector representations. It encodes each part of the sequence through a recursive formula, where the influence of historical information on the current position is controlled by a forgetting factor. Specifically, for a given string of clothing attribute information, each character is converted into a K-dimensional one-hot vector representation through FOFE encoding. FOFE encoding uses a recursive formula for calculation, continuously updating historical information and combining it with the encoding of the current character to finally obtain a fixed-length semantic feature vector. Bidirectional Long Short-Term Memory (Bi-LSTM) is a recurrent neural network structure with hidden states in both forward and backward directions. During text encoding, Bi-LSTM can capture the contextual information of the input sequence and model clothing attribute information. By inputting the semantic feature sequence after FOFE encoding into Bi-LSTM, it can learn feature representations with contextual information. The output of Bi-LSTM is a semantic feature vector of clothing attribute information, which contains important feature information describing the text. The text encoding module constructed in this embodiment includes a text processing unit and a vector encoding unit.

[0110] A feature reconstruction network is constructed based on the U2-NET image segmentation network. For example, the feature reconstruction network includes convolutional layers, activation function layers, and batch normalization layers. The number of convolutional kernels in each convolutional layer is set to 8, 16, or 32, and the standard convolutional kernel size is 3. 3. The step size is 2, and the activation function used in the activation function layer includes the ReLU activation function;

[0111] A decoding module is constructed to take the output of the feature reconstruction network as input and ultimately decode to generate a virtual try-on image. The decoding module typically employs a structure of convolutional layers and upsampling layers. The convolutional layers extract features and perform weighted combinations of these features by learning weights and biases. The upsampling layers enlarge the size of the feature image, thereby restoring image details and resolution. In some embodiments, a pre-trained network model can be used as the decoding module; that is, the parameters of the decoding module are known, and the decoding module does not participate in model training. This reduces the complexity of virtual try-on model training, thereby reducing training time and improving model performance.

[0112] Specifically, step S103 includes steps S1031-S1034:

[0113] S1031: Input the real fitting image into the second encoding module for encoding to obtain the real fitting feature image;

[0114] S1032: Input the deformed clothing image into the second encoding module for encoding to obtain the deformed clothing feature image.

[0115] S1033: Input the model identity image into the second encoding module for encoding to obtain the model identity feature image.

[0116] S1034: Input the clothing attribute information into the text encoding module to obtain the semantic features, wherein the clothing attribute information includes at least one of clothing type, collar type, sleeve length and size.

[0117] Specifically, the text description of clothing attribute information is input into the text processing unit to obtain a semantic feature vector, and the semantic feature vector is input into the vector encoding unit to obtain semantic features.

[0118] In the text encoding part, all text tags are set as a vocabulary of size K, where each character is represented as a K-dimensional one-hot vector. For a given string y = Each character All are represented by corresponding one-hot vectors express.

[0119] In text processing units, each part of the sequence can be encoded using FOFE (Fixed-size Ordinally Encoded) fixed-length sequential forgetting encoding. FOFE encoding is calculated based on a recursive formula, where... Indicates until the character The formula for FOFE encoding of a partial sequence, with a fixed length for sequential forgetting encoding, is as follows:

[0120] ;

[0121] in, Indicates the current character position in the encoding. This represents the forgetting factor, used to control the degree to which history influences the current position. This represents the character currently being encoded. The recursive calculation in the formula adds the previous encoding results to the one-hot vector of the current character with weights to obtain the FOFE encoding of the current position.

[0122] In the vector encoding unit, the vector encoded based on FOFE is fed into the bidirectional Bi-LSTM (Bidirectional Long Short-Term Memory) network to learn features with contextual information and obtain feature vectors. Where i represents the position in the sequence, the encoding formula of the bidirectional long short-term memory network is:

[0123] ;

[0124] in, This represents a bidirectional long short-term memory network. This represents the set of semantic feature vectors output by the fixed-length sequential forgetting encoding.

[0125] By performing the above steps, text labels are converted into feature vectors with contextual information through FOFE encoding and a bidirectional Bi-LSTM network. These feature vectors can improve the performance and effectiveness of the virtual try-on model in subsequent feature reconstruction and decoding processes.

[0126] S1035: Input the deformed clothing feature image, the model identity feature image, and the semantic features into the feature reconstruction network to perform convolution operations on the deformed clothing feature image and the model identity feature image to obtain a fitting feature image, and fuse the fitting feature image and the semantic features to obtain a reconstructed feature image.

[0127] Specifically, through convolution operations, local features are extracted from the deformed clothing feature image and the model identity feature image, and information at different scales and levels of abstraction is gradually fused to obtain a fitting feature image. This fitting feature image contains the shape and texture information of the clothing as well as the model's identity features. The obtained fitting feature image is then fused with semantic features to enhance the expressive power and semantic consistency of the fitting image, resulting in a reconstructed feature image. The fusion formula used is:

[0128] ;

[0129] in, Represents the weight matrix. This indicates the semantic feature. Indicates the deviation value. Represents the hyperbolic tangent function. This represents the feature image of the fitting room.

[0130] S1036: Input the reconstructed feature image into the decoding module to obtain a virtual fitting image.

[0131] In some embodiments, the decoding module employs operations such as deconvolution, upsampling, or unpooling to gradually recover the details and spatial structure of the image. The neural network structure inside the decoder can learn the spatial and semantic information in the reconstructed feature image and transform it into a pixel-level virtual fitting image. The reconstructed feature image is input into the decoding module to finally obtain a virtual fitting image with the same resolution as the real fitting image.

[0132] S104: Based on the reconstructed feature image and the real fitting feature image, calculate the loss using a loss function, and iteratively train the clothing distortion network and the fitting generation network according to the loss until the virtual fitting model converges, thus obtaining the virtual fitting model.

[0133] Loss functions are widely used in machine learning model training to quantify the difference between model predictions and actual values. In this embodiment, instead of using pixel-level perceptual loss, the loss function is calculated based on the reconstructed feature image and the actual fitting feature image, thereby improving the model's generation capability and producing more realistic fitting images. In some embodiments, the loss function is:

[0134] ;

[0135] in, The loss function is defined as the low-dimensional cross-loss between the reconstructed feature image and the real fitting feature image. Indicates the first Zhang's reconstructed feature image Indicates the first The image shows the actual features of the clothing being tried on.

[0136] In some embodiments, the Adaptive Moment Estimation Algorithm (ADAM) is used to optimize the model parameters. For example, the number of iterations is set to 100,000, the initial learning rate is set to 0.001, the learning rate weight decay is set to 0.0005, and the iteration is performed every 1,000 iterations. After obtaining the adjusted model parameters output by the ADAM algorithm, the adjusted model parameters are used for the next training until the clothing distortion network and the virtual try-on generation network converge. The converged model parameters are then output, which yields the virtual try-on model.

[0137] In summary, this application provides a method for training a virtual try-on model, a method for generating try-on images, and related apparatus. The method includes constructing a virtual try-on model using a clothing distortion network and a try-on generation network. Human body keypoint images and clothing images are input into the clothing distortion network to obtain distorted clothing images. Real try-on images, distorted clothing images, and model identity images are input into the try-on generation network. By fusing semantic features from the distorted clothing feature image, the model identity feature image, and clothing attribute information, a reconstructed feature image is generated. The reconstructed feature image is decoded using a decoder to generate a virtual try-on image. The clothing distortion network and the try-on generation network are iteratively trained using a loss function until the virtual try-on model converges, resulting in the virtual try-on model. This method effectively improves the realism and quality of virtual try-on, providing better service quality and user experience for online try-on.

[0138] After training a skin color transfer model using the method provided in this application, the virtual try-on model can be used to perform virtual try-on and generate try-on images. Please see [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating the method for generating fitting images provided in an embodiment of this application, as shown below. Figure 4 As shown, method S200 includes the following steps:

[0139] S201: Obtain the image of the clothing to be tried on and the user image.

[0140] In some embodiments, the image of the clothing to be tried on can be selected by the user, including but not limited to clothing images saved on a local device, clothing images from a clothing website, etc. These clothing images all include the clothing the user wants to try on, with the front of the clothing facing upwards. To achieve a high-resolution virtual try-on effect, the resolution of the clothing image selected by the user needs to be at least 1024. 768. The user image and the beautified reference image are selected by the user. The user image refers to the original human image that needs to be beautified, provided by the user. The user image can be a human photograph taken by different photographing devices (such as mobile phones, cameras, etc.) and under various lighting conditions.

[0141] S202: Input the user's human body key point image and the image of the clothes to be tried on into the clothing distortion network of the virtual fitting model to obtain the deformed image of the clothes to be tried on.

[0142] The methods for obtaining human body analysis images, model identity images, and human body key point images, as well as the structure of the clothing distortion network, have been described in detail in step S102. In some embodiments, these methods can be obtained or constructed with reference to the aforementioned steps, and will not be described in detail here.

[0143] Specifically, the user's human body key point image and the image of the clothes to be tried on are input into the clothing distortion network of the virtual fitting model. By reviewing the network to predict the spatial transformation parameters, the deformation structure is used to distort and deform the image of the clothes to be tried on according to the spatial transformation parameters to obtain the deformed image of the clothes to be tried on.

[0144] S203: Input the deformable clothing image to be tried on and the user's identity image into the virtual clothing model's clothing generation network, so that the deformable clothing image to be tried on and the user's identity image are encoded in the clothing generation network to obtain the deformable clothing feature image and the user's identity feature image respectively, and fuse the semantic features of the deformable clothing feature image, the user's identity feature image and the clothing attribute information to obtain the user clothing reconstruction feature image. Decode the user clothing reconstruction feature image to generate the user clothing image.

[0145] The structural design of the clothing twisted network has been described in detail in step S103 and will not be repeated here.

[0146] Specifically, the deformed image of the clothing to be tried on and the user's identity image are input into the virtual try-on model's try-on generation network. Low-dimensional features of the deformed clothing feature image and the user's identity feature image are extracted respectively. The text description of the clothing attribute information is input into the text processing unit to obtain a semantic feature vector. The semantic feature vector is input into the vector encoding unit to obtain the semantic features of the clothing attribute information. The deformed clothing feature image, the user identity feature image, and the semantic features of the clothing attribute information are input into the feature reconstruction network to perform convolution operations on the deformed clothing feature image and the user identity feature image to obtain the clothing feature image. The semantic features of the clothing feature image and the clothing attribute information are then fused to obtain the user's reconstructed feature image. The user's reconstructed feature image is input into the decoding module to decode and generate the user's try-on image.

[0147] In short, by designing the aforementioned virtual try-on model and its training method, a converged virtual try-on model is obtained through training. Based on this trained model, the user only needs to input the image of the garment to be tried on and the user's image, and the virtual try-on model can output a try-on image to the user. The try-on images generated using the trained virtual try-on model provided in this application embodiment have a realistic and natural try-on effect.

[0148] In some embodiments, after a virtual fitting model is trained using the method provided in this application, the virtual fitting model can be applied to virtual fitting to generate a fitting image. The virtual fitting method provided in this application can be implemented by various types of electronic devices with computing capabilities, such as smart terminals and servers.

[0149] The virtual try-on method provided in this application is described below with reference to exemplary applications and implementations of the terminal provided in the embodiments of this application. See also Figure 5 , Figure 5 This is a schematic flowchart of a method for generating fitting room images provided in an embodiment of this application. Method S300 includes the following steps:

[0150] S301: Train the virtual fitting model to obtain the trained virtual fitting model.

[0151] In some embodiments, a virtual fitting model is trained using servers. There can be multiple servers, which can form a server cluster. For example, the server cluster may include a first server, a second server, ..., an Nth server. Alternatively, the server cluster may be a cloud computing service center, which includes several servers. The servers in this embodiment include, but are not limited to, tower servers, rack servers, blade servers, and cloud servers. Optionally, the server is a cloud server (Elastic Compute Service, ECS).

[0152] It is understandable that the virtual fitting model is trained based on the training method of any of the aforementioned virtual fitting models. After the virtual fitting model is trained on the server, it is stored on the server. Users can establish a communication connection with the server through smart terminals and then use the virtual fitting model.

[0153] S302: Obtain the image of the clothing to be tried on and the user image.

[0154] Specifically, the smart terminal communicates with the aforementioned server, and the server acquires images of the clothing to be tried on and the user's image through the smart terminal. For example, the user inputs images of the clothing to be tried on and the user's image through the smart terminal's input interface. After input, the server automatically acquires the images of the clothing to be tried on and the user's image. Alternatively, the smart terminal has a camera that captures images of the clothing to be tried on and the user's image and uploads them to the server. Or, the user selects an upload server from the clothing to be tried on image library and user image library stored in the smart terminal. Alternatively, the smart terminal uploads images of the clothing to be tried on and the user's image downloaded from the network to the server.

[0155] S303: Input the image of the garment to be tried on and the user image into the virtual fitting model to obtain the fitting effect image corresponding to the image of the garment to be tried on and the user image.

[0156] Specifically, after the server receives the image of the clothes to be tried on and the user image from the user's smart terminal, it inputs the image of the clothes to be tried on and the user image into the trained virtual fitting model, obtains the fitting effect image output by the virtual fitting model corresponding to the image of the clothes to be tried on and the user image, and transmits the fitting effect image to the user's smart terminal.

[0157] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device 10 provided in an embodiment of this application. The electronic device 10 includes at least one processor 11 and a memory 12 that are communicatively connected. Figure 6 (Taking a bus connection and a single processor as an example). Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0158] The processor 11 is used to provide computing and control capabilities to control the electronic device 10 to perform corresponding tasks, such as controlling the electronic device 10 to perform the methods provided in any embodiment of this application.

[0159] It is understood that the processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0160] The memory 12, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the method for training a virtual fitting model in this embodiment, or program instructions / modules corresponding to the method for generating fitting images. The processor 11, by running the non-transitory software programs, instructions, and modules stored in the memory 12, can implement the virtual fitting model training method in any of the above method embodiments, or generate fitting images. The memory 12 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 12 may also include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0161] It is understood that the electronic device 10 also includes a transmission device 13, which is used to receive or send data via a network. For example, the transmission device 13 is used to transmit clothing images and / or user images uploaded by the user, or to return fitting images to the user. Specific examples of the network described above may include a wireless network provided by the electronic device's communication provider. In one example, the transmission device 13 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one example, the transmission device 13 may be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0162] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the target tracking method as described in the foregoing embodiments.

[0163] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, an optical disk, or a read-only memory (ROM). ROM (ROM) or Random Access Memory (RAM), etc.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a virtual fitting model, characterized in that, The virtual try-on model includes a clothing distortion network and a try-on generation network, and the method includes: Obtain a training set, wherein the training set includes multiple training data, the training data including clothing images and real fitting images, wherein the real fitting images show a model wearing the clothing corresponding to the clothing image; The real fitting images are processed to obtain a model identity image and a key point image of the model's body. The model identity image includes information that reflects the model's body identity features. The human body key point image and the clothing image are input into the clothing distortion network to obtain the deformed clothing image; The real fitting image, the deformed clothing image, and the model identity image are input into the fitting generation network. The fitting generation network encodes the real fitting image, the deformed clothing image, and the model identity image respectively to obtain real fitting feature images, deformed clothing feature images, and model identity feature images. The semantic features of the deformed clothing feature image, the model identity feature image, and clothing attribute information are fused to obtain a reconstructed feature image. The reconstructed feature image is decoded to generate a virtual fitting image. The clothing attribute information includes at least one of clothing type, collar type, sleeve length, and size. Based on the reconstructed feature image and the real fitting feature image, the loss is calculated using a loss function, and the clothing distortion network and the fitting generation network are iteratively trained according to the loss until the virtual fitting model converges, thus obtaining the virtual fitting model.

2. The method according to claim 1, characterized in that, The clothing distortion network includes a first encoding module, a regression network, and a deformation structure; The step of inputting the human body key point image and the clothing image into the clothing distortion network to obtain the deformed clothing image includes: The human body key point image is input into the first encoding module for encoding to obtain the first feature map; The clothing image is input into the first encoding module for encoding to obtain a second feature map; By fusing the first feature map and the second feature map, a correlation layer for a single tensor is obtained; The correlation layer is input into the regression network for prediction to obtain spatial transformation parameters; The deformed clothing image is obtained by distorting and deforming the clothing image using the deformable structure according to the spatial transformation parameters.

3. The method according to claim 1, characterized in that, The virtual fitting generation network includes a second encoding module, a text encoding module, a feature reconstruction network, and a decoding module; The process involves inputting the real fitting image, the deformed clothing image, and the model identity image into the fitting generation network. The fitting generation network encodes the real fitting image, the deformed clothing image, and the model identity image respectively, obtaining deformed clothing feature images, model identity feature images, and real fitting image feature images. Semantic features from the deformed clothing feature image, model identity feature image, real fitting image, and clothing attribute information are then fused to obtain a reconstructed feature image. Finally, the reconstructed feature image is decoded to generate a virtual fitting image. This process includes: The real fitting image is input into the second encoding module for encoding to obtain the real fitting feature image; The deformed clothing image is input into the second encoding module for encoding to obtain the deformed clothing feature image; The model's identity image is input into the second encoding module for encoding to obtain the model's identity feature image; The clothing attribute information is input into the text encoding module to obtain the semantic features; The deformed clothing feature image, the model identity feature image, and the semantic features are input into the feature reconstruction network to perform convolution operations on the deformed clothing feature image and the model identity feature image to obtain the fitting feature image. The fitting feature image and the semantic features are then fused to obtain the reconstructed feature image. The reconstructed feature image is input into the decoding module to obtain a virtual fitting image.

4. The method according to claim 3, characterized in that, The text encoding module includes a text processing unit and a vector encoding unit; The step of inputting the clothing attribute information into the text encoding module to obtain semantic features includes: The text description of the clothing attribute information is input into the text processing unit to obtain a semantic feature vector; The semantic feature vector is input into the vector encoding unit to obtain the semantic features.

5. The method according to claim 4, characterized in that, The text processing unit uses a fixed-length sequential forgetting encoding method to encode the text description of the clothing attribute information to obtain the semantic feature vector. The formula for the fixed-length sequential forgetting encoding is as follows: ； in, Indicates the current character position in the encoding. Indicates the forgetting factor, This represents the character currently encoded. A fixed length of character encoding is sequentially forgotten at the current character position; The vector encoding unit uses a bidirectional long short-term memory network to encode the semantic feature vector to obtain the semantic features. The encoding formula of the bidirectional long short-term memory network is as follows: ； in, This represents a bidirectional long short-term memory network. This represents the set of semantic feature vectors output by the fixed-length sequential forgetting encoding, where i represents the position in the sequence. This represents the semantic feature.

6. The method according to claim 3, characterized in that, The fusion formula used to fuse the fitting feature image and the semantic features is as follows: ； in, Represents the weight matrix. This indicates the semantic feature. Indicates the deviation value. Represents the hyperbolic tangent function. This represents the feature image of the fitting room.

7. The method according to any one of claims 1-6, characterized in that, The loss function is: ； in, The loss function is defined as the low-dimensional cross-loss between the reconstructed feature image and the real fitting feature image. This represents the j-th reconstructed feature image. This represents the j-th image of the actual fitting room features.

8. A method for generating fitting room images, characterized in that, include: Obtain images of the clothing to be tried on and the user's image; The user image is processed to obtain the user's identity image and the user's human body key point image. The user's identity image includes information reflecting the user's human body identity characteristics. The user's human body key point image and the image of the clothing to be tried on are input into the clothing distortion network of the virtual fitting model to obtain the deformed image of the clothing to be tried on. The image of the deformable clothing to be tried on and the user's identity image are input into the virtual fitting model's fitting generation network. The virtual fitting generation network encodes the image of the deformable clothing to be tried on and the user's identity image to obtain the feature image of the deformable clothing to be tried on and the feature image of the user's identity. The semantic features of the feature image of the deformable clothing to be tried on, the feature image of the user's identity, and the attribute information of the clothing to be tried on are fused to obtain the user fitting reconstruction feature image. The user fitting reconstruction feature image is decoded to generate the user fitting image. The attribute information of the clothing to be tried on includes at least one of clothing type, collar type, sleeve length, and size. The virtual fitting model is trained using the method described in any one of claims 1-7.

9. An electronic device, characterized in that, include: At least one processor, and The memory communicatively connected to the at least one processor, wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing an electronic device to perform the method as described in any one of claims 1-8.

Images