Method for training acne category prediction model and related device

Abstract

The embodiment of the application relates to the technical field of image processing, and discloses a training method of a pimple category prediction model and related devices, the training method of the pimple category prediction model, on the one hand, by collecting pimple images of various pimple categories, combining several pimple images, generating several pimple image blocks, and processing the pimple image blocks based on a feature extraction module to generate the pimple image blocks after processing, the application can better preprocess the pimple images; on the other hand, the attention mechanism of the Transformer model module is used to extract features of the pimple image blocks after processing, and a loss function including a multi-category weighted cross-entropy loss is constructed to train the model, so that the model can pay more attention to the differences of pimple features in different environments through the attention mechanism, thereby improving the accuracy of pimple category recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a training method and related apparatus for a acne category prediction model. Background Technology

[0002] With the rapid development of mobile communication technology and the improvement of people's living standards, various smart terminals have been widely used in people's daily work and life, making people more and more accustomed to using APP and other software. As a result, the demand for APPs with functions such as beauty selfie and skin analysis is also increasing. Therefore, many users hope that such APPs can automatically analyze the condition of acne on their face and propose targeted skin improvement solutions based on the type of acne.

[0003] Currently, acne category recognition is usually achieved by training neural networks to obtain a trained neural network model for recognition. However, due to the lack of distinct features between acne categories and the low distinguishability, coupled with the low resolution and poor image quality of acne images, the accuracy of acne recognition by the trained model is often low. Summary of the Invention

[0004] This application provides a training method and related apparatus for an acne category prediction model to improve the accuracy of acne category prediction.

[0005] In a first aspect, embodiments of this application provide a training method for an acne category prediction model. The acne category prediction model includes a feature extraction module, a Transformer model module, and a classifier module. The method includes:

[0006] Obtain an image dataset, which includes images of acne of various acne categories;

[0007] Combine several pimple images to generate several pimple image blocks, wherein each pimple image block includes the same number of pimple images;

[0008] Based on the feature extraction module, the pimple image patch is processed to generate a processed pimple image patch;

[0009] Based on the Transformer model module, feature extraction is performed on the processed pimple image patches to obtain the encoded features corresponding to the processed pimple image patches. The Transformer model module extracts features from the processed pimple image patches through an attention mechanism.

[0010] Based on the classifier module, the coded features are used to predict the pimple category in order to obtain the pimple category corresponding to the pimple image;

[0011] Construct a loss function and train the acne category prediction model based on the loss function until the loss function converges. The loss function includes multi-class weighted cross-entropy loss.

[0012] In some embodiments, the feature extraction module includes an embedding layer. Based on the feature extraction module, the acne image patch is processed to generate a processed acne image patch, including:

[0013] The vector corresponding to the pimple image patch is input into the embedding layer to generate the feature vector corresponding to the processed pimple image patch. The size of the feature vector is related to the number of pimple categories. The network structure of the embedding layer is a fully connected layer.

[0014] In some embodiments, the method further includes:

[0015] Generate category blocks, merge the category blocks with the processed pimple image blocks to obtain a stitched block, specifically including:

[0016] Generate category blocks, where each category block corresponds to a random vector. Merge the random vector corresponding to the category block with the feature vector corresponding to the processed pimple image block to obtain the stitching vector corresponding to the stitching block.

[0017] In some embodiments, based on the Transformer model module, feature extraction is performed on the processed acne image patches to obtain the encoded features corresponding to the processed acne image patches, including:

[0018] Based on the Transformer model module, feature extraction is performed on the concatenated blocks to obtain the corresponding encoded features. The Transformer model module extracts features from the concatenated blocks through an attention mechanism.

[0019] In some embodiments, the Transformer model module includes a normalization layer, an attention mechanism layer, and a perceptron layer. Based on the Transformer model module, feature extraction is performed on the concatenated blocks to obtain the encoded features corresponding to the concatenated blocks, including:

[0020] PE′=LN(PE)

[0021] PE″=Attention(PE′)+PE′

[0022] PE″′=MLP(LN(PE″))+PE″

[0023] Where PE is the feature vector corresponding to the concatenation block, LN is the linear transformation, PE′ is the feature vector obtained by performing a linear transformation on the feature vector corresponding to the concatenation block, Attention is the attention network transformation, Attention(PE′) is the feature vector obtained by performing an attention network transformation on the feature vector PE′, PE″ is the vector sum of the feature vector PE′ and the feature vector obtained by performing an attention network transformation on the feature vector PE′, MLP(LN(PE″)) is the feature vector obtained by first performing a linear transformation on the feature vector PE″ and then inputting the feature vector obtained by the linear transformation into the perceptron layer, and PE″′ is the feature vector obtained by the Transformer model module performing one encoding.

[0024] In some embodiments, the calculation formula for the attention mechanism layer includes:

[0025] Q = X * W Q

[0026] K = X * W K

[0027] V = X * W V

[0028]

[0029] Where X is the concatenation vector corresponding to the concatenation block, Q is the query vector, K is the key vector, V is the value vector, and W is the value vector. Q W is the parameter matrix for linearly mapping concatenating vector X to query vector Q. K W is the parameter matrix for linearly mapping the concatenated vector X to the key vector K. V K is the parameter matrix for linearly mapping concatenating vector X to value vector V. T Let d be the transpose of the key vector K. k Let Q be the number of columns in the key vector K, softmax be the function that normalizes the columns, and Attention(Q,K,V) be the relation weights.

[0030] In some embodiments, the method further includes:

[0031] By calling the Transformer model module multiple times, feature extraction is performed on the concatenated block multiple times to obtain the encoded features corresponding to the concatenated block. The input and output vectors of the Transformer model module remain unchanged in each call.

[0032] In some embodiments, the loss function of the pimple category prediction model is:

[0033]

[0034] Where Loss is the multi-class weighted cross-entropy loss, z is the total number of pimple images, n is the number of pimple categories, and W i Let i be the class weight of the i-th class. Predict the probability that the j-th real sample is a pimple of the i-th category. The probability of predicting the j-th sample as a pimple of the i-th category.

[0035] In some embodiments, acne categories include at least one of pimples, post-acne erythema, inflammatory papules, pustules, nodules, and cysts.

[0036] Secondly, embodiments of this application provide a method for predicting acne types, including:

[0037] Obtain images of the pimples to be predicted;

[0038] The pimple image to be predicted is input into the pimple category prediction model to obtain the pimple category corresponding to the pimple image. The pimple category prediction model is trained based on the method in the first aspect.

[0039] Thirdly, embodiments of this application provide an electronic device, including:

[0040] At least one processor, and

[0041] A memory that is communicatively connected to at least one processor, wherein,

[0042] The memory stores instructions that can be executed by at least one processor, such that the instructions are executed by at least one processor to enable the at least one processor to perform the method of the first aspect.

[0043] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer-executable instructions for causing an electronic device to perform the methods of the first or second aspect.

[0044] The beneficial effects of this application's embodiments: Unlike existing technologies, this application provides a training method and related apparatus for a pimple category prediction model. The pimple category prediction model includes a feature extraction module, a Transformer model module, and a classifier module. The method includes: acquiring an image dataset, wherein the image dataset includes pimple images of various pimple categories; combining several pimple images to generate several pimple image patches, wherein each pimple image patch includes the same number of pimple images; processing the pimple image patches based on the feature extraction module to generate processed pimple image patches; extracting features from the processed pimple image patches based on the Transformer model module to obtain the encoded features corresponding to the processed pimple image patches, wherein the Transformer model module extracts features from the processed pimple image patches through an attention mechanism; predicting the pimple category based on the encoded features using the classifier module to obtain the pimple category corresponding to the pimple image; constructing a loss function, and training the pimple category prediction model based on the loss function until the loss function converges, wherein the loss function includes a multi-class weighted cross-entropy loss.

[0045] On the one hand, by collecting acne images of various types of acne, combining several acne images, generating several acne image blocks, and processing the acne image blocks based on the feature extraction module, this application can better preprocess acne images.

[0046] On the other hand, the attention mechanism of the Transformer model module is used to extract features from the processed pimple image patches. Then, the model is trained by constructing a loss function that includes multi-class weighted cross-entropy loss. Through the attention mechanism, the model can pay more attention to the differences in pimple features under different environments, thereby improving the accuracy of pimple category recognition. Attached Figure Description

[0047] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0048] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart illustrating a training method for a pimple category prediction model provided in an embodiment of this application.

[0050] Figure 3 yes Figure 2A detailed flowchart of step S203 in the process;

[0051] Figure 4 This is a schematic diagram of a process for generating splicing blocks provided in an embodiment of this application;

[0052] Figure 5 yes Figure 4 A detailed flowchart of step S402 in the process;

[0053] Figure 6 yes Figure 2 A detailed flowchart of step S204 in the process;

[0054] Figure 7 This is a schematic diagram of the structure of a pimple category prediction model provided in an embodiment of this application;

[0055] Figure 8 This is a schematic diagram of the structure of a Transformer model module provided in an embodiment of this application;

[0056] Figure 9 This is a flowchart illustrating multiple calls to the Transformer model module provided in an embodiment of this application;

[0057] Figure 10 This is a schematic diagram of the training process of a pimple category prediction model provided in an embodiment of this application;

[0058] Figure 11 This is a flowchart illustrating a method for predicting acne types provided in an embodiment of this application;

[0059] Figure 12 This is a schematic diagram of the structure of a training device for a pimple category prediction model provided in an embodiment of this application;

[0060] Figure 13 This is a schematic diagram of the structure of a acne type prediction device provided in an embodiment of this application;

[0061] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0062] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0065] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0066] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0067] Before providing a detailed description of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0068] (1) A neural network, also known as a neural network (NNs) or connection model, is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks to perform distributed parallel information processing. Neural networks rely on the complexity of the system to process information by adjusting the interconnections between a large number of internal nodes. Specifically, a neural network can be composed of neural units, which can be understood as a neural network with an input layer, hidden layers, and an output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the intermediate layers are hidden layers. Neural networks with many hidden layers are called deep neural networks (DNNs). The work of each layer in a neural network can be described by the mathematical expression y = a(W·x + b). From a physical perspective, the work of each layer in a neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space: 1. Dimensional increase / decrease; 2. Magnification / reduction; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are performed by "W·x", operation 4 by "+b", and operation 5 by "a()". The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within that class. W is the weight matrix of each layer in the neural network, where each value represents the weight of a neuron in that layer. This matrix W determines the spatial transformation from the input space to the output space, meaning that the W of each layer in the neural network controls how the space is transformed. The purpose of training the neural network is to ultimately obtain the weight matrices of all layers in the trained neural network. Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0069] It should be noted that, in the embodiments of this application, the models used for machine learning tasks are essentially neural networks. Common components in neural networks include convolutional layers, pooling layers, normalization layers, and deconvolutional layers. By assembling these common components in neural networks, a model is designed. When the model parameters (weight matrices of each layer) are determined such that the model error meets a preset condition or the number of model parameters is adjusted to reach a preset threshold, the model converges.

[0070] The convolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform convolution operations on the image. The purpose of convolution is to extract different features from the input image. The first convolutional layer may only extract some low-level features such as edges, lines, and corners, while deeper convolutional layers can iteratively extract more complex features from low-level features.

[0071] A deconvolutional layer is used to map a low-dimensional space to a high-dimensional space while preserving the connections / patterns between them (the connections during convolution). A deconvolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform deconvolution operations on the image. Generally, framework libraries used for designing neural networks (such as the PyTorch library) have a built-in `upsumple()` function, which allows for low-dimensional to high-dimensional spatial mapping.

[0072] Pooling layers mimic the human visual system, allowing for dimensionality reduction or representing images with higher-level features. Common pooling operations include max pooling, mean pooling, random pooling, median pooling, and combined pooling. Typically, pooling layers are periodically inserted between convolutional layers in neural networks to achieve dimensionality reduction.

[0073] The normalization layer is used to normalize all neurons in the intermediate layer to prevent gradient explosion and gradient vanishing.

[0074] (2) A loss function is a function that maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. The loss function is a non-negative real number function used to quantify the difference between the predicted label and the true label. In applications, the loss function is often used as a learning criterion in relation to optimization problems, i.e., solving and evaluating the model by minimizing the loss function. For example, it is used for parametric estimation in statistics and machine learning. During the training of a neural network, because we want the output of the neural network to be as close as possible to the actual predicted value, we can compare the current network's predicted value with the actual target value, and then update the weight matrix of each layer of the neural network based on the difference between the two (however, there is usually an initialization process before the first update, i.e., pre-configuring the parameters for each layer in the neural network). For example, if the network's predicted value is too high, the weight matrix is ​​adjusted to make it predict lower, and this adjustment continues until the neural network can predict the actual target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value," which is the loss function or objective function. These are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, a higher output value (loss) of the loss function indicates a greater difference, so training the neural network becomes the process of minimizing this loss as much as possible.

[0075] (3) Attention mechanisms, also known as attention models, are a resource allocation scheme that uses limited computing resources to process more important information, and are a major means of solving the problem of information overload. Neural attention mechanisms enable neural networks to focus on a subset of their inputs (or features): selecting specific inputs.

[0076] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0077] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0078] like Figure 1 As shown, the application environment 100 includes an electronic device 101 and a server 102, which communicate with each other via wired or wireless communication.

[0079] The electronic device 101 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The electronic device 101 may include a client, which can be a video client, browser client, online shopping client, instant messaging client, etc. This application does not limit the type of client.

[0080] Electronic device 101 and server 102 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this. Electronic device 101 can acquire image datasets and train a pimple category prediction model, or electronic device 101 can acquire a target image and predict the pimple category of the target image, and display the target image and its pimple category on a visualization interface. The target image can be an image stored in the memory of electronic device 101 or an image sent by other devices.

[0081] Alternatively, electronic device 101 can receive the acne category of the target image sent by server 102 and display the target image and its acne category on a visual interface. Users can browse acne images stored in the electronic device and trigger an acne category prediction command for any acne image by triggering the acne category prediction button corresponding to that acne image. The electronic device can respond to the acne category prediction command by acquiring the acne image through an image acquisition device and using the acne image as the target image. The image acquisition device can be built into electronic device 101 or externally connected to electronic device 101; this application does not limit this.

[0082] Electronic device 101 can send the target image to server 102 and receive the pimple category prediction value of the target image returned by server 102, and then display the target image and its pimple category prediction value on a visualization interface so that users can understand the prediction results of the target image.

[0083] It is understood that electronic device 101 can refer to one of a plurality of electronic devices, and this application embodiment only uses electronic device 101 as an example for illustration. Those skilled in the art will know that the number of the above-mentioned electronic devices can be more or less. For example, there may be only one electronic device, or there may be dozens or hundreds of electronic devices, or more. This application embodiment does not limit the number and type of electronic devices.

[0084] Among them, server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0085] Server 102 and electronic device 101 can be connected directly or indirectly via wired or wireless communication, which is not limited herein. Server 102 can maintain a acne image database to store multiple acne images. Server 102 can receive acne category prediction instructions and acne images sent by electronic device 101, and based on the acne category prediction instructions, perform acne category prediction on the acne images to obtain the acne category prediction value of the acne images, and then send the acne category prediction value of the acne images to electronic device 101.

[0086] It is understood that the number of servers 102 described above may be more or less, and this application embodiment does not limit this. Of course, servers 102 may also include other functional servers in order to provide more comprehensive and diversified services.

[0087] Example 1

[0088] Please see Figure 2 , Figure 2 This is a flowchart illustrating a training method for a pimple category prediction model provided in an embodiment of this application.

[0089] In this embodiment, the training method for the acne category prediction model is applied to an electronic device. Specifically, the training method is executed by one or at least two processors of the electronic device. The acne category prediction model includes a feature extraction module, a Transformer model module, and a classifier module.

[0090] like Figure 2 As shown, the training method for this acne category prediction model includes:

[0091] Step S201: Obtain an image dataset, wherein the image dataset includes acne images of various acne categories;

[0092] Specifically, the image dataset consists of facial images of acne in various categories. That is, the image dataset includes an acne category dataset. For example, each image in the dataset includes a face and acne on the face, and each image is a three-channel color image. Multiple acne images are collected, and each acne image is labeled with a category label to represent the acne category of the image. In some embodiments, the acne categories include comedones, post-acne erythema, inflammatory papules, pustules, nodules, and cysts, totaling six categories. A one-hot class labeling algorithm is used to label the acne images, for example, [0,1,0,0,0,0] represents the acne category as post-acne erythema.

[0093] Furthermore, to provide additional intra-class and inter-class relationships, embodiments of this application also process the category labels of acne categories to obtain soft labels. For example, [0,1,0,0,0,0] represents the acne category as post-acne erythema, and after processing, the resulting soft label is [0,1,0.8,0.02,0.04,0.03,0]. It can be understood that the position corresponding to each acne category in the soft label represents the probability value of that category.

[0094] In this embodiment, the image dataset includes acne images from different application scenarios, with varying shooting qualities, and for different acne categories. It is understood that because the acne images are relatively small, this embodiment performs a normalization operation on them. Specifically, the size of the acne images is adjusted to a preset resolution, for example, adjusting the resolution to 30*30. The size of the acne images in this embodiment can also be other resolutions, and this is not limited here.

[0095] It is understood that this image dataset can be color ID photos or color selfies captured by an image acquisition device. It is also understood that the image dataset can be data from existing open-source face databases, such as the FERET face database, the CMU Multi-PIE face database, or the YALE face database. There are no restrictions on the source of the image samples, as long as the images are color images including faces and blemishes, such as RGB format face images.

[0096] Step S202: Combine several pimple images to generate several pimple image blocks, wherein each pimple image block includes the same number of pimple images;

[0097] Specifically, several pimple images are combined into a single pimple image patch. For example, N pimple images are combined into one pimple image patch. Assuming a pimple image patch is a group of patches, then Patch = {x0, x1, x2, ..., x...} n This method combines multiple pimple images to generate multiple pimple image blocks. The value of N is set according to specific needs and is not limited here, but N ≥ 2; for example, N = 4, 9, 16, etc.

[0098] In this embodiment of the application, the pimple images in a pimple image block may include the same pimple images or different pimple images. It should be noted that the pimples in the same pimple image block are of the same category.

[0099] Understandably, after normalizing each pimple image, the size of each pimple image is 30*30. Taking the pimple image as an RGB format as an example, since the RGB image is a three-channel image, that is, each pixel has three values, namely R, G and B, the size of each pimple image is 30*30*3.

[0100] In this embodiment of the application, after combining several pimple images to generate a pimple image block, the pimple image block is flattened and expanded. For example, if a pimple image block includes 9 pimple images, the pimple image block is flattened and expanded, and the size of the feature map obtained is 9*30*30*3, that is, 9*2700. That is, the vector size corresponding to the pimple image block is 9*2700.

[0101] Step S203: Based on the feature extraction module, process the pimple image patch to generate a processed pimple image patch;

[0102] Specifically, the feature extraction module includes an embedding layer, which has a fully connected network structure.

[0103] Please refer to the following: Figure 3 , Figure 3 yes Figure 2 A detailed flowchart of step S203 in the process;

[0104] like Figure 3 As shown, step S203 includes:

[0105] Step S2031: Input the vector corresponding to the pimple image patch into the embedding layer to generate the feature vector corresponding to the processed pimple image patch. The size of the feature vector is related to the number of pimple categories. The network structure of the embedding layer is a fully connected layer.

[0106] For example, the vector size corresponding to a pimple image patch is 9*2700. The 9*2700 feature vector is passed through the embedding layer to generate the processed feature vector corresponding to the pimple image patch. The size of this feature vector is K*512, where K is the number of pimple categories. For example, if the number of pimple categories is 6, then the size of the feature vector obtained after the embedding layer is 6*512.

[0107] It should be noted that the feature extraction module in this embodiment can be a convolutional network, which includes convolutional layers, pooling layers, and fully connected layers. The feature extraction module is composed of convolutional layers, pooling layers, and fully connected layers stacked together.

[0108] Step S204: Based on the Transformer model module, feature extraction is performed on the processed pimple image patch to obtain the encoded features corresponding to the processed pimple image patch. The Transformer model module extracts features from the processed pimple image patch through an attention mechanism.

[0109] Specifically, after the feature extraction module processes the pimple image patch to obtain the processed pimple image patch, the Transformer model module performs feature extraction on the processed pimple image patch to obtain the encoded features corresponding to the processed pimple image patch. The Transformer model module performs feature extraction on the processed pimple image patch through an attention mechanism.

[0110] In this embodiment of the application, in order to be consistent with Transformer in the NLP field, a category block is generated and then merged with the processed pimple image block to obtain a stitched block.

[0111] Please see Figure 4 , Figure 4 This is a schematic diagram of a process for generating splicing blocks provided in an embodiment of this application;

[0112] like Figure 4As shown, the generated splice blocks include:

[0113] Step S401: Generate category blocks;

[0114] Specifically, each category block corresponds to a random vector, which is used for the classification task, i.e., to classify pimple images. For example, the random vector corresponding to the category block is cls_patch, and the size of the random vector is [1, 512].

[0115] It is understandable that the random vector corresponding to the category block is a category feature vector that can represent the entire stitched block, and extracting this category feature vector can be directly used to classify pimple images.

[0116] Step S402: Merge the category block with the processed pimple image block to obtain a stitched block;

[0117] Please refer to the following: Figure 5 , Figure 5 yes Figure 4 A detailed flowchart of step S402 in the process;

[0118] like Figure 5 As shown, step S402 includes:

[0119] Step S4021: Merge the random vector corresponding to the category block with the feature vector corresponding to the processed pimple image block to obtain the stitching vector corresponding to the stitching block.

[0120] Specifically, for the task of classifying pimple images, the random vector corresponding to the category block is merged with the feature vector corresponding to the processed pimple image block. For example, by inserting a random vector corresponding to the category block before the feature vector corresponding to the processed pimple image block, a spliced ​​block and its corresponding spliced ​​vector are obtained. The random vector corresponding to the category block is used as the category representation of the entire merged feature vector, that is, the category representation of the spliced ​​vector corresponding to the spliced ​​block. The random vector corresponding to the category block is used for the classification task, that is, to determine the pimple category corresponding to the pimple image.

[0121] Please refer to the following: Figure 6 , Figure 6 yes Figure 2 A detailed flowchart of step S204 in the process;

[0122] like Figure 6 As shown, step S204 includes:

[0123] Step S2041: Based on the Transformer model module, feature extraction is performed on the concatenated blocks to obtain the encoded features corresponding to the concatenated blocks. The Transformer model module extracts features from the concatenated blocks through an attention mechanism.

[0124] For details, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of the structure of a pimple category prediction model provided in an embodiment of this application;

[0125] like Figure 7 As shown, the acne category prediction model includes a feature extraction module, a Transformer model module, and a classifier module.

[0126] In this process, pimple images are input into the feature extraction module in the form of pimple image blocks. The feature extraction module generates category blocks and combines these category blocks with the processed pimple image blocks to generate stitched blocks. Each stitched block is input into the Transformer model module for feature extraction. After feature extraction by the Transformer model module, the final encoded features are input into the classifier module for classification.

[0127] Please refer to the following: Figure 8 , Figure 8 This is a schematic diagram of the structure of a Transformer model module provided in an embodiment of this application;

[0128] like Figure 8 As shown, this Transformer model module includes: a normalization layer, an attention mechanism layer, and a perceptron layer. Among them, This indicates vector concatenation.

[0129] Specifically, based on the Transformer model module, feature extraction is performed on the concatenated blocks to obtain the encoded features corresponding to the concatenated blocks, including:

[0130] PE′=LN(PE)

[0131] PE″=Attention(PE′)+PE′

[0132] PE″′=MLP(LN(PE″))+PE″

[0133] Where PE is the feature vector corresponding to the concatenation block, LN is the linear transformation, PE′ is the feature vector obtained by performing a linear transformation on the feature vector corresponding to the concatenation block, Attention is the attention network transformation, Attention(PE′) is the feature vector obtained by performing an attention network transformation on the feature vector PE′, PE″ is the vector sum of the feature vector PE′ and the feature vector obtained by performing an attention network transformation on the feature vector PE′, MLP(LN(PE″)) is the feature vector obtained by first performing a linear transformation on the feature vector PE″ and then inputting the feature vector obtained by the linear transformation into the perceptron layer, and PE″′ is the feature vector obtained by the Transformer model module performing one encoding.

[0134] In this embodiment of the application, the calculation formula for the attention mechanism layer includes:

[0135] Q = X * W Q

[0136] K = X * W K

[0137] V = X * W V

[0138]

[0139] Where X is the concatenation vector corresponding to the concatenation block, Q is the query vector, K is the key vector, V is the value vector, and W is the value vector. Q W is the parameter matrix for linearly mapping concatenating vector X to query vector Q. K W is the parameter matrix for linearly mapping the concatenated vector X to the key vector K. V K is the parameter matrix for linearly mapping concatenating vector X to value vector V. T Let d be the transpose of the key vector K. k Let Q be the number of columns in the key vector K, softmax be the function that normalizes the columns, and Attention(Q,K,V) be the relation weights.

[0140] It is understandable that X is the input vector of the attention mechanism layer, that is, the concatenation vector corresponding to the concatenation block.

[0141] In this embodiment, the Transformer model module extracts features from the stitched blocks using an attention mechanism. This attention mechanism includes soft attention, hard attention, key-value pair attention, multi-head attention, structured attention, or self-attention. Preferably, the attention mechanism in this embodiment is a self-attention mechanism, i.e., a self-attention model. Self-attention models often employ a query-key-value (QKV) pattern.

[0142] In this embodiment, the attention mechanism of the Transformer model module is used to extract features from the stitched blocks, which can better calculate the relationship weight between each pimple category and other pimple categories. This allows the network to better learn the different feature differences of the same pimple category due to changes in environmental differences such as brightness, thereby improving the accuracy of pimple category recognition.

[0143] In this embodiment of the application, the method further includes: repeatedly calling the Transformer model module to extract features.

[0144] Please refer to the following: Figure 9 , Figure 9 This is a flowchart illustrating multiple calls to the Transformer model module provided in an embodiment of this application;

[0145] like Figure 9 As shown, the process of calling the Transformer model module multiple times includes:

[0146] Step S901: By calling the Transformer model module multiple times, feature extraction is performed on the concatenated block multiple times to obtain the encoded features corresponding to the concatenated block. The size of the input vector and output vector remains unchanged in each call to the Transformer model module.

[0147] Specifically, by calling the Transformer model module multiple times, the normalization layer, attention mechanism layer, and perceptron layer of the Transformer model module extract features from the concatenated blocks multiple times. For example, by calling the Transformer model module L times, the final output vector, i.e., the final encoded feature PE, is obtained. L In this approach, the size of the input and output vectors remains constant for each call to the Transformer model module. For example, the size of the input vector for each feature extraction is 7*512, and the size of the output vector for each feature extraction is also 7*512. Clearly, the final output vector, i.e., the final encoded feature PE, will be the largest output vector obtained in the last call. L Its size is also 7*512.

[0148] Step S205: Based on the classifier module, predict the acne category of the encoded features to obtain the acne category corresponding to the acne image;

[0149] In this embodiment, the classifier module includes a Multi-Layer Perceptron (MLP), which is a multilayer perceptron network structure. This MLP network structure includes an input layer, hidden layers, and an output layer. The neurons in the hidden and output layers are functional neurons with activation functions. Information is passed layer by layer through the MLP to obtain the final output. The final output includes each pimple category and its corresponding probability value. Based on the probability value corresponding to each pimple category, the pimple category corresponding to the pimple image is determined; that is, the pimple category with the highest probability value is determined as the pimple category corresponding to the pimple image.

[0150] Step S206: Construct a loss function and train the acne category prediction model based on the loss function until the loss function converges. The loss function includes multi-class weighted cross-entropy loss.

[0151] Specifically, the loss function of this acne category prediction model is:

[0152]

[0153] Where Loss is the multi-class weighted cross-entropy loss, z is the total number of pimple images, n is the number of pimple categories, and W i Let i be the class weight of the i-th class. Predict the probability that the j-th real sample is a pimple of the i-th category. The probability of predicting the j-th sample as a pimple of the i-th category.

[0154] In this embodiment, a multi-class weighted cross-entropy loss is constructed to address the problem of significant differences in the distinguishability of acne category features during actual acne classification, thereby improving the accuracy of acne category prediction.

[0155] Please refer to the following: Figure 10 , Figure 10 This is a schematic diagram of the training process of a pimple category prediction model provided in an embodiment of this application;

[0156] like Figure 10 As shown, the training process for this pimple category prediction model includes:

[0157] Step S1001: Construct a pimple category prediction model;

[0158] Specifically, the acne category prediction model includes a normalization layer, an attention mechanism layer, and a perceptron layer. For details, please refer to the above embodiments, which will not be repeated here.

[0159] Step S1002: Construct the loss function;

[0160] Specifically, the loss function includes multi-class weighted cross-entropy loss. For example, the loss function for a pimple category prediction model is:

[0161]

[0162] Where Loss is the multi-class weighted cross-entropy loss, z is the total number of pimple images, n is the number of pimple categories, and W i Let i be the class weight of the i-th class. Predict the probability that the j-th real sample is a pimple of the i-th category. The probability of predicting the j-th sample as a pimple of the i-th category.

[0163] Step S1003: Iteratively train the acne category prediction model based on the loss function;

[0164] Specifically, the loss function includes a multi-class weighted cross-entropy loss function. By utilizing the image dataset, the image dataset is processed, and then the pimple category prediction model is iteratively trained based on the constructed multi-class weighted cross-entropy loss function.

[0165] Step S1004: Is the number of iterations greater than the threshold for the first iteration?

[0166] Specifically, this application uses the Adam algorithm (Adaptive Moment Estimation Algorithm) to optimize model parameters. For example, the number of iterations is set to 500, the initial learning rate is set to 0.001, the weight decay is set to 0.0005, and the learning rate decays to 1 / 10 of its original value every 50 iterations.

[0167] Understandably, the Adam algorithm (Adaptive Moment Estimation Algorithm) can be seen as a combination of the momentum method and the RMSprop algorithm. It not only uses momentum as a parameter to update the direction, but also can adaptively adjust the learning rate.

[0168] Specifically, it checks whether the number of iterations is greater than the first threshold, which is preset, for example, to 500. If the number of iterations is greater than the first threshold, proceed to step S1006: training complete; if the number of iterations is not greater than the first threshold, proceed to step S1005: whether the loss of the acne category prediction model is less than the first loss threshold.

[0169] It is understandable that the threshold for the first count can be set according to specific needs, and no restrictions are imposed here.

[0170] Step S1005: Is the loss of the pimple category prediction model less than the first loss threshold?

[0171] Specifically, determine whether the loss of the acne category prediction model is less than the first loss threshold. If yes, proceed to step S1006: training complete; otherwise, return to step S1003: iteratively train the acne category prediction model based on the loss function.

[0172] In the embodiments of this application, the first loss threshold can be set to 0.0005 or 0.001. It is understood that the first loss threshold can be set according to specific needs and is not limited here.

[0173] Step S1006: Training complete;

[0174] Understandably, after training is complete, a trained acne category prediction model is obtained. At this point, the acne category prediction model can be called to predict the acne category corresponding to the acne image to be predicted.

[0175] In this embodiment, a training method for a pimple category prediction model is provided. The pimple category prediction model includes a feature extraction module, a Transformer model module, and a classifier module. The method includes: acquiring an image dataset, wherein the image dataset includes pimple images of various pimple categories; combining several pimple images to generate several pimple image patches, wherein each pimple image patch includes the same number of pimple images; processing the pimple image patches based on the feature extraction module to generate processed pimple image patches; extracting features from the processed pimple image patches based on the Transformer model module to obtain the encoded features corresponding to the processed pimple image patches, wherein the Transformer model module extracts features from the processed pimple image patches through an attention mechanism; predicting the pimple category based on the encoded features using the classifier module to obtain the pimple category corresponding to the pimple image; constructing a loss function, and training the pimple category prediction model based on the loss function until the loss function converges, wherein the loss function includes a multi-class weighted cross-entropy loss.

[0176] On the one hand, by collecting acne images of various types of acne, combining several acne images, generating several acne image blocks, and processing the acne image blocks based on the feature extraction module, this application can better preprocess acne images.

[0177] On the other hand, the attention mechanism of the Transformer model module is used to extract features from the processed pimple image patches. Then, the model is trained by constructing a loss function that includes multi-class weighted cross-entropy loss. Through the attention mechanism, the model can pay more attention to the differences in pimple features under different environments, thereby improving the accuracy of pimple category recognition.

[0178] Example 2

[0179] After training the acne category prediction model using the method provided in the above embodiments, a trained acne category prediction model is obtained, which can be used to predict acne categories.

[0180] It is understood that the above embodiment one is the training stage of the acne category prediction model, and embodiment two of this application is the prediction stage of the acne category prediction model. The prediction stage requires the use of a classifier module to predict the acne category in order to obtain the acne category corresponding to the acne image to be predicted.

[0181] For details, please refer to [link / reference]. Figure 11 , Figure 11 This is a flowchart illustrating a method for predicting acne types provided in an embodiment of this application;

[0182] like Figure 11 As shown, the prediction method for this type of acne includes:

[0183] Step S1101: Obtain the image of the pimple to be predicted;

[0184] Specifically, after obtaining the trained acne category prediction model, the acne images to be predicted are acquired.

[0185] The acne category prediction model is trained from the composite neural network (including feature extraction module, Transformer model module and classifier module) in the above embodiments. It has the same structure and function as the composite neural network. The specific processing method of the acne category prediction model on the acne image to be predicted is the same as the processing method of the composite neural network on the acne image to be predicted in the above embodiments, and will not be described in detail here.

[0186] Step S1102: Input the pimple image to be predicted into the pimple category prediction model to obtain the pimple category corresponding to the pimple image to be predicted. The pimple category prediction model is trained based on the training method of the pimple category prediction model.

[0187] Specifically, the pimple image to be predicted is input into the classifier module of the pimple category prediction model. The classifier module outputs a category probability vector, which includes the probability value of each pimple category corresponding to the pimple image to be predicted. Thus, the pimple category with the highest probability value is determined as the pimple category predicted by the pimple category prediction model.

[0188] In this embodiment of the application, a method for predicting acne categories is provided, comprising: acquiring an acne image to be predicted; inputting the acne image to be predicted into an acne category prediction model to obtain the acne category corresponding to the acne image to be predicted, wherein the acne category prediction model is trained based on the training method of the acne category prediction model.

[0189] On the one hand, by collecting acne images of various types of acne, combining several acne images, generating several acne image blocks, and processing the acne image blocks based on the feature extraction module, this application can better preprocess acne images.

[0190] On the other hand, the attention mechanism of the Transformer model module is used to extract features from the processed pimple image patches. Then, the model is trained by constructing a loss function that includes multi-class weighted cross-entropy loss. Through the attention mechanism, the model can pay more attention to the differences in pimple features under different environments, thereby improving the accuracy of pimple category recognition.

[0191] Example 3

[0192] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a training device for a pimple category prediction model provided in an embodiment of this application;

[0193] The training device for the acne category prediction model is applied to an electronic device; specifically, the training device for the acne category prediction model is applied to one or at least two processors of the electronic device.

[0194] The acne category prediction model includes a feature extraction module, a Transformer model module, and a classifier module.

[0195] like Figure 12 As shown, the training device 120 for the acne category prediction model includes:

[0196] The acquisition module 121 is used to acquire an image dataset, wherein the image dataset includes acne images of various acne categories;

[0197] The combination module 122 is used to combine several pimple images to generate several pimple image blocks, wherein each pimple image block includes the same number of pimple images;

[0198] The processing module 123 is used to process the pimple image patch based on the feature extraction module to generate the processed pimple image patch.

[0199] The extraction module 124 is used to extract features from the processed pimple image patch based on the Transformer model module to obtain the encoded features corresponding to the processed pimple image patch. The Transformer model module extracts features from the processed pimple image patch through an attention mechanism.

[0200] The prediction module 125 is used to predict the pimple category based on the encoded features of the classifier module, so as to obtain the pimple category corresponding to the pimple image;

[0201] Training module 126 is used to construct a loss function and train the acne category prediction model based on the loss function until the loss function converges. The loss function includes multi-class weighted cross-entropy loss.

[0202] In the embodiments of this application, the training device for the acne category prediction model can also be built from hardware devices. For example, the training device for the acne category prediction model can be built from one or more chips, and the chips can work together to complete the training method for the acne category prediction model described in the above embodiments. As another example, the training device for the acne category prediction model can also be built from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM processors (Advanced RISC Machines, ARM) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0203] The training device for the acne category prediction model in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0204] The training device for the acne category prediction model in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0205] The training device for the acne category prediction model provided in this application embodiment can achieve... Figure 2 To avoid repetition, the various processes involved will not be described in detail here.

[0206] It should be noted that the training device for the acne category prediction model described above can execute the training method for the acne category prediction model provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the acne category prediction model training device can be found in the training method for the acne category prediction model provided in the embodiments of this application.

[0207] In this embodiment, a training device for a pimple category prediction model is provided, comprising: an acquisition module for acquiring an image dataset, wherein the image dataset includes pimple images of various pimple categories; a combination module for combining several pimple images to generate several pimple image patches, wherein each pimple image patch includes the same number of pimple images; a processing module for processing the pimple image patches based on a feature extraction module to generate processed pimple image patches; an extraction module for extracting features from the processed pimple image patches based on a Transformer model module to obtain the encoded features corresponding to the processed pimple image patches, wherein the Transformer model module extracts features from the processed pimple image patches through an attention mechanism; a prediction module for predicting the pimple category based on the encoded features using a classifier module to obtain the pimple category corresponding to the pimple image; and a training module for constructing a loss function and training the pimple category prediction model based on the loss function until the loss function converges, wherein the loss function includes a multi-class weighted cross-entropy loss.

[0208] On the one hand, by collecting acne images of various types of acne, combining several acne images, generating several acne image blocks, and processing the acne image blocks based on the feature extraction module, this application can better preprocess acne images.

[0209] On the other hand, the attention mechanism of the Transformer model module is used to extract features from the processed pimple image patches. Then, the model is trained by constructing a loss function that includes multi-class weighted cross-entropy loss. Through the attention mechanism, the model can pay more attention to the differences in pimple features under different environments, thereby improving the accuracy of pimple category recognition.

[0210] Example 4

[0211] Please see Figure 13 , Figure 13 This is a schematic diagram of the structure of a acne type prediction device provided in an embodiment of this application;

[0212] like Figure 13 As shown, the acne category prediction device 130 includes:

[0213] Image acquisition module 131 is used to acquire images of pimples to be predicted;

[0214] The acne category prediction module 132 is used to input the acne image to be predicted into the acne category prediction model to obtain the acne category corresponding to the acne image to be predicted. The acne category prediction model is trained based on the training method of the acne category prediction model provided in the above embodiment.

[0215] In the embodiments of this application, the acne category prediction device can also be built from hardware devices. For example, the acne category prediction device can be built from one or more chips, and the chips can work together to complete the acne category prediction method described in the above embodiments. As another example, the acne category prediction device can also be built from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0216] The acne type prediction device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0217] The acne type prediction device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0218] The acne type prediction device provided in this application embodiment can achieve... Figure 11 To avoid repetition, the various processes involved will not be described in detail here.

[0219] It should be noted that the above-described acne category prediction device can execute the acne category prediction method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the embodiments of the acne category prediction device can be found in the acne category prediction method provided in the above embodiments.

[0220] In this embodiment of the application, a device for predicting acne types is provided, including: an image acquisition module for acquiring images of acne to be predicted;

[0221] The acne category prediction module is used to input the acne image to be predicted into the acne category prediction model to obtain the acne category corresponding to the acne image to be predicted. The acne category prediction model is trained based on the training method of the acne category prediction model provided in the above embodiment.

[0222] On the one hand, by collecting acne images of various types of acne, combining several acne images, generating several acne image blocks, and processing the acne image blocks based on the feature extraction module, this application can better preprocess acne images.

[0223] On the other hand, the attention mechanism of the Transformer model module is used to extract features from the processed pimple image patches. Then, the model is trained by constructing a loss function that includes multi-class weighted cross-entropy loss. Through the attention mechanism, the model can pay more attention to the differences in pimple features under different environments, thereby improving the accuracy of pimple category recognition.

[0224] Example 5

[0225] This application also provides an electronic device; please refer to [link / reference]. Figure 14 , Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0226] like Figure 14 As shown, the electronic device 140 includes at least one processor 141 and a memory 142 connected in communication. Figure 14 (Taking a bus connection and a single processor as an example).

[0227] The processor 141 provides computational and control capabilities to control the electronic device 140 to perform corresponding tasks, such as controlling the electronic device 140 to perform the training method of the acne category prediction model in any of the above method embodiments, including:

[0228] Obtain an image dataset, which includes images of acne of various acne categories;

[0229] Combine several pimple images to generate several pimple image blocks, wherein each pimple image block includes the same number of pimple images;

[0230] Based on the feature extraction module, the pimple image patch is processed to generate a processed pimple image patch;

[0231] Based on the Transformer model module, feature extraction is performed on the processed pimple image patches to obtain the encoded features corresponding to the processed pimple image patches. The Transformer model module extracts features from the processed pimple image patches through an attention mechanism.

[0232] Based on the classifier module, the coded features are used to predict the pimple category in order to obtain the pimple category corresponding to the pimple image;

[0233] Construct a loss function and train the acne category prediction model based on the loss function until the loss function converges. The loss function includes multi-class weighted cross-entropy loss.

[0234] Alternatively, the method for predicting acne types in any of the above method embodiments can be implemented, including:

[0235] Obtain images of the pimples to be predicted;

[0236] The pimple image to be predicted is input into the pimple category prediction model to obtain the pimple category corresponding to the pimple image. The pimple category prediction model is trained based on the training method of the pimple category prediction model.

[0237] On the one hand, by collecting acne images of various types of acne, combining several acne images, generating several acne image blocks, and processing the acne image blocks based on the feature extraction module, this application can better preprocess acne images.

[0238] On the other hand, the attention mechanism of the Transformer model module is used to extract features from the processed pimple image patches. Then, the model is trained by constructing a loss function that includes multi-class weighted cross-entropy loss. Through the attention mechanism, the model can pay more attention to the differences in pimple features under different environments, thereby improving the accuracy of pimple category recognition.

[0239] Processor 141 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0240] Memory 142, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the training method or prediction method for the acne category prediction model in the embodiments of this application. Processor 141 can implement the training method or prediction method for the acne category prediction model in any of the following method embodiments by running the non-transitory software programs, instructions, and modules stored in memory 142. Specifically, memory 142 may include volatile memory (VM), such as random access memory (RAM); memory 142 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 142 may also include combinations of the above types of memory.

[0241] In this embodiment, the memory 142 may further include memory remotely configured relative to the processor, and this remote memory may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0242] In this embodiment, the electronic device 140 may also have wired or wireless network interfaces, keyboards, and input / output interfaces for input and output. The electronic device 140 may also include other components for implementing device functions, which will not be described in detail here.

[0243] This application also provides a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the training method of the acne category prediction model or the acne category prediction method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0244] This application also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. A processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the method steps of the training method for the acne category prediction model or the acne category prediction method provided in the above embodiments.

[0245] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0246] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0247] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations as described above in different aspects of this application, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for a pimple category prediction model, characterized in that, The acne category prediction model includes a feature extraction module, a Transformer model module, and a classifier module. The feature extraction module includes an embedding layer. The method includes: Obtain an image dataset, wherein the image dataset includes acne images of various acne categories; Combine several of the pimple images to generate several pimple image blocks, wherein each pimple image block includes the same number of pimple images; Based on the feature extraction module, the pimple image patch is processed to generate a processed pimple image patch; Based on the Transformer model module, feature extraction is performed on the processed acne image patch to obtain the encoded features corresponding to the processed acne image patch. The Transformer model module extracts features from the processed acne image patch through an attention mechanism. Based on the classifier module, the coded features are used to predict the acne category in order to obtain the acne category corresponding to the acne image. Construct a loss function, and train the acne category prediction model based on the loss function until the loss function converges. The loss function includes multi-class weighted cross-entropy loss. The step of processing the pimple image patch based on the feature extraction module to generate a processed pimple image patch includes: The vector corresponding to the pimple image patch is input into the embedding layer to generate the processed feature vector corresponding to the pimple image patch. The size of the feature vector is related to the number of pimple categories. The network structure of the embedding layer is a fully connected layer. The method further includes: generating a category block, merging the category block with the processed acne image block to obtain a stitched block, specifically including: Generate category blocks, wherein each category block corresponds to a random vector. The random vector corresponding to the category block is merged with the feature vector corresponding to the processed pimple image block to obtain the splicing vector corresponding to the splicing block.

2. The method according to claim 1, characterized in that, The step of extracting features from the processed acne image patch based on the Transformer model module to obtain the encoded features corresponding to the processed acne image patch includes: Based on the Transformer model module, feature extraction is performed on the concatenated block to obtain the encoded features corresponding to the concatenated block. The Transformer model module performs feature extraction on the concatenated block through an attention mechanism.

3. The method according to claim 2, characterized in that, The Transformer model module includes a normalization layer, an attention mechanism layer, and a perceptron layer. Based on the Transformer model module, feature extraction is performed on the concatenated blocks to obtain the encoded features corresponding to the concatenated blocks, including: in, The feature vector corresponding to the spliced ​​block. For linear transformation, The feature vector is obtained by performing a linear transformation on the feature vector corresponding to the spliced ​​block. For attention network transformation, For the feature vector The feature vector obtained by performing attention network transformation. For feature vectors and the eigenvectors The sum of the feature vectors obtained by performing attention network transformation. First, for the eigenvectors Perform a linear transformation, and then input the feature vector obtained from the linear transformation into the feature vector obtained from the perceptron layer. The feature vector obtained by encoding the Transformer model module once.

4. The method according to claim 3, characterized in that, The calculation formula for the attention mechanism layer includes: in, This is the concatenation vector corresponding to the concatenation block. For query vector, For key vectors, For value vectors, For concatenating vectors Perform linear mapping to query vector The parameter matrix, For concatenating vectors Perform linear mapping to key vector The parameter matrix, For concatenating vectors Perform linear mapping to value vector The parameter matrix, Key vector transpose, Key vector The number of columns, This is a function that normalizes columns. This represents the relation weight.

5. The method according to claim 3 or 4, characterized in that, The method further includes: By repeatedly calling the Transformer model module, feature extraction is performed on the concatenated block multiple times to obtain the encoded features corresponding to the concatenated block. The input and output vectors of the Transformer model module remain unchanged in each call.

6. The method according to claim 1, characterized in that, The loss function of the acne category prediction model is: in, For multi-class weighted cross-entropy loss, The total number of pimple images. The number of acne types, For the first Category weights For the true first The predicted value for the nth sample is... The probability of different types of acne. For the predicted first The predicted value for the nth sample is... The probability of different types of acne.

7. The method according to any one of claims 1-4, 6, characterized in that, The acne categories include at least one of the following: blackheads, post-acne erythema, inflammatory papules, pustules, nodules, and cysts.

8. A method for predicting acne types, characterized in that, include: Obtain images of the pimples to be predicted; The pimple image to be predicted is input into the pimple category prediction model to obtain the pimple category corresponding to the pimple image to be predicted, wherein the pimple category prediction model is trained based on the method described in any one of claims 1-7.

9. An electronic device, characterized in that, include: At least one processor, and The memory communicatively connected to the at least one processor, wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method according to any one of claims 1-8.

10. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores computer-executable instructions for causing an electronic device to perform the method as described in any one of claims 1-8.

Images