A method for training a color spot detection model and related device

Abstract

The embodiments of the present invention relate to the field of target detection technology, and more particularly to a method for training a color spot detection model and related devices. By cutting images in an image sample set into candidate area images, the interference of regional features (such as background features and facial features) that do not contain color spots on the training model is reduced, thereby improving the accuracy of model training. By performing resolution enhancement processing and size adjustment on each candidate area image containing color spots to obtain multiple training images of consistent sizes, the interference of the size on model learning can be effectively avoided, and the characteristics of the color spots can be enhanced. In addition, during the training process, the number of color spots is introduced to constrain the position of the color spots, so that the predicted position of the color spots can be trained in the same direction as the position of the actual color spots, further improving the accuracy of model training. The improvement of multiple training accuracy makes the detection accuracy of the color spot detection model trained by this method higher.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of target detection technology, and in particular to a method for training a color spot detection model and related devices. Background Art

[0002] With the rapid development of mobile communication technology and the improvement of people's living standards, various smart terminals have been widely used in people's daily work and life, making people more and more accustomed to using APPs and other software. As a result, the demand for APPs with functions such as beauty selfies and photo skin testing has become more and more. Therefore, many users hope that such APPs can automatically analyze the pigmentation of the face and propose targeted skin improvement plans based on the pigmentation.

[0003] Existing facial spot detection mainly uses image processing methods to extract possible spots in the area. This method is too complicated and easily affected by factors such as light, resulting in low detection accuracy. Summary of the invention

[0004] The main technical problem solved by the embodiments of the present invention is to provide a method for training a color spot detection model and a related device, which can accurately classify and locate color spots.

[0005] To solve the above technical problems, in a first aspect, an embodiment of the present invention provides a method for training a wrinkle detection model, comprising:

[0006] Obtaining a sample set of images including faces;

[0007] From each image in the image sample set, respectively intercept at least one facial local image where color spots may exist, so as to obtain a plurality of candidate area images;

[0008] Performing resolution enhancement processing and size adjustment on each candidate region image to obtain a plurality of training images of the same size, wherein the resolution of the training image is higher than the resolution of the candidate region image;

[0009] A plurality of training images annotated with real labels are used as training sets, and a preset convolutional neural network is trained so that the preset convolutional neural network learns the real labels of the plurality of training images to obtain a color spot detection model, wherein the real label of the target training image includes the real position, real category and real number of the color spots in the target training image in the target training image, and the target training image is any training image among the plurality of training images.

[0010] In some embodiments, the preset convolutional neural network includes a feature extraction module, a color spot target detection module and a color spot quantity detection module, and the multiple training images annotated with real labels are used as training sets to train the preset convolutional neural network so that the preset convolutional neural network learns the real labels of the multiple training images to obtain a color spot detection model, including:

[0011] Inputting the target training image into the feature extraction module to obtain training feature maps of at least two sizes;

[0012] Dividing the training feature map into two paths to obtain a first path of training feature maps and a second path of training feature maps;

[0013] Inputting the first training feature map into the color spot target detection module to obtain the predicted position and predicted category of the color spot in the target training image;

[0014] Inputting the second training feature map into the color spot quantity detection module to obtain the predicted number of color spots in the target training image;

[0015] Calculating the error between the predicted label of the target training image and the true label of the target training image according to a preset loss function, wherein the predicted label includes the predicted position, the predicted category, and the predicted quantity;

[0016] According to the error, the model parameters of the preset convolutional neural network are adjusted, and the step of inputting the target training image into the feature extraction module to obtain training feature maps of at least two sizes is returned until the preset convolutional neural network converges to obtain the color spot detection model.

[0017] In some embodiments, the feature extraction module includes multiple feature convolution layers, and the number of convolution kernels of the multiple feature convolution layers tends to first increase and then decrease as the number of feature convolution layers increases.

[0018] In some embodiments, the multiple feature convolution layers include a target feature convolution layer, the number of convolution kernels of the target feature convolution layer is the number of categories of the color spots, and the target feature convolution layer is a feature convolution layer connected to the pigment spot target detection module, which is used to output the training feature map.

[0019] In some embodiments, the preset loss function is a weighted sum of a position loss function, a category loss function, and a quantity loss function. The position loss function is used to calculate the error between the actual position and the predicted position, the category loss function is used to calculate the error between the actual category and the predicted category, and the quantity loss function is used to calculate the error between the actual quantity and the predicted quantity.

[0020] In some embodiments, calculating the error between the predicted label of the target training image and the true label of the target training image according to a preset loss function includes:

[0021] The error between the predicted label and the true label is calculated according to the following formula:

[0022]

[0023] Among them, λ obj is the preset position offset weight, α is the confidence loss weight, β is the quantity weight, M is the preset number of prediction boxes for predicting the color spot position, i is the number of the prediction box, T width is the width of the prediction box, T height is the height of the prediction box, (x, y, w, h) are the coordinates of the upper left corner, width and height of the prediction box respectively, T r Used to indicate the actual position of the color spot, P r is used to indicate the predicted position of the color spot, K is the number of categories of the color spot, T class is the true category of the color spot, P class is the predicted category of the spot, T conf is the confidence of the true label, P conf is the confidence of the predicted label, is the actual number of c-th type spots in K-type spots, is the predicted number of the c-th type of color spots in the K-type color spots.

[0024] In some embodiments, the step of performing resolution enhancement processing and size adjustment on each candidate region image to obtain a plurality of training images of the same size includes:

[0025] Acquire the sizes of a target candidate region image and a target training image, wherein the target candidate region image is any one of the multiple candidate region images, and the target training image is obtained by mapping the target candidate region image;

[0026] According to the first pixel coordinate of the target pixel point in the target training image, the size of the target candidate area image and the size of the target training image, obtaining the second pixel coordinate corresponding to the target pixel point in the target candidate area image, wherein the target pixel point is any pixel point in the target training image;

[0027] Acquire a plurality of neighboring pixel points of the second pixel coordinates, where the neighboring pixel points are pixel points adjacent to the second pixel coordinates in the target candidate area image;

[0028] Obtaining a weight of each of the neighborhood pixel points according to a positional relationship between the second pixel coordinate and the plurality of neighborhood pixel points;

[0029] According to the weight of each of the neighborhood pixels and the pixel value of each of the domain pixels, the pixel value of the target pixel is determined to obtain the target training image.

[0030] In order to solve the above technical problems, in a second aspect, an embodiment of the present invention provides a method for detecting color spots, comprising:

[0031] Obtain the face image to be tested;

[0032] intercepting at least one target facial local image where color spots may exist from the face image to be tested;

[0033] Detecting the at least one target facial local image using the color spot detection model described in the first aspect above, and obtaining the position, type and number of the color spots in the at least one target facial local image;

[0034] Based on the position, category and number of color spots of at least one target facial local image and the position of the target facial local image in the facial image to be tested, the position, category and number of color spots in the facial image to be tested are determined.

[0035] To solve the above technical problems, in a third aspect, an embodiment of the present invention provides an electronic device, including:

[0036] at least one processor, and

[0037] a memory communicatively coupled to the at least one processor, wherein:

[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect and the method described in the second aspect.

[0039] To solve the above technical problems, in a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable an electronic device to execute the method described in the first aspect and the method described in the second aspect.

[0040] Beneficial effects of the embodiments of the present invention: Different from the prior art, the method and related device for training a color spot detection model provided by the embodiments of the present invention can reduce the interference of regional features (such as background features and facial features) that do not contain color spots on the training model during the model training process by cutting images in the image sample set into candidate region images, thereby improving the accuracy of model training; by performing resolution enhancement processing and size adjustment on each candidate region image containing color spots to obtain multiple training images of the same size, the size adjustment can effectively avoid the interference of the size on model learning, and the resolution enhancement can enhance the characteristics of the color spots, thereby further improving the accuracy of model training; in addition, during the training process, by introducing the number of color spots to constrain the color spot position, the predicted position of the color spot can be trained in the same direction as the position of the real color spot, further improving the accuracy of model training; the improvement of multiple training accuracy makes the detection accuracy of the color spot detection model obtained by training through this method higher. In addition, by cutting images in the image sample set into candidate region images, since the size of the candidate region images is smaller than the original image, it is beneficial to reduce the amount of calculation in the training process, improve the speed of model training and calculation, and help to obtain a color spot detection model with small amount of calculation and high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0042] Figure 1 A schematic diagram of an operating environment of a method for training a color spot detection model provided in one embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of an electronic device provided by one embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a flow chart of a method for training a color spot detection model provided in one embodiment of the present invention;

[0045] Figure 4 A schematic diagram of capturing a candidate region image according to one embodiment of the present invention;

[0046] Figure 5 A schematic diagram of the structure of a color spot detection model provided by one embodiment of the present invention;

[0047] Figure 6 for Figure 3 A schematic diagram of a sub-process of step S24 in the method shown;

[0048] Figure 7 for Figure 3 A schematic diagram of a sub-process of step S23 in the method shown;

[0049] Figure 8 A schematic diagram of a neighborhood pixel point provided by one embodiment of the present invention;

[0050] Fig. 9 A schematic flow chart of a method for detecting color spots provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] It should be noted that, if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the flow chart. In addition, the words "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0054] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0055] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] Figure 1 Schematic diagram of the operating environment of the method for training a color spot detection model provided by an embodiment of the present invention. Figure 1, including an electronic device 10 and an image acquisition device 20, wherein the electronic device 10 and the image acquisition device 20 are communicatively connected.

[0057] The communication connection can be a wired connection, such as a fiber optic cable, or a wireless communication connection, such as a WIFI connection, a Bluetooth connection, a 4G wireless communication connection, a 5G wireless communication connection, and the like.

[0058] The image acquisition device 20 is used to acquire an image sample set including human faces, and can also be used to acquire a human face image to be tested. The image acquisition device 20 can be a terminal capable of capturing images, such as a mobile phone, tablet computer, video recorder or camera with a capturing function.

[0059] The electronic device 10 is a device that can run according to a program and automatically and quickly process massive amounts of data. It is usually composed of a hardware system and a software system, such as a computer, a smart phone, etc. The electronic device 10 can be a local device that is directly connected to the image acquisition device 20; it can also be a cloud device, such as a cloud server, a cloud host, a cloud service platform, a cloud computing platform, etc. The cloud device is connected to the image acquisition device 20 through a network, and the two are connected in communication through a predetermined communication protocol. In some embodiments, the communication protocol can be TCP / IP, NETBEUI, IPX / SPX, etc.

[0060] It is understandable that the image acquisition device 20 and the electronic device 10 may also be integrated together as an integrated device, such as a computer or smart phone with a camera.

[0061] The electronic device 10 receives the image sample set including the face sent by the image acquisition device 20, trains the image sample set to obtain a color spot detection model, and uses the color spot detection model to detect the color spot position and category of the face image to be tested sent by the image acquisition device 20. It can be understood that the training of the color spot detection model and the detection of the face image to be tested can also be performed on different electronic devices.

[0062] In the above Figure 1 Based on this, another embodiment of the present invention provides an electronic device 10, see Figure 2 , is a hardware structure diagram of an electronic device 10 provided in an embodiment of the present invention, specifically, as Figure 2 As shown, the electronic device 10 includes at least one processor 11 and a memory 12 ( Figure 2 (a bus connection and a processor are used as an example).

[0063] Among them, the processor 11 is used to provide computing and control capabilities to control the electronic device 10 to perform corresponding tasks, for example, to control the electronic device 10 to execute any one of the methods for training a color spot detection model provided in the following invention embodiments or any one of the methods for detecting color spots provided in the following invention embodiments.

[0064] It can be understood that the processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0065] The memory 12, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the method for training the color spot detection model in the embodiment of the present invention, or the program instructions / modules corresponding to the method for detecting color spots in the embodiment of the present invention. The processor 11 can implement the method for training the color spot detection model in any of the following method embodiments, and can implement the method for detecting color spots in any of the following method embodiments by running the non-transient software programs, instructions and modules stored in the memory 12. Specifically, the memory 12 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory 12 may also include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0066] The following is a detailed description of the method for training a color spot detection model provided by an embodiment of the present invention. Figure 3 , the method S20 includes but is not limited to the following steps:

[0067] S21: Obtain an image sample set including faces.

[0068] S22: from each image in the image sample set, respectively intercept at least one facial local image where color spots may exist, to obtain a plurality of candidate region images.

[0069] S23: performing resolution enhancement processing and size adjustment on each candidate region image to obtain a plurality of training images with the same size, wherein the resolution of the training images is higher than the resolution of the candidate region images.

[0070] S24: Use multiple training images marked with real labels as training sets, and train a preset convolutional neural network so that the preset convolutional neural network learns the real labels of the multiple training images to obtain a color spot detection model, wherein the real label of the target training image includes the real position, real category and real number of the color spots in the target training image in the target training image, and the target training image is any training image among the multiple training images.

[0071] The images in the image sample set include faces, which can be acquired by the above-mentioned image acquisition device. For example, the image sample set can be a ID photo or a selfie captured by the image acquisition device. It is understandable that the image sample set can also be data in an existing open source face database, wherein the open source face database can be a FERET face database, a CMU Multi-PIE face database, or a YALE face database. Here, there is no restriction on the source of the image samples, as long as they include faces.

[0072] It can be understood that the images in the image sample set include human faces and backgrounds, wherein color spots may only exist in local facial regions of human faces, resulting in that the color spot features are not obvious relative to the regional features (such as background features and facial features) that do not contain color spots, thereby resulting in poor color spot detection effects. In order to reduce the interference of regional features that do not contain color spots on color spot detection, and to reduce the training time of subsequent algorithm models, at least one facial local image where color spots may exist is intercepted from each image in the image sample set to obtain multiple candidate region images. For example, the facial local image may be a forehead region, a left face region, a right face region, or a nose region, etc., which are regions where color spots may exist. That is, the candidate region image may include at least one of the forehead region, the left face region, the right face region, or the nose region in the image, thereby retaining the region where color spots may exist in the image, removing the background in the image, and eliminating the interference of background features on model training. In addition, the facial features are disassembled, which can reduce the interference of facial features on model training. It is understandable that the facial local image may also have other division forms, such as T-region, cheek region or chin region, etc. Therefore, in this embodiment, the candidate region image includes at least one of the T-region, cheek region or chin region.

[0073] Among them, each image in the image sample set can be intercepted using an existing face key point algorithm, and the face key point algorithm can be active appearance models (AAMs), constrained local models (CLMs), explicit shape regression (ESR) or supervised descent method (SDM) and the like.

[0074] like Figure 4 As shown in the figure, several key points of the face are located according to the face key point algorithm, including points in the eyebrows, eyes, nose, mouth, face contour and other areas. Therefore, according to these key points, the local face image can be determined, and then, according to the definition of the local face image, the candidate area image can be intercepted. Figure 4 In the figure, the facial local area image is taken as the forehead area, the left face area, the right face area or the nose area for exemplary description. In this embodiment, one image can capture 4 facial local area images to obtain 4 candidate area images.

[0075] It is understandable that the sizes of the above-mentioned candidate region images are different and not completely the same, and there are also subtle color spots, such as lighter color spots. In order to eliminate the interference of the size of the image used for training on the model learning and enhance the color spot characteristics, the resolution enhancement processing and size adjustment are performed on each candidate region image to obtain multiple training images of the same size, wherein the resolution of the training image is higher than the resolution of the candidate region image. Therefore, the sizes of multiple training images are unified, for example, the size of multiple training images can be 416*416*3, which can eliminate the interference of the size on the model learning. In addition, the resolution enhancement processing is also performed while the size is unified, so that the color spot characteristics can be enhanced.

[0076] A plurality of training images annotated with true labels are used as training sets. For any training image among the plurality of training images, i.e., the target training image, its true label includes the true position, true category, and true number of the color spots in the target training image. Among them, the true number of color spots in the target training image is the number of the true positions of the color spots in the target training image, and the true number of the color spots in the target training image can be determined by counting the number of the true positions of the color spots in the target training image. The category of color spots is the classification of color spots. For example, the category of color spots can include at least one of freckles, black spots, chloasma, or age spots. The specific category can be determined according to actual needs and can be divided into one or more categories. It can be understood that the above-mentioned true labels can be manually labeled using existing labeling tools. It can be understood that, since the actual number of the color spots in the target training image can be obtained by counting the actual positions of the color spots in the target training image, in some embodiments, it is not necessary to mark the actual number of the color spots in the target training image, and only the actual positions and categories of the color spots in the target training image need to be marked. In this embodiment, the actual number of the color spots in the target training image is obtained by model statistics.

[0077] A plurality of training images annotated with real labels are used as training sets, and a preset convolutional neural network is trained so that the preset convolutional neural network learns the real labels of the plurality of training images to obtain a color spot detection model. After the preset convolutional neural network learns the features and real labels of the plurality of training images in the training set, the position, category and number of color spots of each training image in the training set are predicted to obtain a prediction result, and then the error between the prediction result and the real label is calculated by a preset loss function, and the model parameters of the preset convolutional neural network are adjusted inversely according to the error. After multiple iterations of training, a color spot detection model with good accuracy is obtained.

[0078] It can be seen that during training, the number of color spots is introduced to constrain the positions of color spots, so that the number of color spot positions can be trained in the direction equal to the actual number of color spots, that is, the predicted positions of color spots are trained in the same direction as the positions of actual color spots, thereby improving the accuracy of the color spot detection model. For example, if the predicted number of color spots is 4, and the predicted number of color spot positions is only 3, it means that the deviation of the predicted color spot positions is large, so that the error of the entire model is large, so the model will be trained in the direction of smaller error, that is, the number of color spot positions is trained in the direction equal to the actual number of color spots, so that the color spot detection model has higher accuracy.

[0079] In summary, by cutting the images in the image sample set into candidate area images, the method can reduce the interference of regional features (such as background features and facial features) that do not contain color spots on the training model during the model training process, thereby improving the accuracy of model training; by performing resolution enhancement processing and size adjustment on each candidate area image containing color spots to obtain multiple training images of the same size, the size adjustment can effectively avoid the interference of the size on model learning, and the resolution enhancement can enhance the characteristics of the color spots, thereby further improving the accuracy of model training; in addition, during the training process, by introducing the number of color spots to constrain the color spot position, the predicted position of the color spot can be trained in the same direction as the position of the real color spot, further improving the accuracy of model training; the improvement of multiple training accuracy makes the detection accuracy of the color spot detection model obtained by training through this method higher. In addition, by cutting the images in the image sample set into candidate area images, since the size of the candidate area image is smaller than the original image, it is beneficial to reduce the amount of calculation in the training process, improve the speed of model training and calculation, and help to obtain a color spot detection model with small amount of calculation and high detection accuracy.

[0080] In some embodiments, see Figure 5 The preset convolutional neural network includes a feature extraction module, a color spot target detection module and a color spot quantity detection module, wherein the feature extraction module is used to extract the features of the image, the color spot target detection module is used to detect the position and category of the color spots in the image, and the color spot quantity detection module is used to detect the number of color spots in the image.

[0081] Based on the above feature extraction module, color spot target detection module and color spot quantity detection module, please refer to Figure 6 , the step S24 specifically includes:

[0082] S241: Input the target training image into the feature extraction module to obtain training feature maps of at least two sizes.

[0083] S242: Divide the training feature image into two paths to obtain a first path of training feature images and a second path of training feature images.

[0084] S243: Input the first training feature image into the color spot target detection module to obtain the predicted position and predicted category of the color spot in the target training image.

[0085] S244: Input the second training feature map into the color spot quantity detection module to obtain the predicted quantity of color spots in the target training image.

[0086] S245: Calculate the error between the predicted label of the target training image and the true label of the target training image according to a preset loss function, wherein the predicted label includes the predicted position, the predicted category and the predicted quantity.

[0087] S246: According to the error, adjust the model parameters of the preset convolutional neural network, and return to execute the step S241 until the preset convolutional neural network converges to obtain the color spot detection model.

[0088] The training set is input into the feature extraction module for convolution feature processing, the image features of each training image (target training image) in the training set, such as shape, edge and other features, are learned, and the image features are extracted to obtain a training feature map.

[0089] In this embodiment, since the convolution feature processing reduces the spatial dimension and resolution of the image, a single-size training feature map cannot meet the requirements of multi-size target detection. In order to take into account the detection of large-size targets (large-area color spots) and small-size targets (small-area color spots), in this embodiment, multi-scale training feature maps are used for training, that is, the target training image is subjected to convolution feature processing by the feature extraction module to output multiple feature maps. Feature maps of at least two sizes are selected from the multiple feature maps as training feature maps for color spot detection.

[0090] In some embodiments, the feature extraction module includes multiple feature convolution layers, and the number of convolution kernels of the multiple feature convolution layers increases first and then decreases as the number of layers of the feature convolution layers increases. It can be understood that a feature convolution layer corresponds to the output of a feature map, and the more convolution kernels a feature convolution layer has, the stronger the feature extraction capability is, the more features there are in the corresponding feature map, and the more it deviates from the image in the original training set. Therefore, in the shallower feature convolution layers, the number of convolution kernels of the multiple feature convolution layers increases as the number of layers of the feature convolution layers increases. However, as the number of convolution kernels increases, the amount of model calculations also increases, making the model's calculation efficiency low. In order to balance the model's calculation efficiency, in the deeper feature convolution layers, the number of convolution kernels decreases as the number of layers of the feature convolution layers increases, so as to reduce the amount of model calculations and improve calculation efficiency.

[0091] In a specific implementation, the feature extraction module includes 16 feature convolution layers and 5 pooling layers, wherein the pooling layer is used for dimensionality reduction and is located after the feature convolution layer that needs dimensionality reduction. The overall structure of the feature extraction module is shown in Table 1 below:

[0092] Table 1 Structure of feature extraction module

[0093]

[0094]

[0095] Among them, the number of convolution kernels in feature convolution layers 1-7 gradually increases to collect more features, and the number of convolution kernels in feature convolution layers 8-16 generally decreases to reduce the amount of model calculation, thereby balancing feature extraction and training efficiency.

[0096] In this embodiment, the feature maps outputted by the 10th, 13th and 16th layers are selected as the training feature maps. That is, the training feature maps used for inputting the model are of sizes 13*13*N, 26*26*N and 52*52*N. Among them, the training feature map of 13*13*N is suitable for the detection of large-sized spots, the training feature map of 26*26*N is suitable for the detection of medium-sized spots, and the training feature map of 52*52*N is suitable for the detection of small-sized spots.

[0097] In some embodiments, the plurality of feature convolution layers include a target feature convolution layer, and the number of convolution kernels of the target feature convolution layer is the number of categories of the color spots. Figure 5 , the target feature convolution layer is a feature convolution layer connected to the pigment spot target detection module, and is used to output the training feature map. For example, if there are 4 categories of the pigment spots, in the above embodiment, the 10th layer, the 13th layer, and the 16th layer are respectively connected to the pigment spot target detection module, and are used to output the training feature map. N is set to 4, so that the target of the final output of the network training, that is, the category and position of the pigment spots, is clarified. Then, in the process of network training, the features learned by the network can be continuously adjusted according to the principle of minimizing the preset loss function, and the features belonging to the pigment spots in the entire training feature image can be learned.

[0098] The training feature map is divided into two paths to obtain a first path of training feature maps and a second path of training feature maps. It can be understood that the first path of training feature maps and the second path of training feature maps are both copied from the training feature maps, and the two paths of training feature maps are exactly the same.

[0099] The first training feature map is input into the color spot target detection module to obtain the predicted position and predicted category of the color spots in the target training image. The second training feature map is input into the color spot quantity detection module to obtain the predicted number of color spots in the target training image. For example, the training feature map of the target training image A is copied and divided into two paths to obtain the first training feature map A1 and the second training feature map A2. Then, the first training feature map A1 is input into the color spot target detection module, so that the color spot target detection module will learn the features of the first training feature map A1, and predict the positions and categories of the color spots in the first training feature map A1, and obtain the corresponding predicted positions and predicted categories of the color spots in the target training image A. The second training feature map A2 is input into the color spot quantity detection module, so that the color spot quantity detection module will learn the features of the second training feature map A2, and predict the number of color spots in the second training feature map A2, and obtain the corresponding predicted number of color spots in the target training image A.

[0100] Based on the fact that the first training feature map A1 and the second training feature map A2 are completely identical, and both correspond to the same target training image A, the predicted position and predicted category of the color spots predicted by the first training feature map A1 and the predicted number of color spots predicted by the second training feature map A2 together constitute the predicted label of the target training image A. Therefore, for the same target training image A, the position and category of the color spots in its predicted label and the number in its predicted label are detected separately and have no association, so the predicted number can constrain the number of predicted positions.

[0101] Specifically, for any target training image in the training set, there are three sizes of training feature maps: 13*13*N, 26*26*N and 52*52*N. The 13*13*N training feature map is divided into 13*13 grids, the 26*26*N training feature map is divided into 26*26 grids, and the 52*52*N training feature map is divided into 52*52 grids. Three prior frames are set for each grid, and each prior frame generates a prediction frame, so there are 13*13*3+26*26*3+52*52*3=10647 prediction frames. The prior frames are rectangular frames of different sizes predefined at each position in the training feature map, and these rectangular frames contain different aspect ratios. The prior frames are used to roughly frame the possible positions of the color spots. These priori frames are mapped and transformed in the prediction convolution layer of the color spot target detection module, and the positions are adjusted to output prediction frames corresponding to the priori frames. The prediction frame reflects the color spot situation in the area where a grid in the training feature map is mapped on the target training image, wherein the color spot situation includes a preset position (i.e., the position of the prediction frame), a preset category, and the confidence of the predicted position and the confidence of the predicted category. For example, the position of the prediction frame 1 of grid B is (x1, y1, w1, h1), the prediction category is chloasma, and the confidence of the prediction category is 90%, that is, in the area mapped by the image grid B on the original training image, the probability that the pixel area framed by the prediction frame 1 (x1, y1, w1, h1) is chloasma is 90%, and in addition, the confidence of the prediction frame 1 is 80%, which means that the prediction frame 1 is 80% close to the real frame, wherein the real frame is the color spot position and color spot category in the real label.

[0102] In addition, the second-way training feature map is input into the color spot quantity detection module, and the color spot data detection module learns the features in the second-way training feature map to generate a regression relationship between the features and the quantity. The regression relationship can fit the relationship between the features and the true quantity to the greatest extent, so that the error between the predicted quantity predicted by the regression relationship and the features and the true quantity is minimized.

[0103] During the training process, the model will set an appropriate confidence level for the predicted position based on the number of predictions. For example, when the number of predictions is 4, if there are 3 predicted positions, the predicted position has a lower confidence level. If there are 4 predicted positions, the predicted position has a higher confidence level.

[0104] Then, the error between the predicted label and the true label is calculated according to the preset loss function, that is, the error between each predicted box and the true box is calculated. It can be understood that when calculating the error between the predicted box and the true box, all predicted boxes corresponding to the target training image can be used to participate in the error calculation.

[0105] Finally, the preset convolutional neural network can reversely adjust the model parameters according to the error, and after determining the new model parameters, the color spot detection model can be obtained.

[0106] In some embodiments, the Adam algorithm can be used to optimize the model parameters, the number of iterations can be set to 500 times, the initial learning rate is set to 0.001, the weight decay is set to 0.0005, and every 50 iterations, the learning rate decays to 1 / 10 of the original. After training, the model parameters of the color spot detection model are output, and the color spot detection model is obtained.

[0107] In this embodiment, on the one hand, the use of multi-size training feature maps for detection can eliminate the influence of the spatial dimension of the image on the detection result, and can take into account the detection of color spots of various sizes, with high detection accuracy. On the other hand, the training feature map is divided into two paths, one of which is used to learn the predicted color spot position and category, and the other is used to learn the predicted number of color spots, so that the predicted number of color spots is independent and unrelated to the predicted position, so that the predicted number can more accurately constrain the number and confidence of the predicted position, so that the number of color spot positions is trained in the direction equal to the actual number of color spots.

[0108] In some embodiments, the preset loss function is a weighted sum of a position loss function, a category loss function, and a quantity loss function. The position loss function is used to calculate the error between the true position and the predicted position, the category loss function is used to calculate the error between the true category and the predicted category, and the quantity loss function is used to calculate the error between the true quantity and the predicted quantity. That is, the predicted quantity and the true quantity are integrated into the loss function, and the parameter error is calculated, so that the error between the true label and the predicted label is more accurate. Since the error is used to reversely adjust the model parameters, the model parameters are made more reasonable, which can improve the accuracy of the color spot detection model.

[0109] In some embodiments, step S245 specifically includes:

[0110] The error between the predicted label and the true label is calculated according to the following formula:

[0111]

[0112] Among them, λ obj is the preset position offset weight, α is the confidence loss weight, β is the quantity weight, M is the preset number of prediction boxes for predicting the color spot position, i is the number of the prediction box, T width is the width of the prediction box, T height is the height of the prediction box, (x, y, w, h) are the coordinates of the upper left corner, width and height of the prediction box respectively, T rUsed to indicate the actual position of the color spot, P r is used to indicate the predicted position of the color spot, K is the number of categories of the color spot, T class is the true category of the color spot, P class is the predicted category of the spot, T conf is the confidence of the true label, P conf is the confidence of the predicted label, is the actual number of c-th type spots in K-type spots, is the predicted number of the c-th type of color spots in the K-type color spots.

[0113] During the training process, for the target training image, the corresponding training feature map of a certain size is divided into S e *S e grids, where e is the number of a training feature map of a certain size. Each grid is set with f prior boxes. Each prior box will generate a corresponding prediction box after passing through the network, thus finally forming prediction box, i.e. Where n is the total number of sizes of the training feature maps. It can be understood that the loss function involves all prediction results reflected by the prediction boxes in calculating the error, so that the error can reflect the difference between all predicted labels and true labels. Therefore, the color spot detection model obtained by reversely adjusting the model parameters through the error has a higher accuracy.

[0114] In the above loss function, the position loss function is in, is the error between the predicted position of one of the M prediction boxes and the actual position of the color spot, (2-T width *T height ) is the proportionality factor used to increase the position loss The weight of . Among them, r∈(x,y,w,h), so, Equivalent to Among them, (x i ,y i ) is the coordinate of the upper left corner of the prediction box, is the coordinate of the upper left corner of the real box (real position), w i is the width of the prediction box, is the width of the real frame, h i is the width of the prediction box, is the height of the real frame, it can be seen that (T r -P r ) 2 Reflects the error between the predicted position and the actual position.

[0115] Thus, the position loss function constrains the relationship between the predicted position (prediction box) output by the preset convolutional neural network and the real position (real box) in the real label, that is, minimizing the predicted position P r With the real position T r The error between them makes the position of the color spot (prediction box) output by the preset convolutional neural network constantly approach the position of the color spot (real box) in the real label to optimize the model parameters.

[0116] Among them, the category loss function is When the predicted category P class is the true category T class When the category loss value is 1, when the predicted category P class Not a real category T class When , the category loss value is category P class Therefore, the category loss function constrains the relationship between the probability of the color spot category output by the preset convolutional neural network and the true probability of the color spot category in the true label, that is, minimizes the error between the true probability of the color spot category in the true label and the probability of the output predicted category, so that the probability of the predicted category output by the preset convolutional neural network is constantly close to the true probability of the color spot category in the true label, so as to optimize the model parameters.

[0117] Among them, the quantity loss function is Among them, β is the quantity weight. By adjusting β, the degree of constraint of quantity on the predicted position can be adjusted. It can be seen that for the target training image, in the color spot quantity detection module, the predicted number of the c-th color spot will be predicted, so the predicted number of the c-th color spot With the real number Compare (find the difference), and add up the difference of each type of color spot to get the loss function reflecting the quantity loss. Therefore, the quantity loss constrains the relationship between the predicted quantity output by the preset convolutional neural network and the actual quantity in the real label, that is, minimizes the error between the actual quantity and the predicted quantity, so that the number of predicted positions is accurate, so that the predicted position of the color spot is trained in the direction equal to the actual position of the color spot, thereby optimizing the model parameters.

[0118] It can be understood that the above loss function also includes a confidence loss function, which is α*(T conf -P conf ) 2 , where T conf is the confidence of the true label, P conf is the confidence of the predicted label. conf When it is low, it means that the prediction frame does not contain color spots or contains fewer color spots, which is consistent with Tconf The difference between them is large, and the difference is included in the loss. The model parameters are adjusted inversely through the loss, so that the predicted box output by the preset convolutional neural network is constantly close to the real box, thereby reducing the error between the predicted label and the real label, so that the model parameters reach the optimal solution to obtain the wrinkle detection model.

[0119] In this embodiment, the error calculated by the preset loss function is gradient-backwarded, back-propagated, and the model parameters are adjusted, so that the predicted label is continuously closer to the true label, so as to improve the accuracy of the color spot detection model.

[0120] In some embodiments, see Figure 7 , the step S23 specifically includes:

[0121] S231: Acquire the sizes of a target candidate region image and a target training image, wherein the target candidate region image is any candidate region image among the multiple candidate region images, and the target training image is obtained by mapping the target candidate region image.

[0122] S232: Obtain a second pixel coordinate corresponding to the target pixel point in the target candidate area image according to the first pixel coordinate of the target pixel point in the target training image, the size of the target candidate area image, and the size of the target training image, wherein the target pixel point is any pixel point in the target training image.

[0123] S233: Acquire a plurality of neighboring pixel points of the second pixel coordinate, where the neighboring pixel points are pixel points adjacent to the second pixel coordinate in the target candidate area image.

[0124] S234: Obtaining a weight of each of the neighborhood pixel points according to a positional relationship between the second pixel coordinates and the plurality of neighborhood pixel points.

[0125] S235: Determine the pixel value of the target pixel point according to the weight of each of the neighborhood pixel points and the pixel value of each of the domain pixel points to obtain the target training image.

[0126] The target candidate region image is any candidate region image among the multiple candidate region images, and the target candidate region image is obtained. Based on the target candidate region image, a target training image is generated, wherein the size of the target training image is preset and can be set specifically according to actual conditions. Thus, the size of the target training image is determined, and then the pixel value of each pixel point of the new target training image is obtained through mapping transformation based on the pixel value of each pixel point in the original target candidate region image.

[0127] For any pixel point in the target training image, i.e., the target pixel point, according to the first pixel coordinate of the target pixel point in the target training image, the second pixel coordinate corresponding to the target pixel point in the target candidate region image is obtained. It can be understood that the first pixel coordinate is the position of the target pixel point in the target training image, and the first pixel coordinate can be described by (row, column), and the second pixel coordinate is the position of the target pixel point mapped in the target candidate region image, and the second pixel coordinate can also be described by (row, column). The second pixel coordinate is obtained by combining the first pixel coordinate with the size of the target candidate region image and the size of the target training image. For example, assuming that the size of the target candidate region image A is m*n and the size of the target training image B is M*N, according to the size ratio between the target candidate region image and the target training image, the second pixel coordinate A(x,y)=A(X*(m / M),Y*(n / N)) corresponding to the target pixel point B(X,Y) in the target training image B in the target candidate region image A can be obtained. It can be understood that x=X*(m / M), y=Y*(n / N) in the second pixel coordinates may be integers or decimals.

[0128] like Figure 8 As shown, point P in the target candidate area image A is the position of the target training image B at (X, Y) corresponding to the target candidate area image A, that is, the coordinates of point P are (X*(m / M), Y*(n / N), so point P is the position of the second pixel coordinate. A plurality of neighborhood pixel points of the second pixel coordinate are obtained, and the neighborhood pixel points are pixel points adjacent to the second pixel coordinate in the target candidate area image, for example, 4*4 pixel points a(i, j) near point P, where i, j=0, 1, 2, 3.

[0129] According to the positional relationship between the second pixel coordinates and the plurality of neighboring pixel points, the weight of each of the neighboring pixel points is obtained, that is, according to the positional relationship between point P and a ij The position relationship of a ij The weight is the influence factor of the neighboring pixel point on the pixel value of the target pixel point. ij The closer to the second pixel coordinate (point P), the greater the impact on the pixel value of the target pixel. ij Based on the above characteristics, a weight function can be constructed through mathematical modeling according to the positional relationship between the second pixel coordinates and the neighboring pixel points.

[0130] It can be understood that, in some embodiments, the positional relationship between the second pixel coordinate and each neighborhood pixel point can be divided into a row positional relationship and a column positional relationship, that is, the weight includes a row weight and a column weight. Thus, in some embodiments, the row weight and the column weight are calculated separately by the weight function. It can be understood that the contribution value of the neighborhood pixel point to the target pixel point is the pixel value of the neighborhood pixel point multiplied by the corresponding row weight and column weight, so that the pixel value of the target pixel point is the sum of the contribution values ​​of each neighborhood pixel point. For example, the pixel value of the target pixel point B (X, Y) is the pixel value of the neighborhood pixel point a. 00 -a 33 The sum of the contribution values.

[0131] Each pixel point in the target training image is processed in the manner of the target pixel point described above, so that the pixel value of each pixel point in the target training image can be obtained, thereby obtaining the target training image.

[0132] In this embodiment, the target pixel point is first mapped to the second pixel coordinate point in the candidate area image according to the size of the target candidate area image and the size of the target training image, and then the pixel value of the target pixel point is determined according to the pixel values ​​and positions of multiple neighborhood pixel points of the second pixel coordinate, so that the boundary between the pixel point corresponding to the color spot and the pixel point corresponding to the surrounding non-color spot is clear, thereby enhancing the color spot feature and obtaining a high-resolution target training image. In addition, the size of the target training image can also be unified.

[0133] In summary, the method for training the color spot detection model can reduce the interference of regional features (such as background features and facial features) that do not contain color spots on the training model during the model training process by cutting the images in the image sample set into candidate area images, thereby improving the accuracy of model training; by performing resolution enhancement processing and size adjustment on each candidate area image containing color spots to obtain multiple training images of the same size, the size adjustment can effectively avoid the interference of the size on model learning, and the resolution enhancement can enhance the characteristics of the color spots, thereby further improving the accuracy of model training; in addition, during the training process, by introducing the number of color spots to constrain the color spot position, the predicted position of the color spot can be trained in the same direction as the position of the real color spot, further improving the accuracy of model training; the improvement of multiple training accuracy makes the detection accuracy of the color spot detection model obtained by training through this method higher. In addition, by cutting the images in the image sample set into candidate area images, since the size of the candidate area image is smaller than the original image, it is beneficial to reduce the amount of calculation in the training process, improve the speed of model training and calculation, and help to obtain a color spot detection model with small amount of calculation and high detection accuracy.

[0134] The following is a detailed description of the method for detecting color spots provided by the embodiment of the present invention. Fig. 9 , the method S30 includes but is not limited to the following steps:

[0135] S31: Obtain a face image to be tested.

[0136] S32: capturing at least one target facial local image where color spots may exist from the face image to be tested.

[0137] S33: Detect the at least one target facial local image using the color spot detection model in any one of the above embodiments to obtain the position, type and number of color spots in the at least one target facial local image.

[0138] S34: Determine the position, category and number of color spots in the face image to be tested based on the position, category and number of color spots of the at least one target face local image and the position of the target face local image in the face image to be tested.

[0139] The face image to be tested is a human face image, which can be obtained by the above-mentioned image acquisition device 20. For example, the face image to be tested can be a certificate photo or a selfie captured by the image acquisition device 20. Here, there is no restriction on the source of the face image to be tested, and it can be a human face image.

[0140] It is understandable that the face image to be tested includes a face and a background, wherein color spots may only exist in a local area of ​​the face, resulting in the color spot features being less obvious than the regional features (such as background features and facial features) that do not contain color spots, thereby affecting the color spot detection. In order to reduce the interference of the regional features that do not contain color spots on the color spot detection, as well as to reduce the detection time and improve the detection efficiency, at least one target facial local image where color spots may exist is respectively captured from the face image to be tested.

[0141] Then, the color spot detection model in any one of the above embodiments is used to detect the at least one target facial local image, and the position, type and number of the color spots in the at least one target facial local image are obtained.

[0142] Finally, based on the position, category and number of the color spots of the at least one target facial local image and the position of the target facial local image in the face image to be tested, at least one target facial local image is mapped into the face image to be tested, so that the position, category and number of the color spots in the face image to be tested can be determined. Among them, the number of color spots in the face image to be tested is equal to the sum of the number of color spots in each target local facial image, the position of the color spots in the face image to be tested is the position when the position of the color spots in each target local facial image is mapped back to the face to be tested, and the category of the color spots in the face image to be tested is the set of categories of the positions of the color spots in each target local facial image.

[0143] It can be understood that the color spot detection model is trained by the method for training the color spot detection model in the above embodiment, and has the same structure and function as the color spot detection model in the above embodiment, which will not be described in detail here.

[0144] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a color spot detection model, characterized in that: include: Obtaining a sample set of images including faces; From each image in the image sample set, respectively intercept at least one facial local image where color spots may exist, so as to obtain a plurality of candidate area images; Performing resolution enhancement processing and size adjustment on each candidate region image to obtain a plurality of training images of the same size, wherein the resolution of the training image is higher than the resolution of the candidate region image; Using multiple training images annotated with real labels as training sets, training a preset convolutional neural network so that the preset convolutional neural network learns the real labels of the multiple training images to obtain a color spot detection model, wherein the real label of the target training image includes the real position, real category and real number of the color spots in the target training image, and the target training image is any training image among the multiple training images; The preset convolutional neural network includes a feature extraction module, a color spot target detection module and a color spot quantity detection module; the plurality of training images annotated with real labels are used as training sets to train the preset convolutional neural network so that the preset convolutional neural network learns the real labels of the plurality of training images to obtain a color spot detection model, including: Inputting the target training image into the feature extraction module to obtain training feature maps of at least two sizes; Dividing the training feature map into two paths to obtain a first path of training feature maps and a second path of training feature maps; Inputting the first training feature map into the color spot target detection module to obtain the predicted position and predicted category of the color spot in the target training image; Inputting the second training feature map into the color spot quantity detection module to obtain the predicted number of color spots in the target training image; Calculating the error between the predicted label of the target training image and the true label of the target training image according to a preset loss function, wherein the predicted label includes the predicted position, the predicted category, and the predicted quantity; According to the error, the model parameters of the preset convolutional neural network are adjusted, and the step of inputting the target training image into the feature extraction module to obtain training feature maps of at least two sizes is returned until the preset convolutional neural network converges to obtain the color spot detection model.

2. The method according to claim 1, characterized in that: The feature extraction module includes multiple feature convolution layers, and the number of convolution kernels of the multiple feature convolution layers tends to first increase and then decrease as the number of feature convolution layers increases.

3. The method according to claim 2, characterized in that The multiple feature convolution layers include a target feature convolution layer, the number of convolution kernels of the target feature convolution layer is the number of categories of the color spots, and the target feature convolution layer is a feature convolution layer connected to the pigment spot target detection module, and is used to output the training feature map.

4. The method according to claim 1, characterized in that: The preset loss function is a weighted sum of a position loss function, a category loss function and a quantity loss function. The position loss function is used to calculate the error between the actual position and the predicted position. The category loss function is used to calculate the error between the actual category and the predicted category. The quantity loss function is used to calculate the error between the actual quantity and the predicted quantity.

5. The method according to claim 4, characterized in that The calculating the error between the predicted label of the target training image and the real label of the target training image according to the preset loss function includes: The error between the predicted label and the true label is calculated according to the following formula: Among them, λ obj is the preset position offset weight, α is the confidence loss weight, β is the quantity weight, M is the preset number of prediction boxes for predicting the color spot position, i is the number of the prediction box, T width is the width of the prediction box, T height is the height of the prediction box, (x, y, w, h) are the coordinates of the upper left corner, width and height of the prediction box respectively, T r Used to indicate the actual position of the color spot, P r is used to indicate the predicted position of the color spot, K is the number of categories of the color spot, T class is the true category of the color spot, P class is the predicted category of the spot, T conf is the confidence of the true label, P conf is the confidence of the predicted label, is the actual number of c-th type spots in K-type spots, is the predicted number of the c-th type of color spots in the K-type color spots.

6. The method according to any one of claims 1 to 5, characterized in that: The step of performing resolution enhancement processing and size adjustment on each candidate region image to obtain a plurality of training images with the same size includes: Acquire the sizes of a target candidate region image and a target training image, wherein the target candidate region image is any one of the multiple candidate region images, and the target training image is obtained by mapping the target candidate region image; According to the first pixel coordinate of the target pixel point in the target training image, the size of the target candidate area image and the size of the target training image, obtaining the second pixel coordinate corresponding to the target pixel point in the target candidate area image, wherein the target pixel point is any pixel point in the target training image; Acquire a plurality of neighboring pixel points of the second pixel coordinates, where the neighboring pixel points are pixel points adjacent to the second pixel coordinates in the target candidate area image; Obtaining a weight of each of the neighborhood pixel points according to a positional relationship between the second pixel coordinate and the plurality of neighborhood pixel points; The pixel value of the target pixel is determined according to the weight of each of the neighborhood pixel points and the pixel value of each of the domain pixel points to obtain the target training image.

7. A method for detecting color spots, characterized in that: include: Obtain the face image to be tested; intercepting at least one target facial local image where color spots may exist from the face image to be tested; Detecting the at least one target facial local image using the color spot detection model according to any one of claims 1 to 6, and obtaining the position, type and number of color spots in the at least one target facial local image; Based on the position, category and number of color spots of at least one target facial local image and the position of the target facial local image in the facial image to be tested, the position, category and number of color spots in the facial image to be tested are determined.

8. An electronic device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor, wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable an electronic device to execute the method according to any one of claims 1 to 7.

Images