Facial spot image detection and recognition method, terminal equipment and system

Abstract

The invention provides a face spot image detection and recognition method, terminal equipment and a face spot image detection and recognition system. The detection and recognition method comprises the following steps: acquiring a first face spot image and a second face spot image, and selecting one of the first face spot image and the second face spot image as an input image; the first face spot image is acquired, and the second face spot image is generated; inputting the input image into a trained first multilayer convolutional network, and extracting a plurality of facial spot feature images from the input image; and extracting and highlighting a spot region from the facial spot feature image by using a self-attention mechanism, and detecting the spots of the input image and the type of the input image. Through the detection and recognition method provided by the invention, the features of the small spot area of the face are enhanced, the spots in the image are detected, and whether the image is a real face spot image is judged.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image detection, and in particular to a detection and recognition method, terminal equipment and system for facial spot images. Background Art

[0002] Facial spot detection has important applications in the fields of dermatology and cosmetology. Traditional image processing and detection methods, such as those based on convolutional neural networks (CNNs), can solve the problem of facial spot detection to a certain extent. However, due to the limitations of local feature extraction, convolutional neural networks cannot enhance the features of small spot areas, resulting in poor results in detecting facial spots. In addition, facial spots vary in shape, size, color, and texture, which brings challenges to the detection method of facial spots.

[0003] Although the conditional generative adversarial network (cGAN) performs well in image generation and segmentation tasks, it is easily limited by the interference of complex background when dealing with small spot detection tasks and pays insufficient attention to small spot areas. Summary of the invention

[0004] The purpose of the present invention is to provide a facial spot image detection and recognition method, terminal equipment and system, so as to enhance the characteristics of small facial spot areas, detect spots in an image and judge whether the image is a real facial spot image.

[0005] The technical solution provided by the present invention is as follows:

[0006] The present invention provides a method for detecting and identifying facial spot images, comprising the following steps:

[0007] A first facial spot image and a second facial spot image are obtained, and one of the first facial spot image and the second facial spot image is selected as an input image; the first facial spot image is collected, and the second facial spot image is generated; the input image is input into a first multi-layer convolutional network that has been trained, and a plurality of facial spot feature images are extracted from the input image; a self-attention mechanism is used to extract and highlight spot areas from the facial spot feature image, and spots of the input image and the type of the input image are detected.

[0008] Specifically, through the facial spot image detection and recognition method proposed in the present invention, the first multi-layer convolutional network introduces a self-attention mechanism, so that more attention is paid to small spots in the input image during detection, which significantly improves the first multi-layer convolutional network's ability to detect the input image type and is more accurate and reliable in judging the authenticity of the image.

[0009] Furthermore, the self-attention mechanism is used to extract and highlight the spot area from the facial spot feature image, including:

[0010] Analyze the pixel similarity between preset local areas and the feature similarity of each facial spot feature image; generate a self-attention matrix according to the pixel similarity between preset local areas and the feature similarity of each facial spot feature image; assign a third weight coefficient to pixels in each facial spot feature image according to the self-attention matrix to enhance the spot area of ​​the input image.

[0011] Specifically, the present invention utilizes the self-attention mechanism to better capture the details of small spots in the image by focusing on feature information at different levels of the local context, so that the first multi-layer convolutional network can utilize richer feature information, thereby improving the detection capability of small facial spots.

[0012] Furthermore, detecting spots of the input image and the type of the input image includes:

[0013] The input image is detected and classified using a normalized exponential function; the fully connected layer of the first multi-layer convolutional network detects spots in the input image and outputs the type of the input image.

[0014] Furthermore, before the input image is input into the first multi-layer convolutional network that has completed training, the following steps are included:

[0015] Using the spatial attention mechanism and Gaussian distributed random noise, the random noise is input into the second multi-layer convolutional network to generate a second facial spot sub-image with enhanced spot areas; using the channel attention mechanism, the second facial spot sub-image is generated as a second facial spot image with highlighted spots in the second multi-layer convolutional network.

[0016] Specifically, by adding spatial attention mechanism and channel attention mechanism to the second multi-layer convolutional network, a generated image is generated that enhances the small spot area and highlights the small spot features, thereby enriching the data set for model training.

[0017] Furthermore, random noise is input into a second multi-layer convolutional network to generate a second facial spot sub-image with enhanced spot areas, including:

[0018] According to the second multi-layer convolutional network, a number of spatial attention images are extracted from random noise; the spatial attention images include spot color attention images, spot texture attention images, and spot shape attention images; the similarity between the pixels of each area in each spatial attention image and the pixels of its adjacent areas is analyzed; a first weight coefficient is assigned to each spatial attention image according to the similarity of the pixels, so as to obtain the second facial spot sub-image of the enhanced spot area.

[0019] Specifically, the present invention utilizes the spatial attention mechanism to strengthen the features of the spot area, significantly improving the generation quality of small facial spots, so that the second multi-layer convolutional network can more accurately reproduce the shape and details of the small spots.

[0020] Further, using the channel attention mechanism, the second facial spot sub-image is generated into a second facial spot image with prominent spots in a second multi-layer convolutional network, including:

[0021] A plurality of facial spot feature channels are extracted from the second facial spot sub-image; the facial spot feature channels include a spot color feature sub-channel and a spot texture sub-channel; each facial spot feature channel is globally averaged and pooled to obtain a channel average value and a channel maximum value of each facial spot feature channel; a channel description vector of each facial spot feature channel is obtained based on the channel average value and the channel maximum value; the similarity between each facial spot feature channel is analyzed according to the channel attention mechanism and the channel description vector of each facial spot feature channel to obtain a channel similarity matrix; in the middle layer of the second multi-layer convolutional network, a second weight coefficient is assigned to each facial spot feature channel according to the channel similarity matrix; a second facial spot image with highlighted spots is obtained by combining each facial spot feature channel and its corresponding second weight coefficient.

[0022] Specifically, the present invention utilizes the channel attention mechanism to further highlight the spots on the basis of strengthening the spot areas, so that the second multi-layer convolutional network generates clear and realistic small spot images.

[0023] Further, including:

[0024] Analyze the first loss value of the first multi-layer convolutional network for distinguishing the types of the first facial blob image and the second facial blob image; analyze the second loss value of the second facial blob image generated by the second multi-layer convolutional network being judged as the first facial blob image by the first multi-layer convolutional network; based on the first loss value and the second loss value, dynamically adjust the training of the first multi-layer convolutional network and the second multi-layer convolutional network.

[0025] Specifically, the present invention continuously conducts adversarial training between a first multi-layer convolutional network and a second multi-layer convolutional network, thereby enhancing the recognition and judgment capabilities of the first multi-layer convolutional network and improving the ability of the second multi-layer convolutional network to generate images that are more like real images.

[0026] The present invention also provides a terminal device for generating and identifying images of facial spots, comprising a processor for calling instructions stored in a memory, wherein when the instructions are executed, the terminal device executes any one of the facial spot image detection and identification methods.

[0027] The present invention also provides a facial spot image detection and recognition system, comprising:

[0028] An acquisition device acquires a first facial spot image and a second facial spot image of a human face; the first facial spot image is acquired, and the second facial spot image is generated; a terminal device detects spots in the first facial spot image and the second facial spot image and determines the types of the first facial spot image and the second facial spot image.

[0029] Through the facial spot image detection and recognition method, terminal device and system provided by the present invention, the second multi-layer convolutional network introduces a self-attention mechanism to more accurately capture the characteristics of the spots, thereby improving the detection accuracy of small facial spots under complex backgrounds; the first multi-layer convolutional network introduces a channel attention mechanism and a spatial attention mechanism to generate diversified facial spot images, thereby reducing the model's dependence on specific data and improving the model's generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The following will explain the preferred implementation mode in a clear and understandable manner with reference to the accompanying drawings, and further explain the above-mentioned characteristics, technical features, advantages and implementation methods of a facial spot image detection and recognition method, terminal device and system.

[0031] Figure 1 It is a structural schematic diagram of an embodiment of a facial spot image detection and recognition system of the present invention;

[0032] Figure 2 It is a flow chart of an embodiment of a method for detecting and identifying facial spot images of the present invention;

[0033] Figure 3 is a flow chart of another embodiment of a method for detecting and identifying facial spot images of the present invention;

[0034] Figure 4 is a flow chart of another embodiment of a method for detecting and identifying facial spot images of the present invention;

[0035] Figure 5 is a flow chart of another embodiment of a method for detecting and identifying facial spot images of the present invention;

[0036] Figure 6 It is a structural schematic diagram of a spatial attention mechanism and a channel attention mechanism proposed in an embodiment of a facial spot image detection and recognition method of the present invention;

[0037] Figure 7 is a schematic diagram of a generated facial blob image identified by the present invention;

[0038] Figure 8 is a schematic diagram of a collected facial spot image identified by the present invention;

[0039] Fig. 9 is a flow chart of another embodiment of a method for detecting and identifying facial spot images of the present invention;

[0040] Fig.10 is a schematic diagram of a facial spot image collected and recognized by the present invention after being processed by a self-attention mechanism;

[0041] Fig.11 is a flow chart of another embodiment of a method for detecting and identifying facial spot images of the present invention;

[0042] Fig.12 It is a flow chart of another embodiment of a method for detecting and identifying facial spot images of the present invention. DETAILED DESCRIPTION

[0043] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0044] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0045] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0046] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0048] Facial spots refer to spots and pigment spots on the face. The detection of facial spots is very important in the field of dermatology and cosmetology, but it does present challenges, especially in the detection of small spots with unclear features in complex backgrounds.

[0049] Although the traditional convolutional neural network (CNN) can solve the problem of spot detection in facial spot images to a certain extent, it is unable to enhance the features of small spot areas due to the limitation of local feature extraction, resulting in poor results in detecting facial small spots. In addition, the shapes, sizes, colors and textures of facial small spots vary, which brings challenges to the detection method of facial small spots.

[0050] Although the conditional generative adversarial network (cGAN) performs well in image generation and segmentation tasks, it is easily limited by the interference of complex background when dealing with small spot detection tasks and pays insufficient attention to small spot areas.

[0051] Therefore, the present invention proposes a facial spot image detection and recognition method, terminal device and system, which introduces a self-attention mechanism in the discriminator to enhance the ability to extract spot features in facial spot images, thereby improving detection accuracy and discrimination ability.

[0052] By adopting channel attention mechanism and spatial attention mechanism in the generator to generate diverse facial spot images, the training data set is increased and the generalization ability of the model in various complex backgrounds is improved; the loss values ​​of the first multi-layer convolutional network and the second multi-layer convolutional network are monitored, and the learning rates of the two multi-layer convolutional networks are dynamically adjusted to prevent the model from overfitting or converging too slowly, ensuring that the first multi-layer convolutional network and the second multi-layer convolutional network remain balanced during the training process.

[0053] The following is a description with reference to the accompanying drawings:

[0054] In one embodiment of the present invention, Figure 1 , which shows a schematic diagram of the structure of a facial spot image detection and recognition system 100 provided in some embodiments of the present disclosure. The detection and recognition system 100 includes: a collection device 110 and a terminal device 120.

[0055] The acquisition device 110 acquires a first facial spot image and a second facial spot image of a human face; the first facial spot image is acquired, and the second facial spot image is generated.

[0056] The terminal device 120 detects spots in the first facial spot image and the second facial spot image and determines the types of the first facial spot image and the second facial spot image.

[0057] Specifically, the acquisition device 110 can be a device with an integrated camera function. The first facial spot image (i.e., the acquired facial spot image) of the human face is captured by the device with the camera function. In order to ensure the data quality, it is recommended to shoot under good lighting conditions to avoid shadows and reflections affecting the image quality. Facial images at different angles and expressions can also be captured to improve the generalization ability of the model. The final collected data set should include diverse samples of multiple subjects. The acquisition device 110 can also be an image generator that generates a realistic second facial spot image (i.e., the generated facial spot image) by using artificial intelligence technology.

[0058] After the terminal device 120 obtains the first facial spot image and the second facial spot image from the acquisition device 110, it extracts and highlights the spot features in the image, detects and identifies the spots in the image, and determines the type of the image.

[0059] The facial spot image detection and recognition system proposed by the present invention can detect spots in an image and determine whether the type of the input image is a real shot image.

[0060] In one embodiment of the present invention, Figure 2 As shown, it shows a flowchart of the facial spot image detection and recognition method performed by the terminal device 120 of the present disclosure, including the following steps:

[0061] S100 obtains a first facial blob image and a second facial blob image, and selects one of the first facial blob image and the second facial blob image as an input image; the first facial blob image is collected, and the second facial blob image is generated.

[0062] S200 inputs the input image into the first multi-layer convolutional network that has completed training, and extracts a number of facial spot feature images from the input image.

[0063] S300 uses the self-attention mechanism to extract and highlight spot areas from facial spot feature images, and detect spots in the input image and the type of the input image.

[0064] Specifically, a first facial blob image taken actually is obtained and a second facial blob image with stronger blob diversity is generated, an input image is randomly selected from the first facial blob image and the second facial blob image, and the input image is input into a first multi-layer convolutional network (i.e., a discriminator) for recognition and judgment.

[0065] Before inputting the input image into the first multi-layer convolutional network, the image needs to be preprocessed to ensure the consistency of the input image data. The preprocessing steps include: image cropping, cropping the facial area of ​​the face to remove background interference; resizing, adjusting all images to the same size (such as 256*256 pixels) for network processing; normalization, scaling the pixel values ​​to the range of [0, 1] to increase the convergence speed of the model; data enhancement, using rotation, translation, rotation and other methods to enhance the data and increase the diversity of image samples.

[0066] The trained first multi-layer convolutional network (discriminator) is used to detect and discriminate the processed input image. An improved self-attention mechanism is introduced into the first multi-layer convolutional network to extract and highlight the spot area in the input image, and to determine the image type of the input image, thereby improving the detection capability of small facial spots.

[0067] In this embodiment, through the facial spot image detection and recognition method proposed in the present invention, a self-attention mechanism is introduced into the discriminator, so that more attention is paid to small spots in the input image during detection, which significantly improves the first multi-layer convolutional network's ability to detect the input image type, and is more accurate and reliable in judging the authenticity of the image.

[0068] In one embodiment of the present invention, Figure 3 As shown, before inputting the input image into the first multi-layer convolutional network that has completed training, it is necessary to train the first convolutional network based on the second multi-layer convolutional network, and improve the second multi-layer convolutional network. The above step S100 may include:

[0069] S110 utilizes the spatial attention mechanism and Gaussian distributed random noise, and inputs the random noise into a second multi-layer convolutional network to generate a second facial spot sub-image of the enhanced spot area.

[0070] S120 utilizes a channel attention mechanism to generate a second facial spot sub-image into a second facial spot image that highlights the spots in a second multi-layer convolutional network.

[0071] Specifically, the second multi-layer convolutional network is a generator, and random noise as the input of the generator can help the model learn a richer data distribution, thereby generating more diverse small spot images. The spatial attention mechanism enables the second multi-layer convolutional network to focus on the spot areas in the feature map, and by assigning different weights to different areas, the second multi-layer convolutional network pays more attention to the spot areas in the facial image.

[0072] The channel attention mechanism focuses on the importance of different channels in the feature map to improve the second multi-layer convolutional network's ability to capture small facial spots. By assigning different weights to different feature channels, the second multi-layer convolutional network can automatically identify which channels contain the most important spot information. In the second multi-layer convolutional network, the feature map consists of multiple channels, each of which may contain different information related to facial spots. In order to increase the focus on facial spot features, it is necessary to allow the second multi-layer convolutional network to automatically identify and strengthen those feature channels related to spots, and then generate a second facial spot image.

[0073] During the generation process, the second multi-layer convolutional network is able to generate different types of second facial blob images by adjusting the input conditional variables (such as the shape, color, position, etc. of the blob). The conditional variables can be generated by an additional conditional network or manually set according to the distribution of the dataset.

[0074] In the second multi-layer convolutional network, the channel attention mechanism and the spatial attention mechanism enable the second multi-layer convolutional network to pay more attention to the characteristic areas of facial spots, rather than generating background information unrelated to facial spots. The second multi-layer convolutional network can generate a large number of spot images with diversity and realism, and add these generated images to the training set of facial spot images, thereby enhancing the diversity of the training data set. The generated facial spot images can be input through different conditions to simulate spots of different shapes, sizes, colors and positions, which expands the coverage of the training data and improves the robustness of the first multi-layer convolutional network and its adaptability to new samples.

[0075] In this embodiment, a spatial attention mechanism and a channel attention mechanism are added to the second multi-layer convolutional network to generate a generated image that enhances the small spot area and highlights the small spot features, thereby enriching the data set for model training.

[0076] In one embodiment of the present invention, Figure 4 As shown, a second facial spot sub-image is generated in a second multi-layer convolutional network, and step S110 includes:

[0077] S111 extracts several spatial attention images from random noise according to the second multi-layer convolutional network; the spatial attention images include spot color attention images, spot texture attention images, and spot shape attention images.

[0078] S112 analyzes the similarity between the pixel points of each area in each spatial attention image and the pixel points of its adjacent areas.

[0079] S113 assigns a first weight coefficient to each spatial attention image according to the similarity of the pixels to obtain a second facial spot sub-image with enhanced spot areas.

[0080] Specifically, the second multi-layer convolutional network performs a series of convolution operations on random noise to extract spatial attention images at different levels, and these spatial attention images capture different information such as color, texture, and shape. For example, the basic color information of the image can be captured in the shallower network layer, and the edge and contour information of the object in the image can be captured in the deeper network layer. In addition to color, texture, and shape, the deep spatial attention image can also capture the spatial relationship between objects such as relative position and size.

[0081] After extracting several spatial attention images, each spatial attention image is divided into multiple regions (i.e., local regions) according to the size of the spots. Dividing the entire spatial attention image into several local regions can facilitate the analysis of the features of different regions in the image, so that the second multi-layer convolutional network can more accurately capture the position and shape of the facial spots.

[0082] For each local region, the similarity between the pixels in the region and the pixels in the surrounding area is calculated. For example, the similarity is calculated by using cosine similarity, Euclidean distance, or other similarity measurement methods. After the similarity calculation, a similarity matrix can be generated, which describes the similarity relationship between regions. For example, if cosine similarity is used, a value can be calculated for each pixel pair and these values ​​can be organized into a matrix, where each element of the matrix represents the similarity of the corresponding pixel pair.

[0083] According to the similarity matrix, a first weight coefficient is assigned to each region in each spatial attention image, so that the local region where the facial spots are located obtains a higher weight, while other background regions obtain a smaller weight.

[0084] Finally, each area in each spatial attention image is multiplied by its corresponding first weight coefficient. By applying the first weight coefficient, the important areas where the facial spots are located are enhanced, and the unimportant background areas are suppressed, so that the eigenvalues ​​of the facial spot areas will be amplified, and the second multi-layer convolutional network will also pay more attention to these areas when generating the second facial spot sub-image.

[0085] In this embodiment, the improved spatial attention mechanism introduces local area feature extraction and regional similarity calculation, so that the generator can more accurately identify areas related to facial spots, autonomously adjust the first weight coefficient of the local area, so that the generated facial spot image has richer details, and can effectively reduce background interference, significantly improve the generation quality of facial spot images, and enable the generator to more accurately reproduce the shape and details of the spots.

[0086] In one embodiment of the present invention, Figure 5As shown, in addition to adding an improved spatial attention mechanism in the second multi-layer convolutional network, an improved channel attention mechanism needs to be introduced. Step S120 includes:

[0087] S121 extracts a plurality of facial spot feature channels from the second facial spot sub-image; the facial spot feature channels include a spot color feature sub-channel and a spot texture sub-channel.

[0088] S122 performs global average pooling on each facial spot feature channel to obtain the channel average value and channel maximum value of each facial spot feature channel.

[0089] S123 obtains a channel description vector of each facial spot feature channel based on the channel average value and the channel maximum value.

[0090] S124 analyzes the similarities between the facial spot feature channels according to the channel attention mechanism and the channel description vector of each facial spot feature channel to obtain a channel similarity matrix.

[0091] S125 assigns a second weight coefficient to each facial spot feature channel according to the channel similarity matrix in the middle layer of the second multi-layer convolutional network.

[0092] S126 combines each facial spot feature channel and its corresponding second weight coefficient to obtain a second facial spot image with highlighted spots.

[0093] Specifically, after the second multi-layer convolutional network extracts each layer of spatial attention image, several facial spot feature channels are extracted from the second facial spot sub-image. Figure 6 As shown in the figure, each layer represents a spatial attention image, and a channel is formed at the same position in each layer. The channel attention mechanism and the spatial attention mechanism can be calculated simultaneously, or the feature calculation of each layer in the spatial attention mechanism can be calculated first, and then the features of each channel are calculated based on each layer. For example, the spatial attention mechanism calculates the weights on the image of each layer, and the channel attention mechanism calculates the weights at the points at the same position in each layer.

[0094] The channel attention mechanism proposed in the present invention is described below from the order of calculation. After the spatial attention mechanism has completed all operations, a second facial spot sub-image is formed, and then several facial spot feature channels are extracted.

[0095] Global average pooling is performed on each facial spot feature channel in order to extract the global information of each channel. Global average pooling calculates the average value of all elements of each channel to obtain a single value, which represents the activity of the channel in the entire image. For example, the channel average value can represent the activity of the channel in the entire image, and the maximum value represents the most significant feature in the channel. For each feature channel, the calculated average and maximum value are combined into a vector, which is the channel description vector of the channel. If a network has n feature channels, then n such channel description vectors can be obtained.

[0096] The channel description vectors are used to calculate the similarity between channels, and the similarity can be calculated by comparing the distance between vectors, such as using cosine similarity or Euclidean distance. Cosine similarity takes into account the angular difference between vectors, while Euclidean distance takes into account the actual distance between vectors.

[0097] A similarity matrix is ​​formed according to the similarity between all channels, and each element in the similarity matrix represents the similarity between the corresponding channels. Based on the similarity matrix, a channel attention weight (i.e., the second weight coefficient) can be assigned to each channel, and the second weight coefficient represents the importance of the channel in generating small spots. For example, the rows or columns of the similarity matrix can be normalized using a softmax function to obtain the second weight coefficient of each channel.

[0098] Finally, the calculated second weight coefficient is applied to the second facial spot sub-image after the spatial attention mechanism is applied, highlighting the spot features on the basis of enhancing the spot area and reducing the influence of background interference, thereby generating a second facial spot image that accurately reproduces the spots. Figure 7 As shown, it is a second facial spot image (generated image) generated by the second multi-layer convolutional network; Figure 8 As shown, it is a first facial spot image (real image). Compared with the first facial spot image, the spot features in the generated second facial spot image are more prominent.

[0099] In this embodiment, by introducing the channel description vector and the similarity matrix into the channel attention mechanism, the second multi-layer convolutional network can more accurately highlight and strengthen the features related to spots, thereby improving the realism of generating the second facial spot image.

[0100] In one embodiment of the present invention, Fig. 9 As shown, using the self-attention mechanism, the spot area is extracted and highlighted from the facial spot feature image, and step S300 includes:

[0101] S310 analyzes the pixel similarity between the preset local areas and the feature similarity of each facial spot feature image.

[0102] S320 generates a self-attention matrix according to the pixel similarity between the preset local areas and the feature similarity of each facial spot feature image.

[0103] S330 assigns a third weight coefficient to each pixel in the facial spot feature image according to the self-attention matrix to enhance the spot area of ​​the input image.

[0104] Specifically, the first multi-layer convolutional network is a discriminator, and an improved self-attention mechanism is introduced into the discriminator to enhance the recognition ability of small facial spots.

[0105] In image processing, self-attention is a mechanism that focuses on the relationship between different parts of the input data, identifying the mutual influence between individual pixels in the image. In this way, the first multi-layer convolutional network is able to consider a wider range of contextual information when determining whether the input image is a real image.

[0106] In the traditional self-attention mechanism, the first multi-layer convolutional network will calculate the similarity between each pixel and all other pixels. However, in the present invention, some preset local areas are first selected (they can be selected based on detection and recognition experience, generally 3*3 local areas), and the features of these areas are used to calculate pixel similarity through the self-attention mechanism, so that each local area in each facial spot feature image can achieve local context focus. In addition, the feature similarity between multiple facial spot feature images at different levels is also calculated, so as to comprehensively utilize the feature information of different layers.

[0107] A self-attention matrix is ​​generated based on the pixel similarity between local regions and the feature similarity between facial spot feature images. Each element of the self-attention matrix represents the attention relationship between a specific pixel and other pixels, ensuring that the first multi-layer convolutional network can identify small spots in the entire image. The third weight coefficient is assigned to each pixel of the facial spot feature image through the self-attention matrix. The third weight coefficient reflects the importance of each pixel in determining the authenticity of the image. For example, small spot areas will receive higher weights, while areas irrelevant to the background will be suppressed. Fig.10 As shown, this is the image after the real image is processed by the self-attention mechanism, which highlights the spot features of the original real image.

[0108] After the improved self-attention mechanism is introduced into the first multi-layer convolutional network, the input image needs to be recognized and judged, such as Fig.11 As shown, the above step S300 also includes:

[0109] S340 detects and classifies the input image using a normalized exponential function.

[0110] S350 The fully connected layer of the first multi-layer convolutional network detects spots in the input image and outputs the type of the input image.

[0111] The normalized exponential function helps the first multi-layer convolutional network (discriminator) to identify small spots in the entire image and perform accurate classification.

[0112] In this embodiment, through the improved self-attention mechanism proposed in the present invention, the first multi-layer convolutional network can better capture the details of the spots in the input image, and combine the feature information at different levels, so as to improve the detection ability of small facial spots, and adaptively adjust the weights to ensure that the first multi-layer convolutional network can still accurately judge the authenticity of small spots under different backgrounds and lighting conditions.

[0113] In one embodiment of the present invention, Fig.12 As shown, before inputting the input image into the first multi-layer convolutional network that has completed training, the first multi-layer convolutional network that introduces the self-attention mechanism and the second multi-layer convolutional network that introduces the spatial attention mechanism and the channel attention mechanism need to be trained. The above step S100 also includes:

[0114] S130 analyzes a first loss value of the first multi-layer convolutional network for distinguishing types of the first facial blob image and the second facial blob image.

[0115] S140 analyzes the second loss value of the second facial blob image generated by the second multi-layer convolutional network and judged by the first multi-layer convolutional network as the first facial blob image.

[0116] S150 dynamically adjusts and trains the first multi-layer convolutional network and the second multi-layer convolutional network based on the first loss value and the second loss value.

[0117] Specifically, before training the first multi-layer convolutional network and the second multi-layer convolutional network, the first facial spot image taken actually and the second facial spot image generated are combined to form a training data set, and a label is assigned to each image to identify it as a captured image or a generated image, which is convenient for subsequent training. The first multi-layer convolution and the second multi-layer convolutional network can be trained before or after recognition detection.

[0118] The first loss value of the first multi-layer convolutional network (discriminator) is calculated by comparing the difference between the predicted results of the first multi-layer convolutional network for the first facial blob image (collected image) and the second facial blob image (generated image) and the actual results.

[0119] For the acquisition image: the first multi-layer convolutional network will try to output a probability value close to 1 (indicating acquisition), and the actual label is also 1. Therefore, the loss of the first multi-layer convolutional network on the acquisition image measures the difference between its predicted value and the actual label 1.

[0120] For generated images: the first multi-layer convolutional network will try to output a probability value close to 0 (indicating fake), but the second multi-layer convolutional network will try to deceive the first multi-layer convolutional network into thinking that these generated images are real. Therefore, the loss of the first multi-layer convolutional network on the generated image measures the difference between its predicted value (close to 0) and the actual label 0 (or the label considered as fake data).

[0121] The second multi-layer convolution network (generator) refers to the second loss value encountered by the second multi-layer convolution network when generating a second facial blob image as close as possible to the first facial blob image (collected image) during training. The first loss value and the second loss value are monitored in real time to ensure that the first multi-layer convolution network and the second multi-layer convolution network remain balanced during training to avoid the "mode collapse" phenomenon.

[0122] In this embodiment, the first multi-layer convolutional network and the second multi-layer convolutional network are continuously trained against each other to enhance the recognition and judgment capabilities of the first multi-layer convolutional network and to enhance the ability of the second multi-layer convolutional network to generate images that are more similar to the acquired images.

[0123] In one embodiment of the present invention, a terminal device includes a processor and a memory, wherein the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory to implement the facial spot image detection and recognition method in the above-mentioned corresponding method embodiment.

[0124] The terminal device may be a desktop computer, a notebook, a PDA, a tablet computer, a mobile phone, a human-computer interaction screen, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that this example does not constitute a limitation on the terminal device, and may include more or fewer components, or a combination of certain components, or different components.

[0125] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0126] The memory may be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the smart device, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory may include both an internal storage unit of the device terminal and an external storage device. The memory is used to store the computer program and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0127] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0128] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed with hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0129] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A method for detecting and identifying facial spot images, characterized in that: The following steps are involved: Obtain a first facial spot image and a second facial spot image, and select one of the first facial spot image and the second facial spot image as an input image; the first facial spot image is acquired, and the second facial spot image is generated; Inputting the input image into a first multi-layer convolutional network that has completed training, and extracting a plurality of facial spot feature images from the input image; By utilizing the self-attention mechanism, spot areas are extracted and highlighted from the facial spot feature image, and spots of the input image and the type of the input image are detected.

2. The facial spot image detection and recognition method according to claim 1, characterized in that: The method of using the self-attention mechanism to extract and highlight the spot area from the facial spot feature image includes: Analyzing the pixel similarity between the preset local areas and the feature similarity of each of the facial spot feature images; Generate a self-attention matrix according to the pixel similarity between the preset local areas and the feature similarity of each of the facial spot feature images; A third weight coefficient is assigned to each pixel in the facial spot feature image according to the self-attention matrix to enhance the spot area of ​​the input image.

3. The facial spot image detection and recognition method according to claim 1, characterized in that: The detecting of spots of the input image and the type of the input image comprises: Using a normalized exponential function, detecting and classifying the input image; The fully connected layer of the first multi-layer convolutional network detects spots of the input image and outputs the type of the input image.

4. The method for detecting and identifying facial spot images according to claim 1, characterized in that: Before inputting the input image into the first multi-layer convolutional network that has completed training, the method includes: Using a spatial attention mechanism and Gaussian distributed random noise, the random noise is input into a second multi-layer convolutional network to generate a second facial spot sub-image with enhanced spot areas; By utilizing the channel attention mechanism, the second facial blob sub-image is generated into the second facial blob image highlighting the blob in the second multi-layer convolutional network.

5. The method for detecting and identifying facial spot images according to claim 4, characterized in that: The step of inputting the random noise into a second multi-layer convolutional network to generate a second facial spot sub-image with an enhanced spot area includes: According to the second multi-layer convolutional network, extracting a plurality of spatial attention images from the random noise; the spatial attention images include a spot color attention image, a spot texture attention image, and a spot shape attention image; Analyzing the similarity between the pixel points of each area in each of the spatial attention images and the pixel points of its adjacent areas; A first weight coefficient is assigned to each of the spatial attention images according to the similarity of the pixel points to obtain the second facial spot sub-image of the enhanced spot area.

6. The method for detecting and identifying facial spot images according to claim 4, characterized in that: The method of using the channel attention mechanism to generate the second facial spot sub-image into a second facial spot image with prominent spots in the second multi-layer convolutional network includes: Extracting a plurality of facial spot feature channels from the second facial spot sub-image; the facial spot feature channels include a spot color feature sub-channel and a spot texture sub-channel; Global average pooling is performed on each facial spot feature channel to obtain a channel average value and a channel maximum value of each facial spot feature channel; Obtaining a channel description vector of each facial spot feature channel based on the channel average value and the channel maximum value; Analyze the similarities between the facial spot feature channels according to the channel attention mechanism and the channel description vector of each facial spot feature channel to obtain a channel similarity matrix; In the middle layer of the second multi-layer convolutional network, a second weight coefficient is assigned to each of the facial spot feature channels according to the channel similarity matrix; The second facial spot image highlighting spots is obtained by combining each of the facial spot feature channels and the second weight coefficient corresponding thereto.

7. The method for detecting and identifying facial spot images according to any one of claims 1 to 6, characterized in that: include: Analyzing a first loss value of the first multi-layer convolutional network for distinguishing types of the first facial blob image and the second facial blob image; Analyze a second loss value of the second facial blob image generated by the second multi-layer convolutional network and determined by the first multi-layer convolutional network to be the first facial blob image; Based on the first loss value and the second loss value, the first multi-layer convolutional network and the second multi-layer convolutional network are dynamically adjusted and trained.

8. A terminal device, characterized in that: A method for generating and identifying an image of facial spots, comprising a processor for calling instructions stored in a memory, wherein when the instructions are executed, the terminal device executes the method for detecting and identifying facial spot images as described in any one of claims 1 to 7.

9. A facial spot image detection and recognition system, characterized in that: include: A collection device for collecting a first facial spot image and a second facial spot image of a human face; The first facial spot image is collected, and the second facial spot image is generated; The terminal device detects spots in the first facial spot image and the second facial spot image and determines the types of the first facial spot image and the second facial spot image.

Images