Model training, incomplete face image recognition and reconstruction methods, equipment and media

Abstract

Embodiments of the present invention relate to the field of face recognition and disclose a method, device, and medium for model training, recognition, and reconstruction of incomplete face images. The model training method includes: sequentially performing block segmentation, random masking, and sorting on a non-incomplete face image to obtain a sequence of masked image blocks of the non-incomplete face image; constructing an incomplete face feature extraction model for extracting incomplete face features from the sequence of masked image blocks of the non-incomplete face image; constructing a loss function based on the feature loss between the incomplete face features output by the incomplete face feature extraction model and the non-incomplete face features of the non-incomplete face image, and training the incomplete face feature extraction model, wherein the non-incomplete face features are obtained based on the non-incomplete face feature extraction model that has been trained for the non-incomplete face image. When applied to the incomplete face recognition process, better incomplete face features can be extracted.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of face recognition, and in particular to a method, device and medium for model training, recognition and reconstruction of incomplete face images. Background Art

[0002] With the development and implementation of facial recognition technology, the requirements for facial recognition systems are becoming increasingly higher. For example, accurate recognition of incomplete faces in situations such as wearing masks, glasses, and accessories. Currently, there are two common methods for recognizing incomplete facial images: the first is cooperative detection, which first determines the integrity of the face. For example, when incomplete facial information is detected, prompts (turn left, turn right, lower the head, raise the head, take off the mask, adjust the hat, etc.) are used to guide the face to adjust to a state that meets the recognition conditions before recognition. The second is non-cooperative detection, which is to build an incomplete face recognition model and collect a large number of incomplete face images under different face occlusion conditions to train the incomplete face recognition model.

[0003] However, the first method requires the user to adjust the face to an unobstructed image and cannot directly recognize incomplete facial images, limiting its applicability. The second method is costly and time-consuming to collect data, and requires multiple specialized models to collaboratively address incomplete face recognition under varying occlusion conditions, making the resulting recognition model large and complex. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device and medium for model training, incomplete facial image recognition and reconstruction, which uses a sequence of mask image blocks of non-incomplete facial images as training data for an incomplete facial feature extraction model to simulate various face missing situations in real scenes, avoiding the long training time and high cost caused by large-scale data collection, and making full use of the connection between local image information, so that the obtained incomplete facial feature extraction model will not be too complicated.

[0005] To solve the above technical problems, an embodiment of the present invention provides a model training method, comprising: sequentially dividing, randomly masking and sorting a non-incomplete face image to obtain a sequence of masked image blocks of the non-incomplete face image; constructing an incomplete face feature extraction model for extracting incomplete face features from the sequence of masked image blocks of the non-incomplete face image; constructing a loss function based on the feature loss between the incomplete face features output by the incomplete face feature extraction model and the non-incomplete face features of the non-incomplete face image, and training the incomplete face feature extraction model, wherein the non-incomplete face features are obtained based on the trained non-incomplete face feature extraction model for the non-incomplete face image.

[0006] To solve the above technical problems, an embodiment of the present invention provides a method for recognizing incomplete facial images, comprising: dividing and sorting the incomplete facial image in sequence to form an image block sequence; inputting the image block sequence into a trained incomplete facial feature extraction model to obtain the incomplete facial features of the incomplete facial image; wherein the incomplete facial feature extraction model is obtained through the model training method as described above; and recognizing the incomplete facial image based on the obtained incomplete facial features to obtain a recognition result.

[0007] An embodiment of the present invention also provides a method for reconstructing an incomplete facial image, comprising: obtaining incomplete facial features of an incomplete facial image, wherein the incomplete facial features are obtained by the incomplete facial image recognition method described in the above embodiment; inputting the incomplete facial features into a trained image reconstruction model to obtain a reconstructed image block sequence, wherein the incomplete facial feature extraction model described in the above embodiment and the image reconstruction model constitute an encoder-decoder model framework; and combining the reconstructed image block sequence to obtain a reconstructed facial image corresponding to the incomplete facial image.

[0008] An embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method mentioned in the above embodiment, or can execute the incomplete facial image recognition method mentioned in the above embodiment, or can execute the incomplete facial image reconstruction method mentioned in the above embodiment.

[0009] An embodiment of the present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the model training method mentioned in the above embodiment, or can execute the incomplete facial image recognition method mentioned in the above embodiment, or can execute the incomplete facial image reconstruction method mentioned in the above embodiment.

[0010] The model training method provided by the embodiments of the present invention uses a sequence of masked image blocks as input data for the incomplete face feature extraction model. On the one hand, it fully utilizes the connection between local image information and, to a certain extent, can avoid over-complexity of the entire face recognition system. On the other hand, using a sequence of masked image blocks as training data can simulate the missing and occluded faces in real scenes, avoiding the long training time and high cost caused by large-scale data collection. During the training process, a loss function is constructed based on the feature loss between the incomplete face features output by the incomplete face feature extraction model and the non-incomplete face features of the non-incomplete face image, and the incomplete face feature extraction model is trained. In other words, the present application performs training and learning based on the trained non-incomplete face feature extraction model, so that the incomplete face features output by the incomplete face feature extraction model are as close as possible to the non-incomplete face features of the corresponding non-incomplete face image, thereby effectively improving the accuracy of subsequent incomplete face recognition, reducing training time, and the resulting incomplete face feature extraction model is not overly complex. The incomplete face feature extraction model obtained through the training method of this application can effectively solve various incomplete face recognition problems for face recognition.

[0011] In addition, the model training method provided by the embodiment of the present invention sequentially divides the non-incomplete face image into blocks, randomly masks, and sorts the blocks to obtain the mask image block sequence, including: dividing the non-incomplete face image into blocks according to a preset image block size to obtain non-incomplete face image blocks; randomly selecting one or more areas from all non-incomplete face image blocks as mask areas, and randomly selecting non-incomplete face image blocks within the mask areas according to a preset mask ratio for masking to obtain mask image blocks; and arranging the mask image blocks in a preset order to obtain a mask image block sequence. When masking the image blocks, the mask areas are first randomly selected, and then image blocks are randomly selected within the mask areas for masking. This can not only effectively simulate face loss situations in various situations, but also quickly obtain training data.

[0012] In addition, in the model training method provided by an embodiment of the present invention, when the non-incomplete face feature extraction model adopts a convolutional neural network structure, the non-incomplete face feature extraction model is a first non-incomplete face feature extraction model trained using a non-incomplete face image as a sample; the non-incomplete face feature used to obtain the feature loss is the non-incomplete face feature obtained by inputting the non-incomplete face image into the first non-incomplete face feature extraction model; when the non-incomplete face feature extraction model adopts a transformer structure, the non-incomplete face feature extraction model is a second non-incomplete face feature extraction model trained using a sequence of image blocks obtained by segmenting and sorting the non-incomplete face image as a sample; the non-incomplete face feature used to obtain the feature loss is the non-incomplete face feature obtained by inputting a sequence of image blocks obtained by segmenting and sorting the non-incomplete face image into the second non-incomplete face feature extraction model. Regardless of the structure of the non-incomplete face feature extraction model, its input data is not masked. The non-incomplete face feature model has been trained, so the training process of the incomplete face feature extraction model is based on the non-incomplete face feature model. The output result of the feature extraction model is as close as possible to the output result of the non-incomplete face feature extraction model. This can not only reduce the training time and the amount of calculation, but also obtain a feature extraction model with a simple structure and good effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0014] Figure 1 is a flow chart of a model training method provided by an embodiment of the present invention;

[0015] Figure 2 is a flow chart of a method for recognizing incomplete facial images provided by an embodiment of the present invention;

[0016] Figure 3 is a flowchart of a method for reconstructing an incomplete face image provided by an embodiment of the present invention;

[0017] Figure 4 It is a schematic structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, each embodiment of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will appreciate that in each embodiment of the present invention, many technical details are provided to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0019] The following examples illustrate the implementation details of the model training of this embodiment. The following content is only provided for the convenience of understanding and is not required for the implementation of this solution.

[0020] The embodiment of the present invention relates to a model training method, such as Figure 1 Shown, including:

[0021] Step 101 : performing block division, random masking, and sorting processing on a non-incomplete face image in sequence to obtain a mask image block sequence of the non-incomplete face image.

[0022] In this embodiment, step 101 specifically includes: dividing the non-incomplete facial image into blocks according to a preset image block size to obtain non-incomplete facial image blocks; randomly selecting one or more mask areas from all non-incomplete facial image blocks, and randomly selecting non-incomplete facial image blocks within the mask areas according to a preset mask ratio to mask, to obtain mask image blocks; and arranging the mask image blocks in a preset order to obtain a mask image block sequence.

[0023] In addition, the mask area described in this embodiment is the upper face area, or the lower face area, or the left face area, or the right face area; the area of ​​the mask area is less than or equal to forty-five percent of the area of ​​the entire face area. That is to say, the area of ​​the upper face area, the lower face area, the left face area, and the right face area does not exceed 45% of the area of ​​the entire face area. Such a setting can fully simulate various face missing scenes without causing the randomly generated mask image to block too many facial key points, making face recognition meaningless. Of course, the shape (circular, rectangular, elliptical, triangular) and position of the mask area can be set at will, and once set, it will no longer change during subsequent training. Furthermore, the mask ratio refers to the proportion of the number of mask image blocks in the mask area to the number of all non-incomplete face image blocks; the setting range of the mask ratio is 30% to 70%.

[0024] Specifically, when segmenting a non-incomplete facial image, the size of each image block must allow for uniform, non-overlapping segmentation. For example, if a non-incomplete facial image is 224×224 in size and a block size of 7×7 is set, 32×32=1024 image blocks will be generated. For the entire non-incomplete facial image block, assuming a randomly selected mask region—the left facial region—should be used. Then, several image blocks within the left facial region are randomly selected for masking. Assuming there are 100 image blocks in the left facial region and the masking ratio is set to 50%, 50 image blocks within the left facial region are masked. The specific locations of these 50 image blocks within the left facial region can be freely selected. Masking involves setting the pixel values ​​of the image blocks to be masked to 0.

[0025] It's important to note that using a sequence of masked image blocks from intact face images as training data not only fully exploits the connections between local images but also simulates various face-missing scenarios found in real-world scenarios. It's understandable that one image block is highly correlated with its neighbors. Therefore, using a sequence of image blocks as training data allows the face-missing recognition model to fully learn the correlations between these blocks, improving its performance in face-missing image recognition.

[0026] Step 102: constructing a defective face feature extraction model for extracting defective face features from a sequence of mask image blocks of a non-defective face image.

[0027] In this embodiment, a model for extracting features of incomplete faces is constructed, and during training, a sequence of mask image blocks of non-incomplete face images is input into the model to extract features of incomplete faces.

[0028] Step 103, constructing a loss function based on the feature loss between the incomplete facial features output by the incomplete facial feature extraction model and the non-incomplete facial features of the non-incomplete facial image, and training the incomplete facial feature extraction model, wherein the non-incomplete facial features are obtained based on the non-incomplete facial feature extraction model that has been trained on the non-incomplete facial image.

[0029] In this embodiment, during training, the non-incomplete facial features extracted by the non-incomplete facial feature extraction model serve as the training guide for the incomplete facial feature extraction model, ensuring that the incomplete facial features output by the incomplete facial feature extraction model are as close as possible to the non-incomplete facial features of the corresponding non-incomplete facial image. Simply put, during training, for the same face, a non-incomplete facial image is obtained, which is then input into the trained non-incomplete facial feature extraction model to extract non-incomplete facial features. The non-incomplete facial image is then segmented, randomly masked, and sorted to obtain a sequence of masked image blocks. This sequence of masked image blocks is then input into the incomplete facial feature extraction model for training, with the training loss function calculated based on the distance between the non-incomplete facial features and the incomplete facial features. In other words, the trained non-incomplete facial feature extraction model serves as the teacher model, and the incomplete facial feature extraction model serves as the student model for training and learning.

[0030] In other words, this application allows the incomplete face feature extraction model to learn based on a well-trained and very complete non-incomplete face feature extraction model. This can not only avoid the incomplete face feature extraction model from being too complicated, but also achieve a good incomplete face recognition effect.

[0031] In one embodiment, when the non-incomplete face feature extraction model adopts a convolutional neural network structure, the non-incomplete face feature extraction model is a first non-incomplete face feature extraction model trained with a non-incomplete face image as a sample; the non-incomplete face feature for obtaining the feature loss is obtained by inputting the non-incomplete face image into the first non-incomplete face feature extraction model; when the non-incomplete face feature extraction model adopts a transformer structure, the non-incomplete face feature extraction model is a second non-incomplete face feature extraction model trained with a sequence of image blocks obtained after the non-incomplete face image is divided and sorted as a sample; the non-incomplete face feature for obtaining the feature loss is obtained by inputting a sequence of image blocks obtained after the non-incomplete face image is divided and sorted into the second non-incomplete face feature extraction model.

[0032] That is to say, when the trained non-incomplete face feature extraction model is a convolutional neural network structure, the non-incomplete face image is input into the trained non-incomplete face feature extraction model, and the mask image block sequence is input into the incomplete face feature extraction model for training; when the trained non-incomplete face feature extraction model is a transformer structure, the image block sequence obtained after the non-incomplete face image is divided into blocks and sorted is input into the trained non-incomplete face feature extraction model, and the mask image block sequence is input into the incomplete face feature extraction model for training. In addition, the incomplete face feature extraction model of the present application can be a transformer structure, or other model structure that can process image block sequences.

[0033] Specifically, regardless of the structure of the non-incomplete face feature extraction model, its input data is not masked. When the trained non-incomplete face feature extraction model is a convolutional neural network structure, the entire non-incomplete face image is directly input into the trained non-incomplete face feature extraction model, and the mask image block sequence is input into the incomplete face feature extraction model for training. During training, the output of the incomplete face feature extraction model is made as close as possible to the output of the non-incomplete face feature extraction model. When the trained non-incomplete face feature extraction model is a transformer structure, the image block sequence obtained after the non-incomplete face image is divided and sorted is input into the trained non-incomplete face feature extraction model, and the mask image block sequence is input into the incomplete face feature extraction model for training. At this time, the image block sequence obtained after the non-incomplete face image is divided and sorted is consistent with the order of the training data during the training of the non-incomplete face feature extraction model.

[0034] Furthermore, the loss function used in the training of the incomplete face feature extraction model in this embodiment is:

[0035]

[0036] Among them, y i The non-incomplete face features output by the non-incomplete face feature extraction model, y i p is the incomplete facial feature output by the incomplete facial feature extraction model, and n is the total number of sample data for training the incomplete facial feature extraction model.

[0037] Specifically, for the same non-incomplete face image, the non-incomplete face image is processed accordingly according to the structure of the non-incomplete face feature extraction model, and then the processed data is input into the trained non-incomplete face feature extraction model to obtain the non-incomplete face features. At the same time, the mask image block sequence after the non-incomplete face image is divided, masked, and sorted is input into the incomplete face feature extraction model to be trained to obtain the incomplete face features. During the training process, the above loss function is used to make the incomplete face features as close as possible to the corresponding non-incomplete face features.

[0038] The model training method provided by the embodiments of the present invention uses a sequence of masked image blocks as input data for the incomplete face feature extraction model. On the one hand, it fully utilizes the connection between local image information and, to a certain extent, can avoid over-complexity of the entire face recognition system. On the other hand, using a sequence of masked image blocks as training data can simulate the missing and occluded faces in real scenes, avoiding the long training time and high cost caused by large-scale data collection. During the training process, a loss function is constructed based on the feature loss between the incomplete face features output by the incomplete face feature extraction model and the non-incomplete face features of the non-incomplete face image, and the incomplete face feature extraction model is trained. In other words, the present application performs training and learning based on the trained non-incomplete face feature extraction model, so that the incomplete face features output by the incomplete face feature extraction model are as close as possible to the non-incomplete face features of the corresponding non-incomplete face image, thereby effectively improving the accuracy of subsequent incomplete face recognition, reducing training time, and the resulting incomplete face feature extraction model is not overly complex. The incomplete face feature extraction model obtained through the training method of this application can effectively solve various incomplete face recognition problems for face recognition.

[0039] The embodiment of the present invention relates to a method for recognizing an incomplete face image, such as Figure 2 Shown, including:

[0040] Step 201 : The incomplete face image is divided into blocks and sorted in sequence to form an image block sequence.

[0041] In this embodiment, the incomplete facial image can be caused by partial face damage due to object occlusion, partial face damage due to lens occlusion or unclear image during shooting, or partial face loss due to partial image loss during image acquisition and transmission.

[0042] It should be noted that the training data for the incomplete face feature extraction model is in the form of an image block sequence. Therefore, when performing incomplete face image recognition, the image block sequence obtained by dividing and sorting the incomplete face image into blocks is consistent with the order of the training data. For example, during training, the order of sorting the mask image blocks is from left to right and from top to bottom. Then, during application, after dividing the incomplete face image into blocks, the image blocks are arranged in the order from left to right and from top to bottom to obtain an image block sequence. Of course, the arrangement order of the training data (image block sequence) in this embodiment can be from left to right and from top to bottom, from left to right and from bottom to top, from top to bottom and from left to right, etc. The specific arrangement order is not limited here, as long as the order of the image block sequence during application is consistent with the order of the image block sequence during training.

[0043] Step 202: input the image block sequence into a trained incomplete face feature extraction model to obtain incomplete face features of the incomplete face image; wherein the incomplete face feature extraction model is obtained by the model training method as described above.

[0044] Step 203: Recognize the incomplete face image based on the acquired incomplete face features to obtain a recognition result.

[0045] Specifically, after extracting the incomplete face features, they are fed into a trained incomplete face classifier to identify the incomplete face image and obtain a recognition result. The incomplete face classifier can be a nearest neighbor classifier, a linear classifier, or other similar classifiers, and the specific classifier model structure is not limited here.

[0046] The incomplete face image recognition method provided in this embodiment inputs the image block sequence of the incomplete face image into the incomplete face feature extraction model to obtain the incomplete face features. Since the incomplete face feature extraction model of this application is based on the trained non-incomplete face feature extraction model and is trained with the mask image block sequence of the non-incomplete face image as training data, the obtained incomplete face features are accurate and reliable, thereby effectively improving the accuracy of incomplete face recognition.

[0047] The embodiment of the present invention relates to a method for reconstructing an incomplete face image, such as Figure 3 Shown, including:

[0048] Step 301: Obtain incomplete facial features of an incomplete facial image, wherein the incomplete facial features are obtained by the incomplete facial image recognition method provided in the above embodiment.

[0049] Specifically, the incomplete face image is divided into blocks and sorted to obtain an image block sequence; the image block sequence is input into the incomplete face feature extraction model provided in the above embodiment to obtain the incomplete face features.

[0050] In step 302, the incomplete facial features are input into the trained image reconstruction model to obtain a reconstructed image block sequence, wherein the image reconstruction model and the incomplete facial feature extraction model described in the above embodiment constitute an encoder-decoder model framework.

[0051] In one embodiment, before step 302, the method further includes: training the image reconstruction model using the following loss function:

[0052]

[0053] Among them, Ω represents the number of training data, x M is the reconstructed image block sequence output by the image reconstruction model, y M is the sequence of original image patches corresponding to the training data of the image reconstruction model.

[0054] Specifically, when training the image reconstruction model, a mask image block sequence (simulating a defective face image) is obtained by dividing, masking, and sorting the non-defective face image. The mask image block sequence is passed through the trained defective face feature extraction model to output the defective face features. The defective face features are then input into the image reconstruction model to be trained to obtain the reconstructed image block sequence x M During the training process, the reconstructed image block sequence is made as close as possible to the original non-incomplete face image block sequence y M .

[0055] Step 303: Combine the reconstructed image block sequences to obtain a reconstructed face image corresponding to the incomplete face image.

[0056] It's worth noting that currently, most 2D face reconstruction or restoration methods rely on network structures like GANs, which take a full face image as input and output a reconstructed face image. However, this approach relies on the global features of the entire face image for reconstruction, ignoring the local characteristics of the face. This results in the reconstructed face having the characteristics of a general face and losing the specificity of the facial identity.

[0057] The incomplete facial image reconstruction method provided by the embodiment of the present invention adopts an incomplete facial feature extraction model to obtain facial features. The incomplete facial feature extraction model is trained and learned based on a trained non-incomplete facial feature extraction model, and a distillation loss function is used for loss calculation, so that the obtained incomplete facial features have more realistic identity information, thereby ensuring that the facial image obtained in subsequent reconstruction is more consistent with the real face appearance, rather than obtaining multiple similar average faces after reconstructing multiple different incomplete facial images, which loses identity specificity.

[0058] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0059] An embodiment of the present invention relates to an electronic device, such as Figure 4 Shown, including:

[0060] At least one processor 401; and a memory 402 communicatively connected to the at least one processor 401; wherein the memory 402 stores instructions that can be executed by the at least one processor 401, and the instructions are executed by the at least one processor 401 so that the at least one processor 401 can execute the model training method mentioned in the above embodiment, or can execute the incomplete facial image recognition method mentioned in the above embodiment, or can execute the incomplete facial image reconstruction method mentioned in the above embodiment.

[0061] The electronic device includes: one or more processors 401 and a memory 402, Figure 4 A processor 401 is used as an example. The processor 401 and the memory 402 may be connected via a bus or other means. Figure 4 The example of a bus connection is taken as an example. Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. For example, the algorithms corresponding to the various processing strategies in the strategy space in the embodiment of the present application are stored in memory 402. Processor 401 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 402, that is, implements the above-mentioned model training method, or the incomplete face image recognition method or the incomplete face image reconstruction method.

[0062] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 402 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely located relative to the processor 401, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0063] One or more modules are stored in the memory 402. When executed by one or more processors 401, they can perform the model training method mentioned in the above embodiment, or can perform the incomplete facial image recognition method mentioned in the above embodiment, or can perform the incomplete facial image reconstruction method mentioned in the above embodiment.

[0064] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0065] The embodiments of the present invention relate to a computer-readable storage medium storing a computer program, which implements the above method embodiments when executed by a processor.

[0066] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0067] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A model training method, characterized in that: include: Sequentially dividing, randomly masking, and sorting the non-incomplete face image to obtain a mask image block sequence of the non-incomplete face image; Constructing a defective face feature extraction model for extracting defective face features from a sequence of mask image blocks of the non-defective face image; Constructing a loss function based on the feature loss between the incomplete facial features output by the incomplete facial feature extraction model and the non-incomplete facial features of the non-incomplete facial image, and training the incomplete facial feature extraction model, wherein the non-incomplete facial features are obtained based on the non-incomplete facial feature extraction model that has been trained for the non-incomplete facial image; Wherein, when the non-incomplete face feature extraction model adopts a convolutional neural network structure, the non-incomplete face feature extraction model is a first non-incomplete face feature extraction model trained using a non-incomplete face image as a sample; the non-incomplete face feature used to obtain the feature loss is the non-incomplete face feature obtained by inputting the non-incomplete face image into the first non-incomplete face feature extraction model; When the non-incomplete facial feature extraction model adopts a transformer structure, the non-incomplete facial feature extraction model is a second non-incomplete facial feature extraction model trained using a sequence of image blocks obtained after the non-incomplete facial image is segmented and sorted as samples; the non-incomplete facial features used to obtain the feature loss are the non-incomplete facial features obtained by inputting a sequence of image blocks obtained after the non-incomplete facial image is segmented and sorted into the second non-incomplete facial feature extraction model.

2. The model training method according to claim 1, characterized in that The step of sequentially performing block division, random masking, and sorting on the non-incomplete face image to obtain the mask image block sequence includes: Dividing the non-incomplete face image into blocks according to a preset image block size to obtain non-incomplete face image blocks; Randomly selecting one or more regions from all non-incomplete face image blocks as mask regions, and randomly selecting non-incomplete face image blocks within the mask regions according to a preset mask ratio for masking, to obtain mask image blocks; The mask image blocks are arranged in a preset order to obtain a mask image block sequence.

3. The model training method according to claim 2, characterized in that The mask area is the upper face area, the lower face area, the left face area, or the right face area; the area of ​​the mask area is less than or equal to 45 percent of the area of ​​the entire face area.

4. The model training method according to claim 2 or 3, characterized in that: The mask ratio is the ratio of the number of mask image blocks to the number of non-incomplete face image blocks in the mask area; the setting range of the mask ratio is 30% to 70%.

5. The model training method according to any one of claims 1 to 3, characterized in that: The loss function used by the incomplete face feature extraction model in the training phase is: Among them, y i The non-incomplete face features output by the non-incomplete face feature extraction model, y i p is the incomplete facial feature output by the incomplete facial feature extraction model, and n is the total number of sample data for training the incomplete facial feature extraction model.

6. A method for recognizing incomplete facial images, characterized in that: include: The incomplete face image is divided into blocks and sorted in sequence to form an image block sequence; Inputting the image block sequence into a trained incomplete face feature extraction model to obtain incomplete face features of the incomplete face image; wherein the incomplete face feature extraction model is obtained by the model training method according to any one of claims 1 to 5 above; The incomplete facial image is recognized based on the acquired incomplete facial features to obtain a recognition result.

7. A method for reconstructing an incomplete face image, characterized in that: include: Obtaining incomplete facial features of the incomplete facial image, wherein the incomplete facial features are obtained by the incomplete facial image recognition method as described in claim 6; Inputting the incomplete facial features into a trained image reconstruction model to obtain a reconstructed image block sequence, wherein the incomplete facial feature extraction model according to any one of claims 1 to 5 and the image reconstruction model constitute an encoder-decoder model framework; The reconstructed image block sequences are combined to obtain a reconstructed face image corresponding to the incomplete face image.

8. The method for reconstructing an incomplete face image according to claim 7, wherein: Before inputting the incomplete facial features into a trained image reconstruction model to obtain a reconstructed image block sequence, the method further includes: The image reconstruction model is trained using the following loss function: Among them, Ω represents the number of training data, x M is the reconstructed image block sequence output by the image reconstruction model, y M is the sequence of original image patches corresponding to the training data of the image reconstruction model.

9. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method as described in any one of claims 1 to 5, or can execute the incomplete facial image recognition method as described in claim 6, or can execute the incomplete facial image reconstruction method as described in any one of claims 7 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the model training method described in any one of claims 1 to 5, or implements the incomplete facial image recognition method described in claim 7, or implements the incomplete facial image reconstruction method described in any one of claims 7 to 8.

Images