Training data construction method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses a training data construction method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a plurality of basic face images and a plurality of basic shielding object images; enhancement transformation processing is carried out on the shelter in the basic shelter image to obtain a plurality of enhanced shelter images; constructing a plurality of to-be-fused image pairs based on a plurality of basic face images, the basic shielding object images and a plurality of enhanced shielding object images; and for each to-be-fused image pair, based on a randomization mechanism, performing fusion processing on the basic face image and the shielding object image in the to-be-fused image pair to obtain a shielding face image. According to the method, diversified sheltered face images including various sheltering conditions can be constructed, and the sheltered face images are taken as training data of the image restoration model, so that the image restoration model can fully learn restoration modes of various sheltering conditions, and the images with sheltered faces can be reliably restored.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a training data construction method, apparatus, device, storage medium and program product. Background Art

[0002] Nowadays, many film and television works have various image quality problems (such as noise, blur, distortion, etc.) due to the long shooting time and aging storage media. These image quality problems will seriously affect the user's viewing experience of film and television works. In order to solve these image quality problems, digital restoration technology is currently usually used to restore the image screen in film and television works.

[0003] In film and television works, the human face is one of the most important elements, and it plays a vital role in promoting the plot and shaping the character's personality. Correspondingly, in digital restoration, face restoration is also a very important link. However, in many cases, the faces in film and television works may be blocked by occluders (such as masks, hats, beards, arms, etc.). For such images with blocked faces, the image restoration models in related technologies often find it difficult to reliably restore the corresponding high-quality images. The reason is that there are many situations in which faces are blocked, and related technologies usually find it difficult to obtain sufficiently diverse training images that cover various occlusion situations. This will cause the image restoration model to be unable to fully learn the restoration methods for various occlusion situations during the training process, and thus it is difficult to reliably repair images with blocked faces. Summary of the invention

[0004] The embodiments of the present application provide a training data construction method, apparatus, device, storage medium and program product, which can construct a variety of occluded face images containing various occlusion situations, and use these occluded face images as training data for the image restoration model, which is conducive to enabling the image restoration model to fully learn the restoration methods of various occlusion situations, so that images with occluded faces can be reliably restored.

[0005] In view of this, the first aspect of the present application provides a method for constructing training data, the method comprising:

[0006] Acquire a plurality of basic face images and a plurality of basic occluder images; the basic face images include unoccluded faces, and the basic occluder images include occluders used to occlude faces;

[0007] Performing enhancement transformation processing on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images;

[0008] Based on the multiple basic face images, the multiple basic occluder images and the multiple enhanced occluder images, construct multiple image pairs to be fused; each of the image pairs to be fused includes one of the basic face images and one of the occluder images, and the occluder image is the basic occluder image or the enhanced occluder image;

[0009] For each pair of images to be fused, based on a randomization mechanism, the basic face image and the occlusion image in the pair of images to be fused are fused to obtain an occluded face image; the randomization mechanism is used to control the random deployment of the occluders in the occlusion image in the occluded face image, and the occluded face image is used as a training image for an image restoration model.

[0010] A second aspect of the present application provides a training data construction device, the device comprising:

[0011] An acquisition module, used to acquire a plurality of basic face images and a plurality of basic occluding object images; the basic face images include unobstructed faces, and the basic occluding object images include occluding objects used to obstruct faces;

[0012] An enhancement module, used for performing enhancement transformation processing on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images;

[0013] A combining module, configured to construct a plurality of image pairs to be fused based on the plurality of basic face images, the plurality of basic occluder images and the plurality of enhanced occluder images; each of the image pairs to be fused includes one of the basic face images and one of the occluder images, and the occluder image is the basic occluder image or the enhanced occluder image;

[0014] A synthesis module is used to fuse the basic face image and the occlusion image in each pair of images to be fused based on a randomization mechanism to obtain an occluded face image; the randomization mechanism is used to control the random deployment of the occluders in the occlusion image in the occluded face image, and the occluded face image is used as a training image for the image restoration model.

[0015] Optionally, the device further comprises:

[0016] A sample determination module, used to determine an annotated image corresponding to the training image, and use the training image and the annotated image to form a training sample; the image quality of the annotated image is higher than the image quality of the training image;

[0017] A sample processing module, used for inputting the training image in the training sample into the image restoration model to obtain a predicted restoration image output by the image restoration model; the image restoration model is used for restoring the input low-quality image into a corresponding high-quality image;

[0018] The model training module is used to train the image restoration model based on the difference between the predicted restoration image and the labeled image in the training sample.

[0019] Optionally, the enhancement module is specifically used for:

[0020] Generate a plurality of different sets of random transformation factors; the random transformation factors are used to determine the transformation mode and degree of the enhanced transformation process;

[0021] For each group of the random transformation factors, construct a corresponding composite affine matrix;

[0022] Based on the composite affine matrices corresponding to each group of the random transformation factors, the enhanced transformation process is performed on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images.

[0023] Optionally, the transformation method of the enhanced transformation processing includes at least one of translation transformation, scaling transformation, rotation transformation, projection transformation, shape transformation, color transformation, and texture transformation.

[0024] Optionally, the synthesis module is specifically used for:

[0025] Performing spatial transformation processing on the occluder image to obtain an aligned occluder image; the aligned occluder image is aligned with the basic face image;

[0026] Based on the randomization mechanism, randomly transform the occluders in the aligned occluder image to obtain a target occluder image;

[0027] The transparency blending technology is adopted to overlay the occluder in the target occluder image onto the basic face image with the target transparency, so as to obtain the occluded face image.

[0028] Optionally, the synthesis module is specifically used for:

[0029] For each pixel position in the initial image, determine a target pixel value of the pixel position in the initial image according to a pixel value of a pixel at the pixel position in the basic face image, a pixel value at the pixel position in the target occluder image, and the target transparency;

[0030] The blocked face image is generated according to the target pixel value of each pixel position in the initial image.

[0031] Optionally, the synthesis module is specifically used for:

[0032] Determine the occluder deployment position according to the target occluder image and the basic face image through the occluder position prediction model; the occluder deployment position is used to indicate the deployment position of the occluder in the target occluder image in the basic face image;

[0033] The transparency blending technology is used to cover the occluder in the target occluder image with the target transparency to the occluder deployment position in the basic face image to obtain the occluded face image.

[0034] Optionally, the acquisition module is specifically used for:

[0035] Acquire an image including an occluder as the basic occluder image;

[0036] and / or,

[0037] A three-dimensional obstruction model is constructed by using three-dimensional modeling software, and an image including the three-dimensional obstruction model is rendered to obtain the basic obstruction image.

[0038] Optionally, the device further comprises:

[0039] A post-processing module is used to perform image post-processing on the occluded face image to obtain a training image for the image restoration model; the image post-processing is used to adjust the occlusion effect between the occluder and the face in the occluded face image.

[0040] Optionally, the image post-processing includes at least one of the following:

[0041] Performing feathering processing on the synthetic boundary of the occluder in the occluded face image;

[0042] According to the illumination condition of the basic face image, adjusting the display parameters corresponding to the occluder in the occluder image;

[0043] Through color correction, the color of the occluder in the occluder image is matched with the color of the basic face image.

[0044] A third aspect of the present application provides an electronic device, the electronic device comprising a processor and a memory;

[0045] The memory is used to store computer programs;

[0046] The processor is used to execute the training data construction method described in the first aspect according to the computer program.

[0047] The fourth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a computer device, it implements the training data construction method described in the first aspect above.

[0048] The fifth aspect of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the training data construction method described in the first aspect above.

[0049] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0050] The embodiment of the present application provides a method for constructing training data, which includes: obtaining multiple basic face images including unobstructed faces, and obtaining multiple basic occluder images including occluders for obstructing faces. Then, the occluders in the basic occluder images are enhanced and transformed to obtain multiple enhanced occluder images; by performing enhanced transformation processing on the occluders in the basic occluder images, richer and more diverse occluders can be enhanced and generated based on limited basic occluder images, providing abundant available occluders for the subsequent generation of occluded face images. Then, based on the obtained basic face images, the basic occluder images and the generated enhanced occluder images, multiple pairs of images to be fused are constructed, each pair of images to be fused includes a basic face image and an occluder image (basic occluder image or enhanced occluder image), thereby constructing different combinations of faces and occluders. Then, for each pair of images to be fused, based on a randomization mechanism, the basic face image and the occluder image in the pair of images to be fused are fused, thereby obtaining an occluded face image including both the face and the occluder; by using a randomization mechanism in the fusion process, the occluders in the occluded image can be controlled to be randomly deployed in the occluded face image to be generated, thereby improving the randomness of the occluder layout, which is conducive to simulating various possible occlusion situations and improving the generalization of the generated occluded face image. Each occluded face image generated by the above method can be used as a training image for the image restoration model. Since the generated occluded face images are relatively diverse (including the diversity of occluders and the diversity of occluder deployment positions), they can basically cover various face occlusion situations. Therefore, using these occluded face images to train the image restoration model can enable the image restoration model to fully learn the restoration methods for various occlusion situations. Accordingly, the image restoration model can be put into practical applications, and images with occluded faces can be more reliably restored to obtain corresponding high-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1A schematic diagram of an application scenario of the training data construction method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of a training data construction method provided in an embodiment of the present application;

[0053] Figure 3 An overall implementation structure diagram of the training data construction method provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a training data construction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] In the related art, for images with occluded faces, image restoration models often have difficulty reliably restoring corresponding high-quality images. The specific reason is that there are many different situations in which faces are occluded, and related technologies usually have difficulty obtaining sufficiently diverse training images that cover various occlusion situations. This results in the image restoration model being unable to fully learn the restoration methods for various occlusion situations during the training process, and thus it is difficult to reliably restore images with occluded faces.

[0058] In order to solve the above problems, an embodiment of the present application provides a training data construction method, which can generate diverse occluded face images that basically cover various occlusion situations by generating diverse occlusion images and fusing the face image with the occlusion image based on a randomization mechanism.

[0059] Specifically, the training data construction method provided in the embodiment of the present application includes: obtaining multiple basic face images including unobstructed faces, and obtaining multiple basic occluder images including occluders for obstructing faces. Then, the occluders in the basic occluder images are enhanced and transformed to obtain multiple enhanced occluder images; by performing enhanced transformation processing on the occluders in the basic occluder images, it is possible to enhance and generate richer and more diverse occluders based on limited basic occluder images, thereby providing abundant available occluders for the subsequent generation of occluded face images. Next, based on the obtained basic face images, the basic occluder images, and the generated enhanced occluder images, multiple pairs of images to be fused are constructed, each pair of images to be fused includes a basic face image and an occluder image (basic occluder image or enhanced occluder image), thereby constructing different combinations of faces and occluders. Then, for each pair of images to be fused, based on a randomization mechanism, the basic face image and the occluder image in the pair of images to be fused are fused, thereby obtaining an occluded face image including both the face and the occluder; by using a randomization mechanism in the fusion process, the occluders in the occluded image can be controlled to be randomly deployed in the occluded face image to be generated, thereby improving the randomness of the occluder layout, which is conducive to simulating various possible occlusion situations and improving the generalization of the generated occluded face image. Each occluded face image generated by the above method can be used as a training image for the image restoration model. Since the generated occluded face images are relatively diverse (including the diversity of occluders and the diversity of occluder deployment positions), they can basically cover various face occlusion situations. Therefore, using these occluded face images to train the image restoration model can enable the image restoration model to fully learn the restoration methods for various occlusion situations. Accordingly, the image restoration model can be put into practical applications, and images with occluded faces can be more reliably restored to obtain corresponding high-quality images.

[0060] It should be noted that the training data construction method provided in the embodiment of the present application can be executed by a computer device, which can be a terminal device or a server. Among them, the terminal device can be a computer or other device with image processing function; the server can be an independent server or a cluster composed of multiple servers. In addition, in the training data construction method provided in the embodiment of the present application, the collection, use and processing of relevant data shall comply with the relevant laws, regulations and standards of relevant countries and regions.

[0061] In order to facilitate understanding of the training data construction method provided in the embodiment of the present application, the application scenario of the training data construction method is exemplarily introduced below by taking the execution subject of the training data construction method as a server as an example.

[0062] See also Figure 1 , Figure 1 Schematic diagram of application scenarios of the training data construction method provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes a server 110 and a database 120. The server 110 can access the database 120 through a network, or the database 120 can also be integrated in the server 110. The server 110 is used to execute the training data construction method provided in the embodiment of the present application, and the database 120 is used to store basic face images and basic occlusion images.

[0063] In actual applications, the server 110 can obtain multiple basic face images and multiple basic occluder images from the database 120. Then, the server 110 can perform enhancement transformation processing on the occluders in the obtained basic face images to obtain multiple enhanced occluder images. Next, the server 110 can construct multiple image pairs to be fused based on each basic face image, each basic occluder image, and each enhanced occluder image; each image pair to be fused includes a basic face image and an occluder image (which is a basic occluder image or an enhanced occluder image). Furthermore, for each image pair to be fused, the server 110 can fuse the basic face image and the occluder image in the image pair to be fused based on a randomization mechanism to obtain an occluded face image; the randomization mechanism here is used to control the random deployment of the occluders in the occluder image in the occluded face image, and the occluded face image can be used as a training image for the image restoration model.

[0064] It should be understood that Figure 1 The application scenarios shown are only examples. In actual applications, the training data construction method provided in the embodiments of the present application can also be applied to other scenarios, and the embodiments of the present application do not impose any limitations on this.

[0065] The following is a detailed introduction to the training data construction method provided in the embodiment of the present application through a method example.

[0066] See also Figure 2 , Figure 2 The following is a flow chart of the training data construction method provided in the embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0067] Step 201: Acquire a plurality of basic face images and a plurality of basic occlusion images; the basic face images include unoccluded faces, and the basic occlusion images include occlusions used to occlude faces.

[0068] The basic face image refers to any image including an unobstructed face, and the basic face image will be used as the basis for constructing the obstructed face image later.

[0069] As an example, multiple basic face images can be obtained from a relevant image database, for example, multiple face images can be obtained from an open source database that stores a large number of face images as basic face images. As another example, a screenshot of a video included in a relevant video database can be taken, that is, a video frame including an unobstructed face can be captured from the video as a basic face image. Of course, in actual applications, multiple basic face images can also be obtained by other means, and the embodiments of the present application do not limit the method for obtaining the basic face images.

[0070] It should be noted that the multiple basic face images obtained need to meet pre-set diversity requirements. For example, the faces in the multiple basic face images obtained need to include different races, genders and ages, thereby ensuring that the faces used when constructing the occluded face image are sufficiently diverse.

[0071] A basic occluder image refers to any image that includes an occluder that can be used to occlude a face. The basic occluder image can subsequently be used as the basis for enhanced transformation processing, and can also be used as the basis for constructing an occluded face image. The occluders included in the basic occluder image may include, but are not limited to, daily necessities occluders (such as masks, glasses, hats, headscarves, headwear, headphones, etc.), natural occluders (such as hair, beards, body parts (hands, arms, etc.), light and shadows), clothing occluders (such as clothing accessories (such as masks, etc.), specific clothing (such as large collar clothing, etc.)), sports equipment occluders, painted makeup occluders, handheld object occluders, transparent occluders (such as transparent glass), moving objects or people in the scene, etc.

[0072] As an example, an image including an occluder can be collected as the above-mentioned basic occluder image. For example, a physical sample of an occluder (such as a hat, glasses, etc.) can be obtained from real life, and the obtained physical sample can be photographed, and then the high-definition image obtained by the photographing can be used as the basic occluder image. For another example, an image search engine can be directly used to search and download an image including an occluder as the basic occluder image.

[0073] As another example, a 3D occluder model can be constructed by 3D modeling software, and an image including the 3D occluder model can be rendered to obtain a basic occluder image. For example, 3D modeling software can be used to construct a 3D occluder model corresponding to a complex and difficult-to-obtain occluder, or 3D modeling software can be used to create a custom 3D occluder model, and then the 3D occluder model constructed by the 3D modeling software is rendered to obtain a 2D image for reflecting the 3D occluder model, and the 2D image is used as the basic occluder image.

[0074] It should be understood that in practical applications, the above two methods can be used simultaneously to obtain a variety of basic occluder images, or only one of the methods can be used to obtain the basic occluder images to reduce the acquisition cost of the basic occluder images. Of course, in practical applications, other methods can also be used to obtain the basic occluder images, and the embodiments of this application do not limit the method of obtaining the basic occluder images.

[0075] Step 202: performing enhancement transformation processing on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images.

[0076] After obtaining multiple basic occluder images, the occluder in each basic occluder image can be enhanced and transformed to obtain multiple variants corresponding to the occluder, and an enhanced occluder image including the variant will be generated based on each variant corresponding to the occluder. That is, the enhanced occluder image is an image including the variant corresponding to the occluder in the basic occluder image, and the variant corresponding to the occluder is obtained by performing the enhanced transformation process on the occluder.

[0077] By performing multiple enhancement transformation processes on the occluders in each basic occluder image, a large number of enhanced occluder images can be quickly generated, and each enhanced occluder image includes rich occluders with different appearances, which provides a rich occluder data source for the subsequent generation of occluded face images.

[0078] In a possible implementation, the enhanced transformation processing performed on the occluder in the basic occlusion image may involve at least one transformation method of translation transformation, scaling transformation, rotation transformation, projection transformation, shape transformation, color transformation, and texture transformation.

[0079] Among them, the translation transformation is used to change the position of the occluder in the basic occlusion image by translation. The scaling transformation is used to enlarge or reduce the occluder in the basic occlusion image to change the size of the occluder. The rotation transformation is used to rotate the occluder in the basic occlusion image to change the angle of the occluder in the basic occlusion image. The projection transformation is used to transform the position of the occluder in the basic occlusion image through the projection matrix. The shape transformation is used to deform the occluder in the basic occlusion image to change the shape of the occluder. The color transformation is used to change the color of the occluder in the basic occlusion image. The texture transformation is used to change the texture of the occluder in the basic occlusion image.

[0080] It should be understood that in practical applications, any one or more of the above transformations may be performed on the occluders in the basic occluder image to increase the diversity of variants corresponding to the occluders.

[0081] In a possible implementation, the above step 202 may specifically include the following steps 2021 to 2023 (not shown in the figure):

[0082] Step 2021: Generate multiple groups of different random transformation factors; the random transformation factors are used to determine the transformation method and degree of the enhanced transformation processing.

[0083] When performing enhanced transformation processing on an occluder in a basic occluder image, multiple sets of different random transformation factors may be generated to obtain occluder variants of different forms through the multiple sets of different random transformation factors.

[0084] The random transformation factor is a parameter used to determine the degree of transformation of the transformation method used. The parameter can be randomly generated or selected according to a distribution algorithm to improve the randomness of the enhanced transformation processing and achieve various occluder changes. For example, assuming that the enhanced transformation processing of the occluder in the basic occluder image includes translation transformation, scaling transformation, rotation transformation and projection transformation, the transformation parameter corresponding to the translation transformation can be (t x ,t y ), where t x Used to indicate the translation distance in the x-axis direction, t y It is used to indicate the translation distance in the y-axis direction. The transformation parameter corresponding to the scaling transformation can be (s x ,s y ), where s x Used to indicate the scaling factor in the x-axis direction, s y It is used to indicate the scaling ratio in the y-axis direction. The transformation parameter corresponding to the rotation transformation can be θ, which is used to indicate the rotation angle of the occluder. The transformation parameter corresponding to the projection transformation can be (p 1 ,p 2), where p 1 and p 2 Used to indicate the two projection matrices involved in the projection transformation.

[0085] Step 2022: For each group of the random transformation factors, construct a corresponding composite affine matrix.

[0086] After the random transformation factors are generated, for each group of random transformation factors generated, a corresponding composite affine matrix can be generated according to each transformation parameter included in the group of random transformation factors. The composite affine matrix can be used to implement enhanced transformation processing of the corresponding random transformation factors.

[0087] Exemplarily, the enhanced transformation processing of the occluder in the basic occluder image includes translation transformation, scaling transformation, rotation transformation and projection transformation, wherein the transformation parameter corresponding to the translation transformation is (t x ,t y ), the transformation parameter corresponding to the scaling transformation is (s x ,s y ), the transformation parameter corresponding to the rotation transformation is θ, and the transformation parameter corresponding to the projection transformation is (p 1 ,p 2 ) as an example, the constructed composite affine matrix M can be specifically shown in the following formula 1:

[0088]

[0089] Step 2023: Based on the composite affine matrices corresponding to each group of the random transformation factors, the enhanced transformation processing is performed on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images.

[0090] After constructing the composite affine matrix corresponding to each group of random transformation factors, the composite affine matrix can be used to perform enhanced transformation processing on the occluder in the basic occluder image to obtain a variant corresponding to the occluder, thereby obtaining an enhanced occluder image including the variant corresponding to the occluder.

[0091] Exemplarily, the enhanced transformation processing of the occluder in the basic occluder image includes translation transformation, scaling transformation, rotation transformation and projection transformation, wherein the transformation parameter corresponding to the translation transformation is (t x ,t y ), the transformation parameter corresponding to the scaling transformation is (s x ,s y ), the transformation parameter corresponding to the rotation transformation is θ, and the transformation parameter corresponding to the projection transformation is (p 1 ,p 2) as an example, the composite affine matrix representing the above-mentioned enhanced transformation processing is used to perform enhanced transformation processing on the occluder as shown in the following formulas (2) and (3):

[0092] P′=M·P (2)

[0093]

[0094] Among them, P' represents the new state of the occluder obtained by the enhanced transformation, and P represents the original state of the occluder in the basic occluder image. (x, y) is the original coordinate of the occluder in the basic occluder image, and (x', y') is the coordinate after the enhanced transformation. It should be noted that the above composite affine matrix combines translation, scaling and rotation in one matrix. The projection transformation is usually more complicated and requires a separate projection matrix to implement. However, in order to simplify the processing, the parameters of these transformations are integrated and merged here.

[0095] Step 203: Based on the multiple basic face images, the multiple basic occlusion images and the multiple enhanced occlusion images, construct multiple image pairs to be fused; each of the image pairs to be fused includes one of the basic face images and one of the occlusion images, and the occlusion image is the basic occlusion image or the enhanced occlusion image.

[0096] Next, each basic face image and each occluder image (including basic occluder images and enhanced occluder images) can be traversed and combined in pairs to construct multiple image pairs to be fused, thereby achieving traversal and combination of different faces and different occluders.

[0097] It should be understood that each image pair to be fused includes a basic face image and an occlusion image, and different image pairs to be fused include at least one different image, that is, different image pairs to be fused need to include different basic face images and / or different occlusion images.

[0098] Step 204: For each of the image pairs to be fused, based on a randomization mechanism, the basic face image and the occlusion image in the image pair to be fused are fused to obtain an occluded face image; the randomization mechanism is used to control the random deployment of the occluders in the occlusion image in the occluded face image, and the occluded face image is used as a training image for the image restoration model.

[0099] For each pair of images to be fused constructed in step 203, a pre-developed image superposition script may be run to superimpose the basic face image and the occlusion image included in the pair of images to be fused based on a randomization mechanism, thereby obtaining an occlusion face image. The occlusion face image is an image that includes both a face and an occlusion.

[0100] It should be noted that the image overlay script is a pre-developed script used to combine the face image and the occlusion image into one image. The image overlay script can realize the automatic overlay and fusion of the face image and the occlusion image, thereby improving the efficiency of generating the occluded face image. The randomization mechanism is a mechanism used to ensure that the angle, position and size of the occlusion in the image generated by the image overlay script are random. The randomization mechanism can make the deployment position of the occlusion in each image have a high degree of randomness, thereby being more conducive to achieving coverage of various face occlusion situations and improving the generalization of the generated face occlusion image.

[0101] The occluded face images generated by the above method can be used as training images for the image restoration model later. The image restoration model refers to a neural network model used to restore low-quality images to corresponding high-quality images. For example, a low-definition image can be restored to a corresponding high-definition image. By using the occluded face images generated by the embodiments of the present application as training images for the image restoration model, the image restoration model can fully learn the restoration methods corresponding to various occlusion situations based on these occluded face images during the training process, thereby acquiring the ability to effectively repair various face occluded images.

[0102] In a possible implementation, the above step 204 may specifically include the following steps 2041 to 2043 (not shown in the figure):

[0103] Step 2041: performing spatial transformation processing on the occluder image to obtain an aligned occluder image; the aligned occluder image is aligned with the basic face image.

[0104] Before fusing the basic face image and the occlusion image, the image overlay script can first use the basic face image in the image pair to be fused as a reference, and perform spatial transformation on the occlusion image in the image pair to be fused through affine transformation or other suitable transformation methods to obtain an aligned occlusion image that is aligned with the basic face image, thereby facilitating the subsequent fusion of the two aligned images and improving the effect of the fused occlusion face image.

[0105] The specific spatial transformation process can refer to the following formula (4):

[0106]

[0107] Where (x, y) is the coordinate of the occluder before the spatial transformation, and (x', y') is the coordinate of the occluder after the spatial transformation. The spatial transformation matrix includes the translation parameter t x and t y , rotation parameter a 11 、a12 、a 21 and a 22 (It is determined according to the rotation angle θ). It should be understood that if other complex transformations are involved in the spatial transformation process, the above-mentioned spatial transformation matrix can be adjusted accordingly.

[0108] Step 2042: Based on the randomization mechanism, perform random transformation processing on the occluders in the aligned occluder image to obtain a target occluder image.

[0109] Then, the occluders in the aligned occluder images obtained through the spatial transformation process are randomly transformed, such as randomly adjusting the angle, position and size of the occluders, thereby obtaining a plurality of target occluder images including occluders deployed at different positions.

[0110] Step 2043: Using transparency blending technology, overlay the occluder in the target occluder image onto the basic face image with target transparency to obtain the occluded face image.

[0111] Then, alpha blending is used to cover the occluder with the target transparency in the target occluder image on the basis of the basic face image, so as to achieve the fusion of the face in the basic face image and the occluder in the target occluder image, and obtain the occluded face image. It should be understood that when multiple target occluder images are generated in step 2042, different target occluder images are respectively covered on the basic face image, and multiple occluded face images will be obtained accordingly.

[0112] As an example, the occluded face image can be generated using transparency blending techniques in the following way:

[0113] For each pixel position in the initial image, determine a target pixel value of the pixel position in the initial image according to a pixel value of a pixel at the pixel position in the base face image, a pixel value at the pixel position in the target occluder image, and a target transparency;

[0114] The blocked face image is generated according to the target pixel value of each pixel position in the initial image.

[0115] Among them, the initial image refers to a blank image that does not yet include any content. The embodiment of the present application needs to determine the pixel value of each pixel position in the initial image based on the pixel value of each pixel position in the basic face image, the pixel value of each pixel position in the target occluder image, and the target transparency used by the transparency blending technology, so as to obtain the occluded face image to be generated.

[0116] Specifically, for a pixel position (x, y) in the initial image, the target pixel value corresponding to the pixel position can be determined by the following formula (5):

[0117] I C (x,y)=α(x,y)·I O (x,y)+(1-α(x,y))·I F (x,y)(5)

[0118] Among them, I C (x, y) is the target pixel value corresponding to the pixel position (x, y); α(x, y) is the transparency of the occluder at the pixel position (x, y), and its value range is [0, 1], where 0 represents complete transparency and 1 represents complete opacity; I O (x, y) is the pixel value corresponding to the pixel position (x, y) in the target occluder image; I F (x, y) is the pixel value corresponding to the pixel position (x, y) in the base face image.

[0119] In addition, in practical applications, the image overlay script can also consider the geometric structure of the face and the physical properties of the occluders (for example, glasses should be located near the eyes, etc.), use machine learning technology to predict the appropriate position of the occluders, and deploy the corresponding occluders here.

[0120] Exemplarily, the occluder position prediction model can be used to determine the occluder deployment position based on the target occluder image and the basic face image, and the occluder deployment position is used to indicate the deployment position of the occluder in the target occluder image in the basic face image; then, the transparency blending technology is used to cover the occluder in the target occluder image with the target transparency to the occluder deployment position in the basic face image to obtain the occluded face image.

[0121] Specifically, an occluder position prediction model can be pre-trained, and the occluder position prediction model is used to combine the face image and the occluder image to predict the deployment position of the occluder in the occluder image in the face image; the occluder position prediction model can specifically be a neural network model based on a convolutional structure. When training the occluder position prediction model, several training samples can be obtained first, each of which includes a training face image, a training occluder image and annotated occluder position information, and the annotated occluder position information is used to indicate the standard deployment position of the occluder in the training occluder image in the training face image; then, the training face image and the training occluder image in the training sample are input into the occluder position prediction model, and the occluder position prediction model can output the predicted occluder position after processing the training face image and the training occluder image, and specifically, the predicted occluder position can be marked in the input training face image; further, a loss function can be constructed based on the difference between the predicted occluder position and the annotated occluder position information in the training sample, and the occluder position prediction model can be trained based on the loss function. In this way, the above process is iteratively performed based on different training samples until the training of the occluder position prediction model is completed.

[0122] Furthermore, the trained occluder position prediction model can be put into practical application. Specifically, the target occluder image and the basic face image can be input into the occluder position prediction model, and the occluder position prediction model can output the occluder deployment position after processing the target occluder image and the basic face image, for example, the basic face image marked with the occluder deployment position can be output, and the occluder deployment position indicates the appropriate deployment position of the occluder in the target occluder image in the basic face image.

[0123] Furthermore, according to the above method, the transparency blending technology can be used to cover the occluder in the target occluder image with the target transparency to the occluder deployment position in the basic face image, thereby obtaining the occluded face image. Therefore, through the above processing, the rationality of the generated occluded face image can be guaranteed, that is, the relationship between the occluder and the face in the occluded face image can be guaranteed to be consistent with the actual situation.

[0124] Optionally, after generating the blocked face image, the method provided in the embodiment of the present application may further include:

[0125] The occluded face image is subjected to image post-processing to obtain a training image for the image restoration model; the image post-processing is used to optimize the occlusion effect between the face and the occluder in the occluded face image.

[0126] That is, after obtaining the occluded face image, the occluded face image can also be subjected to image post-processing to optimize the image quality of the occluded face image, thereby using the image with higher image quality as a training image for the image restoration model.

[0127] As an example, the above-mentioned image post-processing may include at least one of the following implementations:

[0128] The synthetic boundary of the occluder in the occluded face image is feathered. That is, the boundary of the occluder in the occluded face image is feathered, thereby removing the unnatural image synthesis boundary, making the fusion effect of the occluder and the face image more natural.

[0129] According to the lighting conditions of the basic face image, the display parameters corresponding to the occluder in the occluder image are adjusted. For example, according to the lighting conditions of the basic face image, the brightness and contrast corresponding to the occluder in the occluder image are adjusted so that the occluder and the face are in harmony, making the synthesized occluded face image more natural.

[0130] Through color correction, the color of the occluder in the occluder image is made to match the color of the basic face image. For example, through the color correction algorithm, the color of the occluder in the occluder image is adjusted so that the color of the occluder matches the overall color of the basic face image, making the synthesized occluded face image more natural.

[0131] It should be understood that in practical applications, other image post-processing methods may also be used to further optimize the occluded face image, and the embodiments of the present application do not impose any limitation on this.

[0132] Figure 3 This is a diagram of the overall implementation structure of the training data construction method provided in the embodiment of the present application. Figure 3 As shown, the method specifically involves four stages: data collection, occluder variant generation, image fusion, and image post-processing. In the data collection stage, it is used to collect a large number of rich basic face images and basic occluder images; in the occluder variant generation stage, it is used to enhance the occluder in the basic occluder image through an enhanced transformation processing algorithm to generate more occluder variants and obtain the corresponding enhanced occluder image; in the image fusion stage, it is used to combine the basic face image with the basic occluder image or the enhanced occluder image to obtain an occluded face image including the occluded face; in the image post-processing stage, it is used to optimize the generated occluded face image to improve the naturalness and realism of the occluded face image.

[0133] The training data construction method provided in the embodiment of the present application includes: obtaining multiple basic face images including unobstructed faces, and obtaining multiple basic occluder images including occluders for obstructing faces. Then, the occluders in the basic occluder images are enhanced and transformed to obtain multiple enhanced occluder images; by performing enhanced transformation processing on the occluders in the basic occluder images, it is possible to enhance and generate richer and more diverse occluders based on limited basic occluder images, thereby providing abundant available occluders for the subsequent generation of occluded face images. Then, based on the obtained basic face images, the basic occluder images, and the generated enhanced occluder images, multiple pairs of images to be fused are constructed, each pair of images to be fused includes a basic face image and an occluder image (basic occluder image or enhanced occluder image), thereby constructing different combinations of faces and occluders. Then, for each pair of images to be fused, based on a randomization mechanism, the basic face image and the occluder image in the pair of images to be fused are fused, thereby obtaining an occluded face image including both the face and the occluder; by using a randomization mechanism in the fusion process, the occluders in the occluded image can be controlled to be randomly deployed in the occluded face image to be generated, thereby improving the randomness of the occluder layout, which is conducive to simulating various possible occlusion situations and improving the generalization of the generated occluded face image. Each occluded face image generated by the above method can be used as a training image for the image restoration model. Since the generated occluded face images are relatively diverse (including the diversity of occluders and the diversity of occluder deployment positions), they can basically cover various face occlusion situations. Therefore, using these occluded face images to train the image restoration model can enable the image restoration model to fully learn the restoration methods for various occlusion situations. Accordingly, the image restoration model can be put into practical applications, and images with occluded faces can be more reliably restored to obtain corresponding high-quality images.

[0134] After obtaining the training image of the image restoration model by the above method, the training image can be used to train the image restoration model. Specifically, the annotated image corresponding to the training image can be determined first, and then the training image and its corresponding annotated image can be used to form a training sample. Usually, the training image is an image with low image quality, such as an image with low clarity, and the annotated image corresponding to the training image is an image with the same content as the training image, but the annotated object has a higher image quality than the training image; illustratively, the annotated image can be obtained by manually repairing the training image.

[0135] Furthermore, an image restoration model can be trained based on the above training samples, and the image restoration model is used to restore the input low-quality image to a corresponding high-quality image. In the embodiment of the present application, since the images in the training samples are all face images including occluders, the image restoration model is more suitable for restoring low-quality occluded face images to corresponding high-quality images; the image restoration model can specifically be a neural network model based on a convolutional structure, such as a diffusion model.

[0136] When training the image restoration model, the training image in the training sample can be input into the image restoration model, and the image restoration model can output the corresponding predicted restoration image by processing the training image. Furthermore, a loss function can be constructed based on the difference between the predicted restoration image and the annotated image corresponding to the training image, and the model parameters of the image restoration model can be adjusted based on the loss function, thereby realizing the training of the image restoration model. Thus, using different training samples, the above-mentioned training process for the image restoration model is iteratively executed until the image restoration model meets the training end condition. The image restoration model that meets the training end condition can be put into practical application to repair low-quality face occluded images.

[0137] Due to the above Figure 2 The occluded face images generated by the illustrated embodiment are relatively diverse (including the diversity of occluded objects and the diversity of occluded object deployment positions), and can basically cover various face occlusion situations. Therefore, using these occluded face images to train an image restoration model can enable the image restoration model to fully learn the restoration methods for various occlusion situations. Accordingly, putting the image restoration model into practical applications can also more reliably restore images with occluded faces to obtain corresponding high-quality images.

[0138] The present application also provides a training data construction device, see Figure 4 , Figure 4 A schematic diagram of the structure of the training data construction device provided in the embodiment of the present application, such as Figure 4 As shown, the device comprises:

[0139] The acquisition module 401 is used to acquire a plurality of basic face images and a plurality of basic occlusion images; the basic face images include unoccluded faces, and the basic occlusion images include occlusions used to occlude faces;

[0140] An enhancement module 402 is used to perform enhancement transformation processing on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images;

[0141] A combining module 403 is used to construct a plurality of image pairs to be fused based on the plurality of basic face images, the plurality of basic occluder images and the plurality of enhanced occluder images; each of the image pairs to be fused includes one of the basic face images and one of the occluder images, and the occluder image is the basic occluder image or the enhanced occluder image;

[0142] The synthesis module 404 is used to fuse the basic face image and the occlusion image in each pair of images to be fused based on a randomization mechanism to obtain an occluded face image; the randomization mechanism is used to control the random deployment of the occluders in the occlusion image in the occluded face image, and the occluded face image is used as a training image for the image restoration model.

[0143] Optionally, the device further comprises:

[0144] A sample determination module, used to determine an annotated image corresponding to the training image, and use the training image and the annotated image to form a training sample; the image quality of the annotated image is higher than the image quality of the training image;

[0145] A sample processing module, used for inputting the training image in the training sample into the image restoration model to obtain a predicted restoration image output by the image restoration model; the image restoration model is used for restoring the input low-quality image into a corresponding high-quality image;

[0146] The model training module is used to train the image restoration model based on the difference between the predicted restoration image and the labeled image in the training sample.

[0147] Optionally, the enhancement module 402 is specifically used for:

[0148] Generate a plurality of different sets of random transformation factors; the random transformation factors are used to determine the transformation mode and degree of the enhanced transformation process;

[0149] For each group of the random transformation factors, construct a corresponding composite affine matrix;

[0150] Based on the composite affine matrices corresponding to each group of the random transformation factors, the enhanced transformation process is performed on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images.

[0151] Optionally, the transformation method of the enhanced transformation processing includes at least one of translation transformation, scaling transformation, rotation transformation, projection transformation, shape transformation, color transformation, and texture transformation.

[0152] Optionally, the synthesis module 404 is specifically used for:

[0153] Performing spatial transformation processing on the occluder image to obtain an aligned occluder image; the aligned occluder image is aligned with the basic face image;

[0154] Based on the randomization mechanism, randomly transform the occluders in the aligned occluder image to obtain a target occluder image;

[0155] The transparency blending technology is adopted to overlay the occluder in the target occluder image onto the basic face image with the target transparency, so as to obtain the occluded face image.

[0156] Optionally, the synthesis module 404 is specifically used for:

[0157] For each pixel position in the initial image, determine a target pixel value of the pixel position in the initial image according to a pixel value of a pixel at the pixel position in the basic face image, a pixel value at the pixel position in the target occluder image, and the target transparency;

[0158] The blocked face image is generated according to the target pixel value of each pixel position in the initial image.

[0159] Optionally, the synthesis module 404 is specifically used for:

[0160] Determine the occluder deployment position according to the target occluder image and the basic face image through the occluder position prediction model; the occluder deployment position is used to indicate the deployment position of the occluder in the target occluder image in the basic face image;

[0161] The transparency blending technology is used to cover the occluder in the target occluder image with the target transparency to the occluder deployment position in the basic face image to obtain the occluded face image.

[0162] Optionally, the acquisition module 401 is specifically used for:

[0163] Acquire an image including an occluder as the basic occluder image;

[0164] and / or,

[0165] A three-dimensional obstruction model is constructed by using three-dimensional modeling software, and an image including the three-dimensional obstruction model is rendered to obtain the basic obstruction image.

[0166] Optionally, the device further comprises:

[0167] A post-processing module is used to perform image post-processing on the occluded face image to obtain a training image for the image restoration model; the image post-processing is used to adjust the occlusion effect between the occluder and the face in the occluded face image.

[0168] Optionally, the image post-processing includes at least one of the following:

[0169] Performing feathering processing on the synthetic boundary of the occluder in the occluded face image;

[0170] According to the illumination condition of the basic face image, adjusting the display parameters corresponding to the occluder in the occluder image;

[0171] Through color correction, the color of the occluder in the occluder image is matched with the color of the basic face image.

[0172] An embodiment of the present application also provides an electronic device, which includes a processor and a memory; the memory is used to store a computer program; the processor is used to execute the training data construction method described in the above method embodiment according to the computer program.

[0173] An embodiment of the present application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a computer device, the training data construction method described in the above method embodiment is implemented.

[0174] An embodiment of the present application also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the training data construction method described in the above method embodiment is implemented.

[0175] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0176] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store computer programs.

[0180] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0181] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A training data construction method, characterized in that: The method comprises: Acquire a plurality of basic face images and a plurality of basic occluder images; the basic face images include unoccluded faces, and the basic occluder images include occluders used to occlude faces; Performing enhancement transformation processing on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images; Based on the multiple basic face images, the multiple basic occluder images and the multiple enhanced occluder images, construct multiple image pairs to be fused; each of the image pairs to be fused includes one of the basic face images and one of the occluder images, and the occluder image is the basic occluder image or the enhanced occluder image; For each pair of images to be fused, based on a randomization mechanism, the basic face image and the occlusion image in the pair of images to be fused are fused to obtain an occluded face image; the randomization mechanism is used to control the random deployment of the occluders in the occlusion image in the occluded face image, and the occluded face image is used as a training image for an image restoration model.

2. The method according to claim 1, characterized in that The method comprises: Determine an annotated image corresponding to the training image, and use the training image and the annotated image to form a training sample; the image quality of the annotated image is higher than the image quality of the training image; Inputting the training image in the training sample into the image restoration model to obtain a predicted restoration image output by the image restoration model; the image restoration model is used to restore the input low-quality image to a corresponding high-quality image; The image restoration model is trained based on the difference between the predicted restoration image and the labeled image in the training sample.

3. The method according to claim 1, characterized in that The enhancing and transforming processing is performed on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images, including: Generate a plurality of different sets of random transformation factors; the random transformation factors are used to determine the transformation mode and degree of the enhanced transformation process; For each group of the random transformation factors, construct a corresponding composite affine matrix; Based on the composite affine matrices corresponding to each group of the random transformation factors, the enhanced transformation process is performed on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images.

4. The method according to claim 1 or 3, characterized in that: The transformation method of the enhanced transformation processing includes at least one of translation transformation, scaling transformation, rotation transformation, projection transformation, shape transformation, color transformation, and texture transformation.

5. The method according to claim 1, characterized in that The step of fusing the basic face image and the occlusion image in the pair of images to be fused based on a randomization mechanism to obtain an occlusion face image includes: Performing spatial transformation processing on the occluder image to obtain an aligned occluder image; the aligned occluder image is aligned with the basic face image; Based on the randomization mechanism, randomly transform the occluders in the aligned occluder image to obtain a target occluder image; The transparency blending technology is adopted to overlay the occluder in the target occluder image onto the basic face image with the target transparency, so as to obtain the occluded face image.

6. The method according to claim 5, characterized in that The method of using a transparency blending technique to cover the occluder in the target occluder image with a target transparency onto the basic face image to obtain the occluded face image includes: For each pixel position in the initial image, determine a target pixel value of the pixel position in the initial image according to a pixel value of a pixel at the pixel position in the basic face image, a pixel value at the pixel position in the target occluder image, and the target transparency; The blocked face image is generated according to the target pixel value of each pixel position in the initial image.

7. The method according to claim 5, characterized in that The method of using a transparency blending technique to cover the occluder in the target occluder image with a target transparency onto the basic face image to obtain the occluded face image includes: Determine the occluder deployment position according to the target occluder image and the basic face image through the occluder position prediction model; the occluder deployment position is used to indicate the deployment position of the occluder in the target occluder image in the basic face image; The transparency blending technology is used to cover the occluder in the target occluder image with the target transparency to the occluder deployment position in the basic face image to obtain the occluded face image.

8. The method according to claim 1, characterized in that The obtaining of a plurality of basic occlusion images comprises: Acquire an image including an occluder as the basic occluder image; and / or, A three-dimensional obstruction model is constructed by using three-dimensional modeling software, and an image including the three-dimensional obstruction model is rendered to obtain the basic obstruction image.

9. The method according to claim 1, characterized in that: The method further comprises: The occluded face image is subjected to image post-processing to obtain a training image for the image restoration model; the image post-processing is used to adjust the occlusion effect between the occluder and the face in the occluded face image.

10. The method according to claim 9, characterized in that The image post-processing includes at least one of the following: Performing feathering processing on the synthetic boundary of the occluder in the occluded face image; According to the illumination condition of the basic face image, adjusting the display parameters corresponding to the occluder in the occluder image; Through color correction, the color of the occluder in the occluder image is matched with the color of the basic face image.

11. A training data construction device, characterized in that: The device comprises: An acquisition module, used to acquire a plurality of basic face images and a plurality of basic occluding object images; the basic face images include unobstructed faces, and the basic occluding object images include occluding objects used to obstruct faces; An enhancement module, used for performing enhancement transformation processing on the occluders in the basic occluder image to obtain a plurality of enhanced occluder images; A combining module, configured to construct a plurality of image pairs to be fused based on the plurality of basic face images, the plurality of basic occluder images and the plurality of enhanced occluder images; each of the image pairs to be fused includes one of the basic face images and one of the occluder images, and the occluder image is the basic occluder image or the enhanced occluder image; A synthesis module is used to fuse the basic face image and the occlusion image in each pair of images to be fused based on a randomization mechanism to obtain an occluded face image; the randomization mechanism is used to control the random deployment of the occluders in the occlusion image in the occluded face image, and the occluded face image is used as a training image for the image restoration model.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The memory is used to store computer programs; The processor is used to execute the training data construction method according to any one of claims 1 to 10 according to the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a computer device, the training data construction method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the training data construction method according to any one of claims 1 to 10 is implemented.

Images