Image processing method, device, computer equipment and storage medium

Abstract

The present invention discloses an image processing method, device, computer equipment and storage medium. The method comprises: acquiring a first image; analyzing the first image to determine a second image corresponding to a three-dimensional model with a similar posture to a target object and annotation information corresponding to the second image; performing feature extraction processing on the target object to determine a first feature map corresponding to the target object; performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image to determine a second feature map corresponding to the three-dimensional model in the second image; performing image completion processing on the incomplete target object according to the second image, the first feature map and the second feature map to generate a target image corresponding to the complete target object. The method can not only complete the incomplete target object, but also make the features of the complete target object in the target image consistent with the features of the incomplete target object in the first image, and has a good image completion effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, computer equipment and storage medium. Background Art

[0002] In the field of image processing technology, it is usually necessary to train network models with certain image processing capabilities in different application scenarios, for example, to train network models that can identify target characters in multi-character scenes. In the network training process, images with diversity are usually required as data sets for network model training. In the prior art, the image area corresponding to the target object of the image in the data set is usually copied and pasted to other positions of the image or another image, so as to generate new images for the data set and increase the diversity of the data set. However, in practical applications, especially in scenes with dense targets, the target objects that need to be copied and pasted are often partially blocked, resulting in incomplete target objects that are copied and pasted. If the image area corresponding to the incomplete target object is directly copied and pasted to generate an image, an unreasonable image may be generated, affecting the training of the network model. Therefore, how to complete the image containing the incomplete target object and generate an image containing the complete target object to improve the diversity of the images in the data set is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present invention provide an image processing method, apparatus, computer device and storage medium to solve the problem of how to complete an image containing an incomplete target object and generate an image containing a complete target object.

[0004] An image processing method, comprising: Acquire a first image, wherein the first image includes an incomplete target object; Analyze the first image to determine a second image corresponding to a three-dimensional model similar to the posture feature of the target object and annotation information corresponding to the second image; Performing feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object; performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, and determining a second feature map corresponding to the three-dimensional model in the second image; According to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map, the incomplete target object is image completed to obtain a target image, wherein the target image contains the complete target object.

[0005] Preferably, the first image includes two-dimensional key points corresponding to the target object; The analyzing the first image to determine a second image corresponding to a three-dimensional model having a posture feature similar to the target object and annotation information corresponding to the second image includes: Inputting the two-dimensional key points corresponding to the target object into a pre-trained target posture feature extraction model to obtain the target posture features corresponding to the target object; Based on the target posture feature, query a pre-established posture feature database to obtain a similar posture feature corresponding to the three-dimensional model with the highest similarity to the target posture feature, and determine the third image corresponding to the similar posture feature as the second image; A pre-established rendering database is queried based on the second image to determine annotation information corresponding to the second image.

[0006] Preferably, before acquiring the first image, the image processing method further includes: Based on preset configuration parameters, control the 3D rendering engine to render the multiple 3D models to obtain multiple third images and annotation information corresponding to each of the third images, wherein the annotation information includes a mask image and a depth image; Determining the rendering database based on the third image and the annotation information corresponding to the third image; Using a pre-trained target posture feature extraction model to extract features of the three-dimensional models in the plurality of third images in the rendering database, and determining initial posture features corresponding to the three-dimensional models in the plurality of third images; The posture feature database is determined based on the multiple third images and the initial posture feature corresponding to the three-dimensional model in each third image.

[0007] Preferably, before acquiring the first image, the image processing method further includes: Acquire three-dimensional training data, the three-dimensional training data including a first model, a second model, a first original posture corresponding to the first model, a noise posture corresponding to the first model, and a second original posture corresponding to the second model; Extract key points from the three-dimensional training data to determine two-dimensional key points corresponding to the three-dimensional training data, where the two-dimensional key points include a first key point corresponding to the first original posture, a second key point corresponding to the noise posture, and a third key point corresponding to the second original posture; Inputting the two-dimensional key points corresponding to the three-dimensional training data into the original posture feature extraction model to determine the posture features to be trained, wherein the posture features to be trained include a first posture feature corresponding to the first key point, a second posture feature corresponding to the second key point, and a third posture feature corresponding to the third key point; Based on the posture features to be trained, determining a triplet loss function value corresponding to the original posture feature extraction model; If the triplet loss function value satisfies a preset training stop condition, the original posture feature extraction model is determined as the target posture feature extraction model.

[0008] Preferably, performing feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object includes: Performing segmentation processing on the first image to determine a target segmentation image corresponding to the target object, performing target object extraction processing on the first image using the target segmentation image to determine a style image corresponding to the target object, performing feature extraction on the style image using a style feature generation network to determine a style feature map corresponding to the target object; Key points of the target object in the first image are extracted to obtain a posture image corresponding to the target object, and a posture feature generation network is used to extract features from the posture image to determine a posture feature map corresponding to the target object.

[0009] Preferably, the annotation information corresponding to the second image includes a depth image; The performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image to determine a second feature map corresponding to the three-dimensional model in the second image includes: Performing edge extraction on the second image to determine an edge image corresponding to the second image, performing edge feature extraction on the edge image using an edge feature extraction network to determine an edge feature map corresponding to the three-dimensional model in the second image; A deep feature extraction network is used to extract deep features from the depth image to determine a depth feature map corresponding to the three-dimensional model in the second image.

[0010] Preferably, the annotation information corresponding to the second image includes a mask image, the first feature map includes a style feature map and a posture feature map; the second feature map includes a depth feature map and an edge feature map; The performing image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map, and the second feature map to obtain the target image includes: Inputting the second image, the style feature map, the posture feature map, the depth feature map, and the edge feature map into a preset image generation model to generate a fourth image; The mask image is used to perform target object extraction processing on the fourth image to generate a target image corresponding to the complete target object.

[0011] An image processing device, comprising: A first image acquisition module, used to acquire a first image, wherein the first image includes an incomplete target object; A second image determination module, configured to analyze the first image to determine a second image corresponding to a three-dimensional model having a posture feature similar to the target object and annotation information corresponding to the second image; A first feature map determining module, used to perform feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object; A second feature map determining module, configured to perform feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, and determine a second feature map corresponding to the three-dimensional model in the second image; A target image generation module is used to perform image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map to obtain a target image, wherein the target image includes a complete target object.

[0012] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image processing method when executing the computer program.

[0013] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method is implemented.

[0014] The above-mentioned image processing method, device, computer equipment and storage medium obtain a first feature map corresponding to a first image, extract features of a three-dimensional model similar to the posture features of the target object in the second image and the annotation information corresponding to the second image, and obtain a second feature map. According to the second image containing a complete three-dimensional model with the highest similarity to the posture features of the target object in the first image, the annotation information corresponding to the second image, the second feature map containing the features of the complete three-dimensional model, and the first feature map containing the features of the incomplete target object, a target image containing a complete target object is generated, which can not only complete the incomplete target object, but also accurately align the features of the completed complete target object with the features of the incomplete target object in the first image, thereby realizing accurate completion of the incomplete target object. The method can not only complete the incomplete target object, but also make the features of the complete target object in the target image consistent with the features of the incomplete target object in the first image, and has a good image completion effect. The method can be applied to images corresponding to a variety of scenes, and image completion is performed on target objects of different categories, and has a wide range of application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0016] Figure 1 is a flow chart of an image processing method in one embodiment of the present invention; Figure 2 is another flow chart of an image processing method according to an embodiment of the present invention; Figure 3 is another flow chart of an image processing method according to an embodiment of the present invention; Figure 4 is another flow chart of an image processing method according to an embodiment of the present invention; Figure 5 is another flow chart of an image processing method according to an embodiment of the present invention; Figure 6 is another flow chart of an image processing method according to an embodiment of the present invention; Figure 7 is another flow chart of an image processing method according to an embodiment of the present invention; Figure 8 is a schematic diagram of an image processing device in one embodiment of the present invention; Fig. 9 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] An image processing method provided by an embodiment of the present invention is used to complete an image containing an incomplete target object and generate an image containing a complete target object.

[0019] In one embodiment, if Figure 1 As shown, an image processing method is provided, which is applied in Fig. 9 The server in the example is used as an example to illustrate the process, including the following steps: S101: Acquire a first image, where the first image includes an incomplete target object; S102: Analyze the first image to determine a second image corresponding to a three-dimensional model similar to a posture feature of the target object and annotation information corresponding to the second image; S103: performing feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object; S104: performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, and determining a second feature map corresponding to the three-dimensional model in the second image; S105: Performing image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map, and the second feature map to obtain a target image, wherein the target object includes a complete target object.

[0020] The first image refers to an image that needs to be completed. The first image includes an incomplete target object. The target object refers to a target that needs to be completed.

[0021] As an example, in step S101, the computer device obtains the first image in the data set that needs to be completed. The first image includes an incomplete target object that needs to be completed. Understandably, if the first image corresponds to a scene in which multiple characters appear, the target object is a character in the first image, and the target object is occluded by other characters in the scene, resulting in an incomplete image area where the target object is located. In this example, a first image including an incomplete target object is obtained to facilitate image completion of the image area corresponding to the target object, and obtain a target image containing a complete target object, so as to enhance the diversity of the data set where the first image is located.

[0022] The second image refers to the image to which the three-dimensional model with the highest similarity in posture features with the target object in the first image belongs. The annotation information refers to the information corresponding to the three-dimensional model in the second image. The three-dimensional model refers to the complete model corresponding to the category to which the target object belongs in the three-dimensional space. For example, if the category to which the target object belongs is a person, the three-dimensional model refers to a complete person model in the three-dimensional space. If the category to which the target object belongs is an animal other than a person, the three-dimensional model refers to a complete animal model in the three-dimensional space. For example, if the target object is a cat, the three-dimensional model is a complete cat model in the three-dimensional scene.

[0023] As an example, in step S102, the computer device obtains the posture features of the target object in the first image and the posture features corresponding to the three-dimensional model in each image in the preset database, performs similarity analysis on the posture features of the target object and the posture features of each three-dimensional model, obtains the posture features of the three-dimensional model in the preset database with a high similarity to the posture features of the target object, determines the image corresponding to the three-dimensional model in the database as the second image, and obtains the annotation information corresponding to the three-dimensional model in the second image. For example, if the target object is a person to be completed, the three-dimensional model is at least one complete person model in the three-dimensional scene, the computer device obtains the first feature corresponding to the incomplete person in the first image, obtains the second feature corresponding to the complete person model in each image in the database, obtains the similarity between the first feature and each second feature, and determines the image in the database corresponding to the second feature with the highest similarity as the second image with the highest similarity to the first image. Among them, the first feature is the posture feature corresponding to the incomplete person in the first image. The second feature is the posture feature corresponding to the complete person model in the image of the system database. After determining the second image, the computer device further obtains the annotation information corresponding to the second image stored in the database. In this example, a second image corresponding to a three-dimensional model with similar posture features to the target object and annotation information corresponding to the second image are determined so as to complete the incomplete target object in the first image according to the second image and the annotation information corresponding to the second image.

[0024] The first feature map refers to a feature image corresponding to the target object.

[0025] As an example, in step S103, the computer device uses a preset feature extraction algorithm to perform feature extraction processing on the image area corresponding to the target object in the first image to determine at least one first feature map corresponding to the target object. Understandably, the first feature map corresponding to the target object is obtained so that the incomplete target object in the first image can be completed according to the first feature map, so that the completed complete target object is accurately aligned with the incomplete target object in the first image.

[0026] The second feature map refers to the second image and the feature image corresponding to the three-dimensional model in the annotation information of the second image.

[0027] As an example, in step S104, the computer device uses a preset feature extraction algorithm to perform feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, respectively, to determine a plurality of second feature maps corresponding to the three-dimensional model in the second image. Understandably, since the three-dimensional model in the second image is a complete model of the same category as the target object, that is, the three-dimensional model in the second image is not incomplete due to occlusion or the like, and the three-dimensional model in the second image has the highest similarity with the posture features of the target object, therefore, feature extraction is performed on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, so as to accurately complete the target object according to the second feature map corresponding to the extracted complete three-dimensional model.

[0028] The image completion process refers to the process of completing the image of an incomplete target object to obtain a target image corresponding to the complete target object. The target image refers to an image containing the complete target object.

[0029] As an example, in step S105, the computer device guides the process of image completion processing for the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map, and generates a target image corresponding to the complete target object. For example, the computer device inputs the second image, the annotation information corresponding to the second image, the first feature map and the second feature map into a pre-set model for image completion, and the model for image completion recognizes and processes the second image, the annotation information corresponding to the second image, the first feature map and the second feature map, performs image completion processing on the incomplete target object, and outputs a target image corresponding to the complete target object. It can be understood that the three-dimensional model in the second image has the highest similarity with the posture feature of the target object in the first image, and the second image contains the complete three-dimensional model, and the second feature map is obtained by extracting features of the complete three-dimensional model in the annotation information corresponding to the second image and the second image, that is, the second feature map contains the features of the complete three-dimensional model, and the first feature map contains the features of the incomplete target object in the first image. In this example, a target image containing a complete target object is generated based on a second image containing a complete three-dimensional model with the highest similarity to the target object posture features in the first image, the annotation information corresponding to the second image, a second feature map containing features of the complete three-dimensional model, and a first feature map containing features of the incomplete target object. This not only makes it possible to complete the incomplete target object, but also enables the completed complete target object to be accurately aligned with the features of the incomplete target object in the first image, thereby achieving accurate completion of the incomplete target object.

[0030] In this embodiment, a first feature map corresponding to a first image is obtained, and a feature extraction is performed on a three-dimensional model similar to the posture feature of the target object in the second image and the annotation information corresponding to the second image to obtain a second feature map. According to the second image containing a complete three-dimensional model with the highest similarity to the posture feature of the target object in the first image, the annotation information corresponding to the second image, the second feature map containing the features of the complete three-dimensional model, and the first feature map containing the features of the incomplete target object, a target image containing a complete target object is generated, which can not only complete the incomplete target object, but also accurately align the features of the completed complete target object with the features of the incomplete target object in the first image, thereby achieving accurate completion of the incomplete target object. This method can not only complete the incomplete target object, but also make the features of the complete target object in the target image consistent with the features of the incomplete target object in the first image, and has a good image completion effect. This method can be applied to images corresponding to a variety of scenes, and image completion is performed on target objects of different categories, and has a wide range of application prospects.

[0031] In one embodiment, the first image includes two-dimensional key points corresponding to the target object.

[0032] The two-dimensional key point refers to the coordinate point corresponding to the posture of the specific target in the two-dimensional plane of the image to which it belongs. The specific target includes but is not limited to the target object in the first image, the three-dimensional model in the second image, and the three-dimensional model in the third image.

[0033] In one embodiment, if Figure 2 As shown, step S102, i.e., analyzing the first image to determine a second image corresponding to a three-dimensional model with a similar posture to the target object and annotation information corresponding to the second image, includes: S201: inputting the two-dimensional key points corresponding to the target object into a pre-trained target posture feature extraction model to obtain the target posture features corresponding to the target object; S202: Based on the target posture feature, query a pre-established posture feature database to obtain a similar posture feature corresponding to the three-dimensional model with the highest similarity to the target posture feature, and determine the third image corresponding to the similar posture feature as the second image; S203: Query a pre-established rendering database based on the second image to determine annotation information corresponding to the second image.

[0034] The target posture feature refers to the posture feature of the target object.

[0035] As an example, in step S201, the computer device obtains the three-dimensional posture key points corresponding to the target object in the three-dimensional space, extracts the coordinates of the three-dimensional posture key points in the two-dimensional plane corresponding to the first image, determines the two-dimensional key points corresponding to each three-dimensional posture key point in the two-dimensional plane, and inputs the two-dimensional key points into the pre-trained target posture feature extraction model. The target posture feature extraction model extracts posture features of the target object in the first image based on the two-dimensional key points to obtain the target posture features corresponding to the target object. Among them, the three-dimensional posture key points refer to the points used to characterize the spatial position of the posture of a specific target in the image to which it belongs. In this example, the target posture feature extraction model is used to extract posture features of the two-dimensional key points of the target object to determine the target posture features of the target object, so as to accurately obtain similar posture features with the highest similarity to the target posture features based on the target posture features.

[0036] Among them, the similar posture feature refers to the initial posture feature in the posture feature database that has the highest similarity with the target posture feature.

[0037] As an example, in step S202, the computer device calculates the similarity between the target posture feature and the initial posture feature corresponding to the three-dimensional model in each third image in the posture feature database, determines the initial posture feature with the highest similarity as the similar posture feature corresponding to the target posture feature, and determines the third image corresponding to the similar posture feature in the posture feature database as the second image. In this example, according to the similarity between the target posture feature corresponding to the target object in the first image and the initial posture feature corresponding to the three-dimensional model in the third image, the similar posture feature with the highest similarity to the posture feature of the target object in the initial posture feature can be accurately determined, and the corresponding second image is determined according to the similar posture feature, so as to complete the image of the target object in the first image according to the second image.

[0038] As an example, in step S203, after determining the second image corresponding to the similar posture feature, the computer device queries the rendering database for the annotation information corresponding to the second image, so as to subsequently perform more accurate image completion of the target object in the first image based on the annotation information corresponding to the second image.

[0039] In this embodiment, a target posture feature extraction model is used to extract posture features of the two-dimensional key points of the target object, and the target posture features of the target object are determined, so that the similar posture features with the highest similarity to the target posture features can be accurately obtained based on the target posture features. According to the similarity between the target posture features corresponding to the target object in the first image and the initial posture features corresponding to the three-dimensional model in the third image, the similar posture features of the three-dimensional model with the highest similarity to the posture features of the target object in the initial posture features can be accurately determined, and the annotation information corresponding to the second image and the second image corresponding to the three-dimensional model is determined based on the similar posture features, so that the target object in the first image can be accurately completed based on the annotation information corresponding to the second image and the second image.

[0040] In another embodiment, if Figure 3 As shown, before step S101, that is, before acquiring the first image, the image processing method further includes: S301: Based on preset configuration parameters, control a 3D rendering engine to render a plurality of 3D models to obtain a plurality of third images and annotation information corresponding to each third image, where the annotation information includes a mask image and a depth image; S302: determining a rendering database based on the third image and the annotation information corresponding to the third image; S303: extracting features of the three-dimensional models in the plurality of third images in the rendering database using the pre-trained target posture feature extraction model, and determining initial posture features corresponding to the three-dimensional models in the plurality of third images; S304: Determine a posture feature database based on the multiple third images and the initial posture features corresponding to the three-dimensional model in each third image.

[0041] Among them, preset configuration parameters refer to pre-configured parameters for rendering, including but not limited to camera intrinsic parameters and camera extrinsic parameters. A three-dimensional rendering engine refers to an engine used for rendering, including but not limited to Unreal, Unity and Blender. The third image refers to an image obtained by photographing the three-dimensional model. The mask image refers to the binary image corresponding to the three-dimensional model in the second image. The mask image only includes the image with a grayscale value of 255 corresponding to the three-dimensional model, which is a white image corresponding to the three-dimensional model. A depth image refers to an image used to characterize the distance between the three-dimensional model and the optical center of the simulation camera in the three-dimensional rendering engine.

[0042] As an example, in step S301, the computer device controls the 3D rendering engine to shoot and render multiple 3D models in the 3D space according to preset configuration parameters, and obtain multiple third images, as well as the mask image, depth image, 3D posture key points, camera intrinsic parameters, camera extrinsic parameters and other annotation information corresponding to each third image. For example, if the 3D model is a character model, the computer device controls the 3D rendering engine to shoot and render multiple character models according to preset configuration parameters such as camera intrinsic parameters, camera extrinsic parameters, light and 3D scene, and obtain multiple third images corresponding to the multiple character models and the mask image, depth image, 3D posture key points, camera intrinsic parameters and camera extrinsic parameters and other annotation information corresponding to each third image.

[0043] The rendering database refers to a database storing the third image and the annotation information corresponding to the third image.

[0044] As an example, in step S302, the computer device stores the third image and the annotation information corresponding to each third image accordingly, and obtains a rendering database including the third image and the annotation information corresponding to the third image. For example, if the third image is an image obtained by photographing a character model, the annotation information includes a mask image, a depth image, three-dimensional posture key points, camera intrinsic parameters and camera extrinsic parameters corresponding to the character model, and the computer device stores the third image and the mask image, depth image, three-dimensional posture key points, camera intrinsic parameters and camera extrinsic parameters corresponding to each third image accordingly to form a rendering database. In this example, a rendering database is determined so as to determine the second image with the highest similarity to the first image according to the rendering database.

[0045] The target posture feature extraction model refers to a model used to extract features of a specific target in an image. The specific target includes but is not limited to a target object in the first image, a three-dimensional model in the second image, and a three-dimensional model in the third image. The initial posture feature refers to the posture feature of the three-dimensional model in the third image.

[0046] As an example, in step S303, the computer device uses a pre-trained target posture feature extraction model to extract features from the three-dimensional models in the plurality of third images in the rendering database, and determines the initial posture features corresponding to the three-dimensional model in each third image. For example, if the three-dimensional model is a human model, the computer device uses a pre-trained target posture feature extraction model for performing posture feature extraction on the human model to extract features from the human model in each third image in the rendering database, and determines the initial posture features corresponding to the human model in each third image.

[0047] The posture feature database refers to a database that stores the third image and the initial posture features corresponding to the third image.

[0048] As an example, in step S304, the computer device stores the initial posture features corresponding to the three-dimensional model in each of the third images in a corresponding manner to form a posture feature database. For example, if the third image is an image corresponding to a character model, the initial posture feature is the posture feature of the character model in the third image. The computer device stores the initial posture features of the character model in a corresponding manner to the third images corresponding to the multiple character models and each of the third images in a corresponding manner to form a posture feature database.

[0049] In this embodiment, a rendering database is determined based on the third image and the annotation information corresponding to the third image, and a posture feature database is determined based on the third image and the initial posture features corresponding to the three-dimensional model in each third image, so that it is feasible to subsequently determine the second image corresponding to the three-dimensional model with the highest posture feature similarity between the target object in the first image based on the rendering database and the posture feature database.

[0050] In another embodiment, if Figure 4 As shown, before step S101, that is, before acquiring the first image, the image processing method further includes: S401: Acquire three-dimensional training data, where the three-dimensional training data includes a first model, a second model, a first original posture corresponding to the first model, a noise posture corresponding to the first model, and a second original posture corresponding to the second model; S402: extract key points from the three-dimensional training data to determine two-dimensional key points corresponding to the three-dimensional training data, where the two-dimensional key points include a first key point corresponding to the first original posture, a second key point corresponding to the noise posture, and a third key point corresponding to the second original posture; S403: Inputting the two-dimensional key points corresponding to the three-dimensional training data into the original posture feature extraction model to determine the posture features to be trained, where the posture features to be trained include a first posture feature corresponding to the first key point, a second posture feature corresponding to the second key point, and a third posture feature corresponding to the third key point; S404: Determine a triplet loss function value corresponding to the original posture feature extraction model based on the posture feature to be trained; S405: If the triplet loss function value satisfies the preset training stop condition, the original posture feature extraction model is determined as the target posture feature extraction model.

[0051] The three-dimensional training data refers to training data composed of three-dimensional models. It can be understood that the three-dimensional models in the three-dimensional scene are processed to construct multiple sets of triplets to form the three-dimensional training data. The triplets are composed of three three-dimensional models in the three-dimensional scene.

[0052] In this embodiment, the three-dimensional training data includes a first model, a second model, a first original posture corresponding to the first model, a noise posture corresponding to the first model, and a second original posture corresponding to the second model. The first model refers to any three-dimensional model in a three-dimensional scene. The second model refers to any three-dimensional model other than the first model in a three-dimensional scene. The first original posture refers to the posture corresponding to the first model. The second original posture refers to the posture corresponding to the second model. The noise posture refers to the posture corresponding to the three-dimensional model obtained after noise processing of the first model.

[0053] As an example, in step S401, the computer device obtains N three-dimensional models in the three-dimensional scene, where N≥2. The computer device determines any one of the N three-dimensional models as the first model, determines the remaining N-1 three-dimensional models as the second model, performs a small noise process on the first model (for example, a small noise addition process), and obtains a noise model corresponding to the first model. The computer device performs posture recognition on the first model, the second model, and the noise model, respectively, and determines the first original posture corresponding to the first model, the second original posture corresponding to the second model, and the noise posture corresponding to the noise model. The computer device determines the noise model corresponding to the first model as a positive sample of the first model, and determines any second model as a negative sample corresponding to the first model. The first model corresponds to N-1 negative samples. The first model, the positive sample, and any negative sample are combined into a triplet. For the same first model, there are N-1 triplet. By analogy, for the N three-dimensional models in the three-dimensional scene, there are N first models corresponding to N (N-1) triples, N (N-1) triplets are determined as three-dimensional training data. It can be understood that since the noise model is obtained by performing a slight noise processing on the first model, and since the noise processing is very slight, and the three-dimensional model in the first model and the three-dimensional model in the noise model are the same model, the first model and the noise model are determined as a positive sample pair, which is used to minimize the difference between the first model and the noise model during the training process of the original posture feature extraction model, so that the original posture feature extraction model converges. Since the three-dimensional model in the first model and the three-dimensional model in the second model are not the same model, the first model and the second model are determined as a negative sample pair, which is used to maximize the difference between the first model and the noise model during the training process of the original posture feature extraction model, so that the original posture feature extraction model converges.

[0054] The first key point refers to a two-dimensional plane coordinate point used to characterize the posture of the first model. The second key point refers to a two-dimensional plane coordinate point used to characterize the posture of the noise model. The third key point refers to a two-dimensional plane coordinate point used to characterize the posture of the second model.

[0055] As an example, in step S402, the computer device extracts posture key points for the first original posture of the first model, the noise posture of the noise model, and the second original posture of the second model of each triple in the three-dimensional training data, and determines the first key point corresponding to the first original posture, the second key point corresponding to the noise posture, and the third key point corresponding to the second original posture. In this example, the computer device extracts the coordinates of the three-dimensional posture key points corresponding to the first original posture on the two-dimensional plane, determines the two-dimensional key points corresponding to each three-dimensional posture key point on the two-dimensional plane, and determines the two-dimensional key point as the first key point corresponding to the first original posture. Similarly, the second key point corresponding to the second original posture and the third key point corresponding to the third original posture are obtained.

[0056] The posture feature to be trained refers to the posture feature used for model training. The first posture feature refers to the posture feature obtained by extracting the posture feature of the first key point. The second posture feature refers to the posture feature obtained by extracting the posture feature of the second key point. The third posture feature refers to the posture feature obtained by extracting the posture feature of the third key point. The original posture feature extraction model refers to the original model used to train the target posture feature extraction model, including but not limited to the multi-layer perceptron model (MLP).

[0057] As an example, in step S403, the computer device selects any triplet in the three-dimensional training data as input to train the original posture feature extraction model. Specifically, the first key point corresponding to the first model in the triplet is input into the original posture feature extraction model, and the first posture feature corresponding to the first key point is output; the second key point corresponding to the noise model in the triplet is input into the original posture feature extraction model, and the second posture feature corresponding to the second key point is output; the third key point corresponding to the second model in the triplet is input into the original posture feature extraction model, and the third posture feature corresponding to the third key point is output, which is used to determine the triplet loss function value according to the first posture feature, the second posture feature and the third posture feature, so as to realize the training of the original posture feature extraction model.

[0058] The triplet loss function value is to minimize the distance between the posture features corresponding to the positive sample pair and maximize the distance between the posture features corresponding to the negative sample pair. In this example, the positive sample pair is the first model and the noise model, the negative sample pair is the first model and the second model, and the triplet loss function value is used to minimize the distance between the first posture feature and the second posture feature, and maximize the distance between the first posture feature and the third posture feature.

[0059] As an example, in step S404, the computer device minimizes the distance between the first posture feature and the second posture feature, maximizes the distance between the first posture feature and the third posture feature, and determines the triplet loss function value corresponding to the original posture feature extraction model. In this example, the triplet loss function for: in, represents the distance between a and p, represents the distance between a and n, and m is a constant greater than 0. a is the first posture feature, p is the second posture feature, and n is the third posture feature.

[0060] When the computer device trains the original posture feature extraction model, it uses the triplet loss function Determine the corresponding triplet loss function value in real time.

[0061] The preset stop training condition refers to a condition for determining whether the original posture feature extraction model has been trained.

[0062] As an example, in step S405, the computer device obtains the triplet loss function value of each training in the process of training according to the triplet original posture feature extraction model, updates the original posture feature extraction model according to the triplet loss function value, and monitors whether the triplet loss function value meets the preset stop training condition. When it is determined that the triplet loss function value meets the preset stop training condition, the currently updated original posture feature extraction model is determined as the target posture feature extraction model. When it is determined that the triplet loss function value does not meet the preset stop training condition, the original posture feature extraction model continues to be trained according to the three-dimensional training data. In this example, the preset stop training condition includes but is not limited to the triplet loss function value being less than a preset threshold, the triplet loss function value tending to be stable, and the number of acquisitions of the triplet loss function value is greater than the preset number. When the computer device determines that the triplet loss function value is less than the preset threshold, the triplet loss function value tends to be stable, or the number of acquisitions of the triplet loss function value is greater than the preset number, it determines that the triplet loss function value meets the preset stop training condition.

[0063] In this embodiment, the original posture feature extraction model is trained according to the two-dimensional key points corresponding to the three-dimensional training data in the three-dimensional space, so that the trained target posture feature extraction model can accurately extract features from the two-dimensional key points corresponding to the image.

[0064] In one embodiment, if Figure 5 As shown, step S103, i.e., performing feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object, includes: S501: Segment the first image to determine a target segmentation image corresponding to the target object, extract the target object from the first image using the target segmentation image to determine a style image corresponding to the target object, extract features from the style image using a style feature generation network to determine a style feature map corresponding to the target object; S502: extract key points of the target object in the first image to obtain a posture image corresponding to the target object, extract features from the posture image using a posture feature generation network, and determine a posture feature map corresponding to the target object.

[0065] Among them, the target segmentation image refers to an image in which the grayscale value of the image area where the target object is located is 255, and the grayscale value of the remaining image area is 0. The style image refers to an image containing only an incomplete target object, which is used to characterize the style of the target object. The style feature generation network refers to a network that extracts the style features of the target object, which is used to extract features used to characterize the uniqueness of the target object, so that the target object after completion according to the style feature map is the same object as the target object in the first image. The style feature map refers to a feature map obtained by extracting the style features of the target object. Among them, the style feature is a unique feature of the target object, which is used to guide the generation of an image that can be aligned with the target object when completing the image of the target object. It can be understood that if the target object is a certain person, then when completing the image of the person, according to the style feature map of the person, the complete image corresponding to the person is guided to be generated, rather than the image of another person, so as to ensure that the completed target object is the same target object as the target object in the first image, and to ensure the uniqueness of the target object.

[0066] As an example, in step S501, the computer device uses a preset image segmentation algorithm to perform segmentation processing on the first image, determine the target segmentation image corresponding to the target object, use the target segmentation image to perform target object extraction processing on the first image, and determine the style image corresponding to the target object. The image segmentation algorithm includes but is not limited to the SAM (Segment Anything Model) algorithm. For example, the computer device uses the SAM algorithm to perform image segmentation processing on the target object in the first image, obtains a foreground image with a gray value of 255 corresponding to the target object and a black background image with a gray value of 0 corresponding to the first image, and determines the foreground image with a gray value of 255 as the target segmentation image corresponding to the target object. It can be understood that in the target segmentation image, the gray value corresponding to the target object is 255, and the target segmentation image is a white image corresponding to the target object, which is used to extract the incomplete target object in the first image. In this example, the target object in the first image is segmented to obtain a target segmented image, which is used to characterize the image area of ​​the target object in the first image. The computer device uses the target segmented image to perform a mask operation on the first image, extract the image area where the target object is located in the first image, and form a style image containing only an incomplete target object, which is used to guide the completion of the incomplete target object so that the style features of the target object remain unchanged and the completed target object is consistent with the target object in the first image. The computer device uses a preset style feature generation network to extract features from the style image corresponding to the target object, and determines a style feature map corresponding to the target object, which is used as a basis for subsequent image completion, so that the target object completed according to the style feature map is the same object as the target object in the first image. For example, if the target object is a person, the beard features, facial features and facial expression features of the person are all style features of the person, and the style feature map corresponding to the person is determined, so that when the person is completed according to the style feature map, the style features such as the beard features, facial features and facial expression features of the person are referred to to obtain a complete person image consistent with the person in the first image. In this example, the style feature generation network includes but is not limited to the IP-Adapter network.

[0067] The posture feature map refers to a feature map obtained by extracting the posture features of the target object.

[0068] As an example, in step S502, the computer device identifies the three-dimensional posture key points corresponding to the posture of the target object in the first image, and extracts the two-dimensional coordinates of the three-dimensional posture key points in the two-dimensional plane where the first image is located, and obtains a posture image including multiple two-dimensional key points, which is used to extract posture features of the incomplete target object in the first image, as a basis for completing the posture features in the process of completing the image of the target object. The computer device uses a preset posture feature generation network to extract the posture features of the target object in the posture image, and determines the posture feature map corresponding to the incomplete target object. In this example, the posture feature generation network includes but is not limited to the Pose ControlNet network. For example, if the target object is a person, the computer device uses the Pose ControlNet network to extract the posture features of the person in the posture image, and determines the posture feature map corresponding to the person.

[0069] In this embodiment, a style feature map and a posture feature map corresponding to the incomplete target object in the first image are obtained, so as to guide the image completion of the incomplete target object based on the style feature map and the posture feature map, so that the completed complete target object is accurately aligned with the incomplete target in the first image, thereby ensuring the consistency of the target object.

[0070] In one embodiment, the annotation information corresponding to the second image includes a depth image. Figure 6 As shown, step S104, i.e., performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, to determine a second feature map corresponding to the three-dimensional model in the second image, includes: S601: performing edge extraction on the second image to determine an edge image corresponding to the second image, performing edge feature extraction on the edge image using an edge feature extraction network to determine an edge feature map corresponding to the three-dimensional model in the second image; S602: Use a deep feature extraction network to extract depth features from the depth image to determine a depth feature map corresponding to the three-dimensional model in the second image.

[0071] The edge image refers to a model including the edge of the image area corresponding to the three-dimensional model. For example, if the three-dimensional model is a human model, the edge image is an image including the edge of the human model. The edge feature extraction network refers to a network that extracts edge features from the edge image. The edge feature map refers to an image containing edge features corresponding to the edge image.

[0072] As an example, in step S601, the computer device uses a preset edge algorithm to extract the edge of the three-dimensional model in the second image, and determines the edge image corresponding to the second image. In this example, the preset edge algorithm includes but is not limited to the Canny edge detection algorithm and the Laplace algorithm. For example, if the three-dimensional model is a character model, the computer device uses a preset edge detection algorithm to perform edge detection on the image area corresponding to the character model in the second image, and determines the edge image corresponding to the character model. The computer device uses a preset edge feature extraction network to extract the edge features of the three-dimensional model in the edge image corresponding to the second image, and determines the edge feature map corresponding to the second image. In this example, the edge feature extraction network includes but is not limited to Canny ControlNet. For example, when the computer device determines that the three-dimensional model is a character model, the edge image corresponding to the character model is input into Canny ControlNet, and the edge features corresponding to the character model are extracted to obtain an edge feature map. In this example, the edge feature map corresponding to the second image is obtained to guide the generation of the target image.

[0073] The depth feature extraction network refers to a network used to extract depth features from a depth image. A depth feature map refers to an image containing depth features of a depth image. Depth features include but are not limited to the distance between a specific area in an image and the optical center of a simulated camera in a 3D rendering engine. In this example, the specific distance may be the distance between the 3D model and the optical center of the camera when the 3D model is photographed.

[0074] As an example, in step S602, the computer device uses a preset depth feature extraction network to extract the depth features of the three-dimensional model in the depth image corresponding to the second image, and determines the depth feature map corresponding to the second image. In this example, the depth feature extraction network includes but is not limited to Depth ControlNet. For example, when the computer device determines that the three-dimensional model is a character model, the depth image corresponding to the character model is input into Depth ControlNet, and the depth features corresponding to the character model are extracted to obtain a depth feature map. In this example, the depth feature map corresponding to the second image is obtained to guide the generation of the target image.

[0075] In this embodiment, a depth feature map and an edge feature map corresponding to the complete three-dimensional model in the second image are obtained, so as to guide the generation of a target image based on the depth feature map and edge feature map corresponding to the three-dimensional model, so that the complete target object in the target image has accurate depth features and edge features.

[0076] In one embodiment, the annotation information corresponding to the second image includes a mask image, the first feature map includes a style feature map and a posture feature map, and the second feature map includes a depth feature map and an edge feature map. Figure 7As shown, step S105, i.e., performing image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map to obtain the target image, includes: S701: Inputting the second image, the style feature map, the posture feature map, the depth feature map, and the edge feature map into a preset image generation model to generate a fourth image; S702: Using the mask image to extract the target object from the fourth image, to generate a target image corresponding to the complete target object.

[0077] Among them, the image generation model refers to a model that completes the image of the target object.

[0078] As an example, in step S701, the computer device inputs the second image, the style feature map, the posture feature map, the depth feature map and the edge feature map into a preset image generation model, and the image generation model identifies the information corresponding to the second image, the style feature map, the posture feature map, the depth feature map and the edge feature map, and performs image completion processing on the target object according to the identified information to obtain a fourth image including a complete target object. In this example, the image generation model includes but is not limited to a Stable Diffusion network model or a generative adversarial network model. For example, if the target object is a person, the computer device inputs the second image, the style feature map, the posture feature map, the depth feature map and the edge feature map into the Stable Diffusion network model, and the Stable Diffusion network model completes the image area corresponding to the incomplete person in the first image with the information in the second image, the style feature map, the posture feature map, the depth feature map and the edge feature map, and generates a fourth image including a complete person. This method guides the image generation model to complete the target object based on the second image that has the highest similarity to the first image, the depth feature map and edge feature map corresponding to the second image, and the style feature map and posture feature map corresponding to the first image. It can not only complete the missing area of ​​the target object, but also align the complete target object with the incomplete target object in the first image, thereby ensuring the consistency of the target object before and after completion.

[0079] The target image refers to an image that only contains the complete target object.

[0080] As an example, in step S702, the computer device uses a mask image to extract the image area corresponding to the complete target object contained in the fourth image, and generates a target image that only contains the complete target object. It is understandable that the fourth image may include other objects besides the target object. Therefore, the mask image corresponding to the complete three-dimensional model in the second image with the highest similarity to the first image is used to perform image extraction processing on the complete target object in the fourth image, and obtain a target image that only contains the target object.

[0081] In this embodiment, the second image, the style feature map, the posture feature map, the depth feature map and the edge feature map are used to guide the generation of a fourth image including a complete target object, which can not only complete the image of the incomplete target object in the first image, but also obtain a complete target object aligned with the incomplete target object in the first image, thereby ensuring the consistency before and after the target object is completed. The target object in the fourth image is extracted using a mask image corresponding to the three-dimensional model with the highest similarity to the target object, so that a target image containing only the complete target object can be accurately extracted.

[0082] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0083] In one embodiment, an image processing device is provided, which corresponds one-to-one to the image processing method in the above embodiment. Figure 8 As shown, the image processing device includes a first image acquisition module 801, a second image determination module 802, a first feature map determination module 803, a second feature map determination module 804 and a target image generation module 805. Each functional module is described in detail as follows: A first image acquisition module 801 is used to acquire a first image, where the first image includes an incomplete target object; The second image determination module 802 is used to perform similarity analysis on the first image, determine a second image corresponding to a three-dimensional model with a similar posture to the target object, and label information corresponding to the second image; A first feature map determining module 803 is used to perform feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object; A second feature map determining module 804 is used to perform feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, and determine a second feature map corresponding to the three-dimensional model in the second image; The target image generation module 805 is used to perform image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map to obtain a target image, wherein the target image includes the complete target object.

[0084] In one embodiment, the second image determination module 802 includes: The target posture feature acquisition submodule is used to input the two-dimensional key points corresponding to the target object into the pre-trained target posture feature extraction model to obtain the target posture features corresponding to the target object; The second image determination submodule queries a pre-established posture feature database based on the target posture feature to obtain similar posture features corresponding to the three-dimensional model with the highest similarity to the target posture feature, and determines the third image corresponding to the similar posture feature as the second image; The annotation information determination submodule queries a pre-established rendering database based on the second image to determine the annotation information corresponding to the second image.

[0085] In another embodiment, the image processing apparatus further includes: A third image generation module controls a 3D rendering engine to render the multiple 3D models based on preset configuration parameters to obtain multiple third images and annotation information corresponding to each third image, wherein the annotation information includes a mask image and a depth image; a rendering database determination module, which determines a rendering database based on the third image and the annotation information corresponding to the third image; An initial posture feature determination module is used to extract features of the three-dimensional models in the plurality of third images in the rendering database using a pre-trained target posture feature extraction model, and determine initial posture features corresponding to the three-dimensional models in the plurality of third images; The posture feature database determination module determines the posture feature database based on the multiple third images and the initial posture feature corresponding to the three-dimensional model in each third image.

[0086] In another embodiment, the image processing apparatus further includes: A three-dimensional training data acquisition module, used to acquire three-dimensional training data, the three-dimensional training data including a first model, a second model, a first original posture corresponding to the first model, a noise posture corresponding to the first model, and a second original posture corresponding to the second model; A two-dimensional key point determination module is used to extract key points from three-dimensional training data and determine two-dimensional key points corresponding to the three-dimensional training data, where the two-dimensional key points include a first key point corresponding to the first original posture, a second key point corresponding to the noise posture, and a third key point corresponding to the second original posture; A posture feature determination module to be trained, used for inputting the two-dimensional key points corresponding to the three-dimensional training data into the original posture feature extraction model to determine the posture features to be trained, wherein the posture features to be trained include a first posture feature corresponding to the first key point, a second posture feature corresponding to the second key point, and a third posture feature corresponding to the third key point; A triplet loss function value determination module, which determines the triplet loss function value corresponding to the original posture feature extraction model based on the posture feature to be trained; The target posture feature extraction model determination module is used to determine the original posture feature extraction model as the target posture feature extraction model if the triplet loss function value meets the preset training stop condition.

[0087] In one embodiment, the first feature map determining module 803 includes: A style feature map determination submodule is used to segment the first image, determine a target segmentation image corresponding to the target object, use the target segmentation image to extract the target object from the first image, determine a style image corresponding to the target object, use a style feature generation network to extract features from the style image, and determine a style feature map corresponding to the target object; The posture feature map determination submodule is used to extract key points of the target object in the first image to obtain a posture image corresponding to the target object, use a posture feature generation network to extract features from the posture image, and determine the posture feature map corresponding to the target object.

[0088] In one embodiment, the second feature map determining module 804 includes: An edge feature map determination module is used to perform edge extraction on the second image, determine an edge image corresponding to the second image, perform edge feature extraction on the edge image using an edge feature extraction network, and determine an edge feature map corresponding to the three-dimensional model in the second image; The depth feature map determination module is used to extract depth features from the depth image using a depth feature extraction network to determine a depth feature map corresponding to the three-dimensional model in the second image.

[0089] In one embodiment, the target image generation module 805 includes: A fourth image generation submodule, used for inputting the second image, the style feature map, the posture feature map, the depth feature map and the edge feature map into a preset image generation model to generate a fourth image; The target image generation submodule is used to use the mask image to perform target object extraction processing on the fourth image to generate a complete target image corresponding to the target object.

[0090] For the specific definition of the image processing device, please refer to the definition of the image processing method above, which will not be repeated here. Each module in the above image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0091] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data used or generated during the execution of the image processing method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image processing method is implemented.

[0092] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image processing method in the above embodiment is implemented, for example Figure 1 S101-S105 as shown, or Figures 2 to 7 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the image processing device are realized, for example Figure 8 The functions of the first image acquisition module 801, the second image determination module 802, the first feature map determination module 803, the second feature map determination module 804 and the target image generation module 805 are not described here in detail to avoid repetition.

[0093] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method of the above embodiment is implemented, for example Figure 1 S101-S105 as shown, or Figures 2 to 7 Alternatively, when the computer program is executed by a processor, the functions of each module / unit in the above-mentioned image processing device embodiment are realized, for example Figure 8The functions of the first image acquisition module 801, the second image determination module 802, the first feature map determination module 803, the second feature map determination module 804 and the target image generation module 805 are not described here to avoid repetition. The computer-readable storage medium may be non-volatile or volatile.

[0094] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0095] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0096] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire a first image, wherein the first image includes an incomplete target object; Analyze the first image to determine a second image corresponding to a three-dimensional model similar to the posture feature of the target object and annotation information corresponding to the second image; Performing feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object; performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, and determining a second feature map corresponding to the three-dimensional model in the second image; According to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map, the incomplete target object is image completed to obtain a target image, wherein the target image contains the complete target object.

2. The image processing method according to claim 1, characterized in that: The first image includes two-dimensional key points corresponding to the target object; The analyzing the first image to determine a second image corresponding to a three-dimensional model having a posture feature similar to the target object and annotation information corresponding to the second image includes: Inputting the two-dimensional key points corresponding to the target object into a pre-trained target posture feature extraction model to obtain the target posture features corresponding to the target object; Based on the target posture feature, query a pre-established posture feature database to obtain a similar posture feature corresponding to the three-dimensional model with the highest similarity to the target posture feature, and determine the third image corresponding to the similar posture feature as the second image; A pre-established rendering database is queried based on the second image to determine annotation information corresponding to the second image.

3. The image processing method according to claim 2, characterized in that: Before acquiring the first image, the image processing method further includes: Based on preset configuration parameters, control the 3D rendering engine to render the multiple 3D models to obtain multiple third images and annotation information corresponding to each of the third images, wherein the annotation information includes a mask image and a depth image; Determining the rendering database based on the third image and the annotation information corresponding to the third image; Using a pre-trained target posture feature extraction model to extract features of the three-dimensional models in the plurality of third images in the rendering database, and determining initial posture features corresponding to the three-dimensional models in the plurality of third images; The posture feature database is determined based on the multiple third images and the initial posture feature corresponding to the three-dimensional model in each third image.

4. The image processing method according to claim 2, wherein: Before acquiring the first image, the image processing method further includes: Acquire three-dimensional training data, the three-dimensional training data including a first model, a second model, a first original pose corresponding to the first model, a noise pose corresponding to the first model, and a second original pose corresponding to the second model; Extract key points from the three-dimensional training data to determine two-dimensional key points corresponding to the three-dimensional training data, where the two-dimensional key points include a first key point corresponding to the first original posture, a second key point corresponding to the noise posture, and a third key point corresponding to the second original posture; Inputting the two-dimensional key points corresponding to the three-dimensional training data into the original posture feature extraction model to determine the posture features to be trained, wherein the posture features to be trained include a first posture feature corresponding to the first key point, a second posture feature corresponding to the second key point, and a third posture feature corresponding to the third key point; Based on the posture features to be trained, determining a triplet loss function value corresponding to the original posture feature extraction model; If the triplet loss function value satisfies a preset training stop condition, the original posture feature extraction model is determined as the target posture feature extraction model.

5. The image processing method according to claim 1, wherein: The performing feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object includes: Performing segmentation processing on the first image to determine a target segmentation image corresponding to the target object, performing target object extraction processing on the first image using the target segmentation image to determine a style image corresponding to the target object, performing feature extraction on the style image using a style feature generation network to determine a style feature map corresponding to the target object; Key points of the target object in the first image are extracted to obtain a posture image corresponding to the target object, and a posture feature generation network is used to extract features from the posture image to determine a posture feature map corresponding to the target object.

6. The image processing method according to claim 1, characterized in that: The annotation information corresponding to the second image includes a depth image; The performing feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image to determine a second feature map corresponding to the three-dimensional model in the second image includes: Performing edge extraction on the second image to determine an edge image corresponding to the second image, performing edge feature extraction on the edge image using an edge feature extraction network to determine an edge feature map corresponding to the three-dimensional model in the second image; A deep feature extraction network is used to extract deep features from the depth image to determine a deep feature map corresponding to the three-dimensional model in the second image.

7. The image processing method according to claim 1, characterized in that: The annotation information corresponding to the second image includes a mask image, the first feature map includes a style feature map and a posture feature map; the second feature map includes a depth feature map and an edge feature map; The performing image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map, and the second feature map to obtain the target image includes: Inputting the second image, the style feature map, the posture feature map, the depth feature map, and the edge feature map into a preset image generation model to generate a fourth image; The mask image is used to perform target object extraction processing on the fourth image to generate a target image corresponding to the complete target object.

8. An image processing device, characterized in that: include: A first image acquisition module, used to acquire a first image, wherein the first image includes an incomplete target object; A second image determination module, configured to analyze the first image to determine a second image corresponding to a three-dimensional model having a posture feature similar to the target object and annotation information corresponding to the second image; A first feature map determining module, used to perform feature extraction processing on the target object in the first image to determine a first feature map corresponding to the target object; A second feature map determining module, configured to perform feature extraction processing on the three-dimensional model in the second image and the three-dimensional model in the annotation information corresponding to the second image, and determine a second feature map corresponding to the three-dimensional model in the second image; A target image generation module is used to perform image completion processing on the incomplete target object according to the second image, the annotation information corresponding to the second image, the first feature map and the second feature map to obtain a target image, wherein the target image includes a complete target object.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images