Image processing method and device, computer readable storage medium and computer equipment

Abstract

The embodiment of the invention discloses an image processing method and device, a computer readable storage medium and computer equipment. The method comprises the steps of obtaining an object image, wherein the object image comprises an object in an expected posture; in response to the image generation instruction, obtaining a drawing model added with the trained style conversion model for the silhouette style; performing attitude feature extraction on the object image through the attitude detection model to obtain an attitude feature matched with the expected attitude; performing edge feature extraction on the object image through an edge detection model to obtain a target edge feature of the object image; based on the target edge features and the posture features, style conversion is conducted on the object image through a drawing model, a target object image of a silhouette style is obtained, and the target posture of the object in the target object image is matched with the expected posture of the object in the object image. In this way, the image of the silhouette style matched with the expected character action and posture can be generated, and then the image generation effect is improved.

Description

Technical Field

[0001] The present application relates to the field of game technology, and in particular to an image processing method, apparatus, computer-readable storage medium, and computer equipment. Background Art

[0002] With the rapid development of Internet technology, image processing methods have also developed rapidly. For example, artificial intelligence (AI) models such as diffusion models can be used to generate images in styles such as silhouette styles based on images or texts input by users, thereby improving image processing efficiency.

[0003] In the process of research and practice of the prior art, it is found that the action form of the character in the silhouette-style silhouette image generated by the existing image processing method is very likely to be greatly different from the specified action form of the character, resulting in poor image generation effect. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus, computer-readable storage medium, and computer equipment, which can generate silhouette-style images that match desired character movements and postures, thereby improving image generation effects.

[0005] The present application provides an image processing method, including:

[0006] Acquire an object image, wherein the object image includes an object in a desired posture;

[0007] In response to an image generation instruction, obtaining a drawing model to which a trained style transfer model for a silhouette style is added, the drawing model comprising a diffusion control network, the diffusion control network comprising a posture detection model and an edge detection model;

[0008] Extracting posture features from the object image using the posture detection model to obtain posture features that match the expected posture;

[0009] Extracting edge features of the object image using the edge detection model to obtain target edge features of the object image;

[0010] Based on the target edge features and the posture features, the object image is style-converted by the drawing model to obtain the target object image in the silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0011] Accordingly, an embodiment of the present application provides an image processing device, including:

[0012] A first acquisition unit, configured to acquire an object image, wherein the object image includes an object in a desired posture;

[0013] A second acquisition unit is used to acquire, in response to the image generation instruction, a drawing model to which a trained style conversion model for a silhouette style is added, wherein the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model;

[0014] A first extraction unit, configured to extract posture features from the object image using the posture detection model to obtain posture features matching the expected posture;

[0015] A second extraction unit is used to extract edge features of the object image by using the edge detection model to obtain target edge features of the object image;

[0016] An image generating unit is used to perform style conversion on the object image through the drawing model based on the target edge feature and the posture feature to obtain the target object image in the silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0017] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute any image processing method provided in the embodiment of the present application.

[0018] In addition, an embodiment of the present application also provides a computer device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the image processing method provided in the embodiment of the present application.

[0019] The embodiment of the present application obtains an object image, which includes an object in a desired posture; in response to an image generation instruction, obtains a drawing model with a trained style conversion model for silhouette style added, the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; performs posture feature extraction on the object image through the posture detection model to obtain a posture feature matching the desired posture; performs edge feature extraction on the object image through the edge detection model to obtain a target edge feature of the object image; based on the target edge feature and the posture feature, performs style conversion on the object image through the drawing model to obtain a target object image in silhouette style, and the target posture of the object in the target object image matches the desired posture of the object in the object image. In this way, by superimposing the posture detection model and the edge detection model on the diffusion control network of the drawing model superimposed with the style conversion model, respectively extracting posture features and target edge features from the object image using the posture detection model and the edge detection model, and generating a target object image in silhouette style based on the posture features and the target edge features, it is possible to achieve precise control of the posture of the person in the generated image and the action details of the important area, and generate a silhouette style image matching the desired person action and posture, thereby effectively improving the image generation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a schematic diagram of an implementation scenario of an image processing method provided in an embodiment of the present application;

[0022] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0023] Figure 3a is a schematic diagram of a target edge image of an image processing method provided in an embodiment of the present application;

[0024] Figure 3b is another target edge image schematic diagram of an image processing method provided by an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a target object image of an image processing method provided in an embodiment of the present application;

[0026] Figure 5 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0027] Figure 6 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0029] The embodiments of the present application provide an image processing method, apparatus, computer-readable storage medium and computer equipment. The image processing apparatus can be integrated in a computer equipment, which can be a server or a terminal.

[0030] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Terminals may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. Terminals and servers can be directly or indirectly connected via wired or wireless communications, and this application does not limit this.

[0031] In the existing image generation method that uses artificial intelligence (AI) models to generate silhouette-style images based on user-input images, a large model such as a diffusion model is generally used to superimpose a style conversion (lora) model for the silhouette style, and a diffusion control network (ControlNet) is superimposed on the large model to generate a silhouette image of the silhouette style. However, the diffusion control network can only select a preprocessor alone, for example, a posture detection model (Open pose) or an edge detection model (Canny) or other preprocessors are selected alone. However, when the prior art generates a silhouette image containing a specified character's motion form based on a large model's image generation function and in combination with a preprocessor provided by a diffusion control network, the generated silhouette image often has defects, and the character's motion form in the silhouette image is not completely consistent with the specified character's motion form. For example, when a posture detection model is selected to generate a silhouette image, due to the recognition defects of the posture detection model, only the general posture of the character can be recognized. For some complex character motion forms, the posture detection model cannot make an accurate recognition, so that the character's motion form in the generated silhouette image does not match the specified character motion form, making the silhouette image generated by the posture detection model poor; and when an edge detection model is selected to generate a silhouette image, the edge detection model can accurately detect the contour lines of the image. However, since the contour lines are too complex and detailed in describing the character in the image, there will be many unnecessary details in the silhouette image generated by the edge detection model, and the character's motion form in the silhouette image will also be stiff and unnatural, which is not conducive to generating the concise and natural character silhouette image required by the project.

[0032] Therefore, in order to make up for the recognition defects of the preprocessor in the diffusion control network, the image processing method provided in the embodiment of the present application integrates the image recognition advantages of the posture detection model and the edge detection model. By superimposing the posture detection model and the edge detection model on the diffusion control network with the large model (drawing model) superimposed with the style transfer model, the posture detection model and the edge detection model are used to extract the posture features and the target edge features from the object image respectively, so as to generate a silhouette-style target object image based on the posture of the character represented by the posture features and the character movement details represented by the target edge features. It can achieve precise control of the posture of the character and the movement details of the important areas in the generated image, so as to generate a silhouette-style image that is highly consistent with the expected character movement and posture, thereby effectively improving the image generation effect.

[0033] See also Figure 1 , taking the image processing device integrated into a computer device as an example, Figure 1A schematic diagram of an implementation scenario of the image processing method provided in an embodiment of the present application, wherein the computer device may be a terminal or a server, and the computer device may obtain an object image, wherein the object image includes an object in an expected posture; in response to an image generation instruction, a drawing model to which a trained style conversion model for a silhouette style is added is obtained, wherein the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; posture features of the object image are extracted through the posture detection model to obtain posture features matching the expected posture; edge features of the object image are extracted through the edge detection model to obtain target edge features of the object image; based on the target edge features and posture features, style conversion is performed on the object image through the drawing model to obtain a target object image in a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0034] It should be noted that Figure 1 The implementation environment scenario diagram of the image processing method shown is only an example. The implementation environment scenario of the image processing method described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. It is known to those skilled in the art that with the evolution of data processing and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0035] The solutions provided by the embodiments of the present application are specifically described by the following embodiments. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0036] This embodiment will be described from the perspective of an image processing apparatus. The image processing apparatus may be integrated into a computer device. The computer device may be a terminal and / or a server, and this application does not limit this.

[0037] See also Figure 2 , Figure 2 : is a flow chart of an image processing method provided in an embodiment of the present application. The image processing method comprises:

[0038] In step 101, an object image is acquired.

[0039] The object image may include an object in an expected posture, and the object may be a person, an animal, a plant, or the like. The expected posture may be the posture of the object in the object image, and the posture may describe the state of the object's body, and may include the object's action, posture, etc. When performing style conversion based on the object image to generate an image, it is often desired that the object in the generated image has the expected posture. For example, the object image may include an image of a standing person, and the person makes an "OK" gesture. In this case, the expected posture may be a posture of standing and making an "OK" gesture. The object image may also include an image of a sitting person, and the person makes a waving gesture. In this case, the expected posture may be a posture of sitting and making a waving gesture.

[0040] In step 102, in response to an image generation instruction, a drawing model to which a trained style transfer model for a silhouette style is added is acquired.

[0041] The image generation instruction may be an instruction for instructing to generate an image of an object image in a silhouette style. The silhouette style may be the style of an image to be generated, and a silhouette may refer to an image that depicts the outline of an object such as a person. In a silhouette style image for a person, information such as the outline, shape, and action of the person may be depicted. The style conversion model may be a style conversion model for style conversion, and may be used to convert the style of an object image into a silhouette image in a silhouette style. The drawing model may be a model for image drawing, and the drawing model converts the object image into a silhouette image in a silhouette style by calling the style conversion model. The drawing model may include a diffusion control network, and the diffusion control network may include a posture detection model and an edge detection model. The posture detection model (Open pose) may be a model for extracting posture features of an object in an image, and the posture features may be information that characterizes the posture of an object in an object image. The edge detection model (Canny) may be a model for extracting edge features of an image in an image, and the edge features may be information that characterizes the edge distribution in an object image.

[0042] Optionally, the drawing model can be a model for image drawing based on artificial intelligence (AI) technology, for example, a diffusion model such as Stable Diffusion (SD model, AI animation generation tool), and correspondingly, the style transfer model can be a low-rank adaptation model of large language models (Low-Rank Adaptation of Large Language Models, Lora model for short). The diffusion control network can be a control network (ControlNet), which can be used to control a plug-in for AI image generation.

[0043] In one embodiment, a style transfer model for a silhouette style can be trained, wherein a training script (Koyha_ss) can be used to train the style transfer model Lora model, which encapsulates the training task of the Lora model and provides a visual interactive interface, through which the training parameters in the model training process are conveniently configured. Specifically, the training samples of the drawing model can be obtained according to the needs of the project. For example, when it is necessary to generate a silhouette image of a person consistent with the action posture of the person in the object image, a silhouette-style image can be obtained as a training sample with a style transfer model added. The silhouette-style image in the training sample can be cut into a size of 512x512 pixels, leaving a white background except for the main body of the object in the image, so as to unify the training style of the style transfer model, improve the training stability, and reduce the image generation confusion rate. The white background is left to meet the needs of the silhouette style and make the model training more focused on the main body of the image. In the Stable Diffusion model page, you can select the image preprocessing module under the training module, fill in the source directory and target directory corresponding to the training sample, click the reverse prompt word (deepbooru) to generate the description text (tags), and then click the preprocessing button. After the preprocessing process displayed on the page is completed, the image and its annotation file of the training sample can be obtained in the target directory. Optionally, the initial image annotation is often not accurate enough and needs further processing. For example, you can use the tagging tool (BooruDatasetTagManager) to modify the image annotation. The modified annotation method can be: silhouette (wake-up word) + character action name description + human + white_background, etc. After completion, you can see the modified image annotation in the target directory. The modification of the annotation can make the style transfer model better understand the image sample when training the sample, and improve the semantic understanding ability of the model. After the image annotation is modified, the number 10 can be added to the folder prefix of the image sample, indicating that each image will be trained 10 times in each round of training. Next, you can open the Lora training script of koyha_ss, and in the model quick selection block under the source model module, pull down and select "custom". Select the "final-prune.ckpt" model in the name or path of the pre-trained model block. This model has the least negative impact on the final training effect of the style transfer model and can restore the project style to the greatest extent. In the directory module, select the target directory path in the training image directory block, and fill in the corresponding paths for the directory where the model is saved and the log directory.In the training parameter module, you can set the training batch size to 2, the number of iterations (Epoch) to 10, save every epoch to 2, the learning rate scheduler to "cosin_with_restarts", the optimizer to Lion, the text encoder learning rate to 1e-5, the model network size to 32, the model Aplha to 32, the Clip skip to 2, the maximum number of training rounds to 20, and then click Start Training to get the trained Lora model. In this way, the Lora model can be called through the drawing model to get the drawing model with the trained style transfer model for the silhouette style added. Based on the above adjusted training parameters, the drawing model with the trained style transfer model for the silhouette style added can be made to fit the project style to the greatest extent, improve the semantic understanding ability, reduce the image confusion rate, and consolidate the image integrity.

[0044] In step 103, posture features are extracted from the object image using a posture detection model to obtain posture features that match the expected posture.

[0045] The posture feature may be information characterizing the posture of the object in the object image. For example, the posture feature may include information such as the position of the skeleton key points of the object in the object image and the connection relationship between different skeleton key points.

[0046] There are many ways to extract posture features from object images. For example, human posture estimation algorithms such as posture estimation network (Open pose), instance segmentation algorithm (Mask RCNN), regional multi-person pose estimation (RMPE, or AlphaPose) can be used to extract posture features from object images to obtain posture features that match the expected posture.

[0047] Optionally, when it is necessary to generate a silhouette-style character silhouette image based on an object image, since the character silhouette-style image does not require background information in the object image, the object image can be an image that only contains the object but not the background, or it can be an object image with a solid color background. When the object image includes the background, the area where the object is located can be identified in the object image, so that the edge line of the object can be extracted in the object image, thereby obtaining an edge image based on the edge line of the object.

[0048] In step 104, edge features of the object image are extracted using an edge detection model to obtain target edge features of the object image.

[0049] The target edge feature may be information characterizing the edge distribution of the object image.

[0050] There are many ways to extract edge features of an object image through an edge detection model to obtain target edge features of the object image. For example, edge detection can be performed on the object image through an edge detection model to obtain an edge image of the object image, edge adjustment processing can be performed on the edge image to obtain a target edge image, and edge features can be extracted from the target edge image to obtain target edge features.

[0051] The edge image may be an image containing edge lines in the object image, and the edge lines may be lines generated by edges such as the edge between the object and the background in the object image, the edge between objects in the image, etc. For example, they may be line drawings extracted from the object image, etc. The target edge image may be an edge image obtained by adjusting the edge lines of the edge image according to actual needs, for example, the target edge image may be an image obtained by deleting redundant edge lines in the edge image, and the redundant edge lines may be edge lines of unimportant parts of the edge image, or edge lines of unnecessary areas in the edge image. For example, assuming that a silhouette-style image with hand movements that are highly consistent with the hand movements of the object in the object image needs to be generated based on the object image, at this time, the edge lines in other areas of the edge image except the area where the hand is located may be redundant edge lines. For another example, assuming that a silhouette-style silhouette image of a person that is consistent with the posture and hand movements of the object in the object image needs to be generated based on the object image, since the silhouette-style person image does not require too many details, for example, the background, and details such as the person's clothing pattern, spots, and wrinkles are not required, therefore, the edge lines belonging to the background area, the edge lines in the person describing the person's details, and the edge lines in the area where other parts of the person except the hand are located may be deleted as redundant edge lines. In addition, the target edge image can also be an image obtained by supplementing the edge lines of a certain area in the edge image. When there is a blurred or missing area in the edge image, the edge lines of the area in the edge image can be supplemented to improve the accuracy of the edge lines in the edge image. Furthermore, the target edge image can also be an image obtained by modifying the edge lines of a certain part in the edge image. For example, please refer to Figure 3a , Figure 3aThis is a schematic diagram of a target edge image of an image processing method provided in an embodiment of the present application. Assuming that the edge line of the hand area in the target edge image is an edge line representing an "ok" gesture, the edge line of the "ok" gesture can be modified as needed to represent other gestures such as the index finger pointing outward. In this way, the edge lines in the edge image can be modified according to actual business needs to obtain a target edge image containing graphics that meet business needs, thereby improving the diversity of image generation. The target edge feature can be information that characterizes the edge line in the target edge image. Optionally, the encoder in the edge detection model such as Canny can continue to be used to extract edge features from the target edge image to obtain the target edge feature.

[0052] Since the diffusion control network has defects in recognizing the action form of characters, for example, the posture detection model cannot make accurate recognition for some complex action forms of characters, which affects the effect of generating images. The use of the edge detection model to generate silhouette-style images will make the posture of the characters in the generated silhouette-style images stiff and unnatural, and there will be many unnecessary details, which will lead to poor image generation effect. Therefore, in order to correct the recognition defects of the diffusion control network model, the embodiment of the present application superimposes a posture detection model and an edge detection model on the diffusion control network of the drawing model superimposed with the style transfer model, respectively uses the posture detection model and the edge detection model to extract posture features and edge images from the object image, and further adjusts the edge lines in the edge image, so that the important areas in the object image can be more finely and accurately characterized based on the target edge features extracted from the target edge image, and then based on the posture features of the object image and the target edge features, the posture of the characters in the generated image and the action details of the important areas can be precisely controlled, and a silhouette-style image that is relatively consistent with the expected character action and posture is generated, which effectively corrects the recognition defects of the diffusion control network ControlNet for the action form of characters, thereby effectively improving the image generation effect.

[0053] There are many ways to perform edge detection on an object image using an edge detection model. For example, an edge detection model (Canny), an edge detection network (HED) or other edge detection models may be used to perform edge detection on an object image to obtain an edge image of the object image.

[0054] After edge detection is performed on the object image through the edge detection model to obtain the edge image of the object image, the edge image can be edge adjusted to obtain the target edge image. There are many ways to perform edge adjustment on the edge image to obtain the target edge image. For example, the edge lines of non-target parts in the edge image can be deleted to obtain the target edge image containing only the target part.

[0055] The target part may be a part that needs to be accurately drawn in the generated image, that is, the posture and movement of the target part of the object in the image generated based on the object image is highly consistent with the posture and movement of the target part of the object in the object image. For example, the target part may be a hand, head, face, etc. The edge line of the non-target part may be an edge line that does not belong to the target part, and may also include redundant edge lines in the target part. For example, please refer to Figure 3b , Figure 3b This is another target edge image schematic diagram of an image processing method provided by an embodiment of the present application. Assuming that the target part is the hand of the object, the edge lines of other non-target parts except the hand in the edge image can be deleted, so that a target edge image containing only the edge lines of the target part of the hand can be obtained. In this way, by deleting the edge lines of non-target parts in the edge image, a target edge image is obtained in which redundant edge lines that interfere with edge recognition are eliminated, so that based on the target edge features extracted from the target edge image, the important area where the target part is located in the object image can be more finely and accurately characterized, and then based on the posture features of the object image and the target edge features, the posture of the character in the generated image and the action details of important areas such as the target part can be precisely controlled, and a silhouette-style image that is highly consistent with the expected character action and posture is generated, thereby effectively improving the image generation effect.

[0056] There are many ways to delete edge lines of non-target parts in an edge image. For example, users such as relevant technicians can delete redundant edge lines in an edge image. Specifically, an edge image can be displayed, and in response to an edge line deletion operation on a non-target part in the edge image, a deletion position corresponding to the edge line deletion operation is obtained, and edge lines to be deleted in the non-target part are identified in the edge image according to the deletion position, and the edge lines to be deleted are deleted.

[0057] Among them, the edge line deletion operation can be an operation triggered by the user to delete the edge line of the non-target part in the edge image. For example, the user can select the edge line of the non-target part in the edge image to be deleted in the interface displaying the edge image, thereby triggering the edge line deletion operation. The deletion position can be the position of the edge line to be deleted in the edge image, and the edge line to be deleted can be the edge line corresponding to the edge line deletion operation, that is, it can be the edge line of the non-target part to be deleted.

[0058] When displaying the edge image, it can be displayed directly, or it can be displayed by exporting the edge image to other image processing applications, or it can be displayed by exporting the edge image to other devices, etc. This application does not limit this.

[0059] Optionally, there may be a variety of ways to delete the edge lines of non-target parts in the edge image. For example, the area where the target part is located may be identified in the edge image, so that the edge lines of other areas outside the target part may be deleted. Specifically, deletion indication information for the non-target part in the edge image may be obtained, and the target area where the target part is located may be identified in the edge image based on the deletion indication information, and the edge lines of the non-target part outside the target area in the edge image may be deleted.

[0060] Among them, the deletion indication information can be information indicating the deletion of edge lines of non-target parts in the edge image. For example, the deletion indication information can include information of the target part, which is used to indicate the deletion of edge lines of non-target parts in the edge image except for the area where the target part is located. The target area can be the area where the target part is located in the edge image.

[0061] Among them, after deleting the edge lines of non-target parts outside the target area in the edge image, the target edge lines can be identified in the target area based on the target part, and the target edge lines in the target area are deleted to obtain a target edge image containing only the target part.

[0062] Among them, the target edge line can be an edge line in the target area that is invalid or interferes with the motion recognition of the target part, that is, the target edge line can be an edge line in the target area that has no auxiliary effect on the motion recognition of the target part. For example, assuming that the target part is the hand, the edge line in the target area that represents the wrinkles, spots and other details of the hand can be used as the target edge line. The target edge line can also be an edge line in the target area that interferes with the motion recognition of the target part. For example, assuming that the target part is the hand, the edge lines in the target area that represent the decoration of the hand (such as watches, gloves), background, and lines on the hand will interfere with the recognition of hand movements and can be used as the target edge line.

[0063] In one embodiment, there are multiple ways to perform edge adjustment processing on the edge image to obtain the target edge image. For example, the edge line corresponding to the target part in the edge graphic can be modified, supplemented, and redrawn to make the target part meet the ideal state required by actual needs, thereby improving the image generation effect. For example, the edge line corresponding to the target part in the edge graphic can be modified to make the edge line corresponding to the target part more accurate and meet actual business needs, the edge line corresponding to the target part can be supplemented to make the edge line corresponding to the target part more accurate, and the edge line corresponding to the target part can be redrawn to make the action form of the target part in the target edge image meet actual needs.

[0064] In one embodiment, the edge line in the target area may be detected, and when it is detected that the edge line of the target area has defects, the defective edge line may be complemented, thereby obtaining a target edge image with complete edge information of the target part.

[0065] Among them, there is no order of execution between step 103 and step 104. Step 103 can be executed first and then step 104, or step 104 can be executed first and then step 103, or step 103 and step 104 can be executed at the same time, etc. The specific order of execution of the steps can be set according to actual conditions, and the embodiments of the present application are not limited here.

[0066] In step 105, based on the target edge features and posture features, the object image is style-converted by a drawing model to obtain a target object image in a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0067] Among them, the target object image can be an image in a silhouette style generated by performing style conversion on the object image. Since the target edge feature can control the details of the local area in the target object image, and the posture feature can control the posture of the object in the target object image, the target posture of the target object in the target object image is consistent with the expected posture of the object in the object image. The target object can be an object contained in the target object image, and the target posture can be the posture of the target object in the target object image.

[0068] Optional, please refer to Figure 4 , Figure 4 This is a schematic diagram of a target object image of an image processing method provided in an embodiment of the present application. It is assumed that the object image is an image containing a standing person and the person is making an "OK" gesture. It is also assumed that the silhouette style is a silhouette style containing character movements and shapes. At the same time, the number of target object images can be set to 4, so that the object image can be input into a drawing model, an edge detection model, and a posture detection model to which a trained silhouette style style conversion model is added. Thus, four character silhouette images under the silhouette style corresponding to the object image can be generated based on the drawing model. In the character silhouette image, the outline of the character silhouette is the same as the outline of the character in the object image, and the posture of the character silhouette is consistent with the expected posture of the character in the object image. It should be noted that multiple target object images can be generated based on the object image, or only one target object image can be generated based on the object image. The specific setting can be made according to actual needs, and the embodiment of the present application is not limited here.

[0069] There are many ways to perform style conversion on an object image through a drawing model based on target edge features and posture features. For example, the target edge features and posture features can be fused to obtain control features, which are input into the drawing model. The drawing model is controlled based on the control features to perform style conversion on the object image.

[0070] The control feature may be a feature for controlling the drawing process of the drawing model, and may be a feature that combines target edge features and posture features, and is used to control the posture, movement, etc. of the object in the target object image generated by the drawing model.

[0071] There are many ways to fuse the target edge features and the posture features. For example, the target edge features and the posture features may be added to obtain the control features for controlling the drawing of the drawing model.

[0072] There are many ways to perform style conversion on an object image based on controlling a drawing model based on control features. For example, a drawing model can be used to extract features of an object image to obtain object image features corresponding to the object image and silhouette style features corresponding to the silhouette style. The object image features and the silhouette style features are fused to obtain image features after style conversion. Based on the control features and the image features after style conversion, the style of the object image is converted through a drawing model.

[0073] Among them, the object image feature can be a feature that characterizes the object image, the silhouette style feature can be a feature used to generate an image in a silhouette style corresponding to the object image, and can be a silhouette style feature extracted by a style transfer model based on the object image feature; the image feature after style conversion can be a feature obtained by fusing the object image feature and the silhouette style feature.

[0074] There are many ways to fuse the object image features and the silhouette style features. For example, the object image features and the silhouette style features can be added to obtain the image features after style conversion.

[0075] Optionally, there are multiple ways to fuse the object image features and the silhouette style features to obtain the image features after style conversion. For example, the stylization degree of the generated target object image can be adjusted by adjusting the weights of the style conversion model. Specifically, the style weights corresponding to the drawing model can be obtained, and the silhouette style features can be weighted based on the style weights to obtain weighted silhouette style features. The weighted silhouette style features and the object image features are summed to obtain the image features after style conversion.

[0076] The style weight may be the weight of a style conversion model added to the drawing model, and the weighted silhouette style feature may be the silhouette style feature weighted according to the style weight.

[0077] In one embodiment, the object image can be input into a drawing model, a Canny model, and an Open pose model to which a style transfer model for a silhouette style is added. Due to the recognition defects of the Open pose model, the Open pose model cannot make accurate recognition for some complex human action forms, which affects the generated image effect. The use of the Canny model to generate a silhouette-style image will make the posture of the person in the generated silhouette-style image stiff and unnatural, and there will be many unnecessary details, which will lead to poor image generation effect. Therefore, in order to make up for the recognition defects of the Openpose model, the Canny model can be used to make up for the recognition defects of the Open pose model. In the embodiment of the present application, the edge image that has been preprocessed by the Canny model for the first time often cannot be used directly because the Canny model also has recognition defects. Although the Canny model can obtain the contour line of the human action form more accurately, the contour line is too complex and fine, which is not conducive to generating the concise human silhouette image required by the project style. Therefore, it is necessary to perform redundant edge deletion processing on the edge image generated by the initial recognition of the Canny model, for example, to erase the redundant parts that may cause interference or invalid parts to obtain the target edge image, and input the target edge image into the Canny model again, so that the target edge features can be extracted based on the Canny model, and the posture features recognized by the Open pose model are fused with the target edge features extracted by Cannymx to obtain the control features and input them into the drawing model, so that the drawing model can perform image drawing based on the silhouette style conversion based on the extracted silhouette style features, object image features and control features, wherein, due to the edge detail information of the target part represented in the target edge feature, the action morphology details of the target part of the object in the target object image can be accurately controlled, and the expected posture represented in the posture feature can effectively control the posture of the object in the target object image, therefore, the posture of the object in the silhouette image generated by the drawing model can match the expected posture of the object in the object image, and the action morphology of the target part of the object in the silhouette image is highly consistent with the action of the target part of the object in the object image, thereby effectively improving the image generation effect.

[0078] In one embodiment, a target object image can be generated based on an object image in a visual interactive interface provided by the Koyha_ss project. Specifically, after the training of the Lora model and the drawing model is completed, StableDiffusion can be turned on. In the image generation mode, the forward prompt words and reverse prompt words (tags) required by the model can be input, and then the object image that meets the expected character action form can be imported. The length and width of the input object image and the output image can be multiples of 64. For example, the size of the input object image can be 640x768px, and the size of the output image can be 512x512px. Then, the redrawing amplitude of the drawing model can be set to 0.9. In the optional additional network (Lora plug-in), the trained Lora model is started and selected. Then, the control plug-in (ControlNet) can be opened, "openpose" is selected as the first ControlNet preprocessor and enabled, the model selects the corresponding openpose model file, and imports the object image that meets the expected character action form. Select "canny" as the second ControlNet preprocessor and enable it, the model selects the corresponding canny model file, and imports the object image that meets the expected character action form. Open Canny, you can save the preprocessed image of the Canny model locally and import it into the image processing software, retain the part that needs to be controlled and erase the redundant edge lines that will cause interference, get the target edge image, and then re-import the target edge image into the Canny preprocessor to accurately control the generated image, so that you can click the "Generate" button to get a silhouette image that is consistent with the character action form of the object image.

[0079] As can be seen from the above, the embodiment of the present application obtains an object image, which includes an object in an expected posture; in response to an image generation instruction, obtains a drawing model to which a trained style conversion model for a silhouette style is added, the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; performs posture feature extraction on the object image through the posture detection model to obtain a posture feature matching the expected posture; performs edge feature extraction on the object image through the edge detection model to obtain a target edge feature of the object image; based on the target edge feature and the posture feature, performs style conversion on the object image through the drawing model to obtain a target object image with a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image. In this way, by superimposing a posture detection model and an edge detection model on the diffusion control network of the drawing model superimposed with the style transfer model, the posture detection model and the edge detection model are used to extract posture features and target edge features from the object image respectively, so as to generate a silhouette-style target object image based on the posture features and the target edge features. It is possible to achieve precise control of the posture of the characters in the generated image and the action details of important areas, and generate a silhouette-style image that matches the expected character action and posture, thereby effectively improving the image generation effect.

[0080] In order to better implement the above method, an embodiment of the present invention further provides an image processing apparatus, which can be integrated in a computer device, and the computer device can be a terminal or a server.

[0081] For example, Figure 5 As shown, it is a schematic diagram of the structure of the image processing device provided in the embodiment of the present application. The image processing device may include a first acquisition unit 201, a second acquisition unit 202, a first extraction unit 203, a second extraction unit 204 and an image generation unit 205, as follows:

[0082] A first acquisition unit 201 is used to acquire an object image, wherein the object image includes an object in a desired posture;

[0083] A second acquisition unit 202 is used to acquire, in response to the image generation instruction, a drawing model to which a trained style transfer model for a silhouette style is added, wherein the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model;

[0084] A first extraction unit 203 is used to extract posture features from the object image through a posture detection model to obtain posture features that match the expected posture;

[0085] The second extraction unit 204 is used to extract edge features of the object image through an edge detection model to obtain target edge features of the object image;

[0086] The image generation unit 205 is used to perform style conversion on the object image through a drawing model based on the target edge features and posture features to obtain a target object image with a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0087] In some embodiments, the second extraction unit 204 includes:

[0088] An edge detection subunit, used to perform edge detection on the object image through an edge detection model to obtain an edge image of the object image;

[0089] The edge adjustment subunit is used to perform edge adjustment processing on the edge image to obtain a target edge image;

[0090] The edge feature extraction subunit is used to extract edge features from the target edge image to obtain target edge features.

[0091] In some embodiments, the edge adjustment subunit includes a deletion module for;

[0092] The edge lines of non-target parts in the edge image are deleted to obtain a target edge image containing only the target part.

[0093] In some embodiments, the deletion module includes:

[0094] A display submodule, used for displaying edge images;

[0095] A deletion position acquisition submodule, for obtaining a deletion position corresponding to the edge line deletion operation in response to an edge line deletion operation on a non-target part in an edge image;

[0096] The first deletion submodule is used to identify the edge line to be deleted in the edge image at the non-target part according to the deletion position, and delete the edge line to be deleted;

[0097] or,

[0098] A deletion indication information acquisition submodule is used to acquire deletion indication information for non-target parts in the edge image;

[0099] A target region identification submodule, used for identifying a target region where a target part is located in the edge image based on the deletion indication information;

[0100] The second deletion submodule is used to delete edge lines of non-target parts outside the target area in the edge image.

[0101] In some embodiments, the image processing apparatus further includes:

[0102] A target edge line recognition unit, used to recognize a target edge line in a target area based on the target part, wherein the target edge line is an edge line in the target area that is invalid or interferes with the motion recognition of the target part;

[0103] The target edge line deletion unit is used to delete the target edge line in the target area to obtain a target edge image containing only the target part.

[0104] In some embodiments, the image generation unit 205 includes:

[0105] The feature fusion subunit is used to fuse the target edge features and posture features to obtain the control features, and input the control features into the drawing model;

[0106] The style transfer subunit is used to control the drawing model based on the control feature to perform style transfer on the object image.

[0107] In some embodiments, the style conversion subunit includes:

[0108] A feature extraction module, used to extract features from the object image using a drawing model, to obtain object image features corresponding to the object image, and silhouette style features corresponding to the silhouette style;

[0109] A fusion module is used to fuse the object image features and the silhouette style features to obtain the image features after style conversion;

[0110] The style transfer module is used to perform style transfer on the object image through a drawing model based on the control features and the image features after style transfer.

[0111] In some embodiments, the fusion module is used to:

[0112] Get the style weight corresponding to the drawing model;

[0113] Weighting the silhouette style features based on the style weights to obtain weighted silhouette style features;

[0114] The weighted silhouette style features and object image features are summed to obtain the image features after style conversion.

[0115] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0116] As can be seen from the above, the embodiment of the present application obtains an object image through the first acquisition unit 201, and the object image includes an object in an expected posture; the second acquisition unit 202 responds to the image generation instruction, obtains a drawing model to which a trained style transfer model for a silhouette style is added, and the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; the first extraction unit 203 extracts posture features of the object image through the posture detection model to obtain posture features that match the expected posture; the second extraction unit 204 extracts edge features of the object image through the edge detection model to obtain target edge features of the object image; the image generation unit 205 performs style conversion on the object image through the drawing model based on the target edge features and posture features to obtain a target object image with a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image. In this way, by superimposing a posture detection model and an edge detection model on the diffusion control network of the drawing model superimposed with the style transfer model, the posture detection model and the edge detection model are used to extract posture features and target edge features from the object image respectively, and a silhouette-style target object image is generated based on the posture features and the target edge features. It is possible to achieve precise control of the posture of the characters in the generated image and the action details of important areas, and generate a silhouette-style image that matches the expected character action and posture, thereby effectively improving the image generation effect.

[0117] The present application also provides a computer device, such as Figure 6 As shown, it shows a schematic diagram of the structure of a computer device involved in an embodiment of the present application, and the computer device may be a terminal or a server. Specifically:

[0118] The computer device 300 includes a processor 301 having one or more processing cores, a memory 302 having one or more computer-readable storage media, and a computer program stored in the memory 302 and executable on the processor. The processor 301 is electrically connected to the memory 302. Those skilled in the art will appreciate that the computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0119] The processor 301 is the control center of the computer device 300. It uses various interfaces and lines to connect the various parts of the entire computer device 300, executes various functions of the computer device 300 and processes data by running or loading software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, thereby monitoring the computer device 300 as a whole.

[0120] In the embodiment of the present application, the processor 301 in the computer device 300 will load instructions corresponding to the processes of one or more application programs into the memory 302 according to the following steps, and the processor 301 will run the application programs stored in the memory 302 to implement various functions:

[0121] Acquire an object image, wherein the object image includes an object in a desired posture;

[0122] In response to an image generation instruction, obtaining a drawing model to which a trained style transfer model for a silhouette style is added, the drawing model comprising a diffusion control network, the diffusion control network comprising a posture detection model and an edge detection model;

[0123] The posture feature of the object image is extracted through the posture detection model to obtain the posture feature matching the expected posture;

[0124] Extract edge features of the object image through the edge detection model to obtain target edge features of the object image;

[0125] Based on the target edge features and posture features, the object image is style-converted through a drawing model to obtain a target object image with a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0126] In some embodiments, edge features of an object image are extracted by an edge detection model to obtain target edge features of the object image, including:

[0127] Performing edge detection on the object image through an edge detection model to obtain an edge image of the object image;

[0128] Performing edge adjustment processing on the edge image to obtain a target edge image;

[0129] Edge features are extracted from the target edge image to obtain target edge features.

[0130] In some embodiments, performing edge adjustment processing on the edge image to obtain the target edge image includes:

[0131] The edge lines of non-target parts in the edge image are deleted to obtain a target edge image containing only the target part.

[0132] In some embodiments, deleting edge lines of non-target parts in the edge image includes:

[0133] Display edge images;

[0134] In response to an edge line deletion operation on a non-target portion in an edge image, obtaining a deletion position corresponding to the edge line deletion operation;

[0135] According to the deletion position, the edge line to be deleted in the edge image is identified in the non-target part, and the edge line to be deleted is deleted;

[0136] or,

[0137] Obtaining deletion instruction information for non-target parts in the edge image;

[0138] identifying a target area where the target part is located in the edge image based on the deletion indication information;

[0139] The edge lines of the non-target parts outside the target area in the edge image are deleted.

[0140] In some embodiments, after deleting the edge line of the non-target part outside the target area in the edge image, the method further includes:

[0141] Based on the target part, a target edge line is identified in the target area, where the target edge line is an edge line in the target area that is invalid or interferes with the motion recognition of the target part;

[0142] The target edge line in the target area is deleted to obtain a target edge image containing only the target part.

[0143] In some embodiments, based on the edge features and the posture features of the target, style conversion is performed on the object image by using a drawing model, including:

[0144] The target edge features and posture features are fused to obtain control features, and the control features are input into the drawing model;

[0145] Style transfer of object images is performed based on the control feature controlled drawing model.

[0146] In some embodiments, controlling the drawing model to perform style conversion on the object image based on the control feature includes:

[0147] The drawing model is used to extract features of the object image, and object image features corresponding to the object image and silhouette style features corresponding to the silhouette style are obtained;

[0148] The object image features and silhouette style features are combined to obtain the image features after style conversion;

[0149] Based on the control features and the image features after style transfer, the style of the object image is transferred through the drawing model.

[0150] In some embodiments, the object image features and the silhouette style features are fused to obtain the style-converted image features, including:

[0151] Get the style weight corresponding to the drawing model;

[0152] Weighting the silhouette style features based on the style weights to obtain weighted silhouette style features;

[0153] The weighted silhouette style features and object image features are summed to obtain the image features after style conversion.

[0154] The scheme can obtain an object image, which includes an object in a desired posture; in response to an image generation instruction, obtain a drawing model with a trained style conversion model for silhouette style added, the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; extract posture features of the object image through the posture detection model to obtain posture features matching the desired posture; extract edge features of the object image through the edge detection model to obtain target edge features of the object image; based on the target edge features and posture features, perform style conversion on the object image through the drawing model to obtain a target object image in silhouette style, and the target posture of the object in the target object image matches the desired posture of the object in the object image. In this way, by superimposing the posture detection model and the edge detection model on the diffusion control network of the drawing model superimposed with the style conversion model, respectively extracting posture features and target edge features from the object image using the posture detection model and the edge detection model, and generating a target object image in silhouette style based on the posture features and the target edge features, it is possible to achieve precise control of the posture of the person in the generated image and the action details of the important area, and generate a silhouette style image matching the desired person action and posture, thereby effectively improving the image generation effect.

[0155] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0156] Optional, such as Figure 6 As shown, the computer device 300 further includes: a touch screen 303, a radio frequency circuit 304, an audio circuit 305, an input unit 306, and a power supply 307. The processor 301 is electrically connected to the touch screen 303, the radio frequency circuit 304, the audio circuit 305, the input unit 306, and the power supply 307, respectively. Those skilled in the art can understand that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] The touch display screen 303 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 303 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user and various graphical user interfaces of the computer device, and these graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-EmittingDiode) and the like. The touch panel can be used to collect the user's touch operation on or near it (such as the user uses any suitable object or attachment such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 301, and can receive the command sent by the processor 301 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 301 to determine the type of touch event, and then the processor 301 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 303 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 303 can also be used as a part of the input unit 306 to realize the input function.

[0158] The radio frequency circuit 304 may be used to send and receive radio frequency signals, so as to establish wireless communication with a network device or other computer devices through wireless communication, and to send and receive signals between the network device or other computer devices.

[0159] The audio circuit 305 can be used to provide an audio interface between the user and the computer device through a speaker and a microphone. The audio circuit 305 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 305 and converted into audio data, and then the audio data is output to the processor 301 for processing, and then sent to another computer device through the radio frequency circuit 304, or the audio data is output to the memory 302 for further processing. The audio circuit 305 may also include an earphone jack to provide communication between an external headset and the computer device.

[0160] The input unit 306 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0161] The power supply 307 is used to supply power to various components of the computer device 300. Optionally, the power supply 307 can be logically connected to the processor 301 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 307 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0162] although Figure 6 Not shown, the computer device 300 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0163] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. It should be noted that the computer device provided in the embodiment of the present application and the image processing method in the above embodiment belong to the same concept. The specific implementation process is detailed in the above method embodiment, which will not be repeated here.

[0164] As can be seen from the above, the computer device provided in the embodiment of the present application can obtain an object image, wherein the object image includes an object in an expected posture; in response to an image generation instruction, obtain a drawing model to which a style conversion model trained for a silhouette style is added, wherein the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; perform posture feature extraction on the object image through the posture detection model to obtain a posture feature matching the expected posture; perform edge feature extraction on the object image through the edge detection model to obtain a target edge feature of the object image; based on the target edge feature and the posture feature, perform style conversion on the object image through the drawing model to obtain a target object image in a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image. In this way, by superimposing a posture detection model and an edge detection model on the diffusion control network of the drawing model superimposed with the style transfer model, the posture detection model and the edge detection model are used to extract posture features and target edge features from the object image respectively, and a silhouette-style target object image is generated based on the posture features and the target edge features. It is possible to achieve precise control of the posture of the characters in the generated image and the action details of important areas, and generate a silhouette-style image that matches the expected character action and posture, thereby effectively improving the image generation effect.

[0165] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0166] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the image processing methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0167] Acquire an object image, wherein the object image includes an object in a desired posture;

[0168] In response to an image generation instruction, obtaining a drawing model to which a trained style transfer model for a silhouette style is added, the drawing model comprising a diffusion control network, the diffusion control network comprising a posture detection model and an edge detection model;

[0169] The posture feature of the object image is extracted through the posture detection model to obtain the posture feature matching the expected posture;

[0170] Extract edge features of the object image through the edge detection model to obtain target edge features of the object image;

[0171] Based on the target edge features and posture features, the object image is style-converted through a drawing model to obtain a target object image with a silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

[0172] In some embodiments, edge features of an object image are extracted by an edge detection model to obtain target edge features of the object image, including:

[0173] Performing edge detection on the object image through an edge detection model to obtain an edge image of the object image;

[0174] Performing edge adjustment processing on the edge image to obtain a target edge image;

[0175] Edge features are extracted from the target edge image to obtain target edge features.

[0176] In some embodiments, performing edge adjustment processing on the edge image to obtain the target edge image includes:

[0177] The edge lines of non-target parts in the edge image are deleted to obtain a target edge image containing only the target part.

[0178] In some embodiments, deleting edge lines of non-target parts in the edge image includes:

[0179] Display edge images;

[0180] In response to an edge line deletion operation on a non-target portion in an edge image, obtaining a deletion position corresponding to the edge line deletion operation;

[0181] According to the deletion position, the edge line to be deleted in the edge image is identified in the non-target part, and the edge line to be deleted is deleted;

[0182] or,

[0183] Obtaining deletion instruction information for non-target parts in the edge image;

[0184] identifying a target area where the target part is located in the edge image based on the deletion indication information;

[0185] The edge lines of the non-target parts outside the target area in the edge image are deleted.

[0186] In some embodiments, after deleting the edge line of the non-target part outside the target area in the edge image, the method further includes:

[0187] Based on the target part, a target edge line is identified in the target area, where the target edge line is an edge line in the target area that is invalid or interferes with the motion recognition of the target part;

[0188] The target edge line in the target area is deleted to obtain a target edge image containing only the target part.

[0189] In some embodiments, based on the edge features and the posture features of the target, style conversion is performed on the object image by using a drawing model, including:

[0190] The target edge features and posture features are fused to obtain control features, and the control features are input into the drawing model;

[0191] Style transfer of object images is performed based on the control feature controlled drawing model.

[0192] In some embodiments, controlling the drawing model to perform style conversion on the object image based on the control feature includes:

[0193] The drawing model is used to extract features of the object image, and object image features corresponding to the object image and silhouette style features corresponding to the silhouette style are obtained;

[0194] The object image features and silhouette style features are combined to obtain the image features after style conversion;

[0195] Based on the control features and the image features after style transfer, the style of the object image is transferred through the drawing model.

[0196] In some embodiments, the object image features and the silhouette style features are fused to obtain the style-converted image features, including:

[0197] Get the style weight corresponding to the drawing model;

[0198] Weighting the silhouette style features based on the style weights to obtain weighted silhouette style features;

[0199] The weighted silhouette style features and object image features are summed to obtain the image features after style conversion.

[0200] The scheme can obtain an object image, which includes an object in a desired posture; in response to an image generation instruction, obtain a drawing model with a trained style conversion model for silhouette style added, the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; extract posture features of the object image through the posture detection model to obtain posture features matching the desired posture; extract edge features of the object image through the edge detection model to obtain target edge features of the object image; based on the target edge features and posture features, perform style conversion on the object image through the drawing model to obtain a target object image in silhouette style, and the target posture of the object in the target object image matches the desired posture of the object in the object image. In this way, by superimposing the posture detection model and the edge detection model on the diffusion control network of the drawing model superimposed with the style conversion model, respectively extracting posture features and target edge features from the object image using the posture detection model and the edge detection model, and generating a target object image in silhouette style based on the posture features and the target edge features, it is possible to achieve precise control of the posture of the person in the generated image and the action details of the important area, and generate a silhouette style image matching the desired person action and posture, thereby effectively improving the image generation effect.

[0201] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0202] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0203] Since the instructions stored in the computer-readable storage medium can execute the steps in any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0204] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0205] The above is a detailed introduction to an image processing method, apparatus, computer-readable storage medium and computer equipment provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Acquire an object image, wherein the object image includes an object in a desired posture; In response to the image generation instruction, a drawing model to which the trained style transfer model for the silhouette style is added is obtained ， The drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; Extracting posture features from the object image using the posture detection model to obtain posture features that match the expected posture; Extracting edge features of the object image using the edge detection model to obtain target edge features of the object image; Based on the target edge features and the posture features, the object image is style-converted by the drawing model to obtain the target object image in the silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

2. The image processing method according to claim 1, characterized in that: The step of extracting edge features from the object image by using the edge detection model to obtain target edge features of the object image includes: Performing edge detection on the object image by using the edge detection model to obtain an edge image of the object image; Performing edge adjustment processing on the edge image to obtain a target edge image; Edge features are extracted from the target edge image to obtain target edge features.

3. The image processing method according to claim 2, characterized in that: The performing edge adjustment processing on the edge image to obtain a target edge image includes: The edge lines of non-target parts in the edge image are deleted to obtain a target edge image containing only the target part.

4. The image processing method according to claim 3, characterized in that: The deleting of edge lines of non-target parts in the edge image includes: displaying the edge image; In response to an edge line deletion operation on a non-target portion in the edge image, obtaining a deletion position corresponding to the edge line deletion operation; Identifying the edge line to be deleted of the non-target part in the edge image according to the deletion position, and deleting the edge line to be deleted; or, Obtaining deletion instruction information for non-target parts in the edge image; identifying a target area where a target part is located in the edge image based on the deletion instruction information; The edge lines of the edge image that are located in non-target areas outside the target area are deleted.

5. The image processing method according to claim 4, characterized in that: After deleting the edge lines of the edge image in the non-target area outside the target area, the method further includes: identifying a target edge line in the target region based on the target part, the target edge line being an edge line in the target region that is invalid or interferes with motion recognition of the target part; The target edge line in the target area is deleted to obtain a target edge image containing only the target part.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The performing style conversion on the object image by using the drawing model based on the target edge feature and the posture feature comprises: Performing feature fusion on the target edge feature and the posture feature to obtain a control feature, and inputting the control feature into the drawing model; The drawing model is controlled based on the control feature to perform style conversion on the object image.

7. The image processing method according to claim 6, characterized in that: The step of controlling the drawing model to perform style conversion on the object image based on the control feature includes: Using the drawing model to perform feature extraction on the object image, obtaining object image features corresponding to the object image and silhouette style features corresponding to the silhouette style; Fusion the object image features and the silhouette style features to obtain style-converted image features; Based on the control feature and the style-converted image feature, style conversion is performed on the object image through the drawing model.

8. The image processing method according to claim 7, characterized in that: The step of fusing the object image feature and the silhouette style feature to obtain a style-converted image feature includes: Obtaining a style weight corresponding to the drawing model; weighting the silhouette style feature based on the style weight to obtain a weighted silhouette style feature; The weighted silhouette style features and the object image features are summed to obtain style-converted image features.

9. An image processing device, characterized in that: include: A first acquisition unit, configured to acquire an object image, wherein the object image includes an object in a desired posture; A second acquisition unit is used to acquire, in response to the image generation instruction, a drawing model to which a trained style conversion model for a silhouette style is added, wherein the drawing model includes a diffusion control network, and the diffusion control network includes a posture detection model and an edge detection model; A first extraction unit, configured to extract posture features from the object image using the posture detection model to obtain posture features matching the expected posture; A second extraction unit is used to extract edge features of the object image by using the edge detection model to obtain target edge features of the object image; An image generating unit is used to perform style conversion on the object image through the drawing model based on the target edge feature and the posture feature to obtain the target object image in the silhouette style, and the target posture of the object in the target object image matches the expected posture of the object in the object image.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the image processing method according to any one of claims 1 to 8.

11. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method according to any one of claims 1 to 8 when executing the computer program.

Images