Generation method and device of stylized photo image, and electronic equipment

Abstract

The invention relates to the field of AI images, in particular to a stylized photo image generation method and device and electronic equipment, and the method comprises the steps: pre-training a plurality of style models, and enabling any one style model to correspond to one style type; the style model is constructed through a text extraction model, a Stable Diffusion model and an attention mechanism model; inputting the original image into the text extraction model to extract text information associated with the face; inputting the original image into an attention mechanism model to obtain the key point information associated with the face; and inputting the text information and the key point information into the Stable Diffusion model to obtain the style image. Obtaining an original image; inputting the original image into the style model to obtain a style image; and performing posture adjustment on the style image to obtain a stylized photo image. According to the embodiment of the invention, the stylized photo image is generated by combining multiple models, the production is efficient, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of AI images, and more specifically, to a method, device, and electronic device for generating stylized photo images. Background Art

[0002] With the cross-development of artificial intelligence (AI) and computer vision, AI image technology has emerged, also known as artificial intelligence image processing technology, which refers to the technology of using artificial intelligence algorithms to analyze, process and understand image data. At present, in daily life, AI image technology is widely used in multiple social software. People can use AI image technology to generate pictures of the style they want. However, conventional AI image technology requires a lot of picture training in advance, the image generation speed is slow, the labor cost is relatively high, and the generated results are often blurred or distorted. Summary of the invention

[0003] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a method, device and electronic device for generating a stylized photo image, so as to achieve the effect of improving the efficiency of generating stylized photo images.

[0004] According to a first aspect of the present application, a method for generating a stylized photo image is provided, the method comprising:

[0005] A pre-trained style model, wherein the style models include multiple styles, and any style model corresponds to a style type;

[0006] Get the original image;

[0007] Inputting the original image into the style model to obtain a style image;

[0008] The posture of the style image is adjusted to obtain a stylized photo image.

[0009] The method for generating stylized photo images can realize the generation of stylized photo images by using an acquired original image in combination with multiple models and deep learning frameworks, and supports controlling the specified character posture based on the style photo. The generation method has smooth compatibility with style and posture control, and can parse character information from images of poor quality for portraits with exaggerated expressions, thereby ensuring the realism of the generated photo images while adding more visual levels and details. The generation quality of stylized photo images is improved by using a generation model based on deep learning in combination with advanced image processing algorithms.

[0010] Optionally, the step of adjusting the posture of the style image to obtain a stylized photo image includes:

[0011] Preset pose estimation model;

[0012] Inputting the original image into a posture estimation model to extract posture information;

[0013] constructing a corresponding skeleton image based on the posture information;

[0014] The skeleton image is fused with the style image to obtain the stylized photo image.

[0015] The posture estimation module extracts posture information from the input original image, and constructs a skeleton image with the extracted posture information, wherein the skeleton image represents the human body posture, and fuses and aligns the skeleton image with the style image to obtain a stylized photo image.

[0016] Optionally, the fusing the skeleton image with the style image to obtain a stylized photo image includes:

[0017] Extracting posture information features corresponding to the skeleton image;

[0018] Extracting information features of the style image;

[0019] The posture information features are aligned and fused with the information features of the style image to obtain a stylized photo image.

[0020] By aligning the posture information features with the information features of the style image, the detail expression can be improved and the feature representation can be enhanced. A stylized photo image can be obtained by fusion based on the feature alignment, which can optimize the reconstruction effect.

[0021] Optionally, inputting the original image into the style model to obtain a style image includes:

[0022] Select any one of the style models;

[0023] Inputting the original image into any one of the style models to extract key point information associated with the face and text information associated with the face;

[0024] Any one of the style models processes the key point information and the text information to obtain the style image.

[0025] The style model can obtain a style image corresponding to the style model by calculating and processing the key point information and the text information. The generation speed is fast and the style model has the ability to generate a unique style. The face information is sent into the model for processing in parallel with the text information as an independent condition, so that the generated picture has the ability to retain the ID and can correspond to the original image. The generation style of the original image is related to the selected style model.

[0026] Optionally, the style model is constructed by a text extraction model, a Stable Diffusion model and an attention mechanism model;

[0027] Inputting the original image into the text extraction model to extract text information associated with the face;

[0028] Inputting the original image into the attention mechanism model to obtain the key point information associated with the face;

[0029] The text information and the key point information are input into the Stable Diffusion model to obtain the style image.

[0030] The text extraction model is used to extract text information associated with the face in the original image, and the text information is used to describe the features of the original image, including detail features such as texture features of the original image; the attention mechanism model is used to obtain the key point information associated with the face, and the key point information includes eyes, eyebrows, mouth, nose, etc., which can describe the unique key points of the face; the Stable Diffusion model is used to generate a style model, which can generate a corresponding style image based on the text description associated with the face extracted by the text extraction model or user-defined input; the attention mechanism model dynamically adjusts the model's attention to different parts of the input data, so that the model can capture information more effectively and improve its ability to understand the context.

[0031] Optionally, the attention mechanism model includes a Transformer model, and the original image is input into the Transformer model to extract information to obtain facial information features as the key point information.

[0032] The Transformer model includes two parts: an encoder and a decoder. The encoder is used to process input data. The encoder consists of multiple encoder layers, each of which mainly includes a self-attention layer and a feed-forward network (FFN) layer; the decoder is used to output data processed by the encoder. The decoder is also composed of multiple decoder layers, each of which mainly includes a masked self-attention layer, an encoder-decoder attention layer, and a feed-forward network (FFN) layer. In summary, the encoder is responsible for understanding the input data and extracting features, while the decoder is responsible for generating output data based on these features.

[0033] Optionally, the attention mechanism model also includes an attention query model. Before the attention query model is embedded in the output layer of the Transformer model, the attention query model is used to perform feature projection extraction to obtain the facial information features.

[0034] A multi-head attention mechanism is established using the attention query model. The multi-head attention mechanism is combined with the Transformer model when performing feature extraction. The encoder processes the input data, and the decoder decodes and outputs the processed data based on the encoder. After the Transformer model performs image processing, it is connected to the attention query model. The multi-head attention mechanism established based on the attention query model divides the input data into multiple heads, and each head learns different aspects of the input sequence. The efficiency of extracting feature information is improved through the multi-head attention mechanism.

[0035] Optionally, the fusing the skeleton image with the style image to obtain the stylized photo image specifically includes:

[0036] Preprocessing the skeleton image to make it suitable for the input of the ControlNet model;

[0037] The preprocessed skeleton image and the style image are input into the ControlNet model for alignment and fusion to obtain the stylized portrait image.

[0038] The ControlNet model has powerful control capabilities and can accurately control the image generation process through a variety of detection methods (such as edge detection, depth estimation, posture detection, etc.), improve generation quality and diversity, enhance flexibility and controllability, and has a human posture detection function to expand the wide range of application scenarios of generated images.

[0039] According to a second aspect of the present application, a device for generating a stylized photo image is provided, comprising:

[0040] A pre-training module, used for pre-training a style model, wherein the style model includes a plurality of styles, and any style model corresponds to a style type;

[0041] An acquisition module, used for acquiring the original image;

[0042] An input module, used for inputting the original image into the style model to obtain a style image;

[0043] The adjustment module is used to adjust the posture of the style image to obtain a stylized photo image.

[0044] The modules in the generating device cooperate with each other to realize the generation of stylized photo images. The pre-training module is used to pre-train the style model so that the style model has the ability to quickly perform style processing on the image. The pre-trained models include multiple ones, and one style model corresponds to one style type. After the acquisition module acquires the original image, the original image is input into any selected style model through the input module, so that the generated image is given the desired style type. The original image is first input into the style model for style processing to obtain the style image, and then the adjustment module is used to adjust the posture of the style image, so as to generate a stylized photo image.

[0045] According to a third aspect of the present application, an electronic device is provided, including:

[0046] a memory for storing one or more computer programs;

[0047] The processor implements the method for generating a stylized photo image according to the first aspect when the one or more computer programs are executed by the processor.

[0048] The electronic device provides a complete system for generating stylized photo images, stores images, processes the stored images, and stores the processed images in a memory.

[0049] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for generating a stylized photo image described in the first aspect when executed.

[0050] Based on any of the above aspects, the method, device, electronic device and computer storage medium for generating stylized photo images provided by the embodiments of the present application can improve the generation efficiency of stylized photo images and reduce costs. By using multiple models and deep learning frameworks, the time required for a single model is reduced to achieve the generation of stylized photo images, and on the basis of the style photo, it supports controlling the specified character posture, improves the image generation speed and reduces the corresponding cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of an application scenario of the method for generating a stylized photo image provided in this embodiment.

[0053] Figure 2 This is a flowchart of the method for generating a stylized photo image provided in this embodiment.

[0054] Figure 3 A schematic diagram of the functional modules of the apparatus for generating a stylized photo image provided in this embodiment.

[0055] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0056] The drawings of this application are only used for illustrative purposes and should not be construed as limiting the present application. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] Existing stylized photo image technology requires a lot of time to prepare the shooting background and portrait makeup, and use special photography techniques and post-processing software for detailed photo repair; or in order to make the generated image more stable, it has high requirements for portrait images and needs to prepare clear frontal images with multiple angles, multiple expressions, multiple lights, and multiple backgrounds without obstructions or blur. It takes a lot of time and energy to prepare high-quality portrait pictures, and it takes a long time to wait for the model training to be completed, and the cost of generating images is high. Traditional portrait generation methods usually require a lot of training, and the generated results often have a certain degree of blur or distortion, so a simple solution is needed to solve the original problem to reduce labor costs while improving the speed and quality of generation.

[0060] This embodiment provides a technical solution that can solve the above-mentioned problem. The specific implementation methods of this application are described in detail below in conjunction with the accompanying drawings.

[0061] For example, a schematic diagram of an application scenario of a method for generating a stylized photo image provided in an embodiment of the present application is shown. Figure 1 As shown, the application scenario at least includes a server 100 and a terminal 200 that can communicate with the server 100. The server 100 has the function of acquiring images and processing images; the terminal device 200 has the function of processing and displaying images.

[0062] It is understandable that the server 100 can be an independent electronic device or a cluster of multiple electronic devices; the terminal 200 can be a smart phone terminal, a personal computer, a tablet computer, a car terminal, etc., but is not limited thereto.

[0063] In one practicable manner, the server 100 and the terminal 200 may respectively execute the method for generating a stylized photo image provided in the embodiment of the present application, or, optionally, the method for generating a stylized photo image provided in the embodiment of the present application is partially executed in the server 100 and partially executed in the terminal 200.

[0064] like Figure 2 As shown, this embodiment provides a method for generating a stylized photo image, which may include the following steps:

[0065] S110 , pre-training a style model, wherein the style model includes a plurality of style models, and any style model corresponds to a style type.

[0066] In this embodiment, one style model may correspond to multiple styles, or multiple style models may correspond to multiple styles, that is, one style model corresponds to one style. After the style model is established, it is necessary to pre-train the style model. The pre-training can enable the style model to have the ability to generate style images and improve the speed of image generation.

[0067] In an optional implementation, multiple style models are preset, each style model corresponds to a style, and the preset multiple models are trained to enable them to have the ability to generate images. In the specific implementation process, multiple models are combined to obtain a style model, and the style model is constructed by a text extraction model, a Stable Diffusion model and an attention mechanism model. The original image first enters the text extraction model to extract text information associated with the face; then the original image is input into the attention mechanism model to obtain key point information associated with the face, and finally the text information and the key point information are input into the Stable Diffusion model to obtain the style image. The style image generation part of the style model mainly uses the Stable Diffusion model to generate images. The Stable Diffusion model includes three main components: CLIP Text Encoder, U-Net, and Auto Encoder Decoder, wherein CLIP Text Encoder is used for text encoding, U-Net is used to process image information in latent space, and Auto Encoder Decoder is an automatic encoder, which uses the information processed by U-Net to decode and draw the final style image. The specific form of the style model includes but is not limited to this.

[0068] S120: Acquire an original image.

[0069] In this embodiment, the original image is obtained, and the original image may be one or more images.

[0070] In an optional implementation, an original image is obtained, and the original image is input into any one of the style models, wherein one style model corresponds to one style, and the style model processes the original image to obtain a corresponding style image, and the number of original images selected includes but is not limited to this.

[0071] S130: Input the original image into the style model to obtain a style image.

[0072] In this embodiment, the style model may be an independent model or a combination of multiple models.

[0073] In an optional implementation, the style model includes a text extraction model, a Stable Diffusion model and an attention mechanism model, and the style image is generated using the original image. Specifically, the original image is first input into the text extraction model to extract text information associated with the face; then the original image is input into the attention mechanism model to obtain the key point information associated with the face, a face feature extractor based on the Transformer model is built into the attention mechanism model, and a multi-head attention mechanism is set in the Transformer model. The multi-head attention mechanism is based on the attention query model, the face feature extractor extracts all tokens of the penultimate layer of the Transformer model, and then connects to the attention query model to perform feature projection to extract ID features, so that the entire control process controls the face ID information instead of the entire face image in a targeted manner. While extracting the key point information, the correspondence of the image processing is also guaranteed. Finally, the text information and the key point information are input into the Stable Diffusion model to obtain the style image; the Stable Diffusion model obtains the text information and key point information, wherein the CLIP Text Encoder encodes text information into an embedded vector, and the embedded vector and the key point information are input into U-Net. At the same time, the embedded vector corresponding to the text information is used to condition the image generation process. U-Net gradually converts the vector into a potential image representation related to the text information through a diffusion process. Auto EncoderDecoder decodes the potential image representation into a final pixel image and outputs a style image matching the required style type. The Stable Diffusion model has been pre-trained and has the ability to produce corresponding style images. The user selects a favorite style type and inputs the original image into the corresponding style model for processing, thereby obtaining an image of the selected style type. The method of generating a style image includes but is not limited to this.

[0074] S140: Adjust the posture of the style image to obtain a stylized portrait image.

[0075] In this embodiment, the posture of the style image obtained by processing the style model is adjusted, and the adjustment method can use various image editing tools or models.

[0076] In an optional implementation, a ControlNet model is used for posture adjustment. In the specific implementation, a posture estimation model is first preset, and the posture estimation model is used to estimate the posture of the original image. Specifically, the original image is input into the posture estimation model to extract key point information, and the key point information is used to describe the posture information of the human body; a corresponding skeleton image is constructed based on the key point information, and the ControlNet model aligns the skeleton image with the output of the corresponding style model to ensure that the posture information can affect the decision-making process of the generation model at an appropriate level. The ControlNet model designs a specific fusion mechanism, which can combine the posture information with the features of the generation model at different levels to ensure that the generated image is both consistent with the posture information and has high quality. At the same time, the ControlNet model has an adaptive adjustment structure, which can dynamically adjust the influence of the posture information according to different stages of generation. Through the cooperation of the internal structure of the ControlNet model, the posture of the style image can be adjusted. The structure of the ControlNet model is not specifically limited.

[0077] In the specific implementation process, in order to adapt to the processing of the ControlNet model, the skeleton image needs to be preprocessed after it is obtained to obtain a format image that is adapted to the processing of the ControlNet model. Specifically, the skeleton image is converted into a binary image, a heat map or a vector field to adapt to the processing of the ControlNet model.

[0078] The technical solution of the method for generating stylized photo images provided by the present application is described below in conjunction with a specific embodiment.

[0079] like Figure 3 As shown, the embodiment of the present application further provides a device 310 for generating a stylized photo image. Optionally, the device 310 for generating a stylized photo image may include:

[0080] A pre-training module 311 is used for pre-training a style model, wherein the style model includes a plurality of styles, and any style model corresponds to a style type;

[0081] In this embodiment, the pre-training module 311 can be used to perform Figure 2 As shown in step S110, for a detailed description of the pre-training module 311, reference may be made to the description of step S110.

[0082] An acquisition module 312 is used to acquire an original image;

[0083] In this embodiment, the acquisition module 312 can be used to execute Figure 2As shown in step S120, for a detailed description of the acquisition module 312, reference may be made to the description of step S120.

[0084] An input module 313 inputs the original image into the style model to obtain a style image;

[0085] In this embodiment, the input module 313 can be used to execute Figure 2 As shown in step S130, for the specific description of the input module 313, reference may be made to the description of the step S130.

[0086] The adjustment module 314 performs posture adjustment on the style image to obtain a stylized photo image.

[0087] In this embodiment, the adjustment module 314 can be used to perform Figure 2 As shown in step S140 , for a detailed description of the adjustment module 314 , reference may be made to the description of step S140 .

[0088] It can be understood that the above-mentioned device embodiment and the above-mentioned method embodiment can correspond to each other, and similar descriptions of the device embodiment can refer to the method embodiment. To avoid repetition, it will not be repeated here. A stylized photo image generation device provided in an embodiment of the present application can execute a stylized photo image generation method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method. The functional modules of the stylized photo image generation device can be implemented in the form of hardware, can be implemented by instructions in the form of software, and can also be implemented by a combination of hardware and software modules.

[0089] Specifically, each step of the method embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor. The steps of the method for generating stylized photo images in combination with the embodiment of the present application can be directly embodied as being executed by a hardware encoding processor, or being executed by a combination of hardware and software modules in the encoding processor. Optionally, the software module can be located in a random access memory, a read-only memory, a programmable read-only memory, a flash memory, an electrically erasable programmable memory, a register, or other storage media. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in combination with its hardware.

[0090] The present application embodiment provides an electronic device 410, whose structure is as follows: Figure 4 The electronic device 410 may be Figure 1 The server 100 or the terminal 200 is shown.

[0091] like Figure 4As shown, the electronic device 410 includes a memory 411, a processor 412, a communication module 413 and an input / output interface 414, etc. Optionally, the memory 411, the processor 412, the communication module 413 and the input / output interface 414 can be connected and communicated through a bus 415.

[0092] The memory 411 is used to store one or more computer programs and transmit the code of the computer program to the processor 412; when the one or more computer programs are executed by the processor 412, the xx method in the embodiment of the present application is implemented.

[0093] Optionally, the electronic device 410 can be connected to a network via a communication module 413 to communicate with other devices, such as a terminal or a server, through the network to achieve data interaction. The electronic device 410 can be various forms of digital computers, such as desktop computers, servers, workstations, mainframe computers or other types of computers. The electronic device 410 can also be various forms of mobile terminals, such as smart phones, tablet computers, wearable devices (such as helmets, glasses, watches, etc.) and other similar mobile terminals.

[0094] Optionally, the electronic device 410 can be connected to the required input / output devices, such as a keyboard, a display device, etc., through the input / output interface 414. The electronic device 410 itself can have a display device, and can also be connected to other display devices through the input / output interface 414. Optionally, a storage device, such as a hard disk, can also be connected through the input / output interface 414, so that data in the electronic device 410 can be stored in the storage device, or data in the storage device can be read, and data in the storage device can also be stored in the memory 411. It can be understood that the input / output interface 414 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 414 can be a component of the electronic device 410, or it can be an external device connected to the electronic device 410 when needed.

[0095] Optionally, the memory 411 may be a volatile memory and / or a non-volatile memory, the volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory or a flash memory, etc.

[0096] Optionally, the computer program stored in the processor 412 may be divided into one or more modules, which are stored in the memory 411 and executed by the processor 412 to complete the method provided in the embodiment. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the computer program instruction segments are used to describe the execution process of the computer program in the electronic device 410.

[0097] Optionally, the processor 412 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 412 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special artificial intelligence computing chips, various processors running machine learning model algorithms, and may also be any appropriate controller, microcontroller, processor, etc. The processor 412 executes the various methods and processes of this embodiment, illustratively, such as a method for generating a stylized photo image in an embodiment of the present application.

[0098] Optionally, the bus 415 may include a path for transmitting information. The bus 415 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. According to different functions, the bus 415 may be divided into an address bus, a data bus, a control bus, etc.

[0099] In an optional implementation, the embodiment of the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. Part or all of the computer program can be loaded and / or installed on the memory 411 of the electronic device 410. When the computer program is executed by the processor 412, one or more steps of a method for generating a stylized photo image in the embodiment of the present application can be performed.

[0100] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or the like.

[0101] Obviously, the above embodiments of the present application are merely examples for clearly illustrating the technical solution of the present application, and are not intended to limit the specific implementation methods of the present application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A method for generating a stylized photo image, characterized in that: The method comprises: A pre-trained style model, wherein the style models include multiple styles, and any style model corresponds to a style type; Get the original image; Inputting the original image into the style model to obtain a style image; The posture of the style image is adjusted to obtain a stylized photo image.

2. The method for generating a stylized photo image according to claim 1, characterized in that: The step of adjusting the posture of the style image to obtain a stylized photo image includes: Preset pose estimation model; Inputting the original image into a posture estimation model to extract posture information; constructing a corresponding skeleton image based on the posture information; The skeleton image is fused with the style image to obtain the stylized photo image.

3. The method for generating a stylized photo image according to claim 2, characterized in that: The step of fusing the skeleton image with the style image to obtain a stylized photo image includes: Extracting posture information features corresponding to the skeleton image; Extracting information features of the style image; The posture information features are aligned and fused with the information features of the style image to obtain a stylized photo image.

4. The method for generating a stylized photo image according to claim 1, characterized in that: The step of inputting the original image into the style model to obtain a style image includes: Select any one of the style models; Inputting the original image into any one of the style models to extract key point information associated with the face and text information associated with the face; Any one of the style models processes the key point information and the text information to obtain the style image.

5. The method for generating a stylized photo image according to claim 4, characterized in that: The style model is constructed by a text extraction model, a Stable Diffusion model and an attention mechanism model; Inputting the original image into the text extraction model to extract text information associated with the face; Inputting the original image into the attention mechanism model to obtain the key point information associated with the face; The text information and the key point information are input into the Stable Diffusion model to obtain the style image.

6. The method for generating a stylized photo image according to claim 5, characterized in that: The attention mechanism model includes a Transformer model, and the original image is input into the Transformer model to extract information to obtain facial information features as the key point information.

7. The method for generating a stylized photo image according to claim 6, characterized in that: The attention mechanism model also includes an attention query model. Before the attention query model is embedded in the output layer of the Transformer model, the attention query model is used to perform feature projection extraction to obtain the facial information features.

8. A method for generating a stylized photo image according to any one of claims 1 to 7, characterized in that: The fusing the skeleton image with the style image to obtain the stylized photo image specifically includes: Preprocessing the skeleton image to make it suitable for the input of the ControlNet model; The preprocessed skeleton image and the style image are input into the ControlNet model for alignment and fusion to obtain the stylized portrait image.

9. A device for generating a stylized photo image, characterized in that: The device comprises: A pre-training module, used for pre-training a style model, wherein the style model includes a plurality of styles, and any style model corresponds to a style type; An acquisition module, used for acquiring the original image; An input module, used for inputting the original image into the style model to obtain a style image; The adjustment module is used to adjust the posture of the style image to obtain a stylized photo image.

10. An electronic device, characterized in that: include: a memory for storing one or more computer programs; A processor, when the one or more computer programs are executed by the processor, implements the method for generating a stylized photo image as described in any one of claims 1 to 7.

Images