AIGC-based image generation methods, systems, and imaging devices

Abstract

This application discloses an AIGC-based image generation method, system, and imaging device, relating to the field of image generation. It performs parallel deep analysis on user images and style template images, simultaneously extracting high-fidelity constraints such as user identity and pose, as well as multi-dimensional stylized guidance information including clothing, background, and lighting. Furthermore, it uses prior lighting information as a pre-constraint, injecting the lighting environment of the target scene during the foreground subject generation stage, and simultaneously constructing spatial depth information of the foreground figure. Finally, it utilizes this depth information to provide precise spatial constraints and guidance for the background fusion process, and combines it with a high-fidelity inpainting network for detail enhancement, thereby generating a realistic image where the foreground and background are highly consistent in lighting and space, and user identity features are preserved. This enables adaptive control of lighting and spatial relationships during the generation process, effectively improving the fusion quality and visual realism of the final image.

Description

Technical Field

[0001] This application relates to the field of photographic image generation, and more specifically, to a photographic image generation method, system, and photographic device based on AIGC. Background Technology

[0002] With the booming development of digital media and social networks, users' demand for personalized, high-quality visual content is increasing. While traditional photography can produce high-quality images, its processes often involve high equipment costs, professional personnel, specific location selection, and time-consuming post-production, which greatly limits the freedom and efficiency of ordinary users in creating creative images. Against this backdrop, generative artificial intelligence (AIGC) technology, especially the breakthrough of diffusion models in the field of image generation, provides a new technological path for virtual photography and personalized image creation. By utilizing AIGC technology, users only need to provide simple personal photos and style references to potentially generate artistic works comparable to professional photography, greatly lowering the threshold for high-quality image creation and demonstrating broad application prospects and commercial value.

[0003] However, existing technologies still face numerous challenges in applying AIGC technology to high-fidelity photo image generation. Specifically, some mainstream solutions tend to employ a phased generation strategy. First, in a separate stage, a core foreground figure is generated against a relatively neutral or simplified background based on the user-provided pose information and identity features. Then, in a second stage, this generated figure image is used as input to an image-to-image transformation model, combined with background and atmosphere elements from the target style template for secondary rendering and fusion. While this simplifies control to some extent, its inherent decoupling characteristics also bring insurmountable drawbacks. Because the foreground figure generation process lacks prior knowledge of the final scene's lighting environment and spatial layout, the figure's lighting effects and the subsequently integrated background are difficult to match naturally, often resulting in contradictory lighting directions and missing ambient light reflections, giving the final image a noticeable "pasting" feel. Furthermore, the second-stage stylization process may also cause uncontrollable disturbances to the already accurately generated facial features and pose details, thus weakening the fidelity of the user's identity features and making it difficult to balance identity consistency with the realism of environmental integration.

[0004] Therefore, there is an urgent need for an optimized AIGC-based image generation method, system, and imaging device. Summary of the Invention

[0005] This application is made in order to solve the above-mentioned technical problems.

[0006] According to one aspect of this application, an AIGC-based image generation method is provided, comprising: acquiring a user image input by a user and a style template image selected by the user; extracting user pose skeletons and facial feature vectors from the user image; performing parameter parsing on the style template image to obtain clothing description prompts, background description prompts, and lighting description prompts; generating a foreground subject with lighting prior based on the clothing description prompts and lighting description prompts, using the user pose skeletons and facial feature vectors to obtain a foreground person image, a foreground person mask, and a foreground person depth map; performing depth-guided background fusion on the foreground person image, foreground person mask, and foreground person depth map based on the background description prompts to obtain a fused image; and performing high-fidelity detail restoration on the fused image to obtain an enhanced image.

[0007] According to another aspect of this application, an AIGC-based image generation system is provided, comprising: a user input acquisition module for acquiring a user image input by the user and a style template image selected by the user; a user feature extraction module for extracting user pose skeletons and facial feature vectors from the user image; a parameter parsing module for parsing the style template image to obtain clothing description prompts, background description prompts, and lighting description prompts; a foreground subject generation module for generating a foreground subject with lighting prior based on the clothing description prompts and lighting description prompts, using the user pose skeletons and facial feature vectors to obtain a foreground person image, a foreground person mask, and a foreground person depth map; a background fusion module for performing depth-guided background fusion on the foreground person image, foreground person mask, and foreground person depth map based on the background description prompts to obtain a fused image; and a high-fidelity detail restoration module for performing high-fidelity detail restoration on the fused image to obtain an enhanced image.

[0008] According to another aspect of this application, a photographing device is provided, comprising: the photographing device being capable of performing the AIGC-based photographic image generation method as described above.

[0009] Compared with existing technologies, this application provides an AIGC-based image generation method, system, and imaging device. It performs parallel deep analysis on user images and style template images, simultaneously extracting high-fidelity constraints such as user identity and pose, as well as multi-dimensional stylized guidance information including clothing, background, and lighting. Furthermore, it uses prior lighting information as a pre-constraint, injecting the lighting environment of the target scene during the foreground subject generation stage and simultaneously constructing the spatial depth information of the foreground figure. Finally, it utilizes this depth information to provide precise spatial constraints and guidance for the background fusion process, and combines it with a high-fidelity inpainting network for detail enhancement, thereby generating a realistic image where the foreground and background are highly consistent in lighting and space, and user identity features are preserved. This enables adaptive control of lighting and spatial relationships during the generation process, effectively improving the fusion quality and visual realism of the final image. Attached Figure Description

[0010] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is a flowchart of an AIGC-based image generation method according to an embodiment of this application.

[0012] Figure 2 This is a data flow diagram of the AIGC-based image generation method according to an embodiment of this application.

[0013] Figure 3 This is a flowchart of sub-step S2 of the AIGC-based image generation method according to an embodiment of this application.

[0014] Figure 4 This is a flowchart of sub-step S3 of the AIGC-based image generation method according to an embodiment of this application.

[0015] Figure 5 This is a flowchart of sub-step S33 of the AIGC-based image generation method according to an embodiment of this application.

[0016] Figure 6 This is a flowchart of sub-step S4 of the AIGC-based image generation method according to an embodiment of this application.

[0017] Figure 7 This is a flowchart of sub-step S42 of the AIGC-based image generation method according to an embodiment of this application.

[0018] Figure 8 This is a flowchart of sub-step S5 of the AIGC-based image generation method according to an embodiment of this application.

[0019] Figure 9 This is a flowchart of sub-step S6 of the AIGC-based image generation method according to an embodiment of this application.

[0020] Figure 10 This is a block diagram of an AIGC-based image generation system according to an embodiment of this application. Detailed Implementation

[0021] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] To address the problems mentioned above in the background technology, this application proposes a method for generating images based on AIGC. Figure 1 This is a flowchart of an AIGC-based image generation method according to an embodiment of this application. Figure 2 This is a data flow diagram of an AIGC-based image generation method according to an embodiment of this application. For example... Figure 1 and Figure 2 As shown, the AIGC-based image generation method includes the following steps: S1, acquiring a user image input by the user and a style template image selected by the user; S2, extracting user pose skeleton and facial feature vector from the user image; S3, parsing parameters of the style template image to obtain clothing description prompts, background description prompts, and lighting description prompts; S4, based on the clothing description prompts and lighting description prompts, generating a foreground subject with lighting priors from the user pose skeleton and facial feature vectors to obtain a foreground person image, a foreground person mask, and a foreground person depth map; S5, based on the background description prompts, performing depth-guided background fusion on the foreground person image, foreground person mask, and foreground person depth map to obtain a fused image; S6, performing high-fidelity detail restoration on the fused image to obtain an enhanced image.

[0023] In the aforementioned AIGC-based image generation method, step S1 involves acquiring a user-inputted image and a style template image selected by the user. It should be understood that, to ensure the AIGC-driven image generation process has a clear content subject and style orientation, and to avoid deviations from user expectations due to a single image input, this application acquires an image carrying the core content input by the user and a template image defining the artistic style selected by the user, providing a clear source of content features and a reference for style features for subsequent models. This ensures that the generated image retains the key content structure of the user image while accurately matching the style characteristics expected by the user, providing accurate data support for subsequent model processing and avoiding content deviations or style inconsistencies in the generated results. Specifically, the user can take a photo or select an image from local storage, and the system performs preprocessing such as resolution correction and noise reduction. Simultaneously, the user can select a style template image from the system's style library (e.g., categorized by realism, animation, etc.), and the system reads and caches the template image data.

[0024] In the aforementioned AIGC-based image generation method, step S2 involves extracting the user's pose skeleton and facial feature vector from the user image. It should be understood that since AIGC-based image generation needs to simultaneously ensure the accuracy of the user's pose reconstruction and the uniqueness of their identity, a lack of accurate extraction of pose and facial features can easily lead to problems such as pose deviation from the original action and low recognizability of the face compared to the user. Therefore, this application further extracts the pose skeleton representing the human body's action structure and the facial feature vector uniquely identifying the user from the user image, providing core constraints for the subsequent AIGC model generation process. This ensures that the subsequent foreground subject generation strictly follows the user's original pose, avoiding limb misalignment, while accurately preserving the user's unique facial features to prevent facial distortion, laying the foundation for generating images with both pose accuracy and identity fidelity.

[0025] In particular, in one specific embodiment, Figure 3 This is a flowchart of sub-step S2 of the AIGC-based image generation method according to an embodiment of this application. Figure 3 As shown, step S2 includes: S21, inputting the user image into the pose extraction module to obtain the user pose skeleton, the pose extraction module being the Open Pose model; S22, inputting the user image into the face feature extraction module to obtain the face feature vector, the face feature extraction module being the Insight Face model.

[0026] Specifically, in step S21, the user image is input into the pose extraction module to obtain the user pose skeleton. The pose extraction module is the Open Pose model. More specifically, this application further inputs the user image into the Open Pose model, which, through a multi-stage convolutional neural network, can efficiently locate key joints such as the eyes, nose, shoulders, and elbows, calculate their two-dimensional coordinates and confidence levels, and finally output structured data containing joint coordinates and skeletal connection relationships, providing accurate pose constraints for subsequent generation.

[0027] Specifically, in step S22, the user image is input into a face feature extraction module to obtain a face feature vector. This face feature extraction module is the Insight Face model. It should be understood that, to avoid identity confusion due to insufficient feature dimensions or noise interference, this application further inputs the user image into the Insight Face model. This model, through deep metric learning, can effectively capture subtle and unique facial features. Specifically, the face image is first detected and cropped using algorithms such as MTCNN, then aligned and normalized before being input into the Insight Face model. This transforms the image into a high-dimensional (e.g., 512-dimensional) face feature vector that uniquely represents the user's identity, ensuring the consistency of the generated face's identity.

[0028] In the aforementioned AIGC-based image generation method, step S3 involves parsing the style template image to obtain clothing description prompts, background description prompts, and lighting description prompts. It should be understood that since AIGC-based image generation requires obtaining multi-dimensional style information from the style template, directly using the entire template can easily lead to interference between clothing, background, and lighting features, resulting in inconsistent styles in the subsequently generated images. Therefore, this application decomposes and parses the style template image, extracting clothing, background, and lighting features separately and converting them into corresponding clothing description prompts, background description prompts, and lighting description prompts, thereby providing multi-dimensional style guidance for foreground generation and background fusion.

[0029] In particular, in one specific embodiment, Figure 4 This is a flowchart of sub-step S3 of the AIGC-based image generation method according to an embodiment of this application. Figure 4 As shown, step S3 includes: S31, performing foreground-background decoupling on the style template image to obtain a character mask, a segmented character image, and a segmented background image; S32, inputting the segmented character image and the segmented background image into an image-text conversion model to obtain clothing description prompts and background description prompts; S33, performing global illumination environment analysis and quantization on the style template image to obtain illumination description prompts.

[0030] Specifically, step S31 involves foreground-background decoupling of the style template image to obtain a character mask, a segmented character image, and a segmented background image. It should be understood that to avoid background interference during subsequent clothing feature extraction and to prevent background feature extraction from containing character information, thus reducing feature accuracy, this application further decouples the style template image from the foreground and background. For example, the style template image can be input into a general segmentation model (such as the Segment Anything Model, SAM). This type of model can automatically identify and accurately outline the main character in the image, outputting a binary character mask. Based on this mask, pixel-level operations can separate the original template image into a segmented character image containing only the main character and a segmented background image with the background removed (or filled with a neutral color), achieving independent processing of the character and background. This ensures that there is no background interference during subsequent clothing feature extraction and no character residue during background feature extraction. Simultaneously, the character mask can be used for main character protection during background fusion, improving the accuracy of each processing step and laying the foundation for subsequent feature extraction.

[0031] Specifically, in step S32, the segmented person image and the segmented background image are input into an image-to-text conversion model to obtain clothing description prompts and background description prompts, respectively. Specifically, to achieve accurate textual description of the image content, this application further inputs the segmented image into an image-to-text conversion model based on an encoder-decoder architecture. This model first extracts the visual content of the input image into a series of high-level feature embeddings using its visual encoder (such as Vision Transformer, ViT). Subsequently, its language decoder (such as a GPT-based structure) decodes these visual features as contextual conditions in an autoregressive manner, thereby generating descriptive natural language text. For the segmented person image, the model is guided to focus on details such as the material, cut, color, and decoration of the clothing to generate clothing description prompts. For the segmented background image, the model generates background description prompts describing information such as scene type, core elements, and environmental atmosphere.

[0032] Specifically, step S33 involves performing global illumination environment analysis and quantification on the style template image to obtain illumination description prompts. It should be understood that since illumination is a core factor affecting the visual realism of an image, describing the illumination of the style template solely through visual observation can easily lead to vague descriptions, resulting in significant deviations between the subsequently generated illumination effects and the template. Therefore, this application further performs global illumination environment analysis and quantification on the style template image, converting illumination parameters into precise text prompts to provide a clear basis for illumination simulation during subsequent foreground generation. This ensures that the illumination of the subsequently generated foreground figures (such as highlight position, shadow length, and color tendency) perfectly matches the global illumination of the template, avoiding contradictions in illumination direction and color temperature. Simultaneously, quantification analysis improves the accuracy of illumination description, enabling the AIGC model to accurately reproduce the light and shadow atmosphere of the template and enhance the visual realism of the generated image.

[0033] In particular, in one specific embodiment, Figure 5 This is a flowchart of sub-step S33 of the AIGC-based image generation method according to an embodiment of this application. Figure 5 As shown, step S33 includes: S331, inputting the style template image into a pre-trained illumination estimation neural network to obtain an illumination feature vector; S332, inputting the illumination feature vector into a preset mapping function to obtain the illumination description prompt.

[0034] More specifically, in step S331, the style template image is input into a pre-trained illumination estimation neural network to obtain an illumination feature vector. Specifically, this application further inputs the style template image into a pre-trained illumination estimation model built on a deep convolutional neural network. This model, trained on a dataset containing a large number of images and their corresponding real-world illumination parameters, learns to regress the illumination information of a three-dimensional scene from a single two-dimensional image. In one possible embodiment, the output illumination feature vector can be a set of spherical harmonic function coefficients. This set of coefficients can compactly and accurately encode the distribution of ambient light from various directions in the scene, including the direction and intensity of the main light source and complex illumination effects such as ambient diffuse reflection. This ensures that the extracted illumination features cover subtle differences in the light source, avoiding subjective biases from manual analysis. Simultaneously, the high-dimensional feature vector can accurately represent the global distribution of illumination, laying a data foundation for subsequent mapping to text prompts.

[0035] More specifically, in step S332, the illumination feature vector is input into a preset mapping function to obtain the illumination description prompts. It should be understood that since the illumination feature vector is high-dimensional numerical data, the AIGC model cannot directly recognize it and use it for style guidance; it needs to be converted into structured text prompts to play a guiding role. Therefore, this application further inputs the illumination feature vector into a preset, rule-based mapping function. This mapping function contains a set of quantization-to-text conversion rules to establish a precise correspondence between numerical parameters and text descriptions. Specifically, the mapping function first parses the input illumination feature vector into multiple numerical components, including light source direction, light source intensity, and light source color temperature. Then, a lookup table and matching are performed for each component: the light source direction component is looked up in a preset angle-direction dictionary; the light source intensity component is mapped to the description "bright" or "intense" based on its numerical range (e.g., 0.8-1.0); and the light source color temperature component is mapped to cool tones, warm tones, or specific color words, such as "sunset orange light," based on its numerical value. Finally, the text fragments obtained from the conversion of each component are combined to form a coherent and structured lighting description prompt, thereby realizing the conversion of lighting quantization data into AIGC-recognizable text.

[0036] In the aforementioned AIGC-based image generation method, step S4 involves generating a foreground subject with lighting priors based on clothing description prompts and lighting description prompts, using the user's pose skeleton and facial feature vectors to obtain a foreground subject image, a foreground subject mask, and a foreground subject depth map. It should be understood that in traditional AIGC image generation, the foreground subject's clothing and lighting are often disconnected from the subsequent background, easily leading to pose deviations and low facial recognition. Furthermore, background fusion requires subject boundary and spatial depth information; lacking these can result in fusion distortion. Therefore, this application further uses clothing prompts to define the style and lighting prompts to annotate the lighting priors, combining pose skeletons and facial feature vectors to generate a foreground image and its accompanying mask and depth map, thereby achieving uniformity in the foreground subject's style, pose, identity, and lighting. This ensures that clothing fits the template, lighting matches the target scene, the mask protects the subject, and the depth map aids spatial adaptation, enhancing the realism of the final image.

[0037] In particular, in one specific embodiment, Figure 6 This is a flowchart of sub-step S4 of the AIGC-based image generation method according to an embodiment of this application. Figure 6 As shown, step S4 includes: S41, constructing a first-stage aggregated prompt based on clothing description prompts and lighting description prompts; S42, performing multi-condition guided foreground image synthesis on user pose skeleton and face feature vector based on the first-stage aggregated prompts to obtain a foreground person image; S43, inputting the foreground person image into a person segmentation model and a monocular depth estimation model to obtain a foreground person mask and a foreground person depth map.

[0038] Specifically, in step S41, a first-stage aggregated cue word is constructed based on clothing description cue words and lighting description cue words. It should be understood that independently inputting clothing and lighting cue words into the AIGC model can easily distract the model, leading to missing clothing details or mismatches between lighting and materials. Furthermore, the first stage needs to avoid complex backgrounds interfering with foreground generation and wasting model resources. Therefore, this application further integrates clothing and lighting cue words, adding a neutral background constraint to construct the first-stage aggregated cue word, thereby guiding the model to focus on the collaborative generation of characters, clothing, and lighting, eliminating background interference. This ensures that the model considers both clothing details and lighting effects, allowing lighting to naturally affect the clothing, and that a neutral background avoids resource waste, improving the efficiency and quality of foreground generation.

[0039] Specifically, in step S42, based on the aggregated prompts from the first stage, a foreground image is synthesized using multi-condition guidance based on the user's pose skeleton and facial feature vectors to obtain a foreground person image. It should be understood that relying solely on aggregated prompts to generate foreground images can easily lead to problems such as poses deviating from the original actions or significant differences between the face and the user's features. Pose skeletons and facial feature vectors are core constraints for accurate action and identity preservation; their absence results in unexpected outcomes. Therefore, this application further uses aggregated prompts as style guidance, combined with strong constraints from pose skeletons and facial feature vectors, to perform multi-condition foreground synthesis, thereby balancing stylization with the preservation of realistic features. This ensures that the foreground person follows style requirements, restoring the user's pose and unique facial features, avoiding distorted identity and pose despite accurate style.

[0040] In particular, in one specific embodiment, Figure 7 This is a flowchart of sub-step S42 of the AIGC-based image generation method according to an embodiment of this application. Figure 7 As shown, step S42 includes: S421, initializing a latent diffusion model, which includes a ControlNet module and an IP-Adapter module; S422, inputting the user pose skeleton into the ControlNet module to obtain pose conditions; S423, inputting the face feature vector into the IP-Adapter module to obtain identity conditions; S424, inputting the first-stage aggregated prompts into the text encoder to obtain text conditions; S425, based on the pose conditions, identity conditions, and text conditions, performing diffusion denoising on the initial noise tensor to obtain a clean latent representation of the foreground person; S426, inputting the clean latent representation of the foreground person into the decoder of the variational autoencoder to obtain the foreground person image.

[0041] More specifically, step S421 initializes a latent diffusion model, which includes a ControlNet module and an IP-Adapter module. It should be understood that ordinary latent diffusion models can only be generated through text prompts, lacking precise control over user pose and specific constraints on facial identity. This fails to meet the core requirements of preserving pose and identity in AIGC photo generation, easily leading to pose misalignment or the appearance of overly realistic faces. Therefore, this application further initializes a latent diffusion model integrating the ControlNet and IP-Adapter modules, thereby endowing the model with the ability to handle multi-dimensional constraints, balancing stylized generation with the preservation of realistic features. This ensures that the model controls limb structure through ControlNet and locks in identity through the IP-Adapter, while maintaining stylistic advantages, providing support for foreground synthesis guided by multiple conditions.

[0042] More specifically, steps S422, S423, and S424 involve inputting the user pose skeleton into the ControlNet module to obtain pose conditions; inputting the face feature vector into the IP-Adapter module to obtain identity conditions; and inputting the first-stage aggregated prompts into the text encoder to obtain text conditions. It should be understood that since the user pose skeleton (joint data), face feature vector (high-dimensional numerical values), and first-stage aggregated prompts (text) are all in formats not natively recognizable by the model, direct input would prevent the model from parsing them and forming effective constraints. Therefore, this application further transforms the three types of raw data into feature conditions that the model can process through corresponding modules: ControlNet to pose, IP-Adapter to identity, and text encoder to text conditions, thereby constructing a multi-dimensional constraint generation system. This ensures that pose conditions accurately reflect joint relationships (such as shoulder-elbow-wrist connections), identity conditions uniquely match user facial features (such as eye distance and nose shape), and text conditions fully convey style information (such as clothing material and lighting direction). These three types of conditions each perform their respective functions while also working together, providing clear guidance for subsequent diffusion denoising.

[0043] Specifically, in one possible embodiment, steps S422, S423, and S424 are implemented as follows: First, the user pose skeleton is input into the OpenPose branch of ControlNet. The module extracts the spatial correlation features of joints through a convolutional network and outputs a pose conditional feature map that matches the intermediate feature dimension of UNet. To ensure strong constraints on pose, its control weight can be set to a high value, such as between 0.8 and 1.0. Next, the face feature vector is input into the IP-Adapter module. The module transforms it into an identity conditional vector with the same dimension as the text features through a feature mapping network and completes the initial fusion with the text features. To ensure identity fidelity, the control weight of the IP-Adapter can also be set in a high range, such as between 0.7 and 0.9. Finally, the first-stage aggregated prompts are input into the CLIP ViT-L / 14 text encoder. The encoder extracts text semantic features through an attention mechanism and outputs standardized text conditional features. Finally, the three types of conditions are stored for subsequent calls. By configuring the weights of different conditions, the balance between pose, identity, and stylization in the generated results can be flexibly adjusted.

[0044] More specifically, step S425 involves performing diffusion denoising on the initial noise tensor based on posture, identity, and text conditions to obtain a clean latent representation of the foreground person. It should be understood that since the initial noise tensor is random Gaussian distributed data and contains no foreground person-related features, denoising based on a single condition can easily generate a latent representation that deviates from user requirements (such as incorrect posture or style mismatch). Therefore, this application further guides the diffusion denoising process of the initial noise tensor with three types of conditions—posture, identity, and text—to gradually shape foreground person features that conform to all constraints in the latent space. This ensures that limb deviations are corrected, facial features are anchored, and clothing lighting is optimized during denoising, avoiding generation biases and ultimately obtaining a clean latent representation containing complete user features and the target style.

[0045] More specifically, in step S426, the clean latent representation of the foreground person is input into the decoder of the variational autoencoder to obtain the foreground person image. It should be understood that the clean latent representation of the foreground person is a feature tensor in a low-dimensional latent space, rather than a directly visualized pixel image. Therefore, this application utilizes the decoder part of an independently pre-trained variational autoencoder to complete the mapping transformation from the latent space to the pixel space. This VAE itself is independently trained on massive image datasets, aiming to learn an efficient image compression and reconstruction function; the encoder compresses the high-resolution image into a compact latent representation, and the decoder can recover the original image from this representation with high quality. In the framework of this application, the entire diffusion denoising process is performed in the latent space defined by the VAE to significantly reduce computational complexity. Therefore, the decoder receives the final clean latent representation generated by the diffusion model as input, performs upsampling through a series of transposed convolutions, and combines residual connections to finely reconstruct image details, ultimately outputting a foreground person image with the same resolution as the original image, clear content, and conforming to all previous constraints.

[0046] Specifically, in step S43, the foreground person image is input into the person segmentation model and the monocular depth estimation model to obtain a foreground person mask and a foreground person depth map. It should be understood that the subsequent background fusion stage requires precise differentiation between the foreground person and the background area to avoid background generation covering the main subject. Simultaneously, it requires adjusting the background lighting and shadows based on the spatial depth information of the foreground person (e.g., background elements behind the person must conform to perspective relationships). However, the foreground person image itself does not contain this structured information. Therefore, this application further inputs the foreground person image into the person segmentation model and the monocular depth estimation model respectively to extract the foreground person mask (with defined boundaries) and the foreground person depth map (with defined space), thereby providing precise spatial constraints and a basis for subject protection for background fusion. This ensures that during subsequent background fusion, the model only generates content in the background area outside the mask, preventing the main subject from being tampered with. At the same time, the depth map can guide the background lighting and shadows to be naturally projected onto the person's surface (e.g., the shadow formed by the background light source on the person's back conforms to depth relationships), resolving the incongruity of the person floating in the background and improving scene spatial consistency.

[0047] Specifically, in one possible embodiment, step S43 is implemented as follows: First, the foreground person image is preprocessed, adjusting the resolution to 512×512 pixels and correcting brightness deviations to eliminate the interference of illumination on model recognition. The preprocessed image is input into a pre-trained U-2-Net person segmentation model. The model extracts and locates the person region through multi-scale feature extraction and outputs a binary foreground person mask (person region pixel value 255, background 0). Simultaneously, the same preprocessed image is input into a MiDaSv3 monocular depth estimation model. The model analyzes the stereo structure of the person in the image (such as the hands in the foreground and the shoulders in the background) through a residual network and outputs a grayscale foreground person depth map (higher pixel brightness represents closer distance). Finally, Gaussian blurring is applied to the mask edges to eliminate hard edges, and the depth map is normalized to adapt to the input range of the subsequent fusion model.

[0048] In the aforementioned AIGC-based image generation method, step S5 involves performing depth-guided background fusion on the foreground person image, foreground person mask, and foreground person depth map based on background description prompts to obtain a fused image. It should be understood that traditional background fusion often involves simple overlay or image-to-image processing, which can easily cause the foreground to appear "float" on the background, resulting in lighting inconsistencies, spatial distortion, and potentially damaging the foreground's pose and identity features. Therefore, this application further defines the scene style using background description prompts, combining the foreground image subject, mask boundary protection, and depth map spatial guidance to complete the fusion through depth guidance, balancing stylization and spatial consistency. This ensures that the background fits the prompt elements, forms the correct hierarchy based on the depth map, the mask protects the foreground from being covered, and ultimately yields a fused image with unified lighting and spatial consistency.

[0049] In particular, in one specific embodiment, Figure 8 This is a flowchart of sub-step S5 of the AIGC-based image generation method according to an embodiment of this application. Figure 8 As shown, step S5 includes: S51, inputting the foreground person image into the encoder of a variational autoencoder to obtain a clean latent representation of the foreground; S52, adding Gaussian noise corresponding to the denoising time step to the clean latent representation of the foreground to obtain a noisy initial latent tensor; S53, inputting the foreground person mask and the foreground person depth map into the ControlNet module to obtain mask control conditions and depth control conditions; S54, inputting background description prompts into a text encoder to obtain text semantic conditions; S55, based on the text semantic conditions, mask control conditions, and depth control conditions, performing denoising generation under conditions guided by a multi-control network on the noisy initial latent tensor to obtain a fused clean latent tensor; S56, inputting the fused clean latent tensor into the decoder of a variational autoencoder to obtain the fused image.

[0050] Specifically, in step S51, the foreground image of the person is input into the encoder of a variational autoencoder to obtain a clean latent representation of the foreground. It should be understood that the foreground image of the person is pixel-level data, which is computationally intensive when directly used for fusion and is difficult to be compatible with the latent diffusion model process, making it impossible to efficiently inject background features. Therefore, this application further inputs it into the encoder of a variational autoencoder to obtain a clean latent representation through low-dimensional compression, balancing feature preservation and efficiency, and adapting to the diffusion model logic. This significantly reduces subsequent computational load, fully preserves key foreground features, avoids detail loss, and adapts to the diffusion model, laying the foundation for noise addition and background generation.

[0051] Specifically, in step S52, Gaussian noise corresponding to the denoising time step is added to the clean latent representation of the foreground to obtain a noisy initial latent tensor. It should be understood that this application provides a starting point for the subsequent diffusion denoising process that includes both high-level features of the foreground person and sufficient randomness for the model to inject new background information. Specifically, the initial denoising time step t needs to be determined, which is directly related to a user-controllable denoising intensity parameter (typically ranging from 0 to 1). A higher denoising intensity (e.g., 0.75) corresponds to a larger initial time step t; for example, in a total of 1000 time steps, t can be set to 750. This means adding more Gaussian noise to the clean latent representation, giving the model more space to generate a background that blends with the foreground. Conversely, a lower intensity corresponds to a smaller time step and less noise, preserving more of the original foreground image structure. Based on the selected time step t, a noise scheduler is used to add Gaussian noise of the corresponding proportion to the pure latent representation of the foreground, thereby generating a noisy initial latent tensor. This ensures that the core features of the foreground are preserved during denoising, and background information is gradually injected to achieve a smooth fusion of the foreground as the basis for generating the background.

[0052] Specifically, in step S53, the foreground character mask and foreground character depth map are input into the ControlNet module to obtain mask control conditions and depth control conditions. It should be understood that the foreground mask and depth map are pixel-level data, which cannot be directly parsed by the diffusion model, making it difficult for the model to distinguish between foreground and background and understand spatial structure. Therefore, this application further inputs both into the corresponding branch of ControlNet, transforming them into control conditions that the model can process, and constructing a spatial constraint and subject protection mechanism. This ensures that the mask conditions mark the character area, and the model only generates the background externally; the depth conditions transmit spatial hierarchy, preventing the background from penetrating the character and ensuring spatial logic.

[0053] Specifically, in step S54, the background description prompt is input into the text encoder to obtain text semantic conditions. It should be understood that the background prompt is in natural language, and the scene information it contains is difficult to be directly utilized by the diffusion model, and it is easy to deviate from the expected background due to vague expressions. For example, "autumn" may generate a spring scene. Therefore, in this application, the prompt is further input into the pre-trained text encoder to be transformed into high-dimensional semantic conditions, accurately transmitting scene details and styles. In this way, the core information of the prompt can be completely retained, avoiding deviations caused by language ambiguity, and providing clear guidance for the model to generate a background that meets expectations.

[0054] Specifically, in step S55, based on the text semantic conditions, the mask control conditions, and the depth control conditions, denoising generation is performed on the noisy initial latent tensor under the guidance of a multi-control network to obtain a fused pure latent tensor. It should be understood that denoising with a single condition is prone to deviation. For example, only using text may cover the foreground, and only using the mask may generate a background with spatial disorder, making it difficult to balance multiple requirements. Therefore, in this application, the style is determined by the text, the foreground is protected by the mask, and the hierarchy is determined by the depth, collaboratively guiding denoising to construct fused features in the latent space. In this way, it can be ensured that the background is generated according to the prompt during denoising, the foreground is not damaged, and the correct spatial relationship is formed according to the depth, and finally a pure latent representation that fuses the foreground and the background is obtained.

[0055] Specifically, in step S56, the fused pure latent tensor is input into the decoder of the variational autoencoder to obtain the fused image. It should be understood that the fused pure latent tensor is a low-dimensional feature and cannot be directly visualized, and the foreground and background details it contains need to be pixel-level transformed to be perceived. Therefore, in this application, it is further input into the VAE decoder, and the fused image is obtained through feature restoration, realizing the transformation from latent features to a visualized image. In this way, it can be ensured that the decoder completely restores the details, avoiding blurring and missing, and finally outputs an image in which the foreground and the background are naturally fused without synthetic traces.

[0056] Particularly, in a possible embodiment, the implementation process of step S56 is as follows: First, load the pre-trained VAE decoder配套 with the encoder, initialize it and switch to the inference mode; input the fused pure latent tensor; the decoder enlarges the dimension through multiple rounds of upsampling, combines residual convolution to extract details, and performs batch normalization after each round of upsampling to stabilize the features; the activation function maps the pixel values to the visualization range, converts to the RGB space and outputs the fused image; check the details and fusion degree, and after confirming that there are no blurring traces, store it in the result database.

[0057] In the aforementioned AIGC-based image generation method, step S6 involves high-fidelity detail restoration of the fused image to obtain an enhanced image. It should be understood that because the background fusion stage model focuses on the spatial and lighting coordination between the foreground and background, it often prioritizes resource allocation to background generation, resulting in blurring and artifacts in foreground facial details (such as eye wrinkles and lip lines) and clothing textures (such as fabric wrinkles), reducing image realism. Therefore, this application further performs high-fidelity detail restoration on the fused image, precisely optimizing flawed areas to restore the core details of the person and the texture of the clothing. This ensures that the enhanced image retains the consistency of foreground and background fusion while possessing clear facial features and clothing textures, avoiding "overall harmony but rough details," significantly improving image visual appeal and user acceptance.

[0058] In particular, in one specific embodiment, Figure 9 This is a flowchart of sub-step S6 of the AIGC-based image generation method according to an embodiment of this application. Figure 9 As shown, step S6 includes: S61, performing facial region detection and extraction on the fused image to obtain facial slices to be repaired and facial bounding boxes; S62, inputting the facial slices to be repaired into a facial repair model based on a generative adversarial network to obtain a repaired facial image; S63, seamlessly fusing the repaired facial image with the fused image based on the facial bounding boxes to obtain the enhanced image.

[0059] Specifically, in step S61, facial region detection and extraction are performed on the fused image to obtain the facial slice to be repaired and the facial bounding box. It should be understood that since the face is the core area for user identification, background fusion easily leads to distortion of facial details, such as blurred eyes, abnormal eyebrow shape, and uneven skin tone. Furthermore, the repair needs to be precisely applied to the face to avoid affecting the background and clothing areas. Therefore, this application further performs facial region detection and extraction on the fused image to obtain the facial slice to be repaired and its corresponding bounding box coordinates, thereby providing accurate region positioning for subsequent targeted repair. This ensures that subsequent repair focuses only on facial blemishes without interfering with the optimized background and clothing areas. Simultaneously, the bounding box coordinates provide a positional basis for the re-attaching of the face after repair, avoiding problems such as facial displacement and proportional imbalance, and ensuring the accuracy of the repair and the overall image harmony.

[0060] Specifically, in one possible embodiment, step S61 is implemented as follows: First, the fused image is preprocessed to adjust brightness and contrast to improve the distinction between the face and background; the MTCNN face detection model is loaded, and the face is located through a three-layer cascaded network, outputting the bounding box coordinates containing key points; the facial region is cropped according to the coordinates, and the resolution is adjusted to adapt to the repair model to obtain the slice to be repaired. The pixel accuracy information of the bounding box is recorded to ensure that the face can be accurately reattached subsequently, completing the output and storage of the slice and bounding box.

[0061] Specifically, in step S62, the facial slice to be repaired is input into a facial repair model based on a generative adversarial network (GAN) to obtain a repaired facial image. It should be understood that traditional facial repair methods (such as interpolation filling) easily lead to blurred repair areas, lack of realistic texture, and inability to restore the user's unique facial features, such as brow bone height and eye shape. However, GAN-based repair models can generate highly realistic details through adversarial training between the generator and discriminator. Therefore, this application further inputs the facial slice to be repaired into this type of model to restore the fine facial features and natural texture. This ensures that the repaired image eliminates blurring, artifacts, and other imperfections while accurately preserving the user's facial uniqueness, avoiding "looking like someone else after repair" or "becoming an internet celebrity," and guaranteeing identity fidelity and realistic details.

[0062] Specifically, in one possible embodiment, step S62 is implemented as follows: First, the slice to be repaired is preprocessed, the face is aligned with the line connecting the eyes as a reference to eliminate tilt, and the pixel values ​​are normalized to the range of the adaptation model; the GFPGAN repair model is loaded, its U-Net module first removes blur artifacts, and then injects facial features into the StyleGAN2 generator through the channel separation spatial feature transformation layer; the generator combines the user's facial features to output the repair result, the slice is inversely normalized to obtain the repaired facial image, and the consistency with the original facial features is verified before use.

[0063] Specifically, in step S63, based on the facial bounding box, the repaired facial image and the fused image are seamlessly fused to obtain the enhanced image. It should be understood that directly replacing and repairing the face can easily produce compositing marks due to differences in brightness, tone, or hard edge transitions, such as a stiff connection between the face and hair, disrupting the overall harmony of the image and affecting visual realism. Therefore, this application further determines the re-attachment position based on the facial bounding box and uses image fusion technology to achieve seamless connection between the repaired face and the original image, thereby eliminating compositing marks and ensuring overall visual consistency. This ensures that the repaired face perfectly matches the background, hair, and neck in terms of brightness and color temperature, with natural edge transitions and no awkward "attached" appearance, ultimately outputting an enhanced image that is overall harmonious and realistically detailed.

[0064] Specifically, in one possible embodiment, step S63 is implemented as follows: First, based on the bounding box coordinates, the repaired face is adjusted to the same size and angle as the original face; a Poisson fusion algorithm is used to define the fusion region with the edge of the original face as a reference, and a gradient smooth transition is achieved by solving the Poisson equation; the brightness and color temperature matching degree is checked in real time, and the color tone of the repaired face is finely adjusted at the differences; after fusion, the overall image is slightly sharpened, and an enhanced image without compositing traces is output and stored in the final result library.

[0065] In particular, in another possible preferred embodiment, step S63 includes: generating a semantic importance map of the restored facial image, the semantic importance map being used to characterize the importance of different facial regions to identity fidelity; and weighting the gradient field of the restored facial image based on the semantic importance map to solve a weighted Poisson equation within the facial bounding box region of the fused image, thereby preserving key identity features while ensuring a smooth transition between non-key regions and the fused image, resulting in the enhanced image.

[0066] When performing Poisson fusion on the restored facial image and the fused image, it can effectively match low-frequency information such as lighting and color gradations, but it lacks attention to semantic information. For example, Poisson fusion treats smooth gradations on the cheeks and sharp gradations at the eye edges equally. Here, the stylized output image generated by AI, i.e., the fused image and the user's high-fidelity image, i.e., the restored facial image, can express the semantic importance of different facial regions. Therefore, a semantically guided Poisson fusion approach can be adopted, which involves uniformly forcing the gradient field of the source facial image onto the target region by introducing a semantic importance map. Here, the semantic importance map assigns higher weights to pixels that are crucial to maintaining the user's identity (e.g., eyes, corners of the mouth, unique wrinkles), while assigning lower weights to smoother regions where identity features are less critical (e.g., cheeks, forehead). In this way, the fusion process is constructed as a weighted optimization problem, where the influence of the source image gradient is proportional to its semantic importance.

[0067] Specifically, the AIGC output image has the desired target style and lighting consistency, but its facial structure is an AI-stylized interpretation of the user's face. Although the IP-Adapter module can largely preserve the identity, those fine, high-frequency details that define individual uniqueness may be softened, stylized, or slightly altered. That is, its semantic distribution is stylistically coherent, but may lack in identity fidelity. The restored facial image, on the other hand, has high identity fidelity, containing precise and realistic geometric structures of the user's unique facial features. However, its local texture and micro-illumination may not perfectly match the stylized synthetic environment in the fused image.

[0068] Thus, when all gradients from the restored facial image are imposed onto the fused image, conflicts arise. In smooth areas like the cheeks, the subtle textures of the original photo may clash with the painting / rendering style of the AI ​​image, even if the fusion itself is seamless. Furthermore, to minimize overall energy, the solver may unconsciously average out very sharp, identity-defining gradients, resulting in a loss of sharpness in the final image. Introducing a semantic graph guides the solver to be strict in critical areas and lenient in non-critical areas. For example, it aims to actively preserve gradients that define the structure of the eyes, lips, and nose, while allowing gradients in skin areas to be more relaxed to better adapt to the stylized textures of the surrounding fused image.

[0069] For standard Poisson fusion, let This refers to the target region in the target image, i.e., the fused image. Inside, we need to solve for the unknown pixel value function. Let... The source image represents the reconstructed facial image. represent For the target image outside the region, i.e., the fused image, the goal of standard Poisson fusion is to find the function... , so that its gradient With a guiding vector field (i.e., source image) gradient field Minimize the difference: ;in, For the minimization operator of the fusion function, For gradient operators, For the target integration area, For the boundary of the target area, Let be the function for fusing pixel values ​​to be determined. A function for known pixel values ​​outside the boundary. This is a function for the pixel values ​​of the source image.

[0070] This minimization problem is equivalent to finding the solution domain under given boundary conditions. Poisson's equation on: ;in, For divergence operators, For the Laplace operator.

[0071] Specifically, a semantic importance map of the repaired facial image is generated, which characterizes the importance of different facial regions to identity fidelity. The semantic importance map is generated in the process of generating the semantic importance map. hour, (If a stronger retention effect is needed, it can be greater than 1). The larger the value, the higher the semantic importance of that point. This map can be constructed by combining multiple pieces of information related to facial identity. First, facial landmark analysis is performed, and a heatmap is created using a facial landmark detector (such as dlib's 68-point model) on a reference facial slice or a reconstructed facial image. Its intensity is highest at key points (eyes, eyebrows, bridge of the nose, nostrils, and lip contours), and diffuses outward in a Gaussian decay manner. Then, based on identity information, it is also conveyed through high-frequency details such as wrinkles, pores, and hair, and high-frequency detail analysis is performed. For example, this information is captured by calculating the amplitude of the Laplacian operator response of the source face image. ;in, Source image Second-order partial derivatives in the direction, Source image Second-order partial derivatives in the direction, The function is the pixel value of the source image. This is a high-frequency detail map of the face.

[0072] Then, Normalized to the [0,1] interval, and combined with the graph, the final semantic importance graph is obtained by weighted combination of the keypoint graph and the high-frequency detail graph: ;in, A heatmap of key facial features. This is a hyperparameter (e.g., set to 0.7) used to balance the importance between structural keypoints and fine textures. This is a semantic importance map. This ensures that during subsequent fusion, the repaired features of core areas (such as the eyes and lip line) are prioritized for preservation, while non-critical areas (such as the cheeks) can flexibly adapt to the lighting and skin tone of the fused image. This avoids identity distortion and allows for adjustments to ensure a smooth transition, improving the overall naturalness of the fusion. For example, when a user selects an astronaut-style template, it ensures that the repaired eye shape, brow bone, and other core features are highly consistent with the user, avoiding the distortion of a "generic astronaut face" and meeting the core requirement of AIGC's photography equipment to generate personalized style photos.

[0073] Then, based on the semantic importance map, the gradient field of the restored facial image is weighted to solve a weighted Poisson equation within the facial bounding box region of the fused image. This preserves key identity features while ensuring a smooth transition between non-key regions and the fused image, resulting in the enhanced image. Therefore, the semantic importance map is incorporated into the objective function of the Poisson fusion. Inserted as a weighting factor into the integral term, the new objective function becomes: Thus, in Output gradient where the value is very high (e.g., at the corner of the eye). With source gradient Any deviation between them will be severely penalized, forcing the solver to almost perfectly replicate the source gradient; while Where the value is very low (e.g., on a smooth cheek), the penalty is small. The solver has greater degrees of freedom to find a value that might deviate from the expected value. gradient This allows for better matching of boundary conditions and a smoother transition with the texture of the surrounding blended image. Its gradient field can flexibly adapt to the lighting and skin tone of the blended image, ultimately achieving enhancement without distortion of key features or artifacts in non-key areas, avoiding both a synthetic look and identity distortion. For example, after blending against a cyberpunk-style background, key features such as the user's lips and eye wrinkles in a heart-shaped gesture are fully preserved, while the cheek area naturally adapts to the neon cool-toned lighting of the background, with no obvious stitching marks, meeting the practical requirements of AIGC's camera-generated high-quality artistic photos.

[0074] As a result, key facial features from the user's original photo can be transferred with higher fidelity, effectively preventing the "AI face" phenomenon that loses subtle uniqueness. Meanwhile, in secondary areas, the fusion process is more like a gentle diffusion, allowing AI-generated styles and textures to permeate the face, making the final result look like a holistic rendered image rather than a cut-and-paste product. Furthermore, by reducing the weight of gradient transfer in smooth areas, the risk of introducing unnatural-looking textures that clash with the target style is reduced, thus minimizing artifacts.

[0075] In summary, the AIGC-based image generation method based on the embodiments of this application is explained. It performs parallel deep analysis on user images and style template images, simultaneously extracting high-fidelity constraints such as user identity and pose, as well as multi-dimensional stylized guidance information including clothing, background, and lighting. Furthermore, prior lighting information is used as a pre-constraint, injecting the lighting environment of the target scene during the foreground subject generation stage, and simultaneously constructing the spatial depth information of the foreground figure. Finally, this depth information is used to provide precise spatial constraints and guidance for the background fusion process, and combined with a high-fidelity inpainting network for detail enhancement, thereby generating a realistic image where the foreground and background are highly consistent in lighting and space, and user identity features are preserved. This enables adaptive control of lighting and spatial relationships during the generation process, effectively improving the fusion quality and visual realism of the final image.

[0076] Figure 10 This is a block diagram of an AIGC-based image generation system according to an embodiment of this application. Figure 10As shown, the AIGC-based image generation system 100 according to an embodiment of this application includes: a user input acquisition module 110, used to acquire a user image input by the user and a style template image selected by the user; a user feature extraction module 120, used to extract user pose skeleton and facial feature vector from the user image; a parameter parsing module 130, used to perform parameter parsing on the style template image to obtain clothing description prompts, background description prompts, and lighting description prompts; a foreground subject generation module 140, used to generate a foreground subject with lighting prior based on the clothing description prompts and lighting description prompts, based on the user pose skeleton and facial feature vector, to obtain a foreground person image, a foreground person mask, and a foreground person depth map; a background fusion module 150, used to perform depth-guided background fusion on the foreground person image, foreground person mask, and foreground person depth map based on the background description prompts to obtain a fused image; and a high-fidelity detail restoration module 160, used to perform high-fidelity detail restoration on the fused image to obtain an enhanced image.

[0077] Here, those skilled in the art will understand that the specific operations of each step in the AIGC-based image generation system described above have been referenced above. Figures 1 to 9 The AIGC-based image generation method has been described in detail, and therefore, its repeated description will be omitted.

[0078] As described above, the AIGC-based image generation system 100 according to embodiments of this application can be implemented in various wireless terminals, such as servers with gas composition adaptive thermofluid generation control algorithms. In one possible implementation, the AIGC-based image generation system 100 according to embodiments of this application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the AIGC-based image generation system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the AIGC-based image generation system 100 can also be one of many hardware modules of the wireless terminal.

[0079] This application embodiment also provides a photographing device, including: the photographing device is capable of executing the AIGC-based photographing image generation method described above.

Claims

1. An AIGC-based photographed image generation method, characterized by, The method comprises: obtaining a user image input by a user and a style template image selected by the user; extracting a user pose skeleton and a face feature vector from the user image; performing parameter analysis on the style template image to obtain a clothing description prompt word, a background description prompt word and a lighting description prompt word; based on the clothing description prompt word and the lighting description prompt word, performing foreground subject generation with lighting prior on the user pose skeleton and the face feature vector to obtain a foreground figure image, a foreground figure mask and a foreground figure depth map, including: based on the clothing description prompt word and the lighting description prompt word, constructing a first-stage aggregated prompt word; based on the first-stage aggregated prompt word, performing multi-condition guided foreground image synthesis on the user pose skeleton and the face feature vector to obtain the foreground figure image; inputting the foreground figure image into a figure segmentation model and a monocular depth estimation model to obtain the foreground figure mask and the foreground figure depth map; based on the background description prompt word, performing depth-guided background fusion on the foreground figure image, the foreground figure mask and the foreground figure depth map by a latent diffusion model to obtain a fused image, including: inputting the foreground figure image into an encoder of a variational autoencoder to obtain a pure latent representation of the foreground; adding Gaussian noise corresponding to a denoising time step to the pure latent representation of the foreground to obtain a noisy initial latent tensor; inputting the foreground figure mask and the foreground figure depth map into a ControlNet module to obtain a mask control condition and a depth control condition; inputting the background description prompt word into a text encoder to obtain a text semantic condition; based on the text semantic condition, the mask control condition and the depth control condition, performing multi-control network guided conditional denoising generation on the noisy initial latent tensor to obtain a fused pure latent tensor; inputting the fused pure latent tensor into a decoder of the variational autoencoder to obtain the fused image; performing high-fidelity detail repair on the fused image to obtain an enhanced image; wherein, based on the first-stage aggregated prompt word, performing multi-condition guided foreground image synthesis on the user pose skeleton and the face feature vector to obtain the foreground figure image, including: initializing a latent diffusion model, the latent diffusion model comprising a ControlNet module and an IP-Adapter module; inputting the user pose skeleton into the ControlNet module to obtain a pose condition; inputting the face feature vector into the IP-Adapter module to obtain an identity condition; inputting the first-stage aggregated prompt word into a text encoder to obtain a text condition; based on the pose condition, the identity condition and the text condition, performing diffusion denoising on an initial noise tensor to obtain a foreground figure pure latent representation, the initial noise tensor being random Gaussian distribution data; inputting the foreground figure pure latent representation into a decoder of the variational autoencoder to obtain the foreground figure image.

2. The AIGC-based photographed image generation method of claim 1, wherein, The user posture skeleton and the face feature vector are extracted from the user image, including: inputting the user image into a posture extraction module to obtain a user posture skeleton, the posture extraction module being an Open Pose model; and inputting the user image into a face feature extraction module to obtain a face feature vector, the face feature extraction module being an Insight Face model.

3. The AIGC-based photographed image generation method of claim 1, wherein, The style template image is subjected to parameter analysis to obtain clothing description prompt words, background description prompt words and illumination description prompt words, including: performing foreground-background decoupling on the style template image to obtain a person mask, a segmented person image and a segmented background image; inputting the segmented person image and the segmented background image into an image-text conversion model to obtain the clothing description prompt words and the background description prompt words; and performing global illumination environment analysis and quantization on the style template image to obtain the illumination description prompt words.

4. The AIGC-based photographed image generation method of claim 3, wherein, The style template image is subjected to global illumination environment analysis and quantization to obtain illumination description prompt words, including: inputting the style template image into a pre-trained illumination estimation neural network to obtain an illumination feature vector; and inputting the illumination feature vector into a preset mapping function to obtain the illumination description prompt words. 5.The AIGC-based photographed image generation method of claim 1, wherein, The fused image is subjected to high-fidelity detail repair to obtain an enhanced image, including: performing face region detection and extraction on the fused image to obtain a face slice to be repaired and a face bounding box; inputting the face slice to be repaired into a face repair model based on a generative adversarial network to obtain a repaired face image; and based on the face bounding box, seamlessly fusing the repaired face image and the fused image to obtain the enhanced image. 6.A photographing image generation system based on AIGC, characterized by The method comprises the steps of: a user input collection module configured to acquire a user image input by a user and a style template image selected by the user; a user feature extraction module configured to extract a user posture skeleton and a face feature vector from the user image; a parameter analysis module configured to perform parameter analysis on the style template image to obtain clothing description prompt words, background description prompt words and illumination description prompt words; The foreground subject generation module is configured to perform foreground subject generation with light prior based on the clothing description prompt word and the light description prompt word to obtain a foreground figure image, a foreground figure mask, and a foreground figure depth map, including: constructing a first-stage aggregated prompt word based on the clothing description prompt word and the light description prompt word; performing multi-condition guided foreground image synthesis based on the first-stage aggregated prompt word to obtain the foreground figure image; inputting the foreground figure image into a figure segmentation model and a monocular depth estimation model to obtain the foreground figure mask and the foreground figure depth map; the background fusion module is configured to perform depth-guided background fusion based on the background description prompt word to obtain a fused image, including: inputting the foreground figure image into an encoder of a variational autoencoder to obtain a pure latent representation of the foreground; adding Gaussian noise corresponding to a denoising time step to the pure latent representation of the foreground to obtain a noisy initial latent tensor; inputting the foreground figure mask and the foreground figure depth map into a ControlNet module to obtain a mask control condition and a depth control condition; inputting the background description prompt word into a text encoder to obtain a text semantic condition; performing multi-control network guided conditional denoising generation on the noisy initial latent tensor based on the text semantic condition, the mask control condition, and the depth control condition to obtain a fused pure latent representation; inputting the fused pure latent representation into a decoder of the variational autoencoder to obtain the fused image; the high-fidelity detail repair module is configured to perform high-fidelity detail repair on the fused image to obtain an enhanced image; wherein performing multi-condition guided foreground image synthesis based on the first-stage aggregated prompt word to obtain the foreground figure image includes: initializing a latent diffusion model, the latent diffusion model including a ControlNet module and an IP-Adapter module; inputting the user pose skeleton into the ControlNet module to obtain a pose condition; inputting the face feature vector into the IP-Adapter module to obtain an identity condition; inputting the first-stage aggregated prompt word into a text encoder to obtain a text condition; performing diffusion denoising on an initial noise tensor based on the pose condition, the identity condition, and the text condition to obtain a foreground figure pure latent representation, the initial noise tensor being random Gaussian distribution data; inputting the foreground figure pure latent representation into a decoder of the variational autoencoder to obtain the foreground figure image.

7. A photographing apparatus characterized by comprising: The photographing device can perform the AIGC-based photographing image generation method according to any one of claims 1 to 5. The photographing device can perform the AIGC-based photographing image generation method according to any one of claims 1 to 5.

Images