Image scene conversion method and device, equipment and storage medium

Abstract

The invention discloses an image scene conversion method and device, equipment and a storage medium. Firstly, an image to be processed and a conversion cue word used for indicating an image scene conversion direction and a specific attribute adjustment requirement can be obtained, the conversion cue word is input into a pre-trained scene noise optimizer, and structured guide noise accurately matched with the cue word is generated through noise optimization processing and used as target noise. Performing vector encoding on the to-be-processed image and the conversion prompt word through an image encoder and a text encoder to obtain an image feature vector and a text feature vector, and performing feature fusion on the image feature vector and the text feature vector by using a cross attention mechanism to generate a fusion vector; and finally, jointly inputting the fusion vector and the target noise into a pre-trained scene conversion model, and performing conditional decoding generation processing to obtain a scene conversion image corresponding to the to-be-processed image. According to the invention, disordered disturbance caused by noise injection is avoided, and drift of the image content in the conversion process is effectively inhibited.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method, apparatus, device, and storage medium for scene transformation of an image. Background Technology

[0002] In the field of image processing, scene transformation technology is an important method for converting an image from one scene or style to another. This technology plays a vital role in applications such as artistic creation, image enhancement, and scene simulation. However, while traditional image scene transformation techniques can achieve basic image style or scene changes, they still face many challenges in practical applications, especially in ensuring the stability and accuracy of the transformation results.

[0003] Traditional image scene transformation techniques achieve image transformation by injecting random noise into a diffusion model. While this approach can alter the visual effect of an image to some extent, its core problem is the lack of systematic and structured control over the injection of random noise. Specifically, the disordered injection of random noise can lead to unpredictable drift in image content during the transformation process; that is, some key content or features in the image may be incorrectly altered or lost. Furthermore, due to the lack of precise control mechanisms, traditional noise injection techniques easily disrupt the original composition of the image, resulting in the transformed image failing to maintain the structural consistency of the original image. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method, apparatus, device, and storage medium for image scene conversion. This method enables systematic and structured control of noise injection during image scene conversion, preventing image content drift and compositional damage caused by random and disordered noise, thereby improving the stability and accuracy of the conversion effect.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] A method for scene transformation of an image, the method comprising:

[0007] Obtain the image to be processed and the conversion prompt; the conversion prompt is used to indicate the scene conversion direction and specific attribute adjustment requirements of the image to be processed;

[0008] The conversion prompt is input into a pre-trained scene noise optimizer for noise optimization processing to obtain structured guiding noise that matches the conversion prompt and this noise is used as the target noise.

[0009] The image to be processed is vector-encoded using an image encoder to obtain an image feature vector, the conversion prompt word is vector-encoded using a text encoder to obtain a text feature vector, and the image feature vector and the text feature vector are fused using a cross-attention mechanism to obtain a fused vector;

[0010] Using the fusion vector and the target noise as input, a pre-trained scene transformation model is used to perform conditional decoding on the fusion vector to generate the scene transformation image corresponding to the image to be processed.

[0011] In one possible implementation, the pre-training process of the scene transition model includes:

[0012] Construct a first training dataset; the first training dataset includes first sample pairs, each first sample pair including a feature vector of the image before conversion labeled with a scene conversion label and a corresponding scene conversion editing instruction; the scene conversion label includes a feature vector of the image after conversion; the scene conversion editing instruction is used to describe the scene editing achieved by converting the image before conversion into the image after conversion;

[0013] Using the first training dataset as input, the diffusion model is trained to obtain the scene transformation model.

[0014] In one possible implementation, the pre-training process of the scene noise optimizer includes:

[0015] The vector quantization generative adversarial network is initialized to obtain the initial noise generation network;

[0016] Construct a second training dataset; the second training dataset includes second sample pairs, each second sample pair including historical prompt words and corresponding structured guidance noise; the structured guidance noise is scene transition guidance noise determined based on the current historical prompt words;

[0017] Using historical prompts from the second training dataset as input conditions and the corresponding structured guiding noise as the supervision target, the initial noise generation network is trained to obtain the scene noise optimizer.

[0018] In one possible implementation, fusing the image feature vector and the text feature vector using a cross-attention mechanism to obtain a fused vector includes:

[0019] Using the text feature vector as the query vector and the image feature vector as the key vector and value vector, the similarity between the two is calculated and an attention weight matrix is ​​generated.

[0020] Based on the attention weight matrix, the weighted summation of each feature value in the image feature vector is performed to obtain the associated features;

[0021] The associated features are concatenated with each dimension of the text feature vector to obtain the fusion vector.

[0022] In one possible implementation, the step of taking the fusion vector and the target noise as input, and using a pre-trained scene transformation model to perform conditional decoding on the fusion vector to generate a scene transformation image corresponding to the image to be processed includes:

[0023] The fusion vector and the target noise are subjected to dimension matching processing to obtain a fusion vector and target noise with consistent dimensions.

[0024] The fusion vector with consistent dimensions and the target noise are input into the diffusion denoising network of the scene transformation model, and iterative denoising processing is performed based on the output of the diffusion denoising network at each step.

[0025] When the iterative denoising process reaches the preset number of iterations, the output result of the last iteration is used as the initial converted image;

[0026] The initial converted image is then subjected to detail restoration and color adjustment to obtain the scene-converted image.

[0027] An image scene transformation apparatus, the apparatus comprising:

[0028] An acquisition unit is used to acquire the image to be processed and conversion prompts; the conversion prompts are used to indicate the scene conversion direction and specific attribute adjustment requirements of the image to be processed.

[0029] The noise optimization unit is used to input the conversion prompt word into a pre-trained scene noise optimizer for noise optimization processing, to obtain structured guiding noise that matches the conversion prompt word and to use this noise as the target noise;

[0030] The encoding fusion unit is used to perform vector encoding on the image to be processed by an image encoder to obtain an image feature vector, to perform vector encoding on the conversion prompt word by a text encoder to obtain a text feature vector, and to perform feature fusion on the image feature vector and the text feature vector by a cross-attention mechanism to obtain a fusion vector;

[0031] The conversion unit is used to take the fusion vector and the target noise as input, and use a pre-trained scene conversion model to perform conditional decoding and generation processing on the fusion vector to obtain the scene conversion image corresponding to the image to be processed.

[0032] In one possible implementation, the device further includes:

[0033] The first construction unit is used to construct a first training dataset; the first training dataset includes a first sample pair, each first sample pair including a feature vector of the image before conversion labeled with a scene conversion label and a corresponding scene conversion editing instruction; the scene conversion label includes a feature vector of the image after conversion; the scene conversion editing instruction is used to describe the scene editing achieved by converting the image before conversion into the image after conversion;

[0034] The first model training unit is used to train the diffusion model using the first training dataset as input to obtain the scene transformation model.

[0035] In one possible implementation, the device further includes:

[0036] The initialization unit is used to initialize the vector quantization generative adversarial network to obtain the initial noise generation network;

[0037] The second construction unit is used to construct a second training dataset; the second training dataset includes second sample pairs, each second sample pair including historical prompt words and corresponding structured guidance noise; the structured guidance noise is scene transition guidance noise determined based on the current historical prompt words;

[0038] The second model training unit is used to train the initial noise generation network to obtain the scene noise optimizer by taking the historical prompts in the second training dataset as input conditions and the corresponding structured guiding noise as the supervision target.

[0039] An image scene conversion device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image scene conversion method described above.

[0040] A computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the scene transition method for an image as described above.

[0041] Compared with the prior art, this application has the following beneficial effects:

[0042] This application provides a method, apparatus, device, and storage medium for scene transformation of images. Specifically, when executing the scene transformation method for images provided in the embodiments of this application, firstly, the image to be processed and transformation prompts indicating the transformation direction and specific attribute adjustment requirements are obtained, clarifying the target for subsequent processing. Next, the transformation prompts are input into a pre-trained scene noise optimizer to generate structured guiding noise that matches them as target noise. This process replaces the traditional disordered noise injection, allowing the noise to guide the scene transformation in a targeted manner. Then, the image to be processed is encoded into an image feature vector by an image encoder, and the transformation prompts are encoded into a text feature vector by a text encoder. The two are fused through a cross-attention mechanism to obtain a fusion vector. This accurately preserves the key content and features of the image, while enabling a directional association between text semantics and image features, providing a precise feature basis for transformation. Finally, the fusion vector and the target noise are jointly input into a pre-trained scene transformation model, and after conditional decoding, the corresponding scene-transformed image is obtained. This application generates structured guiding noise that precisely matches the conversion prompts using a scene noise optimizer. This provides a clear direction and rules for noise injection, thus avoiding the disorderly disturbances caused by traditional random noise injection. It effectively suppresses image content drift during the conversion process, ensuring the stability of the main structure. Simultaneously, through cross-attention fusion of image and text feature vectors, the collaborative expression of image and text feature vectors is enhanced. This allows the conditional decoding process to guide image conversion more accurately, significantly improving the converted image's performance in terms of semantic consistency, detail preservation, and compositional integrity. Ultimately, this results in a more stable, accurate, and user-relevant scene conversion effect. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating a scene transformation method for an image provided in this application embodiment;

[0045] Figure 2 This is a schematic diagram of the structure of an image scene conversion device provided in an embodiment of this application. Detailed Implementation

[0046] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0047] In the field of image processing, scene transformation technology plays a crucial role in artistic creation. Traditional techniques achieve image scene transformation by injecting random noise into a diffusion model. However, because random noise lacks systematic and structured control, it not only causes unpredictable drift in image content during transformation, leading to the loss of key content or features, but also easily disrupts the original composition of the image, making it difficult to guarantee the stability and accuracy of the transformation effect.

[0048] To address this issue, this application provides a method, apparatus, device, and storage medium for scene conversion of images. First, the image to be processed and conversion prompts for clarifying the scene conversion direction and specific attribute adjustment requirements are acquired. This step establishes the target and basis for the entire conversion process. Subsequently, the conversion prompts are input into a pre-trained scene noise optimizer. After optimization, structured guiding noise matching the prompts, i.e., target noise, is generated. This ensures that the noise injection is no longer random but plays a directional guiding role according to the conversion requirements. Next, the image to be processed is converted into an image feature vector by an image encoder, and the conversion prompts are converted into a text feature vector by a text encoder. These two vectors are fused through a cross-attention mechanism to obtain a fused vector. This process preserves the original key content and features of the image while establishing a close relationship between text semantics and image features, providing accurate feature support for subsequent conversion. Finally, the fused vector and the target noise are input into a pre-trained scene conversion model. The model performs conditional decoding on the fused vector to generate the final scene-converted image corresponding to the image to be processed, thus completing the entire scene conversion process. This application inputs the conversion prompts into a pre-trained scene noise optimizer for noise optimization, generating structured guiding noise that matches the conversion prompts. This structured noise can more accurately guide the direction of image conversion and the specific attribute adjustment requirements, avoiding content drift and compositional damage caused by disordered injection of random noise. Simultaneously, by converting the image to be processed and the conversion prompts into feature vectors and fusing them, the original key content and feature details of the image are accurately preserved, while text semantics and image features form a directional association. This ensures the stability of the image subject during the conversion process from the underlying feature level, thereby improving the quality and content consistency of the converted image.

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0050] See Figure 1This figure is a flowchart of a scene transformation method for an image provided in an embodiment of this application, as shown below. Figure 1 As shown, the scene transformation method for this image may include steps S101-S104:

[0051] S101: Obtain the image to be processed and conversion prompts.

[0052] In the image scene transformation process, the first step is to acquire the original image to be processed and its corresponding transformation prompts. The image to be processed refers to the input image from which the user wishes to transform the scene style or attributes. These images often contain specific content and structural information, forming the basis for subsequent transformation operations. The transformation prompts serve a guiding role, clearly indicating the desired scene or style transformation and specifying the details of the attributes to be adjusted, such as hue, lighting, season, changing background elements, or enhancing certain visual effects. Through these prompts, the system can accurately understand the user's transformation intent, thereby controlling the transformation process to steadily progress towards the expected goal.

[0053] For example, suppose the image to be processed is a photo of a park in spring, and the user wants to convert it into an autumn scene. The conversion prompt could be set to "convert the spring park into an autumn scene, add color to the fallen leaves, and adjust the lighting to a warm golden hue." This prompt not only clearly states the direction of the conversion—from spring to autumn—but also specifies the detailed adjustment requirements for color and lighting. Such instructions allow subsequent image encoding and noise generation to revolve around these clear goals, ensuring that the conversion result meets the user's expectations while maintaining the integrity and naturalness of the image's main structure.

[0054] S102: Input the conversion prompt word into a pre-trained scene noise optimizer for noise optimization processing to obtain structured guiding noise that matches the conversion prompt word and use this noise as the target noise.

[0055] Inputting the conversion prompts into a pre-trained scene noise optimizer for noise optimization to obtain target noise is one of the core steps in achieving precise control of scene transitions. Because the scene noise optimizer has learned and solidified the mapping relationship between conversion prompts and corresponding optimal scene noise through massive amounts of samples during the pre-training phase, it can accurately understand the scene transition intent and attribute adjustment details implied in the conversion prompts, and generate noise with a clear structure and directional guidance function, namely structured guiding noise (target noise). This noise is not the disordered, random interference signal of traditional techniques, but a "guiding signal" highly aligned with the transition requirements. Its noise distribution, intensity changes, and other characteristics all revolve around the requirements of the conversion prompts, enabling it to directionally drive image features towards the target scene during subsequent scene transitions, while avoiding indiscriminate damage to the main content and key structures of the image.

[0056] For example, suppose the conversion prompt is "convert a sunny beach image into a cloudy evening scene, reduce overall brightness, make the seawater appear dark cyan, and preserve the original texture of the sand but with a cooler tone," the pre-trained scene noise optimizer will generate target noise adapted to this requirement based on the learned correspondence. This target noise includes a noise component that guides the overall brightness to decrease by a specific magnitude (matching "reduce overall brightness"), a noise pattern that controls the seawater pixel values ​​to shift towards the dark cyan spectral range (corresponding to "the seawater appears dark cyan"), and a noise structure that constrains the preservation of sand texture features and only adjusts the tone to a cooler tone (matching "the sand preserves the original texture but with a cooler tone"). These noise components work together to form a structured guiding signal, which can accurately drive the image to a "cloudy evening" scene conversion, and ensure that key features such as sand texture are not destroyed during the conversion process through structured design. This provides a precise control basis for the subsequent synergistic effect of the fusion vector and the target noise.

[0057] S103: The image to be processed is vector-encoded by an image encoder to obtain an image feature vector, the conversion prompt word is vector-encoded by a text encoder to obtain a text feature vector, and the image feature vector and the text feature vector are fused by a cross-attention mechanism to obtain a fused vector.

[0058] In the image scene transformation process, extracting features from the image to be processed and the transformation prompts using different encoders is a crucial step in achieving multimodal information fusion. Specifically, the image to be processed is first input into an image encoder, which uses a deep neural network structure to represent the visual information in the image in a high-dimensional vector form, extracting image feature vectors that represent key information such as image content, structure, and texture. Simultaneously, the transformation prompts are input as text information into a text encoder, which is typically based on natural language processing techniques to convert the text into semantically rich vector representations, i.e., text feature vectors. These vectors reflect the scene transformation direction and attribute adjustment requirements carried by the prompts.

[0059] To more effectively utilize these two types of feature information, the system employs a cross-attention mechanism to fuse image and text feature vectors. This mechanism calculates the correlation weights between image and text features, achieving mutual complementarity and reinforcement of information. This ensures that the fused feature vector not only contains the visual details of the image itself but also fully reflects the user's conversion intent expressed through prompts. The resulting fused vector becomes the core basis for subsequent conditional decoding and image generation, guaranteeing a high degree of consistency between visual representation and semantic meaning in the final conversion effect.

[0060] For example, suppose the image to be processed is a summer forest scene, and the transformation prompt is "transform the summer forest into an autumn golden woodland, highlighting fallen leaves and warm tones." The image encoder first extracts the structural features of the forest image, such as tree morphology, light and shadow distribution, and color information. The text encoder then converts keywords such as "autumn," "fallen leaves," and "warm tones" into vectors, conveying the specific scene transformation requirements. Through a cross-attention mechanism, these visual and semantic information interact, allowing the fused vector to both preserve the spatial layout of the original forest and accurately guide the model to enhance the expression of fallen leaf elements and golden tones, thus laying the foundation for subsequently generating an autumn scene image that meets the expected specifications.

[0061] S104: Using the fusion vector and the target noise as input, the fusion vector is conditionally decoded and generated using a pre-trained scene transformation model to obtain the scene transformation image corresponding to the image to be processed.

[0062] In the image scene transformation process, the combination of the fusion vector and the target noise constitutes the key input for generating high-quality scene-transformed images. Specifically, the fusion vector comprehensively expresses the visual features of the original image and the semantic information contained in the transformation prompts, ensuring that the model fully understands the content of the image to be processed and the scene changes expected by the user. The target noise, on the other hand, is a structured guiding signal generated by a pre-trained scene noise optimizer. It injects detailed adjustment directions and style features that highly match the prompts into the transformation process. By simultaneously inputting these two into the pre-trained scene transformation model and through conditional decoding generation, the model can use the fusion vector as a condition to guide the image generation path. Under the guidance provided by the target noise, it selectively adjusts the color, texture, and structural details of the image, thereby achieving accurate transformation of the original image scene.

[0063] This conditional decoding mechanism not only improves the semantic consistency and visual coherence of the generated images, but also effectively avoids random disturbances caused by disordered noise, ensuring stable reproduction and natural transition of the main elements in the converted image. Because the scene transformation model is fully pre-trained on a large amount of diverse data, it possesses strong generalization capabilities, enabling it to handle different types of scene transformation needs and achieve diverse effects, from subtle tonal changes to complex environment replacements.

[0064] Based on S101-S104, the process first involves acquiring the image to be processed and conversion prompts indicating the direction of scene transformation and specific attribute adjustments. These prompts are then input into a pre-trained scene noise optimizer, which generates structured guiding noise matching the prompts, serving as the target noise. Next, the image and prompts are vector-encoded using an image encoder and a text encoder, respectively, and fused using a cross-attention mechanism to obtain a fused vector. Finally, the fused vector and the target noise are input into a pre-trained scene transformation model, and conditional decoding generates the corresponding scene transformation image, achieving accurate and efficient transformation of the image scene. This application generates structured guiding noise that precisely matches the conversion prompts through a scene noise optimizer, providing a clear direction and rules for noise injection, thus avoiding the disordered disturbances caused by traditional random noise injection, effectively suppressing image content drift during the transformation process, and ensuring the stability of the main structure. Meanwhile, by fusing cross-attention between image and text feature vectors, the collaborative expression of image and text feature vectors is enhanced, enabling the conditional decoding process to guide image conversion more accurately. This significantly improves the performance of converted images in terms of semantic consistency, detail preservation, and compositional integrity, thereby achieving a more stable, accurate, and user-friendly scene conversion effect.

[0065] In one possible implementation, the pre-training process of the scene transition model includes steps A1-A2:

[0066] A1: Construct the first training dataset.

[0067] The core objective in constructing the first training dataset is to provide rich and structured sample information for the pre-training of the scene transformation model, enabling the model to learn the transformation rules from the input image feature vector to the target scene image feature vector. Specifically, this training dataset consists of multiple first sample pairs, each containing two key parts: First, the pre-transformation image feature vector labeled with scene transformation tags. This feature vector is a high-dimensional vector representation obtained by encoding the original image to be processed through an image encoder, condensing the image's visual content and structural features. Second, the corresponding scene transformation editing instructions. These instructions typically describe, in text or structured format, how to transform the pre-transformation image into the target scene, including the specific direction of scene change and attribute adjustment requirements. The scene transformation tags here not only record the pre-transformation image features but also include the post-transformation image feature vector, i.e., the high-dimensional representation of the target image after encoder processing. In this way, the entire sample pair fully embodies the mapping relationship from the initial scene to the target scene, providing clear input-output supervision signals for model training.

[0068] A2: Using the first training dataset as input, the diffusion model is trained to obtain the scene transformation model.

[0069] The diffusion model is trained using the first training dataset as input to obtain the scene transformation model. This process is the core step in enabling the model to learn the mapping rules from the pre-transformation scene to the post-transformation scene. The training of the diffusion model essentially involves learning the data distribution through a reverse denoising process, and the first sample pair in the first training dataset provides a precise "input-target" reference for this learning process. In each sample pair, the pre-transformation image feature vector represents the feature distribution of the initial scene, the scene transformation editing command is the conditional signal triggering the scene transformation, and the post-transformation image feature vector (i.e., the scene transformation label) is the target distribution that the model needs to learn to approximate. During training, the diffusion model first adds different levels of noise to the post-transformation image feature vector to simulate the degradation process from sharp to blurry. Then, based on the pre-transformation image feature vector and the scene transformation editing command, it learns how to gradually remove noise and restore the post-transformation image feature vector. Through continuous iterative optimization of the model parameters, the model acquires the ability to accurately map pre-transformation features to post-transformation features under the guidance of specific editing commands.

[0070] For example, if the first training dataset contains sample pairs of "spring cherry blossom forest → autumn maple forest": the image feature vector before conversion carries features such as "pink cherry blossoms, light green leaves, and bright light," and the scene conversion editing instruction is "replace cherry blossoms with red maple leaves, change leaves to orange-yellow, and change light to warm yellow tones," then the image feature vector after conversion corresponds to the feature distribution of "red maple leaves, orange-yellow leaves, and warm yellow light." During training, the diffusion model first injects noise into the feature vector of "autumn maple forest," making it a noisy, blurred feature. Then, using the feature vector of "spring cherry blossom forest" and the above editing instruction as conditions, the model learns to gradually remove noise from the blurred features, ultimately generating a result consistent with the feature vector of "autumn maple forest." After training with a large number of such samples, the model will gradually understand the feature change rules of scene conversions such as "cherry blossom → maple leaves" and "light green → orange-yellow," as well as the correspondence between the descriptions in the editing instructions and the feature conversions. Once training is complete, the diffusion model evolves into a scene transformation model. When a new "spring crabapple forest" image feature vector is input into it, along with editing instructions such as "replace the crabapple with red maple leaves, change the leaves to orange-yellow, and change the light to a warm yellow tone," it can accurately generate the corresponding "autumn maple forest" scene transformation image based on the rules learned during training, achieving a stable scene transformation that meets the instructions.

[0071] In one possible implementation, the pre-training process of the scene noise optimizer includes steps B1-B3:

[0072] B1: Initialize the vector quantization generative adversarial network to obtain the initial noise generation network.

[0073] Initializing the Vector Quantized Generative Adversarial Network (VQGAN) to obtain the initial noise generation network is a crucial step in building the foundational model architecture for the scene noise optimizer. As a model that combines the advantages of Vector Quantization (VQ) and Generative Adversarial Networks (GANs), VQGAN's core structure includes a generator, a discriminator, and a vector quantization module. The generator is responsible for noise generation and feature mapping; the discriminator evaluates the realism and structure of the generated noise; and the vector quantization module enhances the model's ability to structurally represent noise features by mapping continuous features to a discrete codebook. During initialization, random but distributed initial weights are assigned to each component of the VQGAN (such as the generator's convolutional layers, the discriminator's attention mechanism, and the basic parameters of vector quantization) to ensure the model possesses basic noise generation and feature processing capabilities before training, while avoiding training instability caused by extreme parameter values.

[0074] Specifically, the initialized generator can initially output vectors with basic noise features, but at this stage, the output lacks a logical association with the text prompts, and the noise distribution is disordered. The discriminator can initially distinguish between the generated noise and real structured noise, but the discrimination accuracy is low. The codebook of the vector quantization module randomly stores a set of basic noise feature vectors, laying the foundation for subsequent learning of the discrete representation of structured noise. Through this initialization process, VQGAN is transformed into an initial noise generation network that can directly receive text prompt input and output noise vectors. Its architecture design retains VQGAN's ability to capture complex features while reserving parameter optimization space for subsequent training to learn the "prompt-structured noise" mapping relationship, enabling the network to gradually evolve into a scene noise optimizer that accurately generates target noise in subsequent steps.

[0075] B1: Construct the second training dataset.

[0076] Constructing a second training dataset is the core process of providing "text instruction-structured noise" mapping samples for training the scene noise optimizer. Its essence lies in enabling the model to learn the correlation between different scene transformation requirements and corresponding guiding noise through a large number of high-quality second sample pairs. Each second sample pair consists of historical prompts and corresponding structured guiding noise, forming a precise correspondence between "input instructions" and "output noise." The historical prompts are text instructions used in past actual scene transformation tasks, covering diverse scene transformation requirements, such as "transform daytime city streets into a rainy night, increasing road surface reflection" or "change the oil painting style forest to a sketch style, preserving tree outlines." These prompts clearly define the direction of the scene transformation (e.g., time, weather, style) and specific attribute adjustment requirements (e.g., water reflection, outline preservation).

[0077] The corresponding structured guidance noise is a noise signal specifically designed based on these historical prompts to effectively guide the transformation of the target scene. This type of noise is not randomly generated, but has clear structured characteristics: it decomposes the visual dimensions that need to be adjusted (such as color, lighting, texture, and outline) according to the semantics of the prompts, and assigns directional noise parameters to each dimension. For example, for the historical prompt "transform the winter snow scene into a spring flower sea, the snowfield into a green meadow, dotted with pink flowers", the corresponding structured guidance noise will contain three core parts: first, color conversion noise guiding "snowfield → green meadow" (carrying a hue shift parameter from white to green); second, texture generation noise controlling "adding pink flowers" (including the spatial distribution pattern of flower shapes); and third, constraint noise maintaining "the main outlines of trees, mountains, etc. unchanged" (achieved by suppressing pixel fluctuations in the outline area).

[0078] By collecting a large number of second sample pairs covering different scene types and transformation complexities (such as weather changes, seasonal changes, style transfer, attribute adjustments, etc.), the constructed second training dataset can provide comprehensive learning materials for the initial noise generation network. These sample pairs include both simple single-attribute transformations (such as "adjusting image brightness to 50%)" and complex multi-dimensional collaborative transformations (such as "transforming a daytime beach into a nighttime bonfire party, turning the sky dark blue, adding flame lighting, and adding crowds on the beach"), ensuring that the model can gradually master the "cue word-structured noise" mapping logic from simple to complex during training, laying a data foundation for the subsequent generation of target noise that accurately matches new cue words.

[0079] B3: Using the historical prompts in the second training dataset as input conditions and the corresponding structured guiding noise as the supervision target, train the initial noise generation network to obtain the scene noise optimizer.

[0080] Using historical prompts from the second training dataset as input and corresponding structured guiding noise as the supervised input, the initial noise generation network is trained to obtain a scene noise optimizer. This process is the core step in enabling the model to learn the accurate mapping between "text instructions → structured noise". While the initial noise generation network possesses a basic noise generation architecture, it has not yet established a logical association with the text prompts, resulting in noise that lacks directionality and structured features. The second sample pair in the second training dataset provides the model with a clear "input-target" reference: historical prompts, as textual instructions for scene transitions, are the input conditions the model needs to understand. The corresponding structured guiding noise represents the optimal noise pattern that meets this requirement, and is the target output that the model needs to learn to generate.

[0081] During training, the model first encodes historical prompts, transforming the text semantics into processable feature vectors. Then, a generator produces corresponding noise vectors. Subsequently, the discriminator compares the generated noise vectors with the structured guided noise in the sample pairs, assessing the matching degree and providing feedback on the error. The generator adjusts its parameters based on the error, optimizing the noise generation strategy and gradually improving the similarity between the generated noise and the target noise. Simultaneously, the vector quantization module maps the generated noise features to a discrete codebook, enhancing the structured representation of the noise and ensuring that the generated noise accurately corresponds to the semantics of the prompts in dimensions such as color shift, texture distribution, and regional constraints. For example, when the historical prompt is "turn the desert sunset into the aurora borealis, the dunes into glaciers, and add green aurora textures to the sky," the corresponding structured guided noise includes the morphological transformation parameters of "dust → glacier," the texture distribution pattern of "green aurora in the sky," and the constraint signal of "preserving the horizon outline." During training, the model first parses the semantics of the prompt word, generates initial noise, and then uses a discriminator to compare and find problems such as "insufficient continuity of aurora texture" and "low matching degree between glacier morphology and dune outline" in the generated noise. Then, it adjusts the feature weights of the generator so that the noise generated later can accurately cover these requirements.

[0082] Through iterative training with a large number of different types of sample pairs (such as weather transformation, style transfer, attribute adjustment, etc.), the initial noise generation network gradually masters the mapping relationship between key semantics in historical prompts (such as instruction words like "convert to," "add," and "retain," as well as attribute descriptions like "Aurora Borealis" and "green") and various parameters in structured guided noise. After training is complete, the model can not only understand simple prompts (such as "increase image brightness by 30%), but also handle complex multi-dimensional instructions (such as "convert daytime urban streets to a cyberpunk-style rainy night, add neon lights to buildings, add water reflections to the road surface, and retain vehicle outlines"). The generated noise can accurately match each requirement of the prompts. At this point, the initial noise generation network evolves into a scene noise optimizer, possessing the ability to generate matching structured guided noise based on new transformation prompts.

[0083] In one possible implementation, step S103 involves fusing the image feature vector and the text feature vector using a cross-attention mechanism to obtain a fused vector. This can be specifically achieved through steps C1-C3:

[0084] C1: Using the text feature vector as the query vector and the image feature vector as the key vector and value vector, calculate the similarity between the two and generate an attention weight matrix.

[0085] The process of "using text feature vectors as query vectors and image feature vectors as key and value vectors, calculating their similarity, and generating an attention weight matrix" is the core operation in the cross-attention mechanism for establishing a directional association between text semantics and image visual features. Essentially, it allows text instructions to actively "focus" on key areas in the image relevant to the transformation requirements, providing a precise basis for attention allocation in subsequent feature fusion. The text feature vector, as the query vector, carries the semantic intent of the scene transformation (e.g., "change the red roof to blue" or "add ripples to the water surface"). Its role is to actively retrieve features in the image that need to be focused on or modified. The image feature vector serves as both the key and value vectors. The key vector is used for similarity matching with the query vector to identify which features in the image are related to the text semantics, while the value vector corresponds to the specific visual information of these features (e.g., pixel distribution, contour morphology).

[0086] When calculating similarity, the model uses matrix operations (such as dot product and scaled dot product) to quantify the degree of matching between the query vector and each feature dimension of the key vector. Key information in the text semantics (such as "red roof" and "water surface") will generate high similarity scores with corresponding regional features in the image (such as the red tone of the roof and the smooth texture of the water surface), while the similarity scores with irrelevant features (such as distant trees and clouds in the sky) will be low. These similarity scores are normalized (e.g., using the softmax function) to form an attention weight matrix. Each element in the matrix represents the degree of attention the text semantics pays to a certain feature region in the image. The higher the weight, the closer the correlation between that region and the text conversion requirements.

[0087] For example, suppose the image to be processed is "a white house with a red roof, a calm green meadow in front of the house, and a blue pond next to it." The image feature vector contains features such as "red roof," "white wall," "green meadow," and "blue pond." The conversion prompt is "change the roof to gray to make ripples appear on the pond." The corresponding text feature vector highlights the semantics of "roof → gray" and "pond surface → ripples." When this text feature vector is used as the query vector, and the image feature vector is used as the key and value vectors, in the attention weight matrix generated after calculating similarity, the "red roof" and "blue pond" regions will receive higher weights (because they are directly related to the text semantics), while the "white wall" and "green meadow" regions will have lower weights (because the text does not mention the relevant conversion). This weight matrix clearly defines the image regions that the text instruction focuses on, providing clear priority guidance for subsequent weighted summation to generate associated features, ensuring that the association between text semantics and image features is accurate and efficient.

[0088] C2: Based on the attention weight matrix, the feature values ​​in the image feature vector are weighted and summed to obtain the associated features.

[0089] The key step in translating the semantic focus on image features into concrete feature representations is the weighted summation of each feature value in the image feature vector using an attention weight matrix. This process transforms the text's focus on image features into a weighted representation. The core principle is to strengthen image features relevant to the textual needs through weight allocation, while weakening irrelevant features, creating a "text-oriented condensed version of image features." The value of each element in the attention weight matrix directly reflects the correlation between the text feature vector (query vector) and the corresponding feature in the image feature vector (key vector). A higher weight indicates a stronger correlation between the image feature and the textual conversion needs, and it should occupy a more significant position in the final feature representation; a lower weight indicates a weaker correlation, and its proportion in the feature representation should be appropriately reduced. During the weighted summation, each feature value in the image feature vector is multiplied by its corresponding weight value in the attention weight matrix, and all products are summed to obtain the associated feature. This operation is not a simple numerical addition, but rather a selective highlighting of image features relevant to the text through the "amplification" and "reduction" effects of the weights. For example, if an image feature vector contains four features: "red roof," "white wall," "green grass," and "blue lake," the corresponding attention weight matrix assigns a weight of 0.6 to "red roof" (because the text prompt requires "change the roof to brown"), a weight of 0.3 to "blue lake" (because the prompt requires "add ripples to the lake"), and weights of 0.05 each to "white wall" and "green grass" (the prompt doesn't mention any related adjustments). During weighted summation, the "red roof" feature is amplified by 60%, the "blue lake" feature by 30%, while "white wall" and "green grass" retain only 5% of their influence. The resulting associated features emphasize the shape and outline of the roof and the regional features of the lake, while significantly weakening the features of the walls and grass. This makes it an image feature representation highly compatible with the text requirement of "modifying the roof color and adding ripples to the lake," laying the foundation for subsequent fusion with the text feature vector. This ensures that the fused vector contains both the core visual information of the image and closely adheres to the text's intended meaning.

[0090] C3: Concatenate the associated features with each dimension of the text feature vector to obtain the fusion vector.

[0091] The step of "concatenating the dimensions of the associated features and the text feature vector to obtain the fusion vector" is a crucial step in achieving deep fusion of image visual information and text semantic instructions. Its core is to use dimensional concatenation to create a comprehensive feature carrier that integrates the core information of both modalities. The associated features, as the essence of image features filtered by the attention mechanism, condense the key image region features that the text semantics focuses on (such as the objects to be transformed, the structures to be preserved, etc.), ensuring that the fused features do not deviate from the main framework of the original image. The text feature vector, on the other hand, fully retains the semantic instructions such as the direction of scene transformation and specific attribute adjustment requirements (e.g., "turn the leaves red" or "add a snow scene effect"), providing clear target guidance for the transformation. Through dimensional concatenation, the fusion vector inherits both the visual positioning information of "which image regions need to be focused on" from the associated features and the semantic instruction information of "how to transform these regions" from the text feature vector, providing comprehensive and accurate input for the subsequent decoding and generation of the scene transformation model.

[0092] For example, assuming the image to be processed is a "daytime city street scene", and the conversion prompt is "convert the street scene to nighttime and add street lighting effects", the specific process is as follows:

[0093] Vector encoding stage: The image encoder extracts the image feature vector Vimg: Vimg = [0.8 (sky brightness), 0.6 (building outline), 0.3 (road surface details)] (the higher the value, the more significant the feature). The text encoder extracts the text feature vector Vtxt: Vtxt = [0.9 (nighttime semantics), 0.7 (streetlight features), 0.2 (urban scene)].

[0094] Attention weight calculation: Using Vtxt as the query vector and Vimg as the key vector and value vector, the similarity is calculated to generate an attention weight matrix: W=[0.7 (corresponding to sky brightness), 0.2 (corresponding to building outline), 0.1 (corresponding to road surface details)] (because the focus of the prompt is on "sky turning to night", the sky brightness feature has the highest weight).

[0095] Weighted generation of image association features: Based on the weight matrix, Vimg is weighted and summed: text-guided image association features = 0.8×0.7+0.6×0.2+0.3×0.1=0.56+0.12+0.03=0.71 (the result strengthens the sky brightness features associated with "night transition").

[0096] Feature fusion: If dimensional concatenation is used: concatenate 0.71 with Vtxt to obtain the fusion vector [0.71, 0.9, 0.7, 0.2].

[0097] Through this process, the fused vector retains the core structural information of "buildings and roads" in the image, while also incorporating the semantic guidance of the text "night and streetlights," providing accurate cross-modal input for subsequent scene transformation models.

[0098] In one possible implementation, step S104 uses the fusion vector and the target noise as input, and performs conditional decoding on the fusion vector using a pre-trained scene transformation model to obtain the scene transformation image corresponding to the image to be processed. Specifically, this can be achieved through steps D1-D4:

[0099] D1: Perform dimension matching processing on the fusion vector and the target noise to obtain a fusion vector and target noise with consistent dimensions.

[0100] In step D1, the fused vector and the target noise need to undergo dimension matching to ensure their consistency in terms of space and channels. Since the fused vector typically contains multimodal feature information, while the target noise represents potential perturbations after random initialization or optimization, they often differ in size or shape. By using linear transformations, interpolation adjustments, or convolution operations, they are adjusted to the same dimensional format, facilitating the subsequent model to simultaneously receive and effectively utilize both types of information, achieving seamless fusion of multi-source features.

[0101] D2: Input the fusion vector with consistent dimensions and the target noise into the diffusion denoising network of the scene transformation model, and perform iterative denoising processing based on the output of the diffusion denoising network at each step.

[0102] In step D2, the dimension-matched fused vector and the target noise are input together into the diffusion denoising network of the scene transformation model. This network, based on the diffusion model principle, denoises the input through iterative processing, outputting a result that more closely approximates the true image distribution at each step. During each iteration, the model uses a conditional decoding mechanism to guide the noise towards an image space with clear semantics and visual structure, ultimately approximating the desired scene transformation effect, by using the fused vector (containing multimodal information from both text and image) as guiding information.

[0103] D3: When the iterative denoising process reaches the preset number of iterations, the output result of the last iteration is used as the initial converted image.

[0104] When the number of iterations reaches a preset threshold, step D3 treats the output of the last iteration as the preliminary transformed image. At this point, the image has basically completed the transformation from random noise to clear visual content, but there may still be issues such as insufficient detail or poor color reproduction. Therefore, retaining the product of this stage as the basis for subsequent detail enhancement is an important step in ensuring the overall generation quality.

[0105] D4: Perform detail restoration and color adjustment on the preliminary converted image to obtain the scene converted image.

[0106] Finally, in step D4, the initially transformed image undergoes detail restoration and color adjustment. By introducing additional image enhancement modules, such as super-resolution networks, color correction algorithms, or deep learning-based image inpainting techniques, the image's texture detail and color naturalness are further improved. This step not only corrects potential visual imperfections but also makes the final output scene-transformed image more in line with human perception habits and aesthetic standards.

[0107] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an image scene transformation device provided in an embodiment of this application. Figure 2 As shown, the scene transition device for this image includes:

[0108] The acquisition unit 201 is used to acquire the image to be processed and the conversion prompt words; the conversion prompt words are used to indicate the scene conversion direction and specific attribute adjustment requirements of the image to be processed.

[0109] The noise optimization unit 202 is used to input the conversion prompt word into a pre-trained scene noise optimizer for noise optimization processing, to obtain structured guiding noise that matches the conversion prompt word and to use the noise as the target noise;

[0110] Encoding unit 203 is used to perform vector encoding on the image to be processed by an image encoder to obtain an image feature vector, and to perform vector encoding on the conversion prompt word by a text encoder to obtain a text feature vector;

[0111] The fusion unit 204 is used to fuse the image feature vector and the text feature vector through a cross-attention mechanism to obtain a fusion vector;

[0112] The conversion unit 205 is used to take the fusion vector and the target noise as input, and use a pre-trained scene conversion model to perform conditional decoding generation processing on the fusion vector to obtain the scene conversion image corresponding to the image to be processed.

[0113] In one possible implementation, the device further includes:

[0114] The first construction unit is used to construct a first training dataset; the first training dataset includes a first sample pair, each first sample pair including a feature vector of the image before conversion labeled with a scene conversion label and a corresponding scene conversion editing instruction; the scene conversion label includes a feature vector of the image after conversion; the scene conversion editing instruction is used to describe the scene editing achieved by converting the image before conversion into the image after conversion;

[0115] The first model training unit is used to train the diffusion model using the first training dataset as input to obtain the scene transformation model.

[0116] In one possible implementation, the device further includes:

[0117] The initialization unit is used to initialize the vector quantization generative adversarial network to obtain the initial noise generation network;

[0118] The second construction unit is used to construct a second training dataset; the second training dataset includes second sample pairs, each second sample pair including historical prompt words and corresponding structured guidance noise; the structured guidance noise is scene transition guidance noise determined based on the current historical prompt words;

[0119] The second model training unit is used to train the initial noise generation network to obtain the scene noise optimizer by taking the historical prompts in the second training dataset as input conditions and the corresponding structured guiding noise as the supervision target.

[0120] In one possible implementation, the fusion unit 204 specifically includes:

[0121] The attention weight matrix calculation unit is used to calculate the similarity between the text feature vector as the query vector and the image feature vector as the key vector and value vector, and generate an attention weight matrix.

[0122] The weighted summation unit is used to perform weighted summation of each feature value in the image feature vector based on the attention weight matrix to obtain the associated features;

[0123] The vector concatenation unit is used to concatenate the associated features with each dimension of the text feature vector to obtain the fused vector.

[0124] In one possible implementation, the conversion unit 204 specifically includes:

[0125] A dimension matching unit is used to perform dimension matching processing on the fusion vector and the target noise to obtain a fusion vector and target noise with consistent dimensions.

[0126] An iterative denoising unit is used to input a fusion vector with consistent dimensions and target noise into the diffusion denoising network of the scene transformation model, and to perform iterative denoising processing based on the output of the diffusion denoising network at each step.

[0127] The output unit is used to take the output result of the last iteration as the initial converted image when the iterative denoising process reaches a preset number of iterations;

[0128] The repair and adjustment unit is used to perform detail repair and color adjustment on the preliminary converted image to obtain the scene converted image.

[0129] In addition, this application embodiment also provides an image scene conversion device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image scene conversion method as described above.

[0130] In addition, this application embodiment also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the image scene transformation method described above.

[0131] This application generates structured guiding noise that precisely matches the conversion prompts using a scene noise optimizer. This provides a clear direction and rules for noise injection, avoiding the disordered disturbances caused by traditional random noise injection, effectively suppressing image content drift during conversion, and ensuring the stability of the main structure. Simultaneously, through cross-attention fusion of image and text feature vectors, the collaborative expression of image and text feature vectors is enhanced, enabling the conditional decoding process to guide image conversion more accurately. This significantly improves the converted image's performance in terms of semantic consistency, detail preservation, and compositional integrity, thus achieving a more stable, accurate, and user-relevant scene conversion effect.

[0132] The foregoing has provided a detailed description of an image scene conversion method, apparatus, device, and storage medium provided in this application. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0133] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0134] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for scene transformation of an image, characterized in that, The method includes: Obtain the image to be processed and the conversion prompt; the conversion prompt is used to indicate the scene conversion direction and specific attribute adjustment requirements of the image to be processed; The conversion prompt is input into a pre-trained scene noise optimizer for noise optimization processing to obtain structured guiding noise that matches the conversion prompt and this noise is used as the target noise. The image to be processed is vector-encoded using an image encoder to obtain an image feature vector, the conversion prompt word is vector-encoded using a text encoder to obtain a text feature vector, and the image feature vector and the text feature vector are fused using a cross-attention mechanism to obtain a fused vector; Using the fusion vector and the target noise as input, a pre-trained scene transformation model is used to perform conditional decoding on the fusion vector to generate the scene transformation image corresponding to the image to be processed.

2. The method according to claim 1, characterized in that, The pre-training process of the scene transition model includes: Construct a first training dataset; the first training dataset includes first sample pairs, each first sample pair including a feature vector of the image before conversion labeled with a scene conversion label and a corresponding scene conversion editing instruction; the scene conversion label includes a feature vector of the image after conversion; the scene conversion editing instruction is used to describe the scene editing achieved by converting the image before conversion into the image after conversion; Using the first training dataset as input, the diffusion model is trained to obtain the scene transformation model.

3. The method according to claim 1, characterized in that, The pre-training process of the scene noise optimizer includes: The vector quantization generative adversarial network is initialized to obtain the initial noise generation network; Construct a second training dataset; the second training dataset includes second sample pairs, each second sample pair including historical prompt words and corresponding structured guidance noise; the structured guidance noise is scene transition guidance noise determined based on the current historical prompt words; Using historical prompts from the second training dataset as input conditions and the corresponding structured guiding noise as the supervision target, the initial noise generation network is trained to obtain the scene noise optimizer.

4. The method according to claim 1, characterized in that, The step of fusing the image feature vector and the text feature vector through a cross-attention mechanism to obtain a fused vector includes: Using the text feature vector as the query vector and the image feature vector as the key vector and value vector, the similarity between the two is calculated and an attention weight matrix is ​​generated. Based on the attention weight matrix, the weighted summation of each feature value in the image feature vector is performed to obtain the associated features; The associated features are concatenated with each dimension of the text feature vector to obtain the fusion vector.

5. The method according to claim 1, wherein The step of taking the fusion vector and the target noise as input, and using a pre-trained scene transformation model to conditionally decode and generate the fusion vector to obtain the scene transformation image corresponding to the image to be processed includes: The fusion vector and the target noise are subjected to dimension matching processing to obtain a fusion vector and target noise with consistent dimensions. The fusion vector with consistent dimensions and the target noise are input into the diffusion denoising network of the scene transformation model, and iterative denoising processing is performed based on the output of the diffusion denoising network at each step. When the iterative denoising process reaches the preset number of iterations, the output result of the last iteration is used as the initial converted image; The initial converted image is then subjected to detail restoration and color adjustment to obtain the scene-converted image.

6. A scene transition device for an image, characterized in that, The device includes: An acquisition unit is used to acquire the image to be processed and conversion prompts; the conversion prompts are used to indicate the scene conversion direction and specific attribute adjustment requirements of the image to be processed. The noise optimization unit is used to input the conversion prompt word into a pre-trained scene noise optimizer for noise optimization processing, to obtain structured guiding noise that matches the conversion prompt word and to use this noise as the target noise; The encoding fusion unit is used to perform vector encoding on the image to be processed by an image encoder to obtain an image feature vector, to perform vector encoding on the conversion prompt word by a text encoder to obtain a text feature vector, and to perform feature fusion on the image feature vector and the text feature vector by a cross-attention mechanism to obtain a fusion vector; The conversion unit is used to take the fusion vector and the target noise as input, and use a pre-trained scene conversion model to perform conditional decoding and generation processing on the fusion vector to obtain the scene conversion image corresponding to the image to be processed.

7. The apparatus according to claim 6, characterized in that, The device further includes: The first construction unit is used to construct a first training dataset; the first training dataset includes a first sample pair, each first sample pair including a feature vector of the image before conversion labeled with a scene conversion label and a corresponding scene conversion editing instruction; the scene conversion label includes a feature vector of the image after conversion; the scene conversion editing instruction is used to describe the scene editing achieved by converting the image before conversion into the image after conversion; The first model training unit is used to train the diffusion model using the first training dataset as input to obtain the scene transformation model.

8. The apparatus according to claim 6, characterized in that, The device further includes: The initialization unit is used to initialize the vector quantization generative adversarial network to obtain the initial noise generation network; The second construction unit is used to construct a second training dataset; the second training dataset includes second sample pairs, each second sample pair including historical prompt words and corresponding structured guidance noise; the structured guidance noise is scene transition guidance noise determined based on the current historical prompt words; The second model training unit is used to train the initial noise generation network to obtain the scene noise optimizer by taking the historical prompts in the second training dataset as input conditions and the corresponding structured guiding noise as the supervision target.

9. A scene conversion device for images, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the scene transition method for an image as described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the scene transformation method for an image as described in any one of claims 1-5.

Images