An adjustable conditional image generation method, apparatus, computer device, and medium

Abstract

This invention relates to the field of artificial intelligence technology, and discloses an adjustable conditional image generation method, apparatus, computer device, and medium. The method includes encoding input conditions into high-dimensional semantic vectors using a conditional encoder, generating a dynamic computational graph mask via a conditional policy network to allocate local conditional vectors and activation states for each flow layer, selectively transforming initial latent variables using activated flow layers to generate an image, and jointly optimizing network parameters based on a comprehensive reward value. In this invention, addressing the problem that existing normalized flow models cannot adaptively adjust the network architecture according to differences in the complexity of input conditions, the parameters of the conditional policy network and flow layers can be updated based on the calculated comprehensive reward value to obtain the final conditional image generation data. This can be applied to fintech and healthcare fields, effectively reducing redundant computational overhead while ensuring the quality of generated images and the degree of condition matching, achieving an adaptive balance between computational efficiency and generation quality.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an adjustable conditional image generation method, apparatus, computer device, and medium. Background Technology

[0002] Conditional image generation is a core research direction in computer vision, aiming to generate images that conform to semantic constraints based on given conditional information, including but not limited to category labels, text descriptions, or other modal inputs. Traditional conditional image generation methods are mainly based on generative adversarial networks or variational autoencoders; however, these methods have inherent limitations in terms of training stability and generation diversity. In recent years, normalized flow models have gradually become a new paradigm in image generation due to their accurate probability density modeling capabilities and reversible transformation properties, and can be applied to fields such as finance and healthcare. For example, methods such as RealNVP (real-valued non-volume-preserving transform model) and Glow (generative flow model based on reversible 1×1 convolution) achieve high-fidelity image generation by stacking multiple reversible transform layers, demonstrating the significant advantages of normalized flow models in image synthesis tasks.

[0003] However, existing normalized flow models generally employ a fixed computational graph structure. This means that the model uses a pre-determined static network architecture during both training and inference phases, with all flow layers participating in computation in a fixed number and order. This makes it impossible to adaptively adjust to varying input complexity. Specifically, when input conditions are simple, the fixed deep network architecture introduces redundant computation, leading to a waste of computational resources. When input conditions are complex, the fixed network capacity struggles to fully capture the fine-grained semantic features within the conditional information, resulting in insufficient quality of the generated images. This limitation of the static architecture prevents existing conditional normalized flow models from achieving an effective balance between computational efficiency and generation quality, thus hindering the performance of conditional image generation technology in practical applications. Summary of the Invention

[0004] This invention provides an adjustable conditional image generation method, apparatus, computer device, and medium to solve the technical problem that existing normalized flow models use a fixed computation graph structure, which cannot adaptively adjust the network architecture according to the complexity differences of input conditions, resulting in an ineffective balance between computational efficiency and generation quality.

[0005] Firstly, an adjustable conditional image generation method is provided, including: The input conditions are obtained, and the input conditions are encoded and transformed by a preset condition encoder to obtain a high-dimensional semantic vector; The high-dimensional semantic vector is input into the conditional policy network for mask generation calculation to obtain a dynamic computation graph mask. Based on the dynamic computation graph mask, the high-dimensional semantic vector is hierarchically allocated to obtain the local condition vector corresponding to each flow layer, and the activation state of each flow layer is determined. The initial latent variables are obtained, and the initial latent variables are sequentially passed through the flow layer in the active state. The local conditional vector is used to selectively transform the initial latent variables in the invertible residual block and conditional coupling layer of the flow layer to obtain the generated image. The generated image and the input conditions are input into a preset reward calculation module for joint evaluation to obtain a comprehensive reward value; Based on the comprehensive reward value, the parameters of the conditional policy network and the flow layer are updated to obtain the trained conditional image generation data.

[0006] Secondly, an adjustable conditional image generation apparatus is provided, comprising: The data conversion module is used to acquire input conditions and encode and convert the input conditions through a preset condition encoder to obtain a high-dimensional semantic vector. The mask generation module is used to input the high-dimensional semantic vector into the conditional policy network to perform mask generation calculations and obtain a dynamic computation graph mask. The hierarchical allocation module is used to perform hierarchical allocation of the high-dimensional semantic vector based on the dynamic computation graph mask, to obtain the local condition vector corresponding to each flow layer, and to determine the activation state of each flow layer. An image generation module is used to obtain initial latent variables, pass the initial latent variables sequentially through the flow layer in the active state, and use the local conditional vector to selectively transform the initial latent variables in the invertible residual block and conditional coupling layer of the flow layer to obtain the generated image; The joint evaluation module is used to jointly evaluate the generated image and the input conditions by inputting them into a preset reward calculation module to obtain a comprehensive reward value; The parameter update module is used to update the parameters of the conditional policy network and the flow layer based on the comprehensive reward value, so as to obtain the trained conditional image generation data.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described adjustable conditional image generation method.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described adjustable conditional image generation method.

[0009] In the scheme implemented by the above-mentioned adjustable conditional image generation method, apparatus, device, and medium, input conditions can be obtained, and the input conditions can be encoded and transformed by a preset conditional encoder to obtain a high-dimensional semantic vector; the high-dimensional semantic vector is input into a conditional policy network for mask generation calculation to obtain a dynamic computation graph mask; the high-dimensional semantic vector is hierarchically allocated based on the dynamic computation graph mask to obtain local conditional vectors corresponding to each flow layer, and the activation state of each flow layer is determined; initial latent variables are obtained, and the initial latent variables are sequentially passed through the flow layers in the activation state, and the initial latent variables are selectively transformed by the local conditional vectors in the reversible residual blocks and conditional coupling layers of the flow layers to obtain a generated image; the generated image and the input conditions are input into a preset reward calculation module for joint evaluation to obtain a comprehensive reward value; the parameters of the conditional policy network and the flow layers are updated based on the comprehensive reward value to obtain trained conditional image generation data. In this invention, the existing normalized flow model, which uses a fixed computation graph structure, cannot adaptively adjust the network architecture according to the complexity of the input conditions, resulting in an ineffective balance between computational efficiency and generation quality. Instead, the parameters of the conditional policy network and the flow layer can be updated based on the calculated comprehensive reward value to obtain the final conditional image generation data. In this way, the computation path can be flexibly adjusted for input conditions of different complexities, effectively reducing redundant computational overhead while ensuring the quality of the generated image and the degree of condition matching, thus achieving an adaptive balance between computational efficiency and generation quality. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for an adjustable conditional image generation method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an adjustable conditional image generation method according to an embodiment of the present invention. Figure 3 yes Figure 2 A detailed implementation flow diagram of step S20 Figure 1 ; Figure 4 yes Figure 2 A detailed implementation flow diagram of step S40 Figure 2 ; Figure 5 yes Figure 2A detailed implementation flow diagram of step S40 Figure 3 ; Figure 6 yes Figure 2 A detailed implementation flow diagram of step S50 Figure 4 ; Figure 7 yes Figure 2 A detailed implementation flow diagram of step S50 Figure 5 ; Figure 8 yes Figure 2 A detailed implementation flow diagram of step S50 Figure 6 ; Figure 9 This is a schematic diagram of an adjustable conditional image generation device according to an embodiment of the present invention. Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 11 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The adjustable conditional image generation method provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can obtain input conditions from the client, encode and transform the input conditions using a preset conditional encoder to obtain a high-dimensional semantic vector; input the high-dimensional semantic vector into a conditional policy network for mask generation calculation to obtain a dynamic computation graph mask; perform hierarchical allocation of the high-dimensional semantic vector based on the dynamic computation graph mask to obtain local conditional vectors corresponding to each flow layer, and determine the activation state of each flow layer; obtain initial latent variables, and sequentially pass the initial latent variables through the flow layers in the activated state, using the local conditional vectors to selectively transform the initial latent variables in the reversible residual blocks and conditional coupling layers of the flow layers to obtain a generated image; input the generated image and the input conditions into a preset reward calculation module for joint evaluation to obtain a comprehensive reward value; update the parameters of the conditional policy network and the flow layers based on the comprehensive reward value to obtain the trained conditional image generation data. To address the issue that existing normalized flow models cannot adaptively adjust the network architecture based on differences in the complexity of input conditions, this invention updates the parameters of the conditional policy network and flow layer based on the calculated comprehensive reward value, yielding the final conditional image generation data. This data can be applied in fintech and healthcare fields, effectively reducing redundant computational overhead while ensuring image quality and condition matching accuracy, achieving an adaptive balance between computational efficiency and generation quality. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 As shown, Figure 2 A flowchart illustrating an adjustable conditional image generation method provided in an embodiment of the present invention includes the following steps: S10: Obtain the input conditions, and encode and transform the input conditions through a preset condition encoder to obtain a high-dimensional semantic vector.

[0015] First, input conditions are acquired. These input conditions are semantic constraints used to guide the image generation process, and their specific forms may include category labels, text descriptions, or other modal information. For example, in a medical image generation scenario, the input conditions may be pathological description text; in a financial document generation scenario, the input conditions may be document category labels. After acquiring the input conditions, they are input into a preset conditional encoder for encoding conversion. The conditional encoder is a pre-built and trained feature extraction network used to uniformly map different forms of input conditions to a high-dimensional semantic space. The conditional encoder performs feature extraction and dimensionality transformation operations on the input conditions, converting the original conditional information into a high-dimensional semantic vector with rich semantic representation capabilities. The high-dimensional semantic vector carries the core semantic information in the input conditions in the form of a dense numerical vector, providing a unified feature representation basis for subsequent mask generation and conditional injection.

[0016] S20: Input the high-dimensional semantic vector into the conditional policy network to perform mask generation calculation, and obtain a dynamic computation graph mask.

[0017] The high-dimensional semantic vector obtained in step S10 is input into the conditional policy network for mask generation computation. The conditional policy network is a trainable decision network whose core function is to automatically generate computation graph control signals adapted to the semantic features of the input conditions. The conditional policy network receives the high-dimensional semantic vector as input and, through its internal multi-layer mapping and probability sampling mechanism, outputs a set of binary control signals for each flow layer; these control signals are the dynamic computation graph masks. The dynamic computation graph masks, in binary mask form, represent whether each flow layer needs to be activated to participate in computation under the current input conditions, thereby achieving dynamic control of the computation path of the normalized flow model. Through this mechanism, the model can activate more flow layers to enhance generation capabilities for input conditions with high semantic complexity, while reducing the number of flow layers participating in computation to reduce redundancy overhead for input conditions with simpler semantics, thus achieving adaptive allocation of computational resources.

[0018] S30: Based on the dynamic computation graph mask, the high-dimensional semantic vector is hierarchically allocated to obtain the local condition vector corresponding to each flow layer, and the activation state of each flow layer is determined.

[0019] Based on the dynamic computation graph mask generated in step S20, a hierarchical allocation operation is performed on the high-dimensional semantic vector. Each bit of the binary signal in the dynamic computation graph mask corresponds to a flow layer in the normalized flow model. When a bit of the mask is an activation identifier, the corresponding flow layer is determined to be active and will participate in subsequent image generation calculations; when the mask is an inactive identifier, the corresponding flow layer is determined to be inactive and is skipped in the current generation process. After determining the activation state of each flow layer, the high-dimensional semantic vector is further processed by hierarchical allocation. That is, according to the functional positioning and hierarchical position of each active flow layer, the high-dimensional semantic vector is split or transformed into local condition vectors corresponding to each flow layer. Each local condition vector contains the semantic information required to guide the corresponding flow layer to perform condition transformation, so that different flow layers can obtain condition control signals that match their own level, thereby realizing the fine-grained allocation and injection of condition information among the layers of the model.

[0020] S40: Obtain initial latent variables, pass the initial latent variables sequentially through the flow layer in the active state, and use the local conditional vector to selectively transform the initial latent variables in the reversible residual block and conditional coupling layer of the flow layer to obtain the generated image.

[0021] First, initial latent variables are obtained. These initial latent variables are random variables sampled from a preset prior distribution and used as the starting input for the normalized flow model generation process. After obtaining the initial latent variables, they are sequentially fed into each flow layer identified as active in step S30 for selective transformation processing. Within each active flow layer, the latent variables are transformed using the local condition vector corresponding to that flow layer in two sub-modules: the invertible residual block and the conditional coupling layer. Specifically, the conditional coupling layer performs affine transformations on some dimensions of the latent variables using the local condition vectors, injecting conditional information into the latent variable feature space; the invertible residual block, while maintaining the reversibility of the transformation, further enhances the nonlinear features of the latent variables after the transformation by the conditional coupling layer. After sequentially passing through all active flow layers, the initial latent variables are transformed layer by layer and finally mapped to the image data space to obtain the generated image. Since inactive flow layers are skipped in this process, the actual computation path of the model dynamically changes with different input conditions, thereby optimizing computational efficiency while ensuring generation quality.

[0022] S50: The generated image and the input conditions are input into a preset reward calculation module for joint evaluation to obtain a comprehensive reward value.

[0023] The generated image obtained in step S40 and the input conditions obtained in step S10 are jointly input into a preset reward calculation module for evaluation. The reward calculation module is a multi-dimensional quality assessment mechanism used to comprehensively and quantitatively evaluate the quality of the generated image from multiple aspects. Specifically, the reward calculation module comprehensively considers multiple evaluation dimensions, such as the semantic matching degree between the generated image and the input conditions, the visual realism of the generated image, and the diversity among generated images in the same batch. It weights and fuses the evaluation results of each dimension to finally obtain a scalar-form comprehensive reward value. This comprehensive reward value numerically reflects the overall performance level of the current generation result in terms of condition matching degree, image quality, and generation diversity, providing clear gradient guidance signals for subsequent model parameter optimization.

[0024] S60: Update the parameters of the conditional policy network and the flow layer based on the comprehensive reward value to obtain the trained conditional image generation data.

[0025] Based on the comprehensive reward value obtained in step S50, the network parameters of the conditional policy network and each flow layer are jointly optimized and updated. Specifically, the comprehensive reward value is used as the reward signal in the reinforcement learning framework, and the parameters of the conditional policy network are updated using the policy gradient method, enabling it to learn a policy for generating better computational graph masks for different input conditions. Simultaneously, the comprehensive reward value is used as a supervision signal to update the parameters of the invertible residual blocks and conditional coupling layers in each flow layer using gradients, thereby improving the flow model's conditional transformation capability and image generation quality. Through repeated iterations of the above joint optimization process, the conditional policy network gradually learns to allocate the optimal computational path for input conditions of different semantic complexities, and the transformation parameters of each flow layer are continuously optimized to improve the generation effect, ultimately obtaining the trained conditional image generation data. The trained conditional image generation data includes the parameters of the conditional policy network and each flow layer after training convergence, which can be directly used during the inference stage to adaptively generate high-quality images based on new input conditions.

[0026] Combination Figure 3 As shown, step S20 specifically includes: In this embodiment, the process of inputting the high-dimensional semantic vector into the conditional policy network for mask generation calculation to obtain a dynamic computation graph mask in step S20 is specifically implemented through the following steps S201 to S206. The conditional policy network can be constructed using a Transformer architecture, and its core function is to automatically decide whether to activate each flow layer in the normalized flow model based on the semantic features of the input conditions, thereby achieving dynamic configuration of the computation graph. The input to the conditional policy network is a conditional vector, and the dimension of the conditional vector is... The dimension can be, for example, a 768-dimensional CLIP embedding vector; the output of the conditional policy network is a binary mask parameterized by a Bernoulli distribution. ,in This represents the maximum number of layers in the normalized flow model.

[0027] S201: Input the high-dimensional semantic vector as a conditional vector into the conditional policy network, perform matrix multiplication of the conditional vector with the preset first trainable weights and superimpose the first bias term to obtain the first feature vector.

[0028] In this step, the high-dimensional semantic vector obtained in step S10 is used as the condition vector. The input is fed into the conditional policy network. The conditional vector... For one A dense vector of 368 dimensions carries the core semantic information of the input conditions. For example, in a medical image generation scenario, the condition vector can be a 768-dimensional semantic embedding obtained by encoding pathological description text through a conditional encoder; in a financial document generation scenario, the condition vector can be a semantic embedding vector obtained by encoding the document type label through a conditional encoder. The conditional policy network first utilizes a preset first trainable weight. For the condition vector Perform a linear mapping. Specifically, the condition vector... With the first trainable weights Perform matrix multiplication and add the first bias term. The first feature vector is obtained. Here, the first trainable weights... For a dimension The trainable weight matrix, Let be the vector dimension of the hidden layer in the conditional policy network. Conditional vector The dimension; the first bias term To the hidden layer dimension Matching trainable bias parameters. Through this linear mapping operation, the condition vector... From the original The dimensional conditional semantic space is projected to The first feature vector obtained from the dimensional hidden feature space is... This vector initially extracts feature information related to the flow layer activation decision from the input conditions.

[0029] S202: Input the first feature vector into the Gaussian error linear unit for nonlinear activation calculation to obtain the activated feature vector.

[0030] In this step, the first feature vector is input into a Gaussian Error Linear Unit (GeLU) for nonlinear activation calculation to obtain the activated feature vector. The Gaussian Error Linear Unit is a smooth nonlinear activation function. Compared to the traditional ReLU activation function, it introduces a Gaussian cumulative distribution function to probabilistically gate the input, achieving smoother gradient propagation while preserving effective features, which is beneficial for the stable convergence of the conditional policy network during training. The Gaussian Error Linear Unit processes the first feature vector... Each element in the algorithm undergoes a nonlinear transformation, enhancing information highly relevant to the spherical activation decision in the feature space while suppressing redundant information unrelated to it. The resulting activation feature vector after this nonlinear activation calculation is... This vector possesses stronger nonlinear representation capabilities in the hidden feature space, providing a more discriminative feature foundation for subsequent hierarchical probability generation.

[0031] S203: Multiply the activated feature vector by matrix using the preset second trainable weights and add the second bias term to obtain the second feature vector.

[0032] In this step, a pre-set second trainable weight is used. Perform matrix multiplication on the activation feature vectors obtained in step S202, and then add the second bias term. This yields the second feature vector. Wherein, the second trainable weights... For a dimension The trainable weight matrix, In the normalized flow model, the first... The weight parameters corresponding to each flow layer The hidden layer vector dimension; the second bias term This corresponds to the scalar bias parameter. The calculation process in this step will... The activation feature vector of dimension 1 is mapped to a scalar value, that is, the information of each dimension in the activation feature vector is weighted and aggregated to form a value for the first dimension. The activation decision scores of each flow layer. The resulting second feature vector is... This value reflects the first value under the current input conditions. The degree to which each flow layer is activated. It should be noted that the second trainable weight... With the Each flow layer corresponds one-to-one, therefore for the normalized flow model... Each flow layer, the conditional policy network maintains respectively The second group of trainable weights enables fine-grained control over the activation decisions of each flow layer independently.

[0033] S204: Use the activation function to perform probability mapping on the second feature vector to obtain the Bernoulli distribution parameterized hierarchical probability values.

[0034] The second feature vector obtained in step S203 is processed by using the Sigmoid activation function to perform probability mapping, mapping it from the real number domain to the probability space. The Bernoulli distribution parameterized hierarchical probability values ​​are obtained. The Sigmoid activation function is denoted as... The following probability mapping calculation is performed on the second feature vector to obtain the first feature vector. The probability value of each flow layer at the level of the flow layer : in, Indicates that under given input conditions Under the premise that, the first Binary mask for each flow layer The probability of taking a value of 1 (i.e., the flow layer is activated); For the Sigmoid function; For the first The second trainable weights corresponding to each flow layer; This is the first trainable weight; This is the first bias term; This is the second bias term; This is the hierarchical index of the flow layer, with a value range of 100. to The hierarchical probability value Characterizes the conditional policy network for the first... The determination of the necessity of activating each flow layer is based on a probability value. A probability value closer to 1 indicates that the flow layer is more necessary to participate in the image generation calculation under the current conditions, while a probability value closer to 0 indicates that the flow layer can be skipped. For example, when the input condition is a semantically complex medical image pathology description, the conditional policy network tends to output more high-probability values ​​to activate more flow layers and fully capture complex pathological semantic features; when the input condition is a semantically simple financial instrument type label, the conditional policy network tends to output more low-probability values, activating only the necessary flow layers to reduce computational overhead.

[0035] S205: Obtain random noise, perform logarithmic operation on the hierarchical probability value to obtain a probability logarithmic value, and add the probability logarithmic value to the random noise to obtain a noisy logarithmic feature.

[0036] To ensure gradient propagation in the discrete sampling operation during mask generation, a Gumbel-Softmax relaxation mechanism is introduced to randomly perturb the hierarchical probability values ​​obtained in step S204. Specifically, random noise is first acquired. The random noise It follows the standard Gumbel distribution, i.e. The Gumbel distribution is an extreme value distribution, and its introduction provides random perturbation to the mask sampling process, enabling the conditional policy network to have sufficient exploration capabilities in the early stages of training and avoiding premature trapping in suboptimal mask configurations. The hierarchical probability values ​​obtained in step S204... (Right now Perform logarithmic operations to obtain the logarithmic probability value. The logarithmic operation transforms the hierarchical probability values ​​in the probability space to the logarithmic space, enabling it to be operated on at the same scale as random noise following a Gumbel distribution. Finally, the logarithmic probability values ​​are... With the random noise Perform element-wise addition to obtain the noisy logarithmic features. The noisy logarithmic features integrate the deterministic judgment of the necessity of activation of each flow layer by the conditional policy network and the random exploration signal from the Gumbel distribution, laying the foundation for subsequent relaxed discrete sampling through temperature control.

[0037] S206: The noisy logarithmic feature and the annealing temperature coefficient are used to calculate the relaxation feature. The relaxation feature is then subjected to exponential operation and normalization calculation to generate a multi-layer binary mask and combined to obtain a dynamic computation graph mask.

[0038] In this step, the noisy logarithmic features obtained in step S205 are temperature-scaled using an annealing temperature coefficient, and binary masks for each flow layer are generated through exponential and normalization calculations. These masks are then combined to obtain a dynamic computational map mask. First, the noisy logarithmic features... Divide by the annealing temperature coefficient relaxation characteristics were obtained. Subsequently, the relaxed features are subjected to exponential operations and normalization calculations to generate the first... The binary mask corresponding to each flow layer The calculation formula is as follows: in, For the first The binary mask value corresponding to each flow layer; For the first The probability value of each flow layer; For random noise that follows a standard Gumbel distribution; The annealing temperature coefficient; summation operations in the denominator. The relaxed features of the two categories, activation and inactivation, are normalized to make the output values ​​fall within the normal range. Interval.

[0039] The annealing temperature coefficient It is a key hyperparameter controlling the degree of discretization during mask generation. In the early stages of training, the annealing temperature coefficient... Set to a higher value (e.g.) At this point, the output of the normalized calculation is relatively smooth, and the mask values ​​of each flow layer tend to be evenly distributed. The conditional policy network can fully explore different flow layer activation combinations with greater randomness, avoiding convergence to a local suboptimal solution due to insufficient exploration in the early stages of training. As training progresses, the annealing temperature coefficient... As the annealing strategy is gradually reduced to below 0.1, the output of the normalized calculation gradually approaches a discrete binary form. The mask values ​​of each flow layer concentrate at either 0 or 1, causing the output of the conditional policy network to gradually approach the true binary hard mask. This gradual annealing process from soft to hard ensures both the effective propagation of gradients during the training phase and the acquisition of clear binary mask decisions during the inference phase.

[0040] Finally, in the normalized flow model The binary mask generated by each flow layer By combining the results, a dynamic computation graph mask is obtained. The dynamic computation graph mask is based on... The binary vector form fully describes the activation configuration scheme of each flow layer in the normalized flow model under the current input conditions, providing a decision basis for the layer allocation in step S30 and the selective transformation in step S40. Through the above mask generation mechanism, the conditional policy network can adaptively adjust the computational graph structure of the model according to the semantic complexity of different input conditions. For example, when facing medical pathology image generation tasks with high semantic fine-grained requirements, more flow layers are activated to obtain stronger feature modeling capabilities, while when facing financial bill image generation tasks with relatively regular semantic structures, the number of activated flow layers is appropriately reduced to improve computational efficiency.

[0041] Combination Figure 4 As shown, step S40 specifically includes: In this embodiment, step S40, which involves selectively transforming the initial latent variables using the local conditional vector in the conditional coupling layer of the flow layer to obtain the generated image, is specifically implemented through steps S401 to S406. The flow stack of the normalized flow model contains multiple invertible layers, among which the Conditional Affine Coupling Layer is the core component for injecting conditional information. The Conditional Affine Coupling Layer splits the latent variables into two equal variables along the channel dimension and uses a conditional function driven by the local conditional vector to alternately perform affine transformations on the two equal variables, thereby integrating conditional semantic information layer by layer into the feature representation of the latent variables while maintaining the invertibility of the transformation. It should be noted that the local conditional vector... Before entering each flow layer, the dynamic calculation of the map mask has already undergone modulation processing in step S30, i.e. ,in For the first The binary mask value corresponding to each flow layer. Through this mask modulation mechanism, the decision results of the conditional policy network are transmitted to each flow layer, so that only the active state ( The flow layer can receive valid local condition vectors and participate in transformation calculations, while the inactive layer ( The flow layer is effectively skipped because it receives a nulled condition vector.

[0042] S401: Obtain the Gaussian noise corresponding to the initial latent variable, and split the Gaussian noise to obtain the first segmentation variable and the second segmentation variable.

[0043] First, obtain the Gaussian noise corresponding to the initial latent variable. The Gaussian noise For the first The input latent variables of each flow layer, when When the first activated flow layer is, the Gaussian noise That is, from the standard Gaussian distribution The initial latent variables obtained from sampling; when The Gaussian noise is used for subsequent activation of the flow layer. This is the intermediate latent variable output from the previous activated flow layer. The Gaussian noise is then obtained. Then, it is divided equally along the channel dimension to obtain the first segmentation variable. Second split variable The first segmentation variable Second split variable Each contains the Gaussian noise In terms of channel dimension, the feature information in the first and second halves has the same dimension and is both original Gaussian noise. Half of the channel dimension. This operation of equally dividing along the channel dimension is the basic design paradigm of affine coupling layers. Its purpose is to enable the two segmentation variables to alternately transform conditionally, thereby achieving sufficient feature interaction while ensuring the reversibility of the transformation. For example, in the medical image generation scenario, the first segmentation variable and the second segmentation variable can respectively carry the structural features and texture features of different channel levels in the medical image; in the financial document image generation scenario, they can respectively correspond to the layout features and text texture features in the document image.

[0044] S402: Input the second segmentation variable and the local condition vector into a preset condition function mapping to obtain the first condition output and the second condition output respectively.

[0045] In this step, the second segmentation variable obtained in step S401 is... With the local condition vector after masking modulation The inputs are fed into a preset conditional function mapping, and the first conditional outputs are obtained respectively. Second conditional output The condition function corresponding to the first condition output. The conditional function corresponding to the second conditional output All are learnable nonlinear mapping functions implemented by a three-layer Multi-Layer Network (MLN). The three-layer MLN uses a second segmentation variable... and local condition vector The concatenated or fused features are taken as input, and after three layers of fully connected computation and nonlinear activation processing, the output is a feature vector with the same dimension as the segmentation variables. The first conditional output... The scaling parameter vector, whose element values ​​control the magnitude and direction of the element-wise scaling transformation of the first segmentation variable; the second conditional output This is a translation parameter vector, where each element controls the offset by which the first segmentation variable is translated element-by-elementally. The condition function... and Output dimension and segmented same, This is the hierarchy index of the current flow layer. (This is achieved by using the second partition variable.) and local condition vector Both are used as inputs to the conditional function, so that the generated scaling and translation parameters depend on the feature distribution of the latent variables themselves and the semantic information of the input conditions, thereby realizing condition-driven adaptive affine transformation parameter generation.

[0046] S403: Perform an exponential operation on the first conditional output, and combine it with the first segmentation variable and the second conditional output for a fusion calculation to obtain the first updated variable.

[0047] Output the first condition obtained in step S402 Second conditional output For the first splitting variable Perform a conditional affine transformation to obtain the first updated variable. Specifically, first, output the first condition. Perform element-wise exponentiation to obtain a positive scaling factor. The exponential operation ensures that the scaling factor is always positive, thus ensuring the reversibility of the affine transformation; then the first segmentation variable... Element-wise multiplication with the positive scaling factor (Hadamard product, ... (represented), and the multiplication result is output with the second condition. Perform element-by-element addition to obtain the first updated variable. The expression for the above fusion calculation process is as follows: in, This is the first updated variable, i.e., the first partitioning variable after the conditional affine transformation; This is the first dividing variable; For element-wise multiplication (Hadamard product); This is an element-wise exponentiation operation; The first conditional output is the scaling parameter vector; This is the output of the second condition, namely the translation parameter vector; This is the hierarchical index of the current flow layer. Through the above conditional affine transformation, the second segmentation variable... The feature information and local condition vector contained within The semantic information carried in the middle is effectively injected into the update process of the first segmentation variable, so that the first updated variable... This transformation integrates conditional semantic features while preserving the structural information of the original first segmentation variables. For example, in medical image generation, this transformation can integrate lesion morphology information from pathological feature descriptions into the structural channel features of the image; in financial bill generation, this transformation can map bill type information to the layout channel features of the bill image.

[0048] S404: Input the first updated variable and the local condition vector into the preset condition function mapping to obtain the third condition output and the fourth condition output respectively.

[0049] The first update variable obtained in step S403 With local condition vector The inputs are fed into another set of preset conditional function mappings, and the third conditional outputs are obtained respectively. and the fourth conditional output The condition function corresponding to the third condition output. The conditional function corresponding to the fourth conditional output Similarly, it is a learnable nonlinear mapping function implemented by a three-layer multilayer sensing network, and its network structure is the same as the condition function in step S402. and The same, but maintains independent trainable parameters. The condition function... and Update the first variable and local condition vector The fusion features are used as input, and the output dimension is the second segmentation variable. The same feature vector. Specifically, the third conditional output The fourth conditional output is a scaling parameter vector oriented towards the second segmentation variable. This is the translation parameter vector for the second segmentation variable. Unlike step S402, where the second segmentation variable drives the update of the first segmentation variable, this step uses the first update variable to drive the update of the second segmentation variable, thereby achieving bidirectional cross-information interaction between the two segmentation variables. This alternating coupling transformation design ensures sufficient feature fusion between the dimensions of each channel of the latent variable, enabling conditional semantic information to propagate and permeate bidirectionally between the two segmentation variables.

[0050] S405: Perform calculations on the second segmentation variable based on the third and fourth conditional outputs to obtain the second updated variable.

[0051] Output using the third condition obtained in step S404 and the fourth conditional output For the second splitting variable Perform a conditional affine transformation to obtain the second updated variable. The calculation process of this transformation is consistent with the affine transformation structure of the first segmentation variable in step S403. First, the third condition output is... Perform element-wise exponentiation to obtain a positive scaling factor. Then the second splitting variable Multiply the result element-wise with the positive scaling factor, and then combine the result with the fourth condition output. Perform element-by-element addition to obtain the second updated variable. The mathematical expression for the above calculation process is as follows: in, This is the second update variable, that is, the second partitioning variable after conditional affine transformation; This is the second dividing variable; This is an element-wise multiplication operation; This is an element-wise exponentiation operation; This is the output of the third condition, namely the scaling parameter vector oriented towards the second segmentation variable; This is the fourth conditional output, namely the translation parameter vector oriented towards the second segmentation variable. Through the above transformation, the first updated variable... The conditional semantic features that have already been integrated are further passed to the update process of the second segmentation variable, making the second updated variable... Sufficient conditional information was thus injected. At this point, the conditional affine coupling layer completed the alternating bidirectional affine transformation of the two segmentation variables, achieving uniform penetration of conditional information across all channel dimensions of the latent variables.

[0052] S406: Combine the first update variable with the second update variable to obtain the affine transformation output.

[0053] The first update variable obtained in step S403 The second update variable obtained in step S405 By splicing and combining along the channel dimension, the affine transformation output is obtained. The affine transformation output Dimensions and input Gaussian noise Maintaining consistency means restoring the full channel dimension of the original latent variable by re-spoofing the two half-channel features that have undergone conditional affine transformation. The affine transformation output... It integrates the semantic information of the input conditions and the distribution characteristics of the original latent variables, serving as the current... The output of each flow layer is passed to the next active flow layer for further layer-by-layer transformation. Through the splitting, alternating transformation, and combination processes in the conditional affine coupling layers, the normalized flow model achieves condition-driven reversible feature transformation in each active flow layer. This allows latent variables to gradually map from a simple Gaussian distribution to a complex conditional image data distribution as they pass through multiple active flow layers. After all active flow layers have completed the transformation, the final latent variables are the generated image. For example, in a medical image generation scenario, the latent variables after multiple conditional affine transformations are mapped to medical images that conform to pathological description semantics; in a financial document generation scenario, the final output generated image is a financial document sample image that matches the document type label.

[0054] Combination Figure 5As shown, after step S406, the following steps are included: In this embodiment, after obtaining the affine transformation output in step S406, the affine transformation output is further subjected to nonlinear feature enhancement processing through an invertible residual block. The invertible residual block is the second type of invertible layer in the stream stack. Its core design idea lies in constraining the Lipschitz constant of the residual branch network by introducing spectral normalization technology, thereby strictly ensuring the invertibility of the transformation while providing deep nonlinear feature enhancement for the latent variables. The conditional affine coupling layer focuses on injecting conditional semantic information into the feature space of the latent variables, while the invertible residual block focuses on further improving the feature representation ability and detailed modeling accuracy of the latent variables based on conditional injection.

[0055] S407: The affine transformation output and the local conditional vector input are processed by a preset residual branch network to obtain residual features.

[0056] Output the affine transformation obtained in step S406. With the local condition vector after masking modulation The inputs are fed into a pre-defined residual branch network for computation to obtain residual features. This residual branch network is denoted as... This is a learnable nonlinear mapping network used to extract deep residual features after the fusion of latent variables and conditional information. Specifically, the residual branch network... Receive the affine transformation output and local condition vector As a joint input, the two are subjected to feature fusion and multi-layer nonlinear transformation processing, and the output is the same as the affine transformation output. Residual features with the same dimensions The residual features This characterizes the incremental feature information that latent variables need to supplement under the current conditional semantic constraints. By simultaneously inputting the affine transformation output and the local conditional vector into the residual branch network, the generated residual features not only depend on the distribution characteristics of the latent variables themselves but are also modulated and guided by the input conditional semantic information, thereby achieving condition-driven residual feature generation. For example, in medical image generation scenarios, the residual branch network can extract incremental detail features related to lesion morphology and tissue texture based on the conditional semantics of pathological description text; in financial document image generation scenarios, the residual branch network can generate incremental features related to document anti-counterfeiting texture and seal details based on document type conditional information.

[0057] S408: Obtain the weight matrix and perform spectral normalization on the weight matrix to obtain spectral normalized weights. Multiply the spectral normalized weights with the residual features to obtain normalized residual features.

[0058] Obtain the weight matrix in the invertible residual block. and the weight matrix Perform spectral normalization to obtain spectral normalized weights. The weight matrix represents the trainable linear transformation parameters in the invertible residual block, used for linear mapping of the residual features. The spectral normalization operation is denoted as... Spectral normalization is a regularization technique that constrains the spectral norm (i.e., the largest singular value) of the weight matrix to ensure it does not exceed a preset threshold. Specifically, the spectral normalization operation calculates the weight matrix... Maximum Singularity The weight matrix is ​​then divided by this maximum singular value, constraining the spectral norm of the normalized weight matrix to 1. Through this spectral normalization operation, the Lipschitz constants of the linear transformation portion of the residual branch are strictly constrained, thus ensuring the invertibility of the entire invertible residual block transformation. This is crucial for normalized flow models, as these models require each layer of transformation to be an invertible mapping, and unconstrained residual connections can lead to singular Jacobian matrices in the transformation, thereby violating invertibility. Subsequently, the spectral normalization weights are... The residual characteristics obtained in step S407 Perform matrix multiplication to obtain the normalized residual characteristics. The normalized residual feature retains the condition-driven incremental feature information extracted by the residual branch network, while its magnitude is effectively constrained by the spectral normalization weight, thus avoiding numerical instability or reversibility destruction caused by excessively large residual features.

[0059] S409: Obtain the scaling factor, and multiply the scaling factor with the normalized residual feature to obtain the scaled residual feature.

[0060] Get the preset scaling factor The scaling factor The normalized residual characteristics obtained in step S408 Element-wise multiplication is performed to obtain the scaled residual features. The scaling factor This is a preset scalar hyperparameter used to further control the impact of residual branches on the main path features. In this embodiment, the scaling factor... The value is set to 0.8, that is... The design considerations for this value are: on the one hand, This ensures that the contribution magnitude of the residual branch is strictly less than the characteristic magnitude of the main path, and together with the spectral normalization operation, guarantees that the Lipschitz constant of the invertible residual block transform is strictly less than 1, thus providing a double guarantee for the invertibility of the transform; on the other hand, The value of ensures that the residual branch still provides a sufficiently significant feature enhancement effect, avoiding excessive suppression of the feature enhancement capability of the residual branch due to an excessively small scaling factor. Through the synergistic constraint mechanism of spectral normalization and scaling factor, the reversible residual block can provide stable and effective nonlinear feature enhancement for latent variables while strictly ensuring reversibility, thereby improving the detail quality and semantic consistency of the generated image.

[0061] S4010: Add the scaling residual feature to the affine transformation output to obtain the next layer of latent variables.

[0062] In this step, the scaled residual features obtained in step S409 are... With the affine transformation output Perform element-wise addition to obtain the next level of latent variables. The mathematical expression for the above addition calculation process is as follows: in, For the next level of latent variables, i.e., the current [number]th [level] The output of each flow layer after being processed by a reversible residual block; The output of the affine transformation, i.e. the output of the conditional affine coupling layer, is used as the identity mapping input of the main path in the invertible residual block. This is the scaling factor, with a value of 0.8; The weight matrix is ​​after spectral normalization. Indicates spectral normalization operation; For residual branch network Output via affine transformation and local condition vector The residual features are calculated as input; This is the hierarchical index of the current flow layer. The above formula embodies the core idea of ​​residual connections, namely, the latent variables of the next layer are mapped by the identity of the main path. The residual branch output is subject to dual constraints of spectral normalization and scaling factor. It is formed by superposition. Because the spectral normalization operation affects the weight matrix... The spectral norm constraint is 1, and the scaling factor is... Therefore, the global Lipschitz constant of the residual branch is strictly less than 1, and according to the Banach fixed-point theorem, the invertibility of this residual mapping is theoretically guaranteed. The next layer of latent variables... While retaining the conditional semantic features after the conditional affine coupling layer transformation, it further integrates the deep nonlinear incremental features provided by the residual branch network, making the feature expression of latent variables richer and more refined.

[0063] S4011: The next layer latent variable is processed through the corresponding network layer to finally obtain the generated image.

[0064] The next-level latent variables obtained in step S4010 The process continues to be passed to subsequent active flow layers for multi-level processing. In the flow stack of the normalized flow model, each active flow layer can be considered an independent flow, containing two sub-modules: a conditional affine coupling layer and a reversible residual block. The next layer's latent variables... As input to the next activated flow layer, the processing steps S401 to S4010 are repeated in this flow layer. That is, conditional information is first injected and affine transformed through the conditional affine coupling layer, and then residual feature enhancement under spectral normalization constraints is performed through the invertible residual block to obtain higher-level latent variables. This process proceeds layer by layer, with latent variables being transformed and mapped from an initial simple Gaussian distribution to a complex conditional image data distribution space after passing through all activated flow layers in sequence. The final output latent variables are the generated image. In this multi-layer processing, due to the control effect of the dynamically computed graph mask in step S30, only activated flow layers participate in the actual computation, while inactive flow layers are skipped. Therefore, the effective computational depth of the model dynamically changes with the semantic complexity of the input conditions. For input conditions with high semantic complexity, the conditional policy network tends to activate more flow layers to provide stronger feature transformation capabilities and more refined conditional modeling, thereby ensuring high fidelity and semantic consistency of the generated image. For input conditions with relatively simple semantics, the conditional policy network appropriately reduces the number of activated flow layers to avoid redundant computation, significantly improving computational efficiency while ensuring generation quality. For example, in medical image generation scenarios, when the input condition is complex pathological text containing descriptions of various lesion types, the model activates deeper computational paths to accurately capture the morphological features and tissue texture details of various lesions, ultimately generating high-fidelity medical image samples for assisting clinical diagnostic training. In financial bill image generation scenarios, when the input condition is a relatively standardized bill type label, the model can generate bill image samples with standard layout and clear text texture through shallower computational paths, which are used for data augmentation and model verification in financial bill recognition systems, while significantly reducing the generation time of a single image.

[0065] Combination Figure 6 As shown, step S50 specifically includes: In this embodiment, step S50, which involves jointly evaluating the generated image with the input conditions and inputting them into a preset reward calculation module to obtain a comprehensive reward value, includes calculations across multiple evaluation dimensions. The calculation of the perceptual diversity reward value is a crucial component of the comprehensive reward value. Its purpose is to measure the degree of visual difference between multiple generated images within the same batch, encouraging the model to generate image samples with rich diversity while ensuring conditional semantic consistency. The evaluation of perceptual diversity is based on the Learned Perceptual Image Patch Similarity (LPIPS) distance. LPIPS distance is an indicator that uses multi-layer perceptual features extracted by a pre-trained deep neural network to measure the visual perceptual difference between two images. Compared to traditional pixel-level distance metrics (such as mean squared error), LPIPS distance more accurately reflects the human visual system's perceptual judgment of image differences.

[0066] S501: Perform set processing on the generated images to obtain a batch generated image set.

[0067] In each training iteration of the model, the normalized flow model generates multiple images starting from different initial latent variables, guided by the same input conditions. Each generated image is composed of different initial Gaussian noise. After being transformed layer by layer through each active flow layer, it is denoted as... ,in This represents the forward generation mapping of the normalized flow model. For the first There are one initial latent variable. All images generated in the same training iteration are aggregated into a single set, resulting in the batch-generated image set. ,in This refers to the total number of batches, i.e., the number of generated images contained in the current batch. The batch-generated image set is the basic data unit for subsequent calculation of the perceptual diversity reward value. In practical applications, the total number of batches... The batch size setting needs to balance the statistical reliability of diversity assessment with computational overhead. Too small a batch size will result in a large variance in the diversity assessment results, while too large a batch size will increase the computational complexity of pairwise comparisons. For example, in a medical image generation scenario, the batch-generated image set may contain multiple medical image samples with different manifestations generated under the same pathological condition description, used to evaluate whether the model can generate image results with multiple lesion morphological changes; in a financial document image generation scenario, the batch-generated image set may contain multiple document image samples with different layout details generated under the same document type label, used to evaluate whether the diversity of the generated samples meets the requirements of data augmentation.

[0068] S502: For the generated images that are different in the batch generated image set, perform learning perception image block similarity distance calculation between each pair to obtain multiple sets of image difference distance values.

[0069] In step S501, all dissimilar generated images in the batch generated image set are paired up, and the learned perceptual patch similarity (LPIPS) distance is calculated between each pair of images to obtain multiple sets of image difference distance values. For any two dissimilar generated images in the batch generated image set... and (in ), calculate the LPIPS distance between the two The LPIPS distance calculation process is as follows: Two generated images are input into a pre-trained deep feature extraction network (such as a VGG network). Feature maps of each intermediate layer are extracted. Channel-by-channel normalization is performed on the feature maps of each layer for both images, and then the weighted Euclidean distance between them is calculated. Finally, the distances of each layer are summed to obtain the LPIPS distance value. A larger LPIPS distance value indicates a more significant difference between the two generated images at the deep perceptual feature level, i.e., a greater difference in visual perception. Since the batch-generated image set contains… The total number of combinations of two distinct pairwise pairings in the generated image is . Groups (considering the symmetry of LPIPS distance), but in the summation calculation, for all ordered pairs The traversal is performed, therefore the actual calculated image difference distance value is Group. By calculating the LPIPS distance for all different image pairs within a batch, it is possible to comprehensively capture the differential distribution of multi-layer visual perception features among samples in the batch-generated image set.

[0070] S503: Summing the multiple sets of image difference distance values ​​to obtain the total image distance.

[0071] Generate all images in the batch that satisfy the following conditions ordered image pairs The corresponding LPIPS distance values ​​are summed one by one to obtain the total image distance. The sum of image distances reflects the total cumulative perceptual differences among all generated images in the current batch. A larger sum indicates higher overall diversity of generated images within the batch; conversely, a smaller sum indicates that the generated images within the batch tend to be homogeneous at the visual perception level, and the model is at risk of mode collapse. This sum of image distances, as an intermediate calculation result, will be normalized and transformed into a statistically significant average perceptual diversity measure in subsequent steps.

[0072] S504: Obtain the total number of batches of the image set generated in the batch, calculate the difference between the total number of batches and a preset value to obtain the total difference, and multiply the total number of batches and the total difference to obtain the batch combination number.

[0073] Obtain the total number of batches of image sets generated in step S501. And based on the total number of batches Calculate the normalized denominator, i.e., the number of batch combinations. First, calculate the total number of batches. The difference between the value and the preset value of 1 is used to obtain the total difference. The preset value is a constant of 1. Then, the total number of batches... Difference from the total Perform multiplication to obtain the batch combination number. The number of batch combinations Equal to all images in the batch-generated image set that satisfy The total number of ordered image pairs, i.e., from The number of orderly pairings of two images selected from different images. The batch combination number serves as a normalization factor, used to convert the sum of image distances in step S503 into the average perceptual difference value between each pair of images, thereby eliminating the total number of batches. The magnitude of the value affects the perceived diversity reward, making the perceived diversity reward values ​​comparable across different batch sizes.

[0074] S505: Calculate the perceptual diversity reward value by using the sum of the image distances and the number of batch combinations.

[0075] The perceptual diversity reward value is obtained by dividing the sum of image distances obtained in step S503 by the number of batch combinations obtained in step S504. The formula for calculating the reward value for perceived diversity is as follows: in, To perceive diversity reward value; The total number of batches for generating image sets; The number of batch combinations serves as a normalization factor. It means that for all satisfying The ordered image pairs are traversed and summed. For the first Zhang Sheng Image With the Zhang Sheng Image Learning the similarity distance between image patches; Forward generation mapping of the normalized flow model; and The first The and the first One initial latent variable.

[0076] The perceived diversity reward value The meaning of is the arithmetic mean of the LPIPS distances between all different generated image pairs within a batch, and its value range is a non-negative real number. A larger perceptual diversity reward value indicates that the generated images within the current batch are more dispersed and diverse in the deep visual perception feature space; conversely, a smaller perceptual diversity reward value closer to zero indicates that the generated images are highly similar in visual perception, and the model faces the problem of pattern collapse. By incorporating the perceptual diversity reward value into the calculation system of the comprehensive reward value, a continuous positive incentive signal for the diversity of generated images can be provided during the model's training and optimization process, guiding the conditional policy network and flow layer parameters to optimize in a direction that can generate diverse outputs. For example, in the medical image generation scenario, the perceptual diversity reward value can encourage the model to generate diverse medical image samples with different lesion locations, sizes, and morphological features under the same pathological conditions, avoiding the model from generating only a single pattern of lesion manifestations, thereby providing richer and more comprehensive data support for the training of the clinical image analysis system; in the financial bill image generation scenario, the perceptual diversity reward value can prompt the model to generate bill image samples with different font arrangements, seal positions, and background texture details under the same bill type conditions, enhancing the generalization ability and robustness of the financial bill recognition system when facing diverse real bills.

[0077] Combination Figure 7 As shown, step S50 further includes: In this embodiment, step S50, which involves jointly evaluating the generated image and the input conditions using a preset reward calculation module to obtain a comprehensive reward value, also includes calculating a condition matching degree reward value. This condition matching degree reward value measures the semantic alignment between the generated image and the input conditions, i.e., quantitatively assessing the extent to which the generated image faithfully reflects the semantic content described by the input conditions. The calculation of the condition matching degree reward value is based on a pre-trained Contrastive Language-Image Pre-training (CLIP) model. The CLIP model is a multimodal representation model obtained through contrastive learning pre-training on a large-scale image-text pairing dataset. Its core capability lies in mapping images and text to a unified semantic embedding space, ensuring that semantically related images and texts have high similarity in this embedding space, while semantically unrelated image-text pairs have low similarity. Leveraging the powerful cross-modal semantic alignment capabilities of the CLIP model, the semantic consistency between the generated image and the input conditions can be effectively evaluated, providing reward feedback signals for the condition matching dimension of model training.

[0078] S506: Obtain the contrastive language image pre-training model, input the generated image into the image encoder of the contrastive language image pre-training model for encoding extraction, and obtain image encoding features.

[0079] First, obtain the pre-trained contrastive language image pre-trained model (CLIP model). The CLIP model contains two core sub-modules: an image encoder and an image encoder. and text encoder During the pre-training phase, the two are jointly optimized by comparing the learning objectives, thereby enabling them to map images and text to a shared semantic embedding space. After obtaining the CLIP model, the generated image obtained in step S40 is... Image encoder input to the CLIP model Image coding features are extracted during encoding. The image encoder Deep vision network architectures such as VisionTransformer or ResNet can be used to receive and generate images. As input, through multi-layer feature extraction and transformation processing, the visual content of the input image is compressed and encoded into a dense vector of fixed dimensions, namely the image coding feature. The image encoding features reside in the cross-modal shared semantic embedding space constructed during CLIP model pre-training. They carry the core visual semantic information of the generated image in the form of high-dimensional dense vectors, including multi-level visual semantic elements such as object categories, spatial layout, texture details, and overall style. For example, in a medical image generation scenario, the image encoder can extract the morphological features of lesion areas, the contrast features of tissue structures, and the overall modal style features of the image from the generated medical image, and encode this visual semantic information into image encoding feature vectors in the cross-modal embedding space. In a financial document image generation scenario, the image encoder can capture visual semantic information such as document layout, text area distribution, seal markings, and anti-counterfeiting textures from the generated document image and encode them into corresponding image encoding feature vectors. It should be noted that during reward calculation, the parameters of the CLIP model remain frozen and do not participate in gradient updates; they are only used as a pre-trained semantic evaluation tool to ensure the stability and objectivity of the conditional matching evaluation.

[0080] S507: Input the input conditions into the text encoder of the contrastive language image pre-training model for encoding extraction to obtain text encoding features.

[0081] The input conditions obtained in step S10 The text encoder input to the CLIP model The text encoding features are extracted during encoding. The text encoder It is typically built using the Transformer architecture, which receives input conditions. The corresponding text sequence is used as input. Through processing steps such as tokenization, positional encoding, and multi-layer self-attention calculation, the conditional information in text form is encoded into a dense vector of fixed dimensions, namely the text encoding feature. The text encoding features and the image encoding features obtained in step S506 are in the same cross-modal shared semantic embedding space and have the same vector dimension. This is guaranteed by the cross-modal semantic alignment mechanism established by the CLIP model through contrastive learning pre-training. The text encoding features carry the core textual semantic information in the input conditions in the form of high-dimensional dense vectors, including multi-level textual semantic elements such as entity concepts, attribute features, spatial relationships, and abstract semantics involved in the condition description. For example, in the medical image generation scenario, when the input condition is a pathological description text such as "lung CT image - ground-glass nodule", the text encoder can encode the image modality information (CT), anatomical location information (lung), and lesion type information (ground-glass nodule) involved in it into a text encoding feature vector in the cross-modal embedding space; in the financial bill image generation scenario, when the input condition is a bill type label such as "value-added tax special invoice", the text encoder can encode the semantic information of the bill type into the corresponding text encoding feature vector. Since both image-encoded features and text-encoded features reside in the same semantic embedding space constructed during CLIP model pre-training, the distance or similarity between them can effectively reflect the semantic alignment between the generated image and the input conditions.

[0082] S508: Calculate the cosine similarity between the image encoding features and the text encoding features to obtain the image-text alignment score, and use the image-text alignment score as the conditional matching reward value.

[0083] In this step, the image encoding features obtained in step S506 are... The text encoding features obtained in step S507 Cosine similarity is calculated to obtain the image-text alignment score, and this score is used as the conditional matching reward value. The formula for calculating the condition matching degree reward value is as follows: in, This is the conditional matching reward value, i.e., the image-text alignment score; This is the function for calculating cosine similarity. To generate an image Image encoder via CLIP model The encoded features of the image obtained after encoding; For input conditions Text encoder via CLIP model The encoded features of the text obtained after encoding; Forward generation mapping of the normalized flow model; These are the initial latent variables; The input condition is specified. The cosine similarity calculation measures the degree of directional consistency between two feature vectors in the semantic embedding space by calculating the cosine of the angle between them, with a value range of [-1, 1]. Specifically, the cosine similarity calculation process is as follows: the image-encoded feature vector and the text-encoded feature vector are multiplied element-wise and summed to obtain their inner product value, which is then divided by the respective values ​​of the two feature vectors. The cosine similarity value can be obtained by multiplying the norms.

[0084] The condition matching degree reward value The closer the value is to 1, the more consistent the orientation of the generated image's image encoding features with the input condition's text encoding features in the cross-modal semantic embedding space. This means a higher degree of semantic alignment between the generated image and the input condition, and a more faithful reflection of the semantic content described by the input condition. Conversely, a condition matching reward value closer to -1 indicates a significant semantic deviation between the generated image and the input condition. By directly incorporating the image-text alignment score as the condition matching reward value into the calculation system of the comprehensive reward value, continuous quantitative feedback signals regarding the consistency of conditional semantics can be provided for model training, guiding the conditional policy network and flow layer parameters to optimize towards generating images that are highly aligned with the semantics of the input condition. For example, in the medical image generation scenario, the conditional matching degree reward value can accurately measure whether the generated medical image faithfully presents the key semantic elements such as lesion type, anatomical location, and image modality specified in the pathological description text, ensuring the clinical diagnostic reference value of the generated image. When the generated image deviates from the semantics of the pathological description, a low reward value signal is generated in a timely manner to drive model correction. In the financial bill image generation scenario, the conditional matching degree reward value can accurately assess whether the generated bill image is highly matched with the bill category specified by the bill type label in terms of visual semantics, ensuring that the generated value-added tax special invoice image does not present the layout features of a bank draft, thereby ensuring the accuracy of data annotation and the effectiveness of model training when the generated samples are used for training the financial bill recognition system.

[0085] Combination Figure 8 As shown, step S50 further includes: In this embodiment, step S50, which involves jointly evaluating the generated image with the input conditions in a preset reward calculation module to obtain a comprehensive reward value, includes not only the calculation of the perceptual diversity reward value and the condition matching degree reward value, but also the calculation of the adversarial quality reward value and the weighted fusion of the three-dimensional reward values. The adversarial quality reward value is obtained by evaluating the realism of the generated image through a preset Vision Transformer Discriminator, used to measure the degree to which the generated image closely resembles a real image at the visual perception level from the perspective of adversarial learning. The Vision Transformer Discriminator is built using a Vision Transformer architecture. Compared to traditional convolutional neural network discriminators, it can capture the structural dependencies and semantic consistency of the image globally through a self-attention mechanism, thereby making a more accurate and comprehensive judgment on the overall realism of the generated image.

[0086] S509: The generated image and the input conditions are input together into a preset visual transformer discriminator for authenticity evaluation processing to obtain an authenticity score.

[0087] In this step, the generated image obtained in step S40 and the input conditions obtained in step S10 are input together into a preset visual transformer discriminator. The authenticity assessment process is performed to obtain an authenticity score. The visual transformer discriminator This is a conditional discriminative model based on the Vision Transformer architecture. During training, it simultaneously receives images and corresponding conditional information as input, learning to determine whether an input image is a real image under given conditional constraints. Specifically, the Vision Transformer discriminator... First, generate the image. The image is segmented into fixed-size patches, and each patch is mapped to a serialized feature token through a linear embedding layer; simultaneously, the input conditions are... The conditional tags are encoded and concatenated into the image tag sequence. A multi-layer Transformer encoder is then used to perform global self-attention calculation on the hybrid tag sequence, fully capturing the long-range dependencies between image patches and between image patches and conditional information. Finally, the classification head outputs a scalar value in the range [0,1], which is the authenticity score. The authenticity score characterizes the confidence level of the visual transformer discriminator in the closeness of the generated image to a real image under given input conditions. A score closer to 1 indicates that the discriminator considers the generated image closer to the real image, while a score closer to 0 indicates that the discriminator considers the generated image to be significantly different from the real image. By inputting both the input conditions and the generated image into the discriminator, the authenticity assessment considers not only the visual quality of the image itself but also the semantic consistency between the image and the conditions, thus achieving joint authenticity evaluation under conditional constraints. For example, in a medical image generation scenario, the visual transformer discriminator can assess whether the generated medical image, under given pathological description conditions, presents tissue structures, lesion morphology, and image contrast characteristics consistent with clinical cognition; in a financial document image generation scenario, the visual transformer discriminator can assess whether the generated document image, under given document type label conditions, possesses the visual characteristics of a real document, such as format specifications, printing texture, and anti-counterfeiting elements.

[0088] S5010: Perform logarithmic calculation on the authenticity score to obtain the adversarial quality reward value.

[0089] The authenticity score obtained in step S509 Perform logarithmic operations to obtain the adversarial quality bonus value. The calculation formula for the logarithmic operation is as follows: in, For competitive quality bonus value; For natural logarithm operations; For visual transformer discriminator Given input conditions Under the premise of generating images Output authenticity score; Forward generation mapping of the normalized flow model; These are the initial latent variables; As input conditions. Taking the logarithm of the authenticity score has clear theoretical significance and practical value: because the authenticity score... The value range of is (0,1], and its logarithmic value is... The range of values ​​is When the realism score of the generated image is closer to 1, the adversarial quality reward value is closer to 0 (i.e., the maximum value), indicating a higher quality generated image. Conversely, when the realism score is closer to 0, the adversarial quality reward value tends towards negative infinity, generating a strong negative penalty signal on the model. This logarithmic transformation aligns with the design philosophy of the generator loss function in generative adversarial networks, providing a greater gradient driving force when the realism score is low, prompting the model to quickly improve the visual quality of the generated images. Conversely, when the realism score is high, the gradient gradually slows down, avoiding training instability caused by over-optimization. The adversarial quality reward value, from the perspective of adversarial learning, provides a quantitative feedback signal regarding the visual realism of the generated images for model parameter optimization. Together with the perceptual diversity reward value and the conditional matching reward value, it constitutes the three core evaluation dimensions of the comprehensive reward value.

[0090] S5011: Obtain the first weight, the second weight, and the third weight respectively; multiply the perceived diversity reward value by the first weight to obtain the first weighted value; multiply the condition matching degree reward value by the second weight to obtain the second weighted value; and multiply the adversarial quality reward value by the third weight to obtain the third weighted value.

[0091] In this step, the preset first weights are obtained respectively. Second weight and third weight The reward values ​​for the three dimensions are then weighted using the aforementioned weights. The first weight... Second weight and third weight A pre-defined scalar hyperparameter is used to control the relative contribution ratios of perceptual diversity reward, conditional matching reward, and adversarial quality reward in the overall reward value. In this embodiment, the first weight... The value is set to 0.5, that is... = 0.5; Second weight The value is set to 1.0, that is... = 1.0; the third weight The value is set to 0.2, that is... = 0.2. The design consideration for the above weight configuration is that the condition matching degree reward value is assigned the highest weight. = 1.0, reflecting the core position of conditional semantic consistency as the primary optimization objective in the conditional image generation task, that is, the generated image must first meet the semantic constraints of the input conditions; the perceptual diversity reward value is given a medium weight. = 0.5, to moderately encourage the model to generate diverse outputs, avoid pattern collapse, and prevent excessive pursuit of diversity at the expense of semantic consistency with the conditions; the adversarial quality reward value is given a relatively low weight. =0.2, while ensuring the basic visual quality of the generated images, avoid adversarial training signals dominating the optimization process, and prevent the discriminator gradient from being too strong, which could lead to training instability.

[0092] Specifically, the perceived diversity reward value With the first weight Multiply to obtain the first weighted value. The reward value for the condition matching degree With the second weight Multiply to obtain the second weighted value. The adversarial quality reward value With the third weight Multiply to obtain the third weighted value. Through the above weighting process, the reward values ​​of the three dimensions are adjusted to a reasonable relative proportion, preparing for the subsequent fusion and summation calculation. It should be noted that the above weight values ​​are a typical configuration scheme in this embodiment, and can be flexibly adjusted according to the needs of specific task scenarios in actual applications. For example, in medical image generation scenarios, because clinical diagnosis requires extremely high accuracy in the pathological semantics of images, the weight of the conditional matching degree reward value can be appropriately increased. To further enhance the semantic alignment between generated images and pathological descriptions; in the scenario of enhancing financial bill image data, since the training data needs to cover as many bill style variations as possible, the weight of the perceptual diversity reward value can be appropriately increased. This is to encourage the generation of more diverse and abundant sample images of invoices.

[0093] S5012: The first weighted value, the second weighted value, and the third weighted value are combined and summed to obtain the comprehensive reward value.

[0094] The first weighted value obtained in step S5011 Second weighted value and the third weighted value The overall reward value is obtained by performing a fusion summation calculation by adding each item together. The formula for calculating the comprehensive reward value is as follows: in, This is the total reward value; As the first weight, it takes a value of 0.5; The perceptual diversity reward value is used to measure the degree of visual perceptual difference between images generated within the same batch. This is the second weight, with a value of 1.0; The condition matching reward value is used to measure the semantic alignment between the generated image and the input conditions. It is the third weight, with a value of 0.2; The adversarial quality reward value measures how closely the generated image resembles a real image at the visual perception level.

[0095] The comprehensive reward value The quality performance of generated images across three dimensions—diversity, conditional matching, and visual realism—is uniformly quantified in scalar form. A higher overall reward value indicates better overall performance of the generated result across these three dimensions. During model training, the training objective is to maximize this overall reward value. The parameters of the conditional policy network and each flow layer are updated and optimized based on the reward signal. By maximizing the comprehensive reward value, the conditional policy network gradually learns a strategy for generating the optimal dynamic computational graph mask for different input conditions, enabling the activation configuration scheme of each flow layer to achieve the best balance between computational efficiency and generation quality. Simultaneously, the parameters of the conditional affine coupling layer and the invertible residual block in each flow layer are also continuously optimized under the guidance of the comprehensive reward signal, improving the model's conditional transformation capability and image generation quality. The comprehensive reward value... The reward signal used in step S60 as part of the reinforcement learning framework is used to drive the joint optimization and update of the conditional policy network and flow layer parameters through the policy gradient method, ultimately obtaining the trained conditional image generation data. For example, in the medical image generation scenario, maximizing the comprehensive reward value prompts the model to generate a high-quality medical image sample set that not only conforms to the semantics of pathological description but also has the texture of real clinical images and presents diverse lesion manifestations, meeting the actual needs of clinical image data amplification and auxiliary diagnostic system training. In the financial bill image generation scenario, maximizing the comprehensive reward value guides the model to generate financial bill image samples that accurately match the bill type label, possess real bill visual features, and cover multiple format variations, providing high-quality and diverse synthetic data support for the training of financial bill automatic recognition and anti-fraud systems.

[0096] As can be seen, the conditional policy network of this application dynamically generates a computation graph mask based on the semantic features of the input conditions, realizing real-time optimization and adaptive configuration of the computation graph of the normalized flow model. This effectively solves the limitation of the traditional static flow model, which has a fixed architecture when facing conditions of different complexities. When the input conditions are long text descriptions or multi-factor constraints with high semantic complexity, the conditional policy network can automatically activate more flow layers to fully capture fine-grained semantic features, ensuring the quality of the generated image and the degree of condition matching. When the input conditions are category labels or short text descriptions with relatively simple semantics, the conditional policy network adaptively reduces the number of flow layers involved in the computation, significantly improving the inference speed while ensuring the quality of generation. This allows the model to dynamically adjust the computation path for conditions of different complexities, achieving adaptive optimization of generation performance.

[0097] This application utilizes a dynamic computation graph masking mechanism to activate only the flow layers necessary for the current input conditions, effectively avoiding the problem of redundant computations performed by a large number of flow layers in fixed-architecture models. This significantly reduces the computational overhead and energy consumption of the model during the inference phase. This on-demand allocation of computational resources gives the proposed method a significant deployment advantage in resource-constrained applications, enabling efficient conditional image generation on platforms with limited computing power, such as mobile devices or edge computing devices, thus expanding the applicability of normalized flow models in practical applications.

[0098] The framework proposed in this application is designed to be independent of specific conditional input formats. It uses a pre-defined conditional encoder to uniformly encode different forms of input conditions into high-dimensional semantic vectors, decoupling the subsequent policy network decision-making and flow layer transformation processes from the original modal forms of the conditions. Based on this design, the technical solution of this invention can be flexibly extended to various input modalities such as text descriptions, audio signals, and structured data, exhibiting good multimodal compatibility and framework versatility, and capable of meeting the image generation needs of diverse conditional input formats in different application fields.

[0099] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0100] In one embodiment, an adjustable conditional image generation apparatus is provided, which corresponds one-to-one with the adjustable conditional image generation methods in the above embodiments. For example... Figure 9 As shown, the adjustable conditional image generation device includes: a data conversion module 100, a mask generation module 200, a hierarchy allocation module 300, an image generation module 400, a joint evaluation module 500, and a parameter update module 600.

[0101] Detailed descriptions of each functional module are as follows: The data conversion module 100 is used to acquire input conditions and encode the input conditions through a preset condition encoder to obtain a high-dimensional semantic vector. The mask generation module 200 is used to input the high-dimensional semantic vector into the conditional policy network for mask generation calculation to obtain a dynamic computation graph mask. The hierarchical allocation module 300 is used to perform hierarchical allocation of the high-dimensional semantic vector based on the dynamic computation graph mask, to obtain the local condition vector corresponding to each flow layer, and to determine the activation state of each flow layer. The image generation module 400 is used to obtain initial latent variables, pass the initial latent variables sequentially through the flow layer in the active state, and use the local conditional vector to selectively transform the initial latent variables in the invertible residual block and conditional coupling layer of the flow layer to obtain the generated image; The joint evaluation module 500 is used to jointly evaluate the generated image and the input conditions by inputting them into a preset reward calculation module to obtain a comprehensive reward value; The parameter update module 600 is used to update the parameters of the conditional policy network and the flow layer based on the comprehensive reward value, so as to obtain the trained conditional image generation data.

[0102] In one embodiment, the mask generation module 200 is specifically used for: The high-dimensional semantic vector is input as a conditional vector into the conditional policy network. The conditional vector is multiplied by a matrix with a preset first trainable weight and a first bias term is added to obtain the first feature vector. The first feature vector is input into the Gaussian error linear unit for nonlinear activation calculation to obtain the activated feature vector; The activation feature vector is multiplied by a matrix using a preset second trainable weight and a second bias term is added to obtain a second feature vector; The second feature vector is processed by probability mapping using an activation function to obtain the Bernoulli distribution parameterized hierarchical probability values. Obtain random noise, perform logarithmic operation on the hierarchical probability values ​​to obtain probability logarithmic values, and add the probability logarithmic values ​​to the random noise to obtain noisy logarithmic features; The relaxation feature is obtained by calculating the noisy logarithmic feature and the annealing temperature coefficient. The relaxation feature is then subjected to exponential operation and normalization to generate a multi-layer binary mask, which is then combined to obtain a dynamic computation graph mask.

[0103] In one embodiment, the image generation module 400 is specifically used for: Obtain the Gaussian noise corresponding to the initial latent variable, and split the Gaussian noise to obtain the first segmentation variable and the second segmentation variable; The second segmentation variable and the local condition vector are input to a preset condition function mapping to obtain the first condition output and the second condition output, respectively. The first conditional output is subjected to an exponential operation, and then combined with the first segmentation variable and the second conditional output for a fusion calculation to obtain the first updated variable; The first updated variable and the local condition vector are then input into a preset condition function mapping to obtain the third condition output and the fourth condition output, respectively. The second segmentation variable is calculated based on the third and fourth conditional outputs to obtain the second updated variable; The first update variable is combined with the second update variable to obtain the affine transformation output.

[0104] In one embodiment, the adjustable conditional image generation apparatus is further configured to: The affine transformation output and the local conditional vector are input into a preset residual branch network for calculation to obtain residual features; Obtain the weight matrix and perform spectral normalization on the weight matrix to obtain spectral normalized weights. Multiply the spectral normalized weights with the residual features to obtain normalized residual features. Obtain the scaling factor, and multiply the scaling factor with the normalized residual feature to obtain the scaled residual feature; The scaling residual features are added to the affine transformation output to obtain the next layer of latent variables; The next layer of latent variables is processed through the corresponding network layers to finally obtain the generated image.

[0105] In one embodiment, the joint evaluation module 500 is specifically used for: The generated images are processed to obtain a batch generated image set; For each pair of different generated images in the batch generated image set, the similarity distance of the image blocks is calculated by learning the perceptual image block similarity to obtain multiple sets of image difference distance values; The summation of the image difference distance values ​​from multiple sets is performed to obtain the total image distance. Obtain the total number of batches of the image set generated in the batch, calculate the difference between the total number of batches and a preset value to obtain the total difference, and multiply the total number of batches and the total difference to obtain the number of batch combinations; The perceptual diversity reward value is obtained by calculating the sum of the image distances and the number of batch combinations.

[0106] In one embodiment, the joint evaluation module 500 is further specifically used for: A contrastive language image pre-training model is obtained, and the generated image is input into the image encoder of the contrastive language image pre-training model for encoding extraction to obtain image encoding features; The input conditions are input into the text encoder of the contrastive language image pre-training model for encoding extraction to obtain text encoding features; The image encoding features and the text encoding features are subjected to cosine similarity calculation to obtain the image-text alignment score, and the image-text alignment score is used as the conditional matching degree reward value.

[0107] In one embodiment, the joint evaluation module 500 is further specifically used for: The generated image and the input conditions are input together into a preset visual transformer discriminator for authenticity evaluation processing to obtain an authenticity score; The authenticity score is logarithmically processed to obtain the adversarial quality bonus value. The first weight, the second weight, and the third weight are obtained respectively. The perceptual diversity reward value is multiplied by the first weight to obtain the first weighted value. The condition matching degree reward value is multiplied by the second weight to obtain the second weighted value. The adversarial quality reward value is multiplied by the third weight to obtain the third weighted value. The comprehensive reward value is obtained by summing the first weighted value, the second weighted value, and the third weighted value.

[0108] For specific limitations regarding the adjustable conditional image generation device, please refer to the limitations of the adjustable conditional image generation method above, which will not be repeated here. Each module in the aforementioned adjustable conditional image generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0109] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of an adjustable conditional image generation method on the server side.

[0110] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of an adjustable conditional image generation method.

[0111] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, can perform the steps provided in the above embodiments.

[0112] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0115] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0116] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An adjustable conditional image generation method, characterized by, include: The input conditions are obtained, and the input conditions are encoded and transformed by a preset condition encoder to obtain a high-dimensional semantic vector; The high-dimensional semantic vector is input into the conditional policy network for mask generation calculation to obtain a dynamic computation graph mask. Based on the dynamic computation graph mask, the high-dimensional semantic vector is hierarchically allocated to obtain the local condition vector corresponding to each flow layer, and the activation state of each flow layer is determined. The initial latent variables are obtained, and the initial latent variables are sequentially passed through the flow layer in the active state. The local conditional vector is used to selectively transform the initial latent variables in the invertible residual block and conditional coupling layer of the flow layer to obtain the generated image. The generated image and the input conditions are input into a preset reward calculation module for joint evaluation to obtain a comprehensive reward value; Based on the comprehensive reward value, the parameters of the conditional policy network and the flow layer are updated to obtain the trained conditional image generation data.

2. The adjustable conditional image generation method of claim 1, wherein, The step of inputting the high-dimensional semantic vector into the conditional policy network for mask generation calculation to obtain a dynamic computation graph mask includes: The high-dimensional semantic vector is input as a conditional vector into the conditional policy network. The conditional vector is multiplied by a matrix with a preset first trainable weight and a first bias term is added to obtain the first feature vector. The first feature vector is input into the Gaussian error linear unit for nonlinear activation calculation to obtain the activated feature vector; The activation feature vector is multiplied by a matrix using a preset second trainable weight and a second bias term is added to obtain a second feature vector; The second feature vector is processed by probability mapping using an activation function to obtain the Bernoulli distribution parameterized hierarchical probability values. Obtain random noise, perform logarithmic operation on the hierarchical probability values ​​to obtain probability logarithmic values, and add the probability logarithmic values ​​to the random noise to obtain noisy logarithmic features; The relaxation feature is obtained by calculating the noisy logarithmic feature and the annealing temperature coefficient. The relaxation feature is then subjected to exponential operation and normalization to generate a multi-layer binary mask, which is then combined to obtain a dynamic computation graph mask.

3. The adjustable conditional image generation method according to claim 1, characterized in that, The step of selectively transforming the initial latent variables using the local conditional vector in the reversible residual block and conditional coupling layer of the flow layer to obtain the generated image includes: Obtain the Gaussian noise corresponding to the initial latent variable, and split the Gaussian noise to obtain the first segmentation variable and the second segmentation variable; The second segmentation variable and the local condition vector are input to a preset condition function mapping to obtain the first condition output and the second condition output, respectively. The first conditional output is subjected to an exponential operation, and then combined with the first segmentation variable and the second conditional output for a fusion calculation to obtain the first updated variable; The first updated variable and the local condition vector are then input into a preset condition function mapping to obtain the third condition output and the fourth condition output, respectively. The second segmentation variable is calculated based on the third and fourth conditional outputs to obtain the second updated variable; The first update variable is combined with the second update variable to obtain the affine transformation output.

4. The adjustable conditional image generation method according to claim 3, characterized in that, The step of combining the first update variable and the second update variable to obtain the affine transformation output includes: The affine transformation output and the local conditional vector are input into a preset residual branch network for calculation to obtain residual features; Obtain the weight matrix and perform spectral normalization on the weight matrix to obtain spectral normalized weights. Multiply the spectral normalized weights with the residual features to obtain normalized residual features. Obtain the scaling factor, and multiply the scaling factor with the normalized residual feature to obtain the scaled residual feature; The scaling residual features are added to the affine transformation output to obtain the next layer of latent variables; The next layer of latent variables is processed through the corresponding network layers to finally obtain the generated image.

5. The adjustable conditional image generation method according to claim 1, characterized in that, The step of jointly evaluating the generated image and the input conditions in a preset reward calculation module to obtain a comprehensive reward value includes: The generated images are processed to obtain a batch generated image set; For each pair of different generated images in the batch generated image set, the similarity distance of the image blocks is calculated by learning the perceptual image block similarity to obtain multiple sets of image difference distance values; The summation of the image difference distance values ​​from multiple sets is performed to obtain the total image distance. Obtain the total number of batches of the image set generated in the batch, calculate the difference between the total number of batches and a preset value to obtain the total difference, and multiply the total number of batches and the total difference to obtain the number of batch combinations; The perceptual diversity reward value is obtained by calculating the sum of the image distances and the number of batch combinations.

6. The adjustable conditional image generation method according to claim 5, characterized in that, The step of jointly evaluating the generated image and the input conditions in a preset reward calculation module to obtain a comprehensive reward value also includes: A contrastive language image pre-training model is obtained, and the generated image is input into the image encoder of the contrastive language image pre-training model for encoding extraction to obtain image encoding features; The input conditions are input into the text encoder of the contrastive language image pre-training model for encoding extraction to obtain text encoding features; The image encoding features and the text encoding features are subjected to cosine similarity calculation to obtain the image-text alignment score, and the image-text alignment score is used as the conditional matching degree reward value.

7. The adjustable conditional image generation method according to claim 6, characterized in that, The step of jointly evaluating the generated image and the input conditions in a preset reward calculation module to obtain a comprehensive reward value also includes: The generated image and the input conditions are input together into a preset visual transformer discriminator for authenticity evaluation processing to obtain an authenticity score; The authenticity score is logarithmically processed to obtain the adversarial quality bonus value. The first weight, the second weight, and the third weight are obtained respectively. The perceptual diversity reward value is multiplied by the first weight to obtain the first weighted value. The condition matching degree reward value is multiplied by the second weight to obtain the second weighted value. The adversarial quality reward value is multiplied by the third weight to obtain the third weighted value. The comprehensive reward value is obtained by summing the first weighted value, the second weighted value, and the third weighted value.

8. An adjustable conditional image generation apparatus, characterized in that, include: The data conversion module is used to acquire input conditions and encode and convert the input conditions through a preset condition encoder to obtain a high-dimensional semantic vector. The mask generation module is used to input the high-dimensional semantic vector into the conditional policy network to perform mask generation calculations and obtain a dynamic computation graph mask. The hierarchical allocation module is used to perform hierarchical allocation of the high-dimensional semantic vector based on the dynamic computation graph mask, to obtain the local condition vector corresponding to each flow layer, and to determine the activation state of each flow layer. An image generation module is used to obtain initial latent variables, pass the initial latent variables sequentially through the flow layer in the active state, and use the local conditional vector to selectively transform the initial latent variables in the invertible residual block and conditional coupling layer of the flow layer to obtain the generated image; The joint evaluation module is used to jointly evaluate the generated image and the input conditions by inputting them into a preset reward calculation module to obtain a comprehensive reward value; The parameter update module is used to update the parameters of the conditional policy network and the flow layer based on the comprehensive reward value, so as to obtain the trained conditional image generation data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the adjustable conditional image generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the adjustable conditional image generation method as described in any one of claims 1 to 7.

Images