Artificial intelligence-based semantic image generation method and system

Abstract

The application discloses a semantic image generation method and system based on artificial intelligence, relates to the technical field of semantic image generation, and inputs each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score; determines an initial latent variable; classifies all semantic features into main features and secondary features, calculates a conflict coefficient of each pair of main features and secondary features by using an attention graph, and obtains a conflict set through judgment; updates the attention weights of the conflict secondary features and the main features by using an attention weight reduction amount and a compensation amount respectively; calculates a comprehensive context feature by using a scaled attention graph and a semantic feature value matrix; predicts noise by combining the comprehensive context feature and an image feature, updates the latent variable by using a sampling formula of a diffusion model; and finally obtains a final latent variable, which is input into a value decoder to generate a final image.

Description

Technical Field

[0001] This invention relates to the field of semantic image generation technology, specifically to a semantic image generation method and system based on artificial intelligence. Background Technology

[0002] With the rapid development of artificial intelligence technology, semantic image generation, as a core research direction at the intersection of computer vision and natural language processing, has demonstrated extremely high application value in many fields such as content creation, digital cultural and creative industries, intelligent design, and virtual reality, thanks to its ability to transform text descriptions into visual images. It has become one of the important scenarios for the application of artificial intelligence technology. Currently, most mainstream semantic image generation technologies are based on deep learning frameworks such as diffusion models and generative adversarial networks (GANs). Among them, diffusion models have become the mainstream technical solution for semantic image generation due to their significant advantages in image generation quality, detail richness, and stability.

[0003] Current diffusion models typically apply equal or static attention to all text tokens during denoising. When dealing with complex prompts, secondary features may overshadow primary features, stealing their generation weights and leading to a decline in the quality of the generated content. Furthermore, while the U-Net backbone of the diffusion model achieves the fusion of semantic and image features through cross-attention modules, it fails to detect and optimize conflicts in the spatial attention distribution of different semantic features during the fusion process. When multiple semantic features overlap in the image space, spatial conflicts can easily arise, resulting in misaligned, overlapping, and confused elements in the generated image. For example, "white clouds next to blue sky" is generated as a seamless fusion of blue sky and white clouds, and "butterflies next to flowers" is generated as butterflies embedded within flowers, completely violating the spatial logic of the text description. Summary of the Invention

[0004] The purpose of this invention is to provide a semantic image generation method and system based on artificial intelligence to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: An artificial intelligence-based semantic image generation method, the method comprising the following steps: S100: The user inputs descriptive text, and the system uses natural language processing algorithms to extract a set of structured semantic features from the descriptive text. Each semantic feature is then input into a pre-trained lightweight semantic importance discriminator to output an initial importance score. Furthermore, the specific steps for inputting each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score are as follows: S101. The system uses natural language processing algorithms to extract a set of structured semantic features for describing the text, which is F = {f1, f2, f3, ..., f...} n}, f1, f2, f3, ..., f n Let each semantic feature represent the 1st, 2nd, 3rd, ..., nth structured semantic feature describing the text. Each semantic feature is input into a pre-trained lightweight semantic feature importance discriminator to calculate an initial importance score, using the formula: ; In the formula, s i Φ represents the initial importance score, Φ represents the lightweight semantic feature importance discriminator, and T represents the scaling factor. Natural language processing algorithms are used to extract structured semantic features, transforming unstructured descriptive text into a machine-recognizable set of structured features. This makes the representation of semantic features clearer and more standardized, laying a precise semantic foundation for subsequent attention allocation and feature fusion, and avoiding deviations from the text description caused by fuzzy semantic features.

[0006] The lightweight semantic feature importance discriminator employs a lightweight MLP algorithm to construct its discriminator structure. Historical image-text data pairs are collected as the training set, where each data pair represents a data pair consisting of descriptive text and its corresponding image. Based on the descriptive text, an intermediate feature map is extracted from the generated image using a diffusion model. Each feature is treated as a region, and two images are generated: one normally generated image and the other with the features set to zero. Then, the difference between the two images is calculated using the learned perceptual image patch similarity algorithm (LPIPS), and this difference is used as the initial importance score s of the feature's true label. i s i ∈[0,1]; During training using the training set, a loss function is constructed such that the initial importance score output by the lightweight semantic feature importance discriminator is equal to the initial importance score of the true label.

[0007] The semantic importance discriminator built with a lightweight MLP significantly reduces computational load and improves system efficiency while maintaining discrimination accuracy compared to heavyweight networks. Furthermore, the discriminator is trained on historical image and text data pairs and uses the LPIPS algorithm to calculate image differences to obtain true labels, making the initial importance score output more in line with the semantic requirements of actual image generation and objectively reflecting the actual importance of each semantic feature in image generation.

[0008] S200. Use the diffusion model to iteratively denoise and generate images. In each iteration, assign initial attention weights to each semantic feature using the initial importance score and determine the initial latent variables. Furthermore, the specific steps for determining the initial latent variables are as follows: S201. When generating images by iterative denoising using a diffusion model, let the number of iterations be B. At each iteration, the latent variables for the next step are obtained by denoising the current noisy image and the descriptive text. Based on iterative denoising of the diffusion model, the image generation effect is gradually optimized by utilizing the iterative characteristics of the diffusion model from high noise to low noise. Compared with traditional generation models, it can effectively alleviate the blurring and distortion problems of image generation and improve the clarity and realism of the generated image.

[0009] Initial latent variable z is sampled from a standard normal distribution. B The initial latent variables follow a standard multivariate normal distribution N(0, I), where 0 represents the mean vector and I represents the unit covariance matrix; the initial latent variables represent the image representation in the latent space of the diffusion model. The initial latent variables are sampled from the standard normal distribution and follow the multivariate normal distribution N(0, I), which ensures the randomness and normality of the initial latent variables. This provides an initial input that conforms to the model training rules for the iterative denoising of the diffusion model, and avoids generation failure or poor results caused by abnormal initial latent variables.

[0010] S202. Assign initial attention weights to each semantic feature using the initial importance scores, as shown in the formula: ; In the formula, w i B Let s represent the initial attention weights for the i-th semantic feature. j This represents the initial importance score of the j-th semantic feature during summation; it constrains the sum of the initial attention weights of all semantic features to be 1. By allocating initial attention weights according to the proportion of initial importance scores and constraining the sum of weights to 1, differentiated weight allocation for semantic features is achieved, allowing semantic features with high importance to receive higher initial attention and ensuring that core semantics are prioritized in image generation.

[0011] S300. Set the denoising time step, input the current latent variable into the U-Net backbone network of the diffusion model, extract the image features at the corresponding time step at each time step, linearly transform the semantic features and image features to obtain the key matrix and query matrix, and use the query matrix and key matrix to calculate the attention map; classify all semantic features into primary features and secondary features, use the attention map to calculate the conflict coefficient of each pair of primary and secondary features, and obtain the conflict set after judgment. Furthermore, the specific steps for obtaining the conflict set through judgment are as follows: S301. Set the denoising time steps to b = B, B-1, B-2, ..., 1; iterate from the high-noise time step B to the low-noise time step 1 in the diffusion model; for each time step, input the current latent variable into the U-Net backbone network of the diffusion model. The U-Net backbone network consists of an encoder, a bottleneck layer, and a decoder, and cross-attention modules are inserted in all resolution layers to form cross-attention layers; the cross-attention layers are used to integrate semantic features into image features; iterate from high to low noise time steps, and extract image features at each time step, realizing refined temporal optimization of image generation. It can adjust the feature fusion strategy in a targeted manner under different noise levels, making the denoising process more in line with the changing rules of image features.

[0012] In the cross-attention layer, let the input image feature matrix be P. b ∈R H×W×D R H×W×D Let H be the set of real numbers representing the three dimensions of the image features, where H represents height, W represents width, and D represents depth. The semantic features and image features are linearly transformed to obtain the key matrix and query matrix, as shown in the formula: Q=P b ×W Q K i =f i ×W K V i =f i ×W V ; In the formula, W Q W K和 W V K represents the learnable projection matrix, obtained through training; Q represents the query matrix; K represents the query matrix. i V represents the semantic feature key matrix. i Represents the semantic eigenvalue matrix; The attention map for each semantic feature is calculated using the key matrix and the query matrix, as shown in the formula: ; In the formula, A i （b） Let d represent the attention map of the i-th semantic feature at time step b, where d represents the dimension of the key and softmax represents the normalized exponential function. Cross-attention modules are inserted into all resolution layers of the U-Net backbone network to generate key and query matrices through linear transformation of semantic features and image features and to calculate attention maps. This achieves deep fusion of semantic features and image features, enabling the model to accurately capture the attention distribution of semantic features in the image space and ensuring spatial matching between image generation and text semantics.

[0013] S302. Sort all semantic features in descending order based on the initial importance scores, take the top k as the primary feature set M, and the remaining semantic features as the secondary feature set E; plot the initial importance scores after descending order as a curve, and extract the number of semantic features before the inflection point of the curve as the k value; based on the initial importance scores and combined with the inflection point of the curve to determine the k value, the semantic features are divided into primary and secondary features. Compared with manually setting the k value, the feature classification is adaptive, and it can automatically divide according to the semantic feature importance distribution of different descriptive texts, which is more in line with the actual semantic structure of the text.

[0014] For each pair of primary features m∈M and secondary features e∈E, the conflict coefficient is calculated using the following formula: ; In the formula, C （b） m，e A represents the conflict coefficient between the primary feature m and the secondary feature e at time step b. m （b） (p) represents the attention map of the principal feature m at spatial location p at time step b, A e （b） (p) represents the attention map of secondary feature e at spatial location p at time step b; Staff set a conflict threshold θ, when C （b） m，e When the value is greater than θ, the secondary feature e is added to the conflict set C of the current time step. （b） Otherwise, it is determined that there is no conflict and no attention weight needs to be updated. The conflict coefficients of primary and secondary features are calculated and the conflict set is obtained by thresholding. This can accurately identify semantic features with spatial attention conflicts in image generation, avoid semantic confusion and element misalignment in the generated image due to feature conflicts, and provide a clear optimization target for subsequent weight adjustment.

[0015] S400. For each conflicting secondary feature, calculate the attention weight reduction amount. Sum all conflicting secondary features and add a negative sign to obtain the total weight reduction amount. Compensate all primary features with the total weight reduction amount proportionally to obtain the attention weight compensation amount for each conflicting primary feature. Update the attention weights of conflicting secondary features and primary features using the attention weight reduction amount and compensation amount respectively. Furthermore, the specific steps for updating the attention weights of conflicting secondary and primary features using attention weight reduction and compensation amounts are as follows: S401. For each secondary feature of a conflict in the conflict set, calculate the attention weight reduction using the following formula: ; In the formula, △w e（b） w represents the amount of attention weight reduction for the conflicting secondary feature e at time step b. e （b） α represents the attention weights before updating the conflicting minor feature e at time step b; α represents the step size hyperparameter. This represents the ratio of the most severe conflict level to the threshold. Personalized attention weight reduction is calculated for secondary conflict features. By combining the adjustment of the step size hyperparameter α and the conflict level ratio, fine-grained reduction of conflict feature weights is achieved. This can adapt the reduction range according to the severity of the conflict, avoid excessive reduction leading to the loss of secondary semantic features, and effectively reduce the interference of conflict features.

[0016] Calculate the sum of attention weight reductions for all conflicting secondary features and add a negative sign to obtain the total weight reduction; then proportionally compensate all primary features with the total weight reduction to obtain the attention weight compensation for each conflicting primary feature, using the following formula: ; In the formula, △w m （b） Δw represents the attention weight compensation amount for the principal feature m of the conflict at time step b. total w represents the total weight reduction. m （b） This represents the attention weights before updating the main conflict feature m at time step b; The total weight reduction of all conflicting secondary features is proportionally compensated to the primary features, realizing the dynamic redistribution of attention weights. While reducing conflict interference, it further strengthens the attention ratio of the primary semantic features, ensuring the dominance of core semantics in image generation. Moreover, the "reduction and compensation balance" of weights ensures that the attention weights of all features are always reasonable and do not disrupt the attention allocation logic of the model.

[0017] S402. For the primary and secondary features, the attention weight compensation and attention weight reduction are added to the attention weights before the update, respectively, to output the updated attention weights for the primary and secondary features. The updated attention weights are used for image generation during denoising in the next time step. By updating the weights for primary and secondary features separately, the adjusted attention weights can be directly adapted to the denoising generation in the next time step, achieving the immediacy and practicality of weight adjustment, and allowing the conflict optimization strategy to be quickly implemented in the image generation process.

[0018] S500. In the next time step, the updated attention weight set is input into the U-Net backbone network. The attention map of all semantic features is scaled using the updated attention weights. The comprehensive context features are calculated using the scaled attention map and the semantic feature value matrix. Furthermore, the specific steps for calculating the comprehensive contextual features using the scaled attention map and semantic feature value matrix are as follows: S501. Input the updated attention weight set into the U-Net backbone network. For each semantic feature's attention map, multiply it using the updated attention weights to scale the attention map. Calculate the context features of each semantic feature using the scaled attention map and the semantic feature's value matrix, with the formula: G i （b-1） =A i （b-1）’ ×V i G i （b-1） A represents the context feature of the i-th semantic feature at the next time step b-1. i （b-1）’ This represents the scaled attention map of the i-th semantic feature at the next time step b-1; The comprehensive contextual feature G is obtained by summing the contextual features of all semantic features.

[0019] By scaling the attention map using updated attention weights, the spatial distribution of the attention map can be matched with the adjusted semantic feature importance. This strengthens the spatial attention of core semantics and weakens the spatial attention of conflicting semantics, allowing the model to more accurately focus on important and non-conflicting semantic features during feature fusion.

[0020] By calculating individual contextual features using the scaled attention map and semantic feature value matrix, and summing them to obtain comprehensive contextual features, the integration of contextual information of all semantic features is achieved. This allows the model to capture the relationships between semantic features, and the generated image can reflect the overall semantic logic of the text description, rather than a collection of isolated semantic elements.

[0021] S600. By combining comprehensive contextual features and image features with the predicted noise, the latent variables are updated using the sampling formula of the diffusion model after obtaining the predicted noise. Furthermore, the specific steps for updating the latent variables using the sampling formula of the diffusion model after obtaining the predicted noise are as follows: S601. Reshape the integrated contextual features into spatial dimensions H×W×d in the cross-attention layer. v d v The dimension representing the semantic feature value; when d v When the dimension d is not equal to D, linear projection is used to integrate the context features. v The dimensions are then converted to match those of the image features; subsequently, the contextual features and image features are fused together using the formula: P out =P H×W×D +G H×W×dv GH×W×dv This indicates that the comprehensive contextual features are reshaped into a spatial dimension, P H×W×D P represents image features. out This represents the image features output after fusion; The fused image features are input into the U-Net backbone network, which outputs the predicted noise γ at time step b-1. The predicted noise is required to be proportional to the latent variable z input at time step b-1. b-1 Same size; By reshaping the comprehensive contextual features and matching their dimensions with those of the image features through linear projection, and then performing feature fusion, the problem of dimensional heterogeneity between semantic features and image features is solved, allowing the two types of features to be seamlessly integrated. The fused features contain both the spatial information of the image and the semantic contextual information of the text, providing high-quality input for accurate noise prediction.

[0022] S602. Utilizing prediction noise to determine the latent variable z at time step b-1 b-1 To update, the formula is: ; In the formula, z b-2 Let β represent the updated latent variable. b-2 β represents the noise scheduling parameter at time step b-2. b-1 z represents the noise scheduling parameter at time step b-1. b-1 This represents the latent variables before the update at time step b-1. Noise prediction based on the fusion feature output having the same size as the latent variables ensures the fit between the noise prediction and the latent variables, allowing the noise prediction to accurately reflect the image noise distribution at the current time step. Then, the latent variables are updated using a sampling formula specific to the diffusion model, following the denoising rules of the diffusion model. This enables efficient and accurate updating of the latent variables, allowing the denoising process to gradually approach the latent space representation of the true image.

[0023] S700: Input the updated latent variable into the next time step, and repeat steps S300 to S600 until the final latent variable is obtained in the last time step. Input the final latent variable into the value decoder to generate the final image.

[0024] Furthermore, the specific steps for the decoder to generate the final image from the final latent variable input values ​​are as follows: The updated latent variables are used as input latent variables for time step b-2. Steps S300 to S600 are repeated, iteratively updating the latent variables according to the time steps until the final latent variable z1 is obtained in the last time step. The final latent variable is then input into the decoder to generate the final image I. The updated latent variables are cyclically input into the next time step, and the feature fusion, noise prediction, and latent variable update process is repeated. This achieves multi-round iterative optimization of the latent variables, allowing them to gradually approach the latent space representation of the real image through continuous denoising and semantic fusion, significantly improving the quality and semantic consistency of the final generated image.

[0025] The semantic image generation system based on artificial intelligence includes a semantic feature importance module, an attention weight module, a conflict judgment module, an attention weight update module, a comprehensive context feature calculation module, a latent variable update module, and an iterative output module. The semantic feature importance module is used by the system to extract a set of structured semantic features describing the text using natural language processing algorithms, and inputs each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score; The attention weight module is used to generate images by iteratively denoising using a diffusion model. In each iteration, initial attention weights are assigned for each semantic feature using an initial importance score, and initial latent variables are determined. The conflict judgment module is used to set the denoising time step, extract the image features at the corresponding time step at each time step, linearly transform the semantic features and image features to obtain the key matrix and query matrix, and use the query matrix and key matrix to calculate the attention map; classify all semantic features into primary features and secondary features, use the attention map to calculate the conflict coefficient of each pair of primary and secondary features, and obtain the conflict set after judgment. The attention weight update module is used to calculate the attention weight reduction amount for each conflicting secondary feature, sum up all conflicting secondary features and add a negative sign to obtain the total weight reduction amount, and compensate all primary features proportionally to obtain the attention weight compensation amount for each conflicting primary feature. The attention weights of the conflicting secondary features and primary features are updated using the attention weight reduction amount and compensation amount, respectively. The comprehensive context feature calculation module is used to scale the attention map of all semantic features using the updated attention weights in the next time step, and to calculate the comprehensive context features using the scaled attention map and the semantic feature value matrix. The latent variable update module is used to combine comprehensive context features and image features with the predicted noise, obtain the predicted noise, and then update the latent variables using the sampling formula of the diffusion model. The iterative output module is used to input the updated latent variables into the next time step, repeat steps S300 to S600 until the final latent variables are obtained in the last time step, and input the final latent variables into the value decoder to generate the final image.

[0026] The conflict determination module includes an attention map calculation unit and a conflict coefficient calculation unit; The attention map calculation unit is used to set the input image feature matrix as P in the cross-attention layer. b ∈R H ×W×D R H×W×D The set of real numbers represents the three-dimensional dimension of image features; the semantic features and image features are linearly transformed to obtain the key matrix and query matrix; the attention map of each semantic feature is calculated using the key matrix and query matrix; The conflict coefficient calculation unit is used to sort all semantic features in descending order according to the initial importance score, take the first k as the main feature set M, and the remaining semantic features as the secondary feature set E; plot the initial importance score after descending order as a curve, and extract the number of semantic features before the inflection point of the curve as the k value; for each pair of main features m∈M and secondary features e∈E, calculate the conflict coefficient.

[0027] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention systematically solves the problems of semantic feature conflict, semantic-image space mismatch, and core semantic loss in traditional semantic image generation by quantifying the importance of semantic features, dynamically adjusting attention weights, and identifying and optimizing conflict features. It enables the generated image to accurately match all the core semantics of the descriptive text, while reasonably presenting secondary semantics, avoiding semantic confusion or missing elements.

[0028] 2. This invention integrates the attention mechanism throughout the entire image generation process, from initial weight allocation based on importance scores to dynamic weight adjustment based on conflict, and then to scaling optimization of the attention map, achieving refined management of attention weights across the entire chain from initial allocation to dynamic adjustment to precise application. This allows semantic features with different levels of importance and conflict states to receive differentiated attention resources, ensuring that image generation reflects the semantic hierarchy of the text description and that the generated results are more consistent with human language expression logic. Attached Figure Description

[0029] Figure 1 This is a schematic diagram illustrating the steps of the semantic image generation method based on artificial intelligence according to the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example: Figure 1 As shown, the present invention provides a technical solution. An artificial intelligence-based semantic image generation method, the method comprising the following steps: S100: The user inputs descriptive text, and the system uses natural language processing algorithms to extract a set of structured semantic features from the descriptive text. Each semantic feature is then input into a pre-trained lightweight semantic importance discriminator to output an initial importance score. The specific steps for inputting each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score are as follows: S101. The system uses natural language processing algorithms to extract a set of structured semantic features for describing the text, which is F = {f1, f2, f3, ..., f...} n}, f1, f2, f3, ..., f n Let each semantic feature represent the 1st, 2nd, 3rd, ..., nth structured semantic feature describing the text. Each semantic feature is input into a pre-trained lightweight semantic feature importance discriminator to calculate an initial importance score, using the formula: ; In the formula, s i Φ represents the initial importance score, Φ represents the lightweight semantic feature importance discriminator, and T represents the scaling factor. Natural language processing algorithms are used to extract structured semantic features, transforming unstructured descriptive text into a machine-recognizable set of structured features. This makes the representation of semantic features clearer and more standardized, laying a precise semantic foundation for subsequent attention allocation and feature fusion, and avoiding deviations from the text description caused by fuzzy semantic features.

[0032] The lightweight semantic feature importance discriminator employs a lightweight MLP algorithm to construct its discriminator structure. Historical image-text data pairs are collected as the training set, where each data pair represents a data pair consisting of descriptive text and its corresponding image. Based on the descriptive text, an intermediate feature map is extracted from the generated image using a diffusion model. Each feature is treated as a region, and two images are generated: one normally generated image and the other with the features set to zero. Then, the difference between the two images is calculated using the learned perceptual image patch similarity algorithm (LPIPS), and this difference is used as the initial importance score s of the feature's true label. i s i∈[0,1]; During training using the training set, a loss function is constructed such that the initial importance score output by the lightweight semantic feature importance discriminator is equal to the initial importance score of the true label.

[0033] The semantic importance discriminator built with a lightweight MLP significantly reduces computational load and improves system efficiency while maintaining discrimination accuracy compared to heavyweight networks. Furthermore, the discriminator is trained on historical image and text data pairs and uses the LPIPS algorithm to calculate image differences to obtain true labels, making the initial importance score output more in line with the semantic requirements of actual image generation and objectively reflecting the actual importance of each semantic feature in image generation.

[0034] S200. Use the diffusion model to iteratively denoise and generate images. In each iteration, assign initial attention weights to each semantic feature using the initial importance score and determine the initial latent variables. The specific steps for determining the initial latent variables are as follows: S201. When generating images by iterative denoising using a diffusion model, let the number of iterations be B. At each iteration, the latent variables for the next step are obtained by denoising the current noisy image and the descriptive text. Based on iterative denoising of the diffusion model, the image generation effect is gradually optimized by utilizing the iterative characteristics of the diffusion model from high noise to low noise. Compared with traditional generation models, it can effectively alleviate the blurring and distortion problems of image generation and improve the clarity and realism of the generated image.

[0035] Initial latent variable z is sampled from a standard normal distribution. B The initial latent variables follow a standard multivariate normal distribution N(0, I), where 0 represents the mean vector and I represents the unit covariance matrix; the initial latent variables represent the image representation in the latent space of the diffusion model. The initial latent variables are sampled from the standard normal distribution and follow the multivariate normal distribution N(0, I), which ensures the randomness and normality of the initial latent variables. This provides an initial input that conforms to the model training rules for the iterative denoising of the diffusion model, and avoids generation failure or poor results caused by abnormal initial latent variables.

[0036] S202. Assign initial attention weights to each semantic feature using the initial importance scores, as shown in the formula: ; In the formula, w i B Let s represent the initial attention weights for the i-th semantic feature. jThis represents the initial importance score of the j-th semantic feature during summation; it constrains the sum of the initial attention weights of all semantic features to be 1. By allocating initial attention weights according to the proportion of initial importance scores and constraining the sum of weights to 1, differentiated weight allocation for semantic features is achieved, allowing semantic features with high importance to receive higher initial attention and ensuring that core semantics are prioritized in image generation.

[0037] S300. Set the denoising time step, input the current latent variable into the U-Net backbone network of the diffusion model, extract the image features at the corresponding time step at each time step, linearly transform the semantic features and image features to obtain the key matrix and query matrix, and use the query matrix and key matrix to calculate the attention map; classify all semantic features into primary features and secondary features, use the attention map to calculate the conflict coefficient of each pair of primary and secondary features, and obtain the conflict set after judgment. The specific steps for obtaining the conflict set after judgment are as follows: S301. Set the denoising time steps to b = B, B-1, B-2, ..., 1; iterate from the high-noise time step B to the low-noise time step 1 in the diffusion model; for each time step, input the current latent variable into the U-Net backbone network of the diffusion model. The U-Net backbone network consists of an encoder, a bottleneck layer, and a decoder, and cross-attention modules are inserted in all resolution layers to form cross-attention layers; the cross-attention layers are used to integrate semantic features into image features; iterate from high to low noise time steps, and extract image features at each time step, realizing refined temporal optimization of image generation. It can adjust the feature fusion strategy in a targeted manner under different noise levels, making the denoising process more in line with the changing rules of image features.

[0038] In the cross-attention layer, let the input image feature matrix be P. b ∈R H×W×D R H×W×D Let H be the set of real numbers representing the three dimensions of the image features, where H represents height, W represents width, and D represents depth. The semantic features and image features are linearly transformed to obtain the key matrix and query matrix, as shown in the formula: Q=P b ×W Q K i =f i ×W K V i =f i ×W V ; In the formula, W Q W K和 W V K represents the learnable projection matrix, obtained through training; Q represents the query matrix; K represents the query matrix. i V represents the semantic feature key matrix.i Represents the semantic eigenvalue matrix; The attention map for each semantic feature is calculated using the key matrix and the query matrix, as shown in the formula: ; In the formula, A i （b） Let d represent the attention map of the i-th semantic feature at time step b, where d represents the dimension of the key and softmax represents the normalized exponential function. Cross-attention modules are inserted into all resolution layers of the U-Net backbone network to generate key and query matrices through linear transformation of semantic features and image features and to calculate attention maps. This achieves deep fusion of semantic features and image features, enabling the model to accurately capture the attention distribution of semantic features in the image space and ensuring spatial matching between image generation and text semantics.

[0039] S302. Sort all semantic features in descending order based on the initial importance scores, take the top k as the primary feature set M, and the remaining semantic features as the secondary feature set E; plot the initial importance scores after descending order as a curve, and extract the number of semantic features before the inflection point of the curve as the k value; based on the initial importance scores and combined with the inflection point of the curve to determine the k value, the semantic features are divided into primary and secondary features. Compared with manually setting the k value, the feature classification is adaptive, and it can automatically divide according to the semantic feature importance distribution of different descriptive texts, which is more in line with the actual semantic structure of the text.

[0040] For each pair of primary features m∈M and secondary features e∈E, the conflict coefficient is calculated using the following formula: ; In the formula, C （b） m，e A represents the conflict coefficient between the primary feature m and the secondary feature e at time step b. m （b） (p) represents the attention map of the principal feature m at spatial location p at time step b, A e （b） (p) represents the attention map of secondary feature e at spatial location p at time step b; Staff set a conflict threshold θ, when C （b） m，e When the value is greater than θ, the secondary feature e is added to the conflict set C of the current time step. （b） Otherwise, it is determined that there is no conflict and no attention weight needs to be updated. The conflict coefficients of primary and secondary features are calculated and the conflict set is obtained by thresholding. This can accurately identify semantic features with spatial attention conflicts in image generation, avoid semantic confusion and element misalignment in the generated image due to feature conflicts, and provide a clear optimization target for subsequent weight adjustment.

[0041] S400. For each conflicting secondary feature, calculate the attention weight reduction amount. Sum all conflicting secondary features and add a negative sign to obtain the total weight reduction amount. Compensate all primary features with the total weight reduction amount proportionally to obtain the attention weight compensation amount for each conflicting primary feature. Update the attention weights of conflicting secondary features and primary features using the attention weight reduction amount and compensation amount respectively. The specific steps for updating the attention weights of the secondary and primary features of the conflict using attention weight reduction and compensation are as follows: S401. For each secondary feature of a conflict in the conflict set, calculate the attention weight reduction using the following formula: ; In the formula, △w e （b） w represents the amount of attention weight reduction for the conflicting secondary feature e at time step b. e （b） α represents the attention weights before updating the conflicting minor feature e at time step b; α represents the step size hyperparameter. This represents the ratio of the most severe conflict level to the threshold. Personalized attention weight reduction is calculated for secondary conflict features. By combining the adjustment of the step size hyperparameter α and the conflict level ratio, fine-grained reduction of conflict feature weights is achieved. This can adapt the reduction range according to the severity of the conflict, avoid excessive reduction leading to the loss of secondary semantic features, and effectively reduce the interference of conflict features.

[0042] Calculate the sum of attention weight reductions for all conflicting secondary features and add a negative sign to obtain the total weight reduction; then proportionally compensate all primary features with the total weight reduction to obtain the attention weight compensation for each conflicting primary feature, using the following formula: ; In the formula, △w m （b） Δw represents the attention weight compensation amount for the principal feature m of the conflict at time step b. total w represents the total weight reduction. m （b） This represents the attention weights before updating the main conflict feature m at time step b; The total weight reduction of all conflicting secondary features is proportionally compensated to the primary features, realizing the dynamic redistribution of attention weights. While reducing conflict interference, it further strengthens the attention ratio of the primary semantic features, ensuring the dominance of core semantics in image generation. Moreover, the "reduction and compensation balance" of weights ensures that the attention weights of all features are always reasonable and do not disrupt the attention allocation logic of the model.

[0043] S402. For the primary and secondary features, the attention weight compensation and attention weight reduction are added to the attention weights before the update, respectively, to output the updated attention weights for the primary and secondary features. The updated attention weights are used for image generation during denoising in the next time step. By updating the weights for primary and secondary features separately, the adjusted attention weights can be directly adapted to the denoising generation in the next time step, achieving the immediacy and practicality of weight adjustment, and allowing the conflict optimization strategy to be quickly implemented in the image generation process.

[0044] S500. In the next time step, the updated attention weight set is input into the U-Net backbone network. The attention map of all semantic features is scaled using the updated attention weights. The comprehensive context features are calculated using the scaled attention map and the semantic feature value matrix. The specific steps for calculating the comprehensive contextual features using the scaled attention map and semantic feature value matrix are as follows: S501. Input the updated attention weight set into the U-Net backbone network. For each semantic feature's attention map, multiply it using the updated attention weights to scale the attention map. Calculate the context features of each semantic feature using the scaled attention map and the semantic feature's value matrix, with the formula: G i （b-1） =A i （b-1）’ ×V i G i （b-1） A represents the context feature of the i-th semantic feature at the next time step b-1. i （b-1）’ This represents the scaled attention map of the i-th semantic feature at the next time step b-1; The comprehensive contextual feature G is obtained by summing the contextual features of all semantic features.

[0045] By scaling the attention map using updated attention weights, the spatial distribution of the attention map can be matched with the adjusted semantic feature importance. This strengthens the spatial attention of core semantics and weakens the spatial attention of conflicting semantics, allowing the model to more accurately focus on important and non-conflicting semantic features during feature fusion.

[0046] By calculating individual contextual features using the scaled attention map and semantic feature value matrix, and summing them to obtain comprehensive contextual features, the integration of contextual information of all semantic features is achieved. This allows the model to capture the relationships between semantic features, and the generated image can reflect the overall semantic logic of the text description, rather than a collection of isolated semantic elements.

[0047] S600. By combining comprehensive contextual features and image features with the predicted noise, the latent variables are updated using the sampling formula of the diffusion model after obtaining the predicted noise. The specific steps for updating the latent variables using the sampling formula of the diffusion model after obtaining the predicted noise are as follows: S601. Reshape the integrated contextual features into spatial dimensions H×W×d in the cross-attention layer. v d v The dimension representing the semantic feature value; when d v When the dimension d is not equal to D, linear projection is used to integrate the context features. v The dimensions are then converted to match those of the image features; subsequently, the contextual features and image features are fused together using the formula: P out =P H×W×D +G H×W×dv G H×W×dv This indicates that the comprehensive contextual features are reshaped into a spatial dimension, P H×W×D P represents image features. out This represents the image features output after fusion; The fused image features are input into the U-Net backbone network, which outputs the predicted noise γ at time step b-1. The predicted noise is required to be proportional to the latent variable z input at time step b-1. b-1 Same size; By reshaping the comprehensive contextual features and matching their dimensions with those of the image features through linear projection, and then performing feature fusion, the problem of dimensional heterogeneity between semantic features and image features is solved, allowing the two types of features to be seamlessly integrated. The fused features contain both the spatial information of the image and the semantic contextual information of the text, providing high-quality input for accurate noise prediction.

[0048] S602. Utilizing prediction noise to determine the latent variable z at time step b-1 b-1 To update, the formula is: ; In the formula, z b-2 Let β represent the updated latent variable. b-2 β represents the noise scheduling parameter at time step b-2. b-1 z represents the noise scheduling parameter at time step b-1. b-1 This represents the latent variables before the update at time step b-1. Noise prediction based on the fusion feature output having the same size as the latent variables ensures the fit between the noise prediction and the latent variables, allowing the noise prediction to accurately reflect the image noise distribution at the current time step. Then, the latent variables are updated using a sampling formula specific to the diffusion model, following the denoising rules of the diffusion model. This enables efficient and accurate updating of the latent variables, allowing the denoising process to gradually approach the latent space representation of the true image.

[0049] S700: Input the updated latent variable into the next time step, and repeat steps S300 to S600 until the final latent variable is obtained in the last time step. Input the final latent variable into the value decoder to generate the final image.

[0050] The specific steps for the decoder to generate the final image from the final latent variable input value are as follows: The updated latent variables are used as input latent variables for time step b-2. Steps S300 to S600 are repeated, iteratively updating the latent variables according to the time steps until the final latent variable z1 is obtained in the last time step. The final latent variable is then input into the decoder to generate the final image I. The updated latent variables are cyclically input into the next time step, and the feature fusion, noise prediction, and latent variable update process is repeated. This achieves multi-round iterative optimization of the latent variables, allowing them to gradually approach the latent space representation of the real image through continuous denoising and semantic fusion, significantly improving the quality and semantic consistency of the final generated image.

[0051] The semantic image generation system based on artificial intelligence includes a semantic feature importance module, an attention weight module, a conflict judgment module, an attention weight update module, a comprehensive context feature calculation module, a latent variable update module, and an iterative output module. The semantic feature importance module is used by the system to extract a set of structured semantic features describing the text using natural language processing algorithms, and inputs each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score; The attention weight module is used to generate images by iteratively denoising using a diffusion model. In each iteration, initial attention weights are assigned for each semantic feature using an initial importance score, and initial latent variables are determined. The conflict judgment module is used to set the denoising time step, extract the image features at the corresponding time step at each time step, linearly transform the semantic features and image features to obtain the key matrix and query matrix, and use the query matrix and key matrix to calculate the attention map; classify all semantic features into primary features and secondary features, use the attention map to calculate the conflict coefficient of each pair of primary and secondary features, and obtain the conflict set after judgment. The attention weight update module is used to calculate the attention weight reduction amount for each conflicting secondary feature, sum up all conflicting secondary features and add a negative sign to obtain the total weight reduction amount, and compensate all primary features proportionally to obtain the attention weight compensation amount for each conflicting primary feature. The attention weights of the conflicting secondary features and primary features are updated using the attention weight reduction amount and compensation amount, respectively. The comprehensive context feature calculation module is used to scale the attention map of all semantic features using the updated attention weights in the next time step, and to calculate the comprehensive context features using the scaled attention map and the semantic feature value matrix. The latent variable update module is used to combine comprehensive context features and image features with the predicted noise, obtain the predicted noise, and then update the latent variables using the sampling formula of the diffusion model. The iterative output module is used to input the updated latent variables into the next time step, repeat steps S300 to S600 until the final latent variables are obtained in the last time step, and input the final latent variables into the value decoder to generate the final image.

[0052] The conflict determination module includes an attention map calculation unit and a conflict coefficient calculation unit; The attention map calculation unit is used to set the input image feature matrix as P in the cross-attention layer. b ∈R H ×W×D R H×W×D The set of real numbers represents the three-dimensional dimension of image features; the semantic features and image features are linearly transformed to obtain the key matrix and query matrix; the attention map of each semantic feature is calculated using the key matrix and query matrix; The conflict coefficient calculation unit is used to sort all semantic features in descending order according to the initial importance score, take the first k as the main feature set M, and the remaining semantic features as the secondary feature set E; plot the initial importance score after descending order as a curve, and extract the number of semantic features before the inflection point of the curve as the k value; for each pair of main features m∈M and secondary features e∈E, calculate the conflict coefficient. Example

[0053] With the goal of generating a semantic image from the descriptive text "A small yellow butterfly is beside a cluster of red roses under a blue sky and white clouds," this paper details the entire execution process of the implementation plan, clarifying the input, calculation, processing logic, and output results of each step, making the application of the method and system more concrete. In this embodiment, the diffusion model iteration step number B=100 (denoising time steps b=100,99,...,1), the step size hyperparameter α=0.2, and the conflict threshold θ=0.6 are adjusted. All calculation processes follow the formulas and rules in the plan.

[0054] The user inputs the target text: "There is a small yellow butterfly next to a cluster of red roses under a blue sky and white clouds."

[0055] The system extracts a set of structured semantic features F={f1: blue sky, f2: white clouds, f3: red rose bushes, f4: yellow butterflies} using NLP algorithms. There are four core semantic features and no redundant features.

[0056] Input f1-f4 into a lightweight MLP semantic importance discriminator, and combine it with the real label mapping of the LPIPS algorithm to output the initial importance scores of each feature: s1 (blue sky) = 0.85, s2 (white clouds) = 0.80, s3 (red rose bush) = 0.95, s4 (yellow butterfly) = 0.70; the scores reflect the core importance of each feature in image generation, with the red rose bush being the most core feature and the yellow butterfly being a secondary feature.

[0057] The initial latent variable z is obtained by sampling from the standard normal distribution N(0, I). 100 As the initial input to the high-noise time step 100 of the diffusion model, the latent variable is a three-dimensional matrix that meets the model requirements, representing the initial noise latent space of the image.

[0058] Calculated as "single feature importance score / sum of all feature scores", with a constraint weight sum of 1. Calculation process: Score sum = 0.85 + 0.80 + 0.95 + 0.70 = 3.3; Initial attention weight: w1 100 =0.85 / 3.3≈0.258, w2 100 =0.80 / 3.3≈0.242, w3 100 =0.95 / 3.3≈0.288, w4 100 =0.70 / 3.3≈0.212; weight sum =0.258+0.242+0.288+0.212=1, which satisfies the constraint condition, and the core feature, the red rose bush, has the highest weight.

[0059] The initial latent variable z 100 Input the U-Net backbone network and extract the image feature matrix P across the attention layer. 100 ∈R H ×W×D (Set H=256, W=256, D=512, which are the height, width, and depth dimensions of the image).

[0060] Calculate the attention map for each semantic feature to obtain four attention maps A1. (100) (Blue Sky), A2 (100) (White Clouds), A3 (100) (Red Rose), A4 (100) (Yellow Butterfly), each attention map is a 256×256 spatial matrix, representing the spatial attention distribution of features in the image.

[0061] Sort the initial importance scores in descending order: s3(0.95)>s1(0.85)>s2(0.80)>s4(0.70).

[0062] Draw a score sorting curve and extract the number of features before the inflection point of the curve, k=3. Therefore, the main feature set M={f1: blue sky, f2: white clouds, f3: red rose bushes} and the secondary feature set E={f4: yellow butterflies}.

[0063] Calculate the conflict coefficient of each pair of primary and secondary features according to the formula, and take the average value across all spatial locations to obtain the conflict coefficient of a unique pair of primary and secondary features: C 1,4 (100) ≈0.55, C 2,4 (100) ≈0.58, C 3,4 (100) ≈0.72; Compared to the conflict threshold θ=0.6: C 3,4 (100) Since ≈0.72 > 0.6, the minor feature f4 (the yellow butterfly) is added to the conflict set C. (100) ={f4}, where f4 is the conflict minor feature at time step 100.

[0064] For the conflict set C (100) ={f4}, and complete the weight reduction, compensation and update according to the scheme formula, still taking b=100 as an example.

[0065] Calculate the weight reduction of conflicting secondary features, Δw4 (100) =-0.212×0.2×1.2=-0.05088; that is, the attention weight reduction of f4 is -0.05088.

[0066] Total weight reduction △w total =0.05088 (with a leading minus sign); The compensation amount for the main features is allocated according to the ratio of "weight of a single main feature / sum of weights of all main features", and the sum of weights of the main features = 0.258 + 0.242 + 0.288 = 0.788; therefore, Δw1 is calculated. (100) ≈0.01664, △w2 (100) ≈0.01562, △w3 (100) ≈0.01857; Update the attention weights for primary and secondary features. The conflicting secondary feature (f4: yellow butterfly) = 0.212 - 0.05088 = 0.16122; The primary feature (f1: blue sky) = 0.258 + 0.01664 = 0.27464; Principal feature (f2: cloud) = 0.242 + 0.01562 = 0.25762; Principal feature (f3: red rose bush) = 0.288 + 0.01857 = 0.30657; Moving to the next time step b=99, input the updated attention weights from S400 into the U-Net cross-attention layer and perform the following operations: The original attention map of each semantic feature is multiplied by the updated weights to obtain a scaled attention map, and the comprehensive contextual feature G is calculated. Reshape G to 256×256×64, then transform it to 256×256×512 via linear projection (consistent with the image feature dimension D), and press P. out =P 99 +G 256×256×512 Fusion of image features and contextual semantic features; Fusing feature P out Input U-Net, output and latent variable z 99 Size-consistent noise prediction γ （256×256×512） ; The updated latent variable z is calculated using the document diffusion model sampling formula. 98 As input for time step b=98, the semantic and spatial matching of the latent variables is more accurate than before the correction.

[0067] z 98 Given time step b=98, repeat all steps from S300 to S600 to complete the iteration for 97 time steps from b=98 to b=1: Conflict-free time step: The weights updated in the previous time step are directly used, and there is no need to calculate the reduction amount; If new conflict characteristics emerge: recalculate the reduction amount and update the weights according to the revised S400 formula; Iteration results: As the time step decreases (noise decreases), the spatial representation of the core feature (red rose bush) is continuously strengthened, the interference of conflicting features is continuously suppressed, and the spatial distribution of semantic features is more consistent with the text description "there are butterflies next to the rose bush".

[0068] After completing the b=1 time step, the final latent variable z1 is obtained; z1 is input into the latent variable decoder, and converted into a pixel space image through deconvolution and feature mapping, and the final generated image is output.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A semantic image generation method based on artificial intelligence, characterized in that: The method includes the following steps: S100. The system uses natural language processing algorithms to extract a set of structured semantic features describing the text, and inputs each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score. S200. Use the diffusion model to iteratively denoise and generate images. In each iteration, assign initial attention weights to each semantic feature using the initial importance score and determine the initial latent variables. S300. Set a denoising time step, extract the image features at each time step, linearly transform the semantic features and image features to obtain the key matrix and query matrix, and use the query matrix and key matrix to calculate the attention map; classify all semantic features into primary features and secondary features, use the attention map to calculate the conflict coefficient of each pair of primary and secondary features, and obtain the conflict set after judgment. S400. For each conflicting secondary feature, calculate the attention weight reduction amount. Sum all conflicting secondary features and add a negative sign to obtain the total weight reduction amount. Compensate all primary features with the total weight reduction amount proportionally to obtain the attention weight compensation amount for each conflicting primary feature. Update the attention weights of conflicting secondary features and primary features using the attention weight reduction amount and compensation amount respectively. S500. In the next time step, the attention map of all semantic features is scaled using the updated attention weights, and the comprehensive context features are calculated using the scaled attention map and the semantic feature value matrix. S600. By combining comprehensive contextual features and image features with the predicted noise, the latent variables are updated using the sampling formula of the diffusion model after obtaining the predicted noise. S700: Input the updated latent variable into the next time step, and repeat steps S300 to S600 until the final latent variable is obtained in the last time step. Input the final latent variable into the value decoder to generate the final image.

2. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: The lightweight semantic importance discriminator pre-trained in S100 is specifically as follows: The lightweight semantic feature importance discriminator employs a lightweight MLP algorithm to construct its discriminator structure. Historical image-text data pairs are collected as the training set, where each data pair represents a data pair consisting of descriptive text and its corresponding image. Based on the descriptive text, an intermediate feature map is extracted from the generated image using a diffusion model. Each feature is treated as a region, and two images are generated: one normally generated image and the other with the features set to zero. Then, the difference between the two images is calculated using the learned perceptual image patch similarity algorithm (LPIPS), and this difference is used as the initial importance score s of the feature's true label. i s i ∈[0,1]; During training using the training set, a loss function is constructed such that the initial importance score output by the lightweight semantic feature importance discriminator is equal to the initial importance score of the true label.

3. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: The determination of the initial latent variables in S200 specifically involves: When generating an image by iterative denoising using a diffusion model, let the number of iterations be B. At each iteration, the latent variables for the next step are obtained based on the current noisy image and descriptive text. Initial latent variable z is sampled from a standard normal distribution. B The initial latent variables follow a standard multivariate normal distribution N(0, I), where 0 represents the mean vector and I represents the unit covariance matrix; the initial latent variables represent the image representation in the latent space of the diffusion model. The method of allocating initial attention weights using initial importance scores specifically involves: calculating the proportion of the initial importance score of each semantic feature to the initial importance scores of all semantic features as the initial attention weight, and constraining the sum of the initial attention weights of all semantic features to 1.

4. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: The calculation of the attention map using the query matrix and key matrix in S300 is specifically as follows: The denoising time steps are set to b = B, B-1, B-2, ..., 1; the diffusion model iterates from the high-noise time step B to the low-noise time step 1; for each time step, the current latent variable is input into the U-Net backbone network of the diffusion model. The U-Net backbone network consists of an encoder, a bottleneck layer, and a decoder, and cross-attention modules are inserted in all resolution layers to form cross-attention layers; the cross-attention layers are used to integrate semantic features into image features; In the cross-attention layer, let the input image feature matrix be P. b ∈R H×W×D R H×W×D Let H be the set of real numbers representing the three dimensions of the image features, where H represents height, W represents width, and D represents depth. The semantic features and image features are linearly transformed to obtain the key matrix and query matrix, as shown in the formula: Q=P b ×W Q ,K i =f i ×W K ,V i =f i ×W V ; In the formula, W Q W K和 W V K represents the learnable projection matrix, obtained through training; Q represents the query matrix; K represents the query matrix. i V represents the semantic feature key matrix. i f represents the semantic eigenvalue matrix. i Represents the i-th semantic feature; Calculate the attention map for each semantic feature using the key matrix and query matrix; The conflict set obtained after judgment is specifically as follows: All semantic features are sorted in descending order based on the initial importance scores. The top k features are taken as the primary feature set M, and the remaining semantic features are taken as the secondary feature set E. The initial importance scores after descending order are plotted as a curve, and the number of semantic features before the inflection point of the curve is extracted as the k value. For each pair of primary features m∈M and secondary features e∈E, calculate the conflict coefficient C. （b） m，e C （b） m，e This represents the conflict coefficient between the primary feature m and the secondary feature e at time step b. Staff set a conflict threshold θ, when C （b） m，e When the value is greater than θ, the secondary feature e is added to the conflict set C of the current time step. （b） Otherwise, it is determined that there is no conflict and there is no need to update the attention weights.

5. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: In step S400, the attention weights of the secondary and primary features of the conflict are updated using attention weight reduction and compensation, respectively, as follows: For each secondary feature of a conflict within the conflict set, calculate the attention weight reduction using the following formula: ； In the formula, △w e （b） w represents the amount of attention weight reduction for the conflicting secondary feature e at time step b. e （b） This represents the attention weights before updating the conflicting minor feature e at time step b; α represents the adjustment step size hyperparameter; This represents the proportion of the most severe conflict level relative to a threshold. Calculate the sum of attention weight reductions for all conflicting secondary features and add a negative sign to obtain the total weight reduction; then proportionally compensate all primary features with the total weight reduction to obtain the attention weight compensation for each conflicting primary feature. For the primary and secondary features, the attention weight compensation and attention weight reduction are added to the attention weights before the update, respectively, and the updated attention weights for the primary and secondary features are output. The updated attention weights are used for image generation during the denoising process in the next time step.

6. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: The S500 process specifically calculates the comprehensive contextual features using the scaled attention map and semantic feature value matrix as follows: The updated attention weight set is input into the U-Net backbone network. For each semantic feature's attention map, the updated attention weights are multiplied to scale the attention map. The scaled attention map and the semantic feature's value matrix are then used to calculate the context features of each semantic feature, as shown in the formula: G i （b-1） =A i （b-1）’ ×V i G i （b-1） A represents the context feature of the i-th semantic feature at the next time step b-1. i （b-1）’ This represents the scaled attention map of the i-th semantic feature at the next time step b-1; The comprehensive contextual feature G is obtained by summing the contextual features of all semantic features.

7. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: The latent variables updated using the sampling formula of the diffusion model in S600 are specifically as follows: In the cross-attention layer, the integrated contextual features are reshaped into spatial dimensions H×W×d. v d v The dimension representing the semantic feature value; when d v When the dimension d is not equal to D, linear projection is used to integrate the context features. v Convert to be consistent with the dimensions of image features; Then, the contextual features and image features are fused together, using the formula: P out =P H×W×D +G H×W×dv G H×W×dv This indicates that the comprehensive contextual features are reshaped into a spatial dimension, P H×W×D P represents image features. out This represents the image features output after fusion; The fused image features are input into the U-Net backbone network, which outputs the predicted noise γ at time step b-1. The predicted noise is required to be proportional to the latent variable z input at time step b-1. b-1 Same size; Using prediction noise to evaluate the latent variable z at time step b-1 b-1 To update, the formula is: ； In the formula, z b-2 Let β represent the updated latent variable. b-2 β represents the noise scheduling parameter at time step b-2. b-1 z represents the noise scheduling parameter at time step b-1. b-1 This represents the latent variable before it is updated at time step b-1.

8. The semantic image generation method based on artificial intelligence according to claim 1, characterized in that: In S700, the decoder generates the final image from the final latent variable input value as follows: The updated latent variable is used as the input latent variable for time step b-2. Steps S300 to S600 are repeated to iterate and update the latent variable according to the time step until the final latent variable z1 is obtained in the last time step. The final latent variable is then input into the decoder to generate the final image I.

9. A semantic image generation system based on artificial intelligence, characterized in that: The semantic image generation system includes a semantic feature importance module, an attention weight module, a conflict judgment module, an attention weight update module, a comprehensive context feature calculation module, a latent variable update module, and an iterative output module. The semantic feature importance module is used by the system to extract a set of structured semantic features describing the text using natural language processing algorithms, and inputs each semantic feature into a pre-trained lightweight semantic importance discriminator to output an initial importance score; The attention weight module is used to generate images by iteratively denoising using a diffusion model. In each iteration, initial attention weights are assigned for each semantic feature using an initial importance score, and initial latent variables are determined. The conflict judgment module is used to set the denoising time step, extract the image features at the corresponding time step at each time step, linearly transform the semantic features and image features to obtain the key matrix and query matrix, and use the query matrix and key matrix to calculate the attention map; classify all semantic features into primary features and secondary features, use the attention map to calculate the conflict coefficient of each pair of primary and secondary features, and obtain the conflict set after judgment. The attention weight update module is used to calculate the attention weight reduction amount for each conflicting secondary feature, sum up all conflicting secondary features and add a negative sign to obtain the total weight reduction amount, and compensate all primary features proportionally to obtain the attention weight compensation amount for each conflicting primary feature. The attention weights of the conflicting secondary features and primary features are updated using the attention weight reduction amount and compensation amount, respectively. The comprehensive context feature calculation module is used to scale the attention map of all semantic features using the updated attention weights in the next time step, and to calculate the comprehensive context features using the scaled attention map and the semantic feature value matrix. The latent variable update module is used to combine comprehensive context features and image features with the predicted noise, obtain the predicted noise, and then update the latent variables using the sampling formula of the diffusion model. The iterative output module is used to input the updated latent variables into the next time step, repeat steps S300 to S600 until the final latent variables are obtained in the last time step, and input the final latent variables into the value decoder to generate the final image.

10. The artificial intelligence-based semantic image generation system according to claim 9, characterized in that: The conflict determination module includes an attention map calculation unit and a conflict coefficient calculation unit; The attention map calculation unit is used to set the input image feature matrix as P in the cross-attention layer. b ∈R H×W×D R H ×W×D The set of real numbers represents the three-dimensional dimension of image features; the semantic features and image features are linearly transformed to obtain the key matrix and query matrix; the attention map of each semantic feature is calculated using the key matrix and query matrix; The conflict coefficient calculation unit is used to sort all semantic features in descending order according to the initial importance score, take the first k as the main feature set M, and the remaining semantic features as the secondary feature set E; plot the initial importance score after descending order as a curve, and extract the number of semantic features before the inflection point of the curve as the k value; for each pair of main features m∈M and secondary features e∈E, calculate the conflict coefficient.

Images