An AI-based picture generation and optimization method

Abstract

The application discloses an AI-based picture generation and optimization method, and relates to the technical field of intelligent image generation; key phrases are obtained by performing character recognition and semantic analysis on a requirement document, and are substituted into a full connection layer network to obtain phrase vector representation, and then are substituted into a long short-term memory network to obtain a target relationship prompt; the target relationship prompt is substituted into a preset model to obtain an initial image; the target relationship prompt is substituted into a horizontal LSTM network and a vertical LSTM network respectively to obtain horizontal correlation conditions and vertical correlation conditions, and the requirement document is substituted into a CLIP model to obtain global correlation conditions; and the initial image is subjected to enhancement processing according to the horizontal correlation conditions, the vertical correlation conditions and the global correlation conditions to obtain a target requirement image. Key phrases are extracted, and the logical relationship implied in the text is converted into calculable structured information through an LSTM network, and the global correlation conditions are combined to ensure that the overall style and content of the image are consistent with the requirements, and the operation efficiency of the system is improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent image generation technology, specifically relating to an AI-based image generation and optimization method. Background Technology

[0002] In the traditional field of holiday image production, it has long relied on professional designers investing significant time and effort in handcrafted creation. From conceiving holiday themes and selecting suitable elements to outlining the composition and coordinating color schemes, each step requires designers to rely on rich experience and a refined aesthetic sense. However, this purely manual production model is not only inefficient and unable to handle the large volume of images required in a short period, but more importantly, due to differences in designers' creative styles and understandings, it is difficult to accurately meet the personalized needs of each user.

[0003] With the rapid development of artificial intelligence technology, AI has demonstrated remarkable potential in the field of image generation and optimization. Text-based AI tools can quickly generate images with certain visual effects by learning from massive amounts of image data. Although many methods for text-to-image generation have been proposed and applied, in practice, these methods often struggle to effectively and accurately integrate textual information into the entire image generation process. The rich semantics contained in text, including specific object shapes, scene atmosphere, and emotional tendencies, are difficult for AI systems to fully capture and transform into visual elements in images. This results in significant discrepancies between the generated images and the user's intended images, greatly reducing image quality and causing a serious waste of computer resources, leading to low overall system efficiency. Summary of the Invention

[0004] The purpose of this invention is to solve the problem that there is a significant error between the image generated from the text and the image actually desired by the user. This not only greatly reduces the quality of the generated image, but also causes a serious waste of computer resources and makes the entire system inefficient. Therefore, this invention proposes an AI-based image generation and optimization method.

[0005] This invention proposes an AI-based image generation and optimization method, the method comprising:

[0006] Obtain a requirements document, perform text recognition and semantic parsing on the requirements document to obtain key phrases, and substitute the key phrases into a fully connected layer network to obtain phrase vector representations; the key phrases include topic phrases, object phrases, and relation phrases; the phrase vector representations include topic representations, object representations, and relation representations;

[0007] Substituting the phrase vector representation into a long short-term memory network yields target relation hints;

[0008] Substitute the target relationship hints into the preset model to obtain the initial image;

[0009] The target relationship hints are substituted into the horizontal LSTM network and the vertical LSTM network respectively to obtain the horizontal correlation conditions and the vertical correlation conditions. The requirement document is substituted into the CLIP model to obtain the global correlation conditions.

[0010] The target image is obtained by enhancing the initial image based on the horizontal correlation condition, the vertical correlation condition, and the global correlation condition.

[0011] Optionally, the key phrases obtained by performing text recognition and semantic parsing on the requirement document include:

[0012] The requirement document is preprocessed to obtain an initial draft, and the initial draft is segmented to obtain a short word set;

[0013] The effective short word set is obtained by removing the stop words from the short word set.

[0014] The effective short word set is processed by syntactic analysis and semantic role labeling to obtain topic phrases, object phrases, and relational phrases; the topic phrases, object phrases, and relational phrases are used as key phrases.

[0015] Optionally, substituting the target relationship hints into a preset model to obtain an initial image includes:

[0016] Initialize Gaussian noise, and substitute the Gaussian noise into the fully connected layer to perform dimension adjustment to obtain the target Gaussian noise;

[0017] The initial fused features are obtained by concatenating the target Gaussian noise and the target relationship hints;

[0018] The initial fusion features are substituted into the image generation module to obtain the first image;

[0019] Global max pooling and global average pooling are performed on the first image to obtain the global maximum feature and global average feature;

[0020] The global fused feature is obtained by concatenating the global maximum feature and the global average feature along the channel dimension;

[0021] The initial image is obtained by performing a 1×1 convolution operation on the global fusion features.

[0022] Optionally, substituting the initial fusion features into the image generation module to obtain the first image includes:

[0023] After upsampling the initial fused features, a convolution operation is performed to obtain the first feature. The initial fused features and the first feature are then substituted into the improved text fusion layer to obtain the second feature.

[0024] The second feature is convolved to obtain the third feature, and the third feature and the initial fusion feature are substituted into the improved text fusion layer to obtain the first image.

[0025] Optionally, the improved text fusion layer comprises two improved affine transformation layers and two activation layers stacked alternately, followed by a convolutional layer; the working principle of the improved text fusion layer specifically includes:

[0026] Obtain the first input feature and the initial fused feature, and substitute the initial fused feature into the improved affine transformation layer to obtain the first affine output;

[0027] The first affine feature is obtained by adjusting the first input feature based on the first affine output; the first input feature is either the first feature or the third feature.

[0028] Substitute the first affine feature into the activation layer to obtain the first activation feature;

[0029] The first activation feature is substituted into the improved affine transformation layer to obtain the second affine output, and the first activation feature is adjusted according to the second affine output to obtain the second affine feature.

[0030] The first image is obtained by substituting the second affine feature into the activation layer and then performing a 1×1 convolution operation.

[0031] Optionally, the improved affine transformation layer includes:

[0032] The input features are acquired, and scaling and translation parameters are learned from the input features using two multilayer perceptrons.

[0033] The input features are adjusted according to the scaling parameters and the translation parameters to obtain the output features.

[0034] Optionally, enhancing the initial image based on the horizontal correlation condition, the vertical correlation condition, and the global correlation condition to obtain the target image includes:

[0035] Based on the horizontal and vertical correlation conditions, word-level radial transformations are performed on the initial image to obtain a first radial transformation map.

[0036] The global correlation conditions are used to perform global radiometric transformation on the first radiometric transformation map to obtain the second radiometric transformation map;

[0037] The initial image and the second radiometric variation map are fused to obtain an initial fused variation map;

[0038] The discriminator verifies the matching degree between the initial fused change map and the horizontal correlation condition, the vertical correlation condition, and the global correlation condition. If the matching degree is greater than a preset threshold, the initial fused change map is recorded as the target requirement image.

[0039] Optionally, performing word-level radial transformation on the initial image based on the horizontal correlation condition and the vertical correlation condition to obtain a first radial transformation map includes:

[0040] The objects are determined based on the horizontal correlation conditions and the vertical correlation conditions, and the scaling factor and offset of each object are generated by the first multilayer perceptron and the second multilayer perceptron.

[0041] The dynamic weights for each object are determined using a first-level multilayer perceptron and Softmax.

[0042] The objects in the initial image are adjusted according to the scaling factor, offset, and dynamic weight of each object to obtain the first radial variation map.

[0043] Optionally, the global correlation conditions for performing global radiometric transformation on the first radiometric transformation map to obtain the second radiometric transformation map include:

[0044] Extract the sentence-level vectors from the global relevance conditions to obtain the global affine parameters;

[0045] The first radiation variation map is adjusted according to the global affine parameters to obtain the second radiation variation map.

[0046] Optionally, after verifying the matching degree between the initial fused change map and the horizontal correlation condition, the vertical correlation condition, and the global correlation condition through a discriminator, the method further includes:

[0047] If the matching degree is less than or equal to the preset value, the scaling factor and offset of each object are adjusted through backpropagation to re-optimize the first radial change map.

[0048] The beneficial effects of this invention are:

[0049] This invention proposes an AI-based image generation and optimization method. The method involves acquiring a requirement document, performing text recognition and semantic parsing to obtain key phrases, and then substituting these key phrases into a fully connected layer network to obtain phrase vector representations. Key phrases include topic phrases, object phrases, and relational phrases. Phrase vector representations include topic representations, object representations, and relational representations. These phrase vector representations are then substituting into a Long Short-Term Memory (LSTM) network to obtain target relational hints. These hints are then substituting into a preset model to obtain an initial image. The target relational hints are further substituting into horizontal and vertical LSTM networks to obtain horizontal and vertical relevance conditions. The requirement document is then substituting into a CLIP model to obtain global relevance conditions. Based on these conditions, the initial image is enhanced to obtain the target requirement image. By extracting key phrases through text recognition and semantic parsing, the core information of the requirement document can be accurately captured. Furthermore, by converting these key phrases into vector representations and using an LSTM network to transform the implicit logical relationships in the text into computable structured information, combined with global relevance conditions, the overall style and content of the image are ensured to be consistent with the requirements. This reduces the error between the text-generated image and the user's actual desired image, improving the system's operating efficiency. Attached Figure Description

[0050] The invention will now be further described with reference to the accompanying drawings.

[0051] Figure 1 A flowchart illustrating an AI-based image generation and optimization method provided in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of a method for obtaining an initial image through a target relationship, provided as an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0054] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention provides an AI-based image generation and optimization method. See also... Figure 1 , Figure 1 This is a flowchart illustrating an AI-based image generation and optimization method provided in an embodiment of the present invention. The method includes the following steps:

[0056] S101, Obtain the requirements document, perform text recognition and semantic parsing on the requirements document to obtain key phrases, and substitute the key phrases into the fully connected layer network to obtain phrase vector representations;

[0057] S102, Substitute the phrase vector representation into the long short-term memory network to obtain target relation cues;

[0058] S103, Substitute the target relationship prompts into the preset model to obtain the initial image;

[0059] S104, Substitute the target relationship prompts into the horizontal LSTM network and the vertical LSTM network respectively to obtain the horizontal correlation conditions and the vertical correlation conditions, and substitute the requirement document into the CLIP model to obtain the global correlation conditions.

[0060] S105, the initial image is enhanced based on horizontal correlation conditions, vertical correlation conditions and global correlation conditions to obtain the target image;

[0061] Key phrases include topic phrases, object phrases, and relation phrases; phrase vector representations include topic representations, object representations, and relation representations.

[0062] Based on the AI-based image generation and optimization method provided in this invention, key phrases are extracted through text recognition and semantic parsing, which can accurately capture the core information of the requirement document. Then, by converting the key phrases into vector representations and using an LSTM network to transform the implicit logical relationships in the text into computable structured information, combined with global correlation conditions, the overall style and content of the image are ensured to be consistent with the requirements, reducing the error between the image generated from the text and the image actually desired by the user, and improving the system's operating efficiency.

[0063] In one implementation, key phrases are extracted through text recognition and semantic parsing, which can accurately capture the core information of the requirements document. By using a fully connected layer and LSTM, the implicit logical relationships (spatial location, action association) in the text can be transformed into computable structured information, ensuring that the semantics of the text are fully encoded into a computable vector representation, avoiding subjective bias or omissions that may occur when humans understand the requirements.

[0064] In one implementation, the initial image is generated by target relation mapping to avoid the ambiguity of manually designed features in traditional methods; the horizontal and vertical structural constraints of the image are captured by horizontal and vertical LSTM respectively, and the global semantic consistency extracted by CLIP is combined to perform local-global joint optimization on the initial image, thereby improving the accuracy of image generation.

[0065] In one implementation, the global relevant conditions ensure the global semantic consistency of the overall style, content, and requirements of the image; the entire process from parsing the requirements document to image generation depends on automated model processing, reducing the links of manual annotation, design sketches, and repeated modifications, achieving efficient and accurate conversion from text requirements to high-quality images, and improving system efficiency.

[0066] In one embodiment, the key phrases obtained by performing word recognition and semantic parsing on the requirements document include:

[0067] Preprocess the requirements document to obtain an initial manuscript, and perform word segmentation on the initial manuscript to obtain a short word set;

[0068] Remove the stop words in the short word set to obtain an effective short word set;

[0069] Process the effective short word set through syntactic analysis and semantic role labeling to obtain subject phrases, object phrases, and relationship phrases; the subject phrases, object phrases, and relationship phrases are used as key phrases.

[0070] In one implementation, preprocessing and word segmentation break down the text into basic language units. After removing stop words, the effective short word set filters out meaningless words such as "de", "le", "zai", etc., reducing the interference of redundant information and making subsequent analysis more focused on valuable content.

[0071] In one implementation, by extracting subject phrases, object phrases, and relationship phrases through syntactic analysis and semantic role labeling, the logical structure of "who (subject)", "to whom / what (object)", and "did what / has what relationship (relationship)" in the text can be clearly sorted out.

[0072] In one implementation, preprocessing is to remove redundant symbols (such as punctuation marks, special characters), correct spelling mistakes (perform context correction through language models such as BERT), unify case, format, and other existing technical means.

[0073] In one implementation, the extraction of key phrases compresses long texts into structured core elements. Subsequently, tasks such as information retrieval, text classification, sentiment analysis, or knowledge graph construction can be efficiently carried out based on these key phrases, avoiding repeated processing of the complete text and saving time and computing power costs.

[0074] In one implementation, word segmentation, syntactic analysis, and semantic role labeling are implemented through natural language processing (NLP) techniques.

[0075] In one implementation, compared to simple keyword extraction, the division of topic, object, and relational phrases is more in line with the semantic and syntactic rules of language, can more comprehensively reflect the connotation of the text, reduce ambiguity caused by isolated keywords, and improve the accuracy of understanding the text's topic, core content, and logical relationships; the extracted key phrases can be directly used to construct text summaries, generate tags, and perform similar text matching, etc.

[0076] In one embodiment, see Figure 2 , Figure 2 A flowchart providing a method for obtaining an initial image through target relationships also includes:

[0077] S1031, initialize Gaussian noise, substitute the Gaussian noise into the fully connected layer and adjust the dimension to obtain the target Gaussian noise;

[0078] S1032, Initial fused features are obtained by splicing the target Gaussian noise and target relationship clues;

[0079] S1033, Substitute the initial fusion features into the image generation module to obtain the first image;

[0080] S1034, Perform global max pooling and global average pooling on the first image to obtain the global maximum feature and global average feature;

[0081] S1035, global fusion features are obtained by concatenating the global maximum feature and the global average feature along the channel dimension;

[0082] S1036, perform a 1×1 convolution operation on the global fusion features to obtain the initial image.

[0083] In one implementation, Gaussian noise is initialized as the starting point of input. Its randomness provides rich initial images for image generation, avoiding the generation of homogeneous results. The dimensional adjustment of the fully connected layer can map the noise to feature dimensions that are suitable for subsequent tasks, ensuring that the noise information can effectively participate in the image generation process and laying the foundation for diversified image output.

[0084] In one implementation, the dimension is adjusted to the scale of the final image. By concatenating the target Gaussian noise with the target relationship cues, abstract semantic relationships can be embedded in the initial stage of image generation. This fusion allows the generation model to clearly define what relationship content to generate, avoiding the generation result from being disconnected from the target semantics and improving the matching degree between the image and the expected relationship.

[0085] In one implementation, global max pooling and global average pooling are performed on the first image to capture the most prominent features (the areas with the highest brightness and the strongest contrast) and the overall trend features (average color and overall structure) in the image, respectively. The global fusion feature after splicing the two along the channel dimension can comprehensively reflect the global information of the image and avoid the model from focusing too much on local details and ignoring the overall structure.

[0086] In one embodiment, substituting the initial fusion features into the image generation module to obtain the first image includes:

[0087] The first feature is obtained by upsampling the initial fused features and then performing a convolution operation. The second feature is obtained by substituting the initial fused features and the first feature into the improved text fusion layer.

[0088] The third feature is obtained by performing a convolution operation on the second feature. The third feature and the initial fusion feature are then substituted into the improved text fusion layer to obtain the first image.

[0089] In one implementation, the improved text fusion layer intervenes twice at different feature stages (initial fusion feature and first feature, third feature and initial fusion feature), which allows text information (such as semantics in target relationship prompts) to penetrate into visual features in stages and at multiple levels. Unlike the shallow association of information that may be caused by single fusion, this progressive fusion ensures that text semantics continue to be effective in the process of feature evolution from low dimension to high dimension and from coarse to fine, avoids visual features from deviating from text guidance, and improves the consistency between image and text semantics; the convolution operation is performed by using a 1×1 convolution kernel.

[0090] In one implementation, the initial fused features are upsampled and convolved to obtain the first feature, which is essentially an enhancement of the features in terms of dimensionality and detail. The initial fused features are then fused with the first feature to obtain the second feature, which is equivalent to using the underlying semantics retained in the initial features to constrain the evolution direction of the detailed features, thus avoiding semantic distortion caused by excessive expansion of details after upsampling.

[0091] In one implementation, the second feature is convolved to obtain the third feature, which is then fused with the initial fusion feature a second time. The global structure of the high-order feature is further calibrated through the core semantics of the initial feature, ensuring that while the richness of details is improved during feature iteration, the overall semantic coherence is not compromised. The final generated image contains both rich details and conforms to the core semantics.

[0092] In one implementation, the text fusion layer is improved by performing two applications, and the features at different stages (low-order initial features, mid-order first features, and high-order third features) are fused differently. The early fusion focuses on injecting text semantics into the basic features to guide the general direction of feature generation; the later fusion focuses on using text semantics to constrain the detailed expression of high-order features, so as to avoid the text information being diluted after it only plays a role in the initial stage, and enhance the text's guiding power on image generation throughout the process.

[0093] In one embodiment, the improved text fusion layer comprises two improved affine transformation layers and two activation layers stacked alternately, followed by a convolutional layer; the working principle of the improved text fusion layer specifically includes:

[0094] Obtain the first input feature and the initial fused feature, and substitute the initial fused feature into the improved affine transformation layer to obtain the first affine output;

[0095] The first affine feature is obtained by adjusting the first input feature based on the first affine output; the first input feature is either the first feature or the third feature.

[0096] Substituting the first affine feature into the activation layer yields the first activation feature;

[0097] The first activation feature is substituted into the improved affine transformation layer to obtain the second affine output. The first activation feature is then adjusted based on the second affine output to obtain the second affine feature.

[0098] The first image is obtained by substituting the second affine feature into the activation layer and then performing a 1×1 convolution operation.

[0099] In one implementation, the improved affine transformation layer generates affine transformation parameters (first and second affine outputs) based on the initial fusion features (including text semantics and noise features). These parameters dynamically adjust the first input feature (first feature or third feature) and the first activation feature. Essentially, this adjustment calibrates the visual features (input features and activation features) under the guidance of text semantics. For example, based on information such as "color" and "shape" in the text description, the scale, offset, or distribution of the features are dynamically adjusted to make the visual features more consistent with the text semantics, reduce the deviation between visual and textual features, and improve their consistency.

[0100] In one implementation, the alternating use of affine transformation layers and activation layers (affine adjustment, activation, re-affine adjustment, re-activation) forms a progressive feature optimization chain of linear transformation and nonlinear mapping. Affine transformation adjusts the basic distribution of features through linear scaling and offset, while activation layers (such as ReLU, Swish, etc.) introduce nonlinearity to enhance the expressive power of features for complex patterns. Secondary affine adjustment further refines the calibration based on nonlinear features, allowing features to overcome linear limitations while maintaining direction under semantic constraints, thus preventing features from deviating from the core semantics during nonlinear transformation.

[0101] In one implementation, the initial fusion feature is reused twice as a semantic benchmark for affine transformation, ensuring that the text semantics continue to play a role in different stages of feature processing (input feature adjustment, post-activation feature re-adjustment), and avoiding the dilution of semantic information after a single adjustment; at the same time, the first input feature (first feature or third feature) undergoes two affine calibrations and activation optimizations, and the visual details (such as texture and edges) contained therein are gradually refined and fused with semantics, realizing the efficient flow and integration of visual and semantic information.

[0102] In one implementation, the parameters of the improved affine transformation layer are dynamically generated from the initial fused features, rather than fixed parameters. This means that the transformation rules will adaptively adjust according to changes in the semantics of the input text or the initial noise, making the feature processing scene-adaptive and improving the model's response accuracy to different text conditions.

[0103] In one implementation, after two affine adjustments, nonlinear enhancement of the activation layer, and a final 1×1 convolution, the output first image is based on the features of "semantic calibration + nonlinear optimization + channel compression". This not only preserves key visual details but also deeply integrates text semantics. Furthermore, the 1×1 convolution reduces redundant channels, improves the compactness of features, and can be directly transformed into a clearer and more expected image, reducing the burden of subsequent processing and improving generation efficiency and quality.

[0104] In one embodiment, the improved affine transformation layer includes:

[0105] The input features are acquired, and scaling and translation parameters are learned from the input features through two multilayer perceptrons.

[0106] The input features are adjusted based on scaling and translation parameters to obtain the output features.

[0107] In one implementation, the scaling and translation parameters are not fixed values, but are dynamically learned from the input features through a multilayer perceptron (MLP). This means that the adjustment rules will adapt to the specific content of the input features. For example, when the input features contain bright areas, the MLP may learn a larger scaling parameter to enhance the contrast of that area; when the input features have background interference, the feature distribution may be shifted by the translation parameter to highlight the subject. This dynamism allows the feature adjustment to better fit the characteristics of the input data, avoiding the limitations of fixed rules.

[0108] In one embodiment, enhancing the initial image based on horizontal correlation conditions, vertical correlation conditions, and global correlation conditions to obtain the target image includes:

[0109] The first radiometric map is obtained by performing word-level radiometric transformation on the initial image based on horizontal and vertical correlation conditions.

[0110] The second radiation change map is obtained by applying global radiation change to the first radiation change map under global correlation conditions.

[0111] The initial fused change map is obtained by fusing the initial image and the second radiometric change map.

[0112] The discriminator verifies the matching degree between the initial fused change map and the horizontal correlation condition, vertical correlation condition, and global correlation condition. If the matching degree is greater than the preset threshold, the initial fused change map is recorded as the target required image.

[0113] In one implementation, horizontal correlation conditions (such as the horizontal position of the object and its left-right relationship) and vertical correlation conditions (such as the vertical position of the object and its hierarchical relationship) perform word-level affine transformations on the initial image from the local spatial dimension to ensure that the image conforms to specific constraints in the detailed spatial layout; global correlation conditions (such as the overall scene layout and the field of view range) adjust the first radial transformation map from the global dimension to ensure that the overall structure of the image meets the macroscopic requirements; the discriminator is an MA-GP discriminator.

[0114] In one implementation, word-level affine transformation focuses on the precise adjustment of local elements (correction of the position and angle of individual objects), while global affine transformation coordinates the overall layout (avoiding overlap and disproportion between objects after local adjustments). The combination of the two forms an optimization chain of "local calibration and global coordination", which not only preserves the accurate response of local details to conditions, but also maintains the rationality and coherence of the overall image through global adjustment, so that the final image maintains high quality at both the detail and global levels.

[0115] In one implementation, the discriminator verifies the matching degree between the initial fused transformation map and three types of conditions. Only when the image simultaneously meets the horizontal, vertical, and global conditions is it recognized as the target image. This mechanism can filter out unqualified images caused by transformation errors, condition conflicts, etc., avoid outputting results that do not meet the requirements, and improve the reliability and effectiveness of the final image.

[0116] In one implementation, horizontal, vertical, and global conditions can be set or adjusted independently. Layered processing of word-level and global transformations allows the model to flexibly respond to needs at different granularities. For example, modifying only the horizontal condition allows for individual adjustment of the object's left and right positions without affecting the overall layout; modifying the global condition allows for a complete switch of the scene's perspective while preserving the relative relationships of local objects. This controllability enables image generation to adapt to diverse and personalized needs.

[0117] In one implementation, the initial image and the second radial transformation map are fused instead of the transformed image directly. This not only preserves the valid information in the initial image that meets the conditions, but also corrects the parts that do not meet the conditions through the second radial transformation map, achieving the dual effect of "preserving valid information and correcting deviations", thereby improving the efficiency of feature utilization and the accuracy of transformation.

[0118] In one embodiment, obtaining a first radiometric map by performing word-level radiometric transformation on an initial image based on horizontal and vertical correlation conditions includes:

[0119] The objects are determined based on horizontal and vertical correlation conditions, and the scaling factor and offset of each object are generated by the first and second multilayer perceptrons.

[0120] The dynamic weights for each object are determined using a first-level multilayer perceptron and Softmax.

[0121] The first radiometric transformation map is obtained by adjusting the objects in the initial image according to the scaling factor, offset, and dynamic weight of each object.

[0122] In one implementation, a first multilayer perceptron is used to generate scaling factors; a second multilayer perceptron is used to generate offsets; the first multilayer perceptron, combined with Softmax, generates dynamic weights for each object, and different objects are assigned adjustment priorities based on the importance of spatial conditions or the correlation between objects; when there are potential conflicts in spatial conditions, objects with higher weights can prioritize ensuring that their own conditions are met, while objects with lower weights can make appropriate compromises to maintain overall rationality, thereby improving the flexibility and feasibility of adjustments.

[0123] In one implementation, the first radiometric map is obtained by adjusting the objects in the initial image according to the scaling factor, offset, and dynamic weight of each object. Specifically, this includes using the formula... For each adjusted object, a first radial transformation map is obtained based on all adjusted objects. For each object's region's pixel set M, each pixel is adjusted using the formula described above, where AFF is the adjusted object, M is the object's pixel set, and m is the number of pixels. Let m be the dynamic weight of pixel m. Let m be the scaling factor for pixel m. This is the original value of pixel m. This is the offset of pixel m.

[0124] In one embodiment, obtaining a second radiation change map by applying global correlation conditions to a first radiation change map includes:

[0125] Extract sentence-level vectors from global relevance conditions to obtain global affine parameters;

[0126] The second radiation variation map is obtained by adjusting the first radiation variation map based on the global affine parameters.

[0127] In one implementation, the global relevant conditions typically include a description of the overall scene. Extracting the sentence-level vectors (i.e., the overall encoding of global semantics) can transform the abstract global requirements into quantifiable global affine parameters.

[0128] In one implementation, global semantic constraints are applied directly to the overall spatial adjustment of the image, ensuring that the first radial change map (the image after local object adjustment) conforms to the global requirements at the macro level in terms of overall layout, perspective, and scale, thus avoiding the problem that local objects meet the conditions but the overall scene deviates.

[0129] In one implementation, the global affine parameters include a first coefficient A and a second coefficient B; the second affine transformation diagram ADF is obtained by modifying all objects in the first affine transformation diagram using the formula ADF=A*T(AFF)+B.

[0130] In one implementation, the first affine transformation map focuses on adjusting the horizontal / vertical relationship of local objects, which may result in local optimization but overall inconsistency. The adjustment of global affine parameters, on the other hand, coordinates the overall process from a macro perspective. The combination of the two forms a closed loop of "precise local adjustment and coordinated global optimization", ensuring that the image maintains consistency and rationality in both local details and global structure.

[0131] In one implementation, sentence-level vectors contain rich information about global semantics, and the global affine parameters generated based on this can adapt to the characteristics of different scenarios. Compared with fixed global adjustment rules, this semantically driven parameter generation method allows the model to automatically optimize and adjust its strategy according to different global needs, thereby improving its adaptability to diverse scenarios.

[0132] In one embodiment, after verifying the matching degree between the initial fused change map and the horizontal correlation condition, vertical correlation condition, and global correlation condition by a discriminator, the method further includes:

[0133] If the matching degree is less than or equal to the preset value, the scaling factor and offset of each object are adjusted through backpropagation to re-optimize the first radial change map.

[0134] In one implementation, if the matching degree is less than or equal to a preset value, the backpropagation mechanism can trace back the "error signal" of the matching degree (i.e., the degree of deviation between the image and the conditions) along the parameter generation path (multilayer perceptron) to accurately locate which objects' scaling factors or offsets need to be corrected; by adjusting the parameters in a targeted manner and regenerating the first radiometric transformation map, a closed loop of "generation, verification, correction, and regeneration" is formed, gradually reducing the deviation between the image and the conditions until the matching degree reaches the standard, significantly improving the accuracy of the final result.

[0135] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. An AI-based picture generation and optimization method, characterized in that, The method comprises: obtaining a requirement document, performing character recognition and semantic analysis on the requirement document to obtain key phrases, and substituting the key phrases into a full connection layer network to obtain a phrase vector representation; the key phrases include a subject phrase, an object phrase and a relationship phrase; the phrase vector representation includes a subject representation, an object representation and a relationship representation; substituting the phrase vector representation into a long short-term memory network to obtain a target relationship prompt; substituting the target relationship prompt into a preset model to obtain an initial image; substituting the target relationship prompt into a horizontal LSTM network and a vertical LSTM network to obtain a horizontal correlation condition and a vertical correlation condition, and substituting the requirement document into a CLIP model to obtain a global correlation condition; performing enhancement processing on the initial image according to the horizontal correlation condition, the vertical correlation condition and the global correlation condition to obtain a target requirement image; substituting the target relationship prompt into a preset model to obtain an initial image comprises: initializing Gaussian noise, substituting the Gaussian noise into a full connection layer for dimension adjustment to obtain a target Gaussian noise; performing feature splicing on the target Gaussian noise and the target relationship prompt to obtain an initial fusion feature; substituting the initial fusion feature into an image generation module to obtain a first image; performing global maximum pooling and global average pooling on the first image to obtain a global maximum feature and a global average feature; splicing the global maximum feature and the global average feature along the channel dimension to obtain a global fusion feature; performing 1x1 convolution operation on the global fusion feature to obtain an initial image; performing enhancement processing on the initial image according to the horizontal correlation condition, the vertical correlation condition and the global correlation condition to obtain a target requirement image comprises: performing word-level radiation change on the initial image according to the horizontal correlation condition and the vertical correlation condition to obtain a first radiation change image; the global correlation condition performs global radiation change on the first radiation change image to obtain a second radiation change image; fusing the initial image and the second radiation change image to obtain an initial fusion change image; verifying the matching degree of the initial fusion change image with the horizontal correlation condition, the vertical correlation condition and the global correlation condition through a discriminator, and if the matching degree is greater than a preset threshold, recording the initial fusion change image as a target requirement image.

2. The AI-based picture generation and optimization method of claim 1, wherein, Performing character recognition and semantic analysis on the requirement document to obtain key phrases comprises: preprocessing the requirement document to obtain an initial manuscript, and performing word segmentation processing on the initial manuscript to obtain a short word set; removing stop words in the short word set to obtain an effective short word set; processing the effective short word set through syntax analysis and semantic role labeling to obtain a subject phrase, an object phrase and a relationship phrase; the subject phrase, the object phrase and the relationship phrase are key phrases.

3. The AI-based picture generation and optimization method of claim 1, wherein, Substituting the initial fusion feature into an image generation module to obtain a first image comprises: performing convolution operation on the initial fusion feature after upsampling to obtain a first feature, and substituting the initial fusion feature and the first feature into an improved text fusion layer to obtain a second feature; The second feature is convoluted to obtain a third feature, and the third feature and the initial fusion feature are substituted into the improved text fusion layer to obtain a first image.

4. The AI-based picture generation and optimization method of claim 3, wherein, The improved text fusion layer comprises two improved affine transformation layers and two activation layers stacked alternately followed by a convolution layer. The working principle of the improved text fusion layer specifically comprises: The first input feature and the initial fusion feature are obtained, and the initial fusion feature is substituted into the improved affine transformation layer to obtain a first affine output. The first input feature is adjusted according to the first affine output to obtain a first affine feature; the first input feature is the first feature or the third feature. The first affine feature is substituted into the activation layer to obtain a first activated feature. The first activated feature is substituted into the improved affine transformation layer to obtain a second affine output, and the first activated feature is adjusted according to the second affine output to obtain a second affine feature. The second affine feature is substituted into the activation layer and then subjected to 1×1 convolution to obtain a first image.

5. The AI-based picture generation and optimization method of claim 4, wherein, The improved affine transformation layer comprises: An input feature is obtained, and scaling parameters and translation parameters are learned from the input feature through two multi-layer perceptrons; The input feature is adjusted according to the scaling parameters and the translation parameters to obtain an output feature.

6. The AI-based picture generation and optimization method of claim 1, wherein, The initial image is subjected to word-level radial variation according to the horizontal correlation condition and the vertical correlation condition to obtain a first radial variation image, which comprises: Objects are determined according to the horizontal correlation condition and the vertical correlation condition, and scaling factors and offsets of each object are generated through a first multi-layer perceptron and a second multi-layer perceptron; The dynamic weight corresponding to each object is determined through the first multi-layer perceptron and Softmax; The objects in the initial image are adjusted according to the scaling factors, offsets and dynamic weights of each object to obtain a first radial variation image.

7. The AI-based picture generation and optimization method of claim 1, wherein, The first radial variation image is subjected to global radial variation according to the global correlation condition to obtain a second radial variation image, which comprises: Global affine parameters are obtained by extracting sentence-level vectors in the global correlation condition; The first radial variation image is adjusted according to the global affine parameters to obtain a second radial variation image.

8. The AI-based picture generation and optimization method of claim 6, wherein, After verifying the matching degree of the initial fusion variation image with the horizontal correlation condition, the vertical correlation condition and the global correlation condition through the discriminator, the method further comprises: If the matching degree is less than or equal to a preset value, the scaling factors and offsets of each object are adjusted through back propagation to re-optimize the first radial variation image.

Images