An image generation method and system based on style feature injection

Abstract

The application belongs to the technical field of artificial intelligence image generation, and particularly relates to an image generation method and system based on style feature injection, which comprises the following steps: performing multi-dimensional feature analysis on a reference image specified by a user end through a pre-trained multi-level style extraction model, and extracting a style feature vector, wherein the style feature vector comprises a color feature vector, a texture feature vector, a composition feature vector and a lighting feature vector; obtaining a text description input by the user end, and outputting triple information comprising scene entities, entity attributes and spatial relationships; performing weighted fusion on the triple information and the style feature vector to obtain a style-enhanced semantic embedding vector; and inputting the style-enhanced semantic embedding vector into a generative adversarial network, performing splicing, and outputting an image. The application has the effect of significantly improving the similarity between the generated image and the reference style.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence image generation, and particularly relates to an image generation method and system based on style feature injection. BACKGROUND

[0002] In the prior art, the following problems exist: 1. Insufficient cross-modal mapping: the alignment of text semantics and image style in the feature space is insufficient, resulting in a mismatch between the generated image content and the style; 2. Coarse style control granularity: traditional methods only adjust the style through weight distribution or pixel fusion, and cannot achieve fine-grained control of multiple levels (such as color, texture, composition); 3. Low generation robustness: lacking an iterative optimization mechanism, it is difficult to meet the generation requirements of complex scenes.

[0003] Therefore, improvements are needed. SUMMARY

[0004] To solve the above technical problems, the application provides an image generation method and system based on style feature injection.

[0005] The application aims to achieve the following technical solutions:

[0006] An image generation method based on style feature injection, comprising:

[0007] extracting a style feature vector by performing multi-dimensional feature analysis on a reference image specified by a user terminal through a pre-trained multi-level style extraction model, wherein the style feature vector includes a color feature vector, a texture feature vector, a composition feature vector, and a lighting feature vector;

[0008] obtaining a text description input by the user terminal and outputting triple information including scene entities, entity attributes, and spatial relationships;

[0009] performing weighted fusion on the triple information and the style feature vector to obtain a style-enhanced semantic embedding vector;

[0010] inputting the style-enhanced semantic embedding vector into a generative adversarial network, performing splicing, and outputting an image.

[0011] In a preferred embodiment, the step of extracting a style feature vector by performing multi-dimensional feature analysis on a reference image specified by a user terminal through a pre-trained multi-level style extraction model, wherein the style feature vector includes a color feature vector, a texture feature vector, a composition feature vector, and a lighting feature vector, comprises:

[0012] The pre-trained multi-level style extraction model includes a color feature extraction layer, a texture feature extraction layer, a composition feature extraction layer, and a lighting feature extraction layer.

[0013] The color feature extraction layer includes:

[0014] Color space conversion is performed on the reference image, from RGB color space to HSV color space or Lab color space;

[0015] A pixel distribution histogram is constructed to count the pixel distribution of each color channel in the image;

[0016] Based on the pixel distribution histogram, color feature parameters are calculated, including the dominant color tone, color contrast, and color gradient rule;

[0017] The color feature parameters are encoded into a color feature vector, and the color feature vector is outputted;

[0018] The texture feature extraction layer includes:

[0019] A Gabor filter bank is designed for the reference image, and the Gabor filter bank includes multiple Gabor filters with different directions and scales;

[0020] The reference image is convolved with the Gabor filter bank to obtain multiple directional texture response maps;

[0021] Post-processing is performed on the texture response map, including normalization and binarization operations;

[0022] The statistical characteristics of the texture response map are counted, and texture feature parameters are outputted, including the mean, variance, and energy;

[0023] The texture feature parameters are encoded into a texture feature vector, and the texture feature vector is outputted.

[0024] In a preferred embodiment, the step of performing multi-dimensional feature analysis on the reference image specified by the user terminal through the pre-trained multi-level style extraction model to extract a style feature vector, including a color feature vector, a texture feature vector, a composition feature vector, and a lighting feature vector, further includes:

[0025] The composition feature extraction layer includes:

[0026] A deep learning segmentation network is used to perform semantic segmentation on the reference image to obtain each subject region in the picture;

[0027] Morphological processing is performed on the subject region, including erosion, dilation, opening operation, and closing operation;

[0028] The position information of the subject region is calculated, including the center coordinates, the bounding box, and the relative position relationship between regions;

[0029] Based on the position information of the subject area, a visual guide line and a balance feature in the picture are identified, the visual guide line including diagonal lines, trisection lines, and the balance feature including symmetry and barycentric position;

[0030] The position information, the visual guide line and the balance feature are encoded into a composition feature vector, and the composition feature vector is outputted;

[0031] The light feature extraction layer includes:

[0032] A reference image is analyzed for light by using an ambient light estimation model to estimate light source direction, light source intensity and light source type;

[0033] A shadow area of the reference image is detected, including the position, shape and projection direction of the shadow;

[0034] A highlight area of the reference image is identified, and the brightness value and distribution range of the highlight area are calculated;

[0035] The light source direction, the light source intensity, the shadow area and the highlight area are encoded into a light feature vector, and the light feature vector is outputted.

[0036] In a preferred embodiment, the step of obtaining a text description input by a user terminal and outputting triple information including scene entities, entity attributes and spatial relationships includes:

[0037] The text description is preprocessed, including removing stop words, punctuation marks, and performing word segmentation and part-of-speech tagging;

[0038] Based on natural language processing technology, the preprocessed text description is parsed, including:

[0039] Scene entities including characters, objects and background elements are identified and extracted;

[0040] Entity attributes of each scene entity are extracted, including color, shape, size and material;

[0041] Spatial relationships between scene entities are identified and extracted, including position relationship, direction relationship and distance relationship;

[0042] The extracted scene entities, entity attributes and spatial relationships are structured according to a preset format to generate triple information.

[0043] In a preferred embodiment, the step of weighting and fusing the triple information and the style feature vector to obtain a style-enhanced semantic embedding vector includes:

[0044] inputting the triple information into a graph neural network, generating a semantic vector E of dimension d through node embedding and relation aggregation semantic ;

[0045] mapping the style feature vector to a d-dimensional space through a fully connected layer, and obtaining a style vector E after splicing style ;

[0046] calculating cross-attention weights between the semantic vector and the style vector:

[0047] wherein W Q ,W K are learnable parameter matrices

[0048] weighting the style vector based on the cross-attention weights: E style\_attn =A·E style .

[0049] introducing a learnable dynamic weight parameter α, adjusting the contribution ratio of the style vector and the semantic vector through a gating mechanism: α=σ(W g [E semantic ;E style ]), wherein σ is a Sigmoid function, and W g is a gating weight matrix

[0050] generating a style-enhanced semantic embedding vector through weighted fusion: E fused =α·E style\_attn +(1-α)·E semantic .

[0051] In a preferred embodiment, the step of inputting the style-enhanced semantic embedding vector into a generative adversarial network, splicing, and outputting an image comprises:

[0052] the generative adversarial network comprises a generator

[0053] the generator comprises an initial convolutional layer, a residual block sequence, an up-sampling layer, and an output layer

[0054] inputting the style-enhanced semantic embedding vector into the initial convolutional layer to generate a low-resolution feature map, wherein the low-resolution feature map contains high-frequency details, and the high-frequency details include edge features, local textures, and noise patterns

[0055] inserting AdaIN after the initial convolutional layer to perform style modulation on the low-resolution feature map and output a shallow feature map

[0056] In a preferred embodiment, the step of inputting the style-enhanced semantic embedding vector into a generative adversarial network, splicing, and outputting an image further comprises:

[0057] inputting the shallow feature map into the residual block sequence, improving resolution through an upsampling layer, and outputting a high-resolution feature map, the high-resolution feature map containing global semantics, the global semantics including object contours, spatial layouts, and illumination distributions;

[0058] inserting an AdaIN layer after each upsampling layer, performing style re-injection on the high-resolution feature map, and outputting a deep feature map;

[0059] upsampling the shallow feature map to a target resolution through transpose convolution, and concatenating the deep feature map in the channel dimension to generate a fusion feature map;

[0060] performing channel compression on the fusion feature map, and outputting an RGB image.

[0061] The second application purpose is achieved through the following technical solutions:

[0062] An image generation system based on style feature injection, comprising:

[0063] an extraction module: a pre-trained multi-level style extraction model is used to perform multi-dimensional feature analysis on a reference image specified by a user terminal, and a style feature vector is extracted, the style feature vector including a color feature vector, a texture feature vector, a composition feature vector, and an illumination feature vector;

[0064] an acquisition module: a text description input by a user terminal is acquired, and triple information including scene entities, entity attributes, and spatial relationships is output;

[0065] a fusion module: the triple information and the style feature vector are weighted and fused to obtain a style-enhanced semantic embedding vector;

[0066] a concatenation module: the style-enhanced semantic embedding vector is input into a generative adversarial network for concatenation, and an image is output.

[0067] The third application purpose is achieved through the following technical solutions:

[0068] A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described image generation method based on style feature injection.

[0069] The fourth application purpose is achieved through the following technical solutions:

[0070] A computer readable storage medium stores a computer program, the computer program is executed by a processor to implement the steps of the image generation method based on style feature injection.

[0071] In summary, the present application includes at least one of the following beneficial technical effects:

[0072] By decoupling feature analysis of the reference image through the multi-level style extraction model, style vectors are extracted from four dimensions of color, texture, composition and lighting to realize fine-grained style control; combined with the structured triplets generated by the text analysis module, the style feature vectors and the triplet information are weighted and fused to obtain a style-enhanced semantic embedding vector; the style-enhanced semantic embedding vector is input into the generative adversarial network, the generative adversarial network adopts a multi-scale residual architecture, and style parameters are injected through a deformable AdaIN module between multiple resolution levels to generate shallow feature maps and deep feature maps, and then the shallow feature maps and the deep feature maps are spliced to output an RGB image. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 is an implementation flowchart of an embodiment of the image generation method based on style feature injection of the present application;

[0074] Figure 2 is an implementation flowchart of step S20 in an embodiment of the image generation method based on style feature injection of the present application;

[0075] Figure 3 is an implementation flowchart of step S40 in an embodiment of the image generation method based on style feature injection of the present application;

[0076] Figure 4 is another implementation flowchart of step S40 in an embodiment of the image generation method based on style feature injection of the present application;

[0077] Figure 5 is a principle block diagram of a computer device of the present application. DETAILED DESCRIPTION

[0078] The following will be described in detail below Figures 1-5 The present application will be further described in detail.

[0079] In an embodiment, as shown in Figure 1 The present application discloses an image generation method based on style feature injection, specifically including the following steps:

[0080] S10: extracting a style feature vector from a reference image specified by a user terminal by using a pre-trained multi-level style extraction model, the style feature vector including a color feature vector, a texture feature vector, a composition feature vector, and a lighting feature vector;

[0081] S20: obtaining text description input by the user terminal and outputting triple information including scene entities, entity attributes, and spatial relationships;

[0082] S30: performing weighted fusion of the triple information and the style feature vector to obtain a style-enhanced semantic embedding vector;

[0083] S40: inputting the style-enhanced semantic embedding vector into a generative adversarial network, performing splicing, and outputting an image.

[0084] In this embodiment, the reference image is subjected to decoupled feature analysis by using a multi-level style extraction model to extract a style vector from four dimensions of color, texture, composition, and lighting, thereby achieving fine-grained style control. The structured triple generated by the text analysis module is combined with the style feature vector to perform weighted fusion, thereby obtaining a style-enhanced semantic embedding vector. The style-enhanced semantic embedding vector is input into a generative adversarial network. The generative adversarial network adopts a multi-scale residual architecture, injects style parameters between multiple resolution levels through a deformable AdaIN module, generates shallow feature maps and deep feature maps, splices the shallow feature maps and the deep feature maps, and outputs an RGB image.

[0085] S10 includes:

[0086] SB1: The pre-trained multi-level style extraction model includes a color feature extraction layer, a texture feature extraction layer, a composition feature extraction layer, and a lighting feature extraction layer.

[0087] SB2: The color feature extraction layer includes:

[0088] The reference image is subjected to color space conversion from an RGB color space to an HSV color space or a Lab color space.

[0089] SB3: A pixel distribution histogram is constructed to count pixel distribution of each color channel in the image.

[0090] SB4: Based on the pixel distribution histogram, color feature parameters are calculated, the color feature parameters including a dominant color, a color contrast, and a color gradient rule.

[0091] SB5: The color feature parameters are encoded into a color feature vector, and the color feature vector is output.

[0092] SB6: The texture feature extraction layer includes:

[0093] designing a Gabor filter bank for the reference image, the Gabor filter bank comprising a plurality of Gabor filters of different directions and different scales;

[0094] SB7: performing convolution operation on the reference image and the Gabor filter bank to obtain a plurality of directional texture response maps;

[0095] SB8: performing post-processing on the texture response maps, the post-processing comprising normalization and binarization operation;

[0096] SB9: calculating statistical properties of the texture response maps to output texture feature parameters, the texture feature parameters comprising mean, variance and energy;

[0097] SB10: encoding the texture feature parameters into a texture feature vector and outputting the texture feature vector.

[0098] In the embodiment, the reference image is analyzed for fine-grained features by a pre-trained multi-level style extraction model. The color feature extraction layer (SB1-SB5) converts the image from RGB space to Lab / HSV space, constructs a pixel distribution histogram to quantify the dominant color tone, color contrast and gradient rule, encodes into a color feature vector, and realizes accurate capture of color style. The texture feature extraction layer (SB6-SB10) generates a texture response map through a multi-directional and multi-scale Gabor filter bank convolution, encodes into a texture feature vector after normalization and statistical property (mean, variance, energy) analysis, and effectively represents the complexity and directionality of local texture.

[0099] The S10 step further comprises:

[0100] SH1: the composition feature extraction layer:

[0101] adopting a deep learning segmentation network to perform semantic segmentation on the reference image to obtain each subject region in the picture;

[0102] SH2: performing morphological processing on the subject region, the morphological processing comprising erosion, dilation, opening operation and closing operation;

[0103] SH3: calculating position information of the subject region, the position information comprising center coordinates, a bounding box, and relative position relationships between regions;

[0104] SH4: identifying visual guide lines and balance features in the picture based on the position information of the subject region, the visual guide lines comprising diagonal lines and trisection lines, and the balance features comprising symmetry and barycenter position;

[0105] SH5: encode the position information, the visual guide line and the balance feature into a composition feature vector, and output the composition feature vector;

[0106] SH6: the illumination feature extraction layer:

[0107] adopting an ambient light estimation model to perform illumination analysis on the reference image, estimating the light source direction, the light source intensity and the light source type;

[0108] SH7: detecting a shadow area of the reference image, the shadow area including the position, shape and projection direction of the shadow;

[0109] SH8: identifying a highlight area of the reference image, and calculating the brightness value and distribution range of the highlight area;

[0110] SH9: encode the light source direction, the light source intensity, the shadow area and the highlight area into an illumination feature vector, and output the illumination feature vector.

[0111] In this embodiment, the reference image is structurally analyzed by a pre-trained multi-level style extraction model. In the composition feature extraction layer, a deep learning segmentation network is used to perform semantic segmentation on the main body area of the picture, and morphological processing (such as erosion and dilation) is used to optimize the region boundary. By calculating the center coordinates, boundary box and relative position relationship of the main body area, visual guide lines (such as trisection lines and diagonal lines) and balance features (such as symmetry index and barycentric offset) are identified and encoded into a composition feature vector to accurately represent the spatial layout rules of the picture. In the illumination feature extraction layer, the direction and intensity of the light source are analyzed based on the ambient light estimation model. By geometric shape detection of the shadow area and brightness distribution calculation of the highlight area, parameters such as shadow projection direction and highlight coverage range are extracted and encoded into an illumination feature vector to dynamically restore the real illumination scene.

[0112] Figure 2 , the S20 step, comprising:

[0113] S201: preprocessing the text description, the preprocessing including removing stop words, punctuation, and performing word segmentation and part-of-speech tagging;

[0114] S202: based on natural language processing technology, analyzing the preprocessed text description, the analysis including:

[0115] S203: identifying and extracting scene entities, the scene entities including characters, objects and background elements;

[0116] S204: extracting entity attributes of each scene entity, the entity attributes including color, shape, size and material;

[0117] S205: Identify and extract the spatial relationship between the scene entities, including the position relationship, direction relationship, and distance relationship.

[0118] S206: Structure the extracted scene entities, entity attributes, and spatial relationships in a preset format to generate triple information.

[0119] In this embodiment, first, the input text is preprocessed (S201) to remove stop words and punctuation and perform word segmentation and part-of-speech tagging to provide standardized input for semantic analysis; based on dependency syntax analysis and named entity recognition technology, scene entities (such as characters, objects, and background elements) and their attributes (including color, shape, material, and other physical characteristics) are accurately extracted from the text, and the relative position, direction, and distance relationships between entities are identified through a spatial relationship reasoning model (such as a position resolver based on rules or deep learning) (S202-S205), and finally structured triple information is generated (S206). This process converts free text into a machine-readable semantic framework, effectively eliminating the ambiguity of natural language descriptions by explicitly defining entity attributes and spatial constraints, providing a precise semantic control basis for image generation. After combining the style feature vector, the generation model can accurately restore the visual attributes and spatial layout of scene entities based on structured triples, ensuring that the generated image is highly matched with the text description in terms of object details (such as material texture), spatial logic (such as relative position consistency), and global composition (such as entity proportion relationship), significantly improving the content accuracy and scene rationality of the generated results.

[0120] S30 step, including:

[0121] S301: Input the triple information into the graph neural network to generate a semantic vector E of dimension d through node embedding and relationship aggregation semantic ;

[0122] S302: Map the style feature vector to a d-dimensional space through a fully connected layer, and concatenate to obtain a style vector E style ;

[0123] S303: Calculate the cross-attention weight of the semantic vector and the style vector:

[0124] where W Q ,W K are learnable parameter matrices;

[0125] S304: Based on the cross-attention weight, weight the style vector: E style\_attn = A·E style ;

[0126] S305: Introducing a learnable dynamic weight parameter a to adjust the contribution ratio of style vector and semantic vector through a gating mechanism: a = σ(W g [EQ. 1] semantic ; EQ. 2 style ), where σ is the Sigmoid function, W g is the gating weight matrix.

[0127] S306: Weighted fusion to generate style-enhanced semantic embedding vector: E fused = a · E style\_attn + (1-a) · E semantic .

[0128] In this embodiment, the efficient alignment of semantic and style features is achieved through a cross-modal fusion mechanism (S30). First, the structured triplets are input into the graph neural network to capture entity semantics through node embedding, and dimensionally aligned semantic vectors are generated based on the relationship aggregation layer to integrate spatial constraints. At the same time, the multi-dimensional style features (color, texture, composition, lighting) are mapped to a unified latent space through a fully connected layer to form a style vector. The cross-attention mechanism is used to calculate the correlation weight between the semantic vector and the style vector, dynamically identify the key style elements (such as the "material" entity preferentially associated with the texture feature), and weight the style vector. Further, a learnable gating weight parameter is introduced to adaptively adjust the fusion ratio of style and semantics using the Sigmoid function (e.g., increasing the style weight to enhance artistic expressiveness in complex scenes). The final style-enhanced semantic embedding vector not only retains the precise semantic constraints of the text description (such as reducing the object position error to pixel-level accuracy), but also deeply integrates the style characteristics of the reference image, enabling the generative adversarial network to output images with content accuracy, style consistency, and visual naturalness.

[0129] Figure 3 , S40 steps, including:

[0130] S401: The generative adversarial network includes a generator;

[0131] S402: The generator includes an initial convolutional layer, a residual block sequence, an upsampling layer, and an output layer;

[0132] S403: Input the style-enhanced semantic embedding vector into the initial convolutional layer to generate a low-resolution feature map containing high-frequency details, including edge features, local textures, and noise patterns;

[0133] S404: Insert AdaIN after the initial convolutional layer to perform style modulation on the low-resolution feature map and output a shallow feature map.

[0134] In this embodiment, the multi-level synthesis of images is achieved by the generator of the generative adversarial network (S401-S402), the core of which is to combine style modulation and cross-level feature fusion. The style-enhanced semantic embedding vector is first input into the initial convolutional layer to generate a low-resolution feature map (S403), which captures high-frequency details such as edge sharpness, local texture, and noise disturbance, and dynamically adjusts the feature distribution through the AdaIN layer (S404) to align the shallow features with the reference style.

[0135] Figure 4 , the S40 step further comprises:

[0136] S405: inputting the shallow feature map into the residual block sequence, improving the resolution through the up-sampling layer, and outputting a high-resolution feature map, the high-resolution feature map containing global semantics, the global semantics including object contour, spatial layout, and light distribution;

[0137] S406: inserting an AdaIN layer after each up-sampling layer to perform style re-injection on the high-resolution feature map and output a deep feature map;

[0138] S407: up-sampling the shallow feature map to the target resolution through transposed convolution, and concatenating the deep feature map in the channel dimension to generate a fusion feature map;

[0139] S408: compressing the fusion feature map in the channel dimension to output an RGB image.

[0140] In this embodiment, the shallow features are up-sampled through the residual block sequence (S405), and an AdaIN layer is inserted after each level to perform style re-injection on the high-resolution feature map (S406), gradually building global semantics such as geometric precision of object contour, topological relationship of spatial layout, and natural transition of light distribution. Finally, the shallow features are up-sampled to the target resolution through transposed convolution, and concatenated with the deep features in the channel dimension (S407) to fuse high-frequency details such as fine depiction of fabric texture with global structure such as reasonable arrangement of scene objects, and then compressed in the channel dimension to output an RGB image (S408). This method combines multi-level style modulation and cross-scale feature fusion to preserve local authenticity while ensuring global style consistency, significantly improving the problem of blurred details and fragmented styles in traditional methods, generating images that visually have both delicate texture and overall harmony, and is suitable for high-precision artistic creation and scene modeling needs.

[0141] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0142] In an embodiment, a style feature injection based image generation system is provided, which corresponds to the style feature injection based image generation method in the above embodiments. The style feature injection based image generation system comprises:

[0143] The specific limitation of the style feature injection based image generation system can refer to the limitation of the style feature injection based image generation method in the above, which will not be repeated here. Each module in the above style feature injection based image generation system can be realized by software, hardware and combination thereof in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operation corresponding to each module by the processor.

[0144] In an embodiment, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in Figure 5 The computer device comprises a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store shallow feature maps and deep feature maps. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement a style feature injection based image generation method.

[0145] In an embodiment, a computer device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements a style feature injection based image generation method when executing the computer program.

[0146] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program implements a style feature injection based image generation method when executed by a processor.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. An image generation method based on style feature injection, characterized in that, include: By using a pre-trained multi-level style extraction model, multi-dimensional feature analysis is performed on the reference image specified by the user to extract style feature vectors, which include color feature vectors, texture feature vectors, composition feature vectors, and lighting feature vectors. Obtain the text description input by the user and output a triplet of information including scene entities, entity attributes, and spatial relationships; The triplet information and the style feature vector are weighted and fused to obtain a style-enhanced semantic embedding vector; The triplet information is input into a graph neural network, and through node embedding and relation aggregation, a dimension of [dimensionality missing] is generated. semantic vectors ; The style feature vector is mapped to... through a fully connected layer. The style vector is obtained by concatenating the 3D space. ; Calculate the cross-attention weights between the semantic vector and the style vector: ,in, The learnable parameter matrix; The style vector is weighted based on the cross-attention weights: ; Introduce learnable dynamic weight parameters The contribution ratio of style vectors to semantic vectors is adjusted through a gating mechanism: ,in, For the Sigmoid function, This is the gate weight matrix; Weighted fusion generates style-enhanced semantic embedding vectors: ; The style-enhanced semantic embedding vector is input into a generative adversarial network to generate shallow and deep feature maps. The shallow and deep feature maps are then concatenated to output an image.

2. The image generation method based on style feature injection according to claim 1, characterized in that, The step involves using a pre-trained multi-level style extraction model to perform multi-dimensional feature analysis on a reference image specified by the user, extracting style feature vectors. These style feature vectors include color feature vectors, texture feature vectors, composition feature vectors, and illumination feature vectors. include: The pre-trained multi-level style extraction model includes a color feature extraction layer, a texture feature extraction layer, a composition feature extraction layer, and a lighting feature extraction layer. The color feature extraction layer: The reference image is converted to a color space from RGB color space to HSV color space or Lab color space. Construct a pixel distribution histogram to statistically analyze the pixel distribution of each color channel in the image; Based on the pixel distribution histogram, color feature parameters are calculated, including the dominant hue, color contrast, and color gradation pattern. The color feature parameters are encoded into a color feature vector, and the color feature vector is output. The texture feature extraction layer: A Gabor filter bank is designed for the reference image, the Gabor filter bank comprising multiple Gabor filters of different directions and scales; The reference image is convolved with a Gabor filter bank to obtain texture response maps in multiple directions; Post-processing is performed on the texture response map, including normalization and binarization operations; The statistical properties of the statistical texture response map are analyzed, and the texture feature parameters are output, including mean, variance, and energy. The texture feature parameters are encoded into a texture feature vector, and the texture feature vector is output.

3. The image generation method based on style feature injection according to claim 2, characterized in that, The step of using a pre-trained multi-level style extraction model to perform multi-dimensional feature analysis on a reference image specified by the user and extract style feature vectors, wherein the style feature vectors include color feature vectors, texture feature vectors, composition feature vectors, and illumination feature vectors, further includes: The composition feature extraction layer: A deep learning segmentation network is used to perform semantic segmentation on the reference image to obtain the various main regions in the image; Morphological processing is performed on the main region, including erosion, dilation, opening operation, and closing operation; Calculate the location information of the main region, which includes the center coordinates, bounding box, and relative positional relationships between regions; Based on the positional information of the main area, visual guide lines and balance features in the image are identified. The visual guide lines include diagonals and third lines, and the balance features include symmetry and center of gravity position. The location information, visual guide lines, and balance features are encoded into a composition feature vector, and the composition feature vector is output. The illumination feature extraction layer: An ambient light estimation model is used to perform illumination analysis on the reference image to estimate the direction, intensity, and type of the light source. Detect the shadow region of the reference image, wherein the shadow region includes the position, shape and projection direction of the shadow; Identify the highlight regions of the reference image and calculate the brightness value and distribution range of the highlight regions; The light source direction, light source intensity, shadow area, and highlight area are encoded into a lighting feature vector, and the lighting feature vector is output.

4. The image generation method based on style feature injection according to claim 1, characterized in that, The step of obtaining the text description input by the user and outputting a triplet of information including scene entities, entity attributes, and spatial relationships includes: The text description is preprocessed, including removing stop words and punctuation marks, and performing word segmentation and part-of-speech tagging. Based on natural language processing technology, the preprocessed text description is parsed, and the parsing includes: Identify and extract scene entities, including people, objects, and background elements; Extract the entity attributes of each scene entity, including color, shape, size, and material; Identify and extract spatial relationships between scene entities, including positional relationships, directional relationships, and distance relationships; The extracted scene entities, entity attributes, and spatial relationships are structured and represented according to a preset format to generate triplet information.

5. The image generation method based on style feature injection according to claim 1, characterized in that, The step of inputting the style-enhanced semantic embedding vector into a generative adversarial network, concatenating them, and outputting an image includes: The generative adversarial network includes a generator; The generator includes an initial convolutional layer, a residual block sequence, an upsampling layer, and an output layer; The style-enhanced semantic embedding vector is input into the initial convolutional layer to generate a low-resolution feature map, which contains high-frequency details, including edge features, local textures, and noise patterns. AdaIN is inserted after the initial convolutional layer to perform style modulation on the low-resolution feature map and output a shallow feature map.

6. The image generation method based on style feature injection according to claim 5, characterized in that, The step of inputting the style-enhanced semantic embedding vector into a generative adversarial network, concatenating them, and outputting an image further includes: The shallow feature map is input into the residual block sequence. The resolution is improved by an upsampling layer, and a high-resolution feature map is output. The high-resolution feature map contains global semantics, which includes object contours, spatial layout, and illumination distribution. An AdaIN layer is inserted after each upsampling layer to perform style re-injection on the high-resolution feature map and output a deep feature map. The shallow feature map is upsampled to the target resolution through transposed convolution and then concatenated with the deep feature map along the channel dimension to generate a fused feature map. The fused feature map is then compressed to output an RGB image.

7. An image generation system based on style feature injection, characterized in that, include: Extraction module: Through a pre-trained multi-level style extraction model, multi-dimensional feature analysis is performed on the reference image specified by the user to extract style feature vectors, which include color feature vectors, texture feature vectors, composition feature vectors, and lighting feature vectors. Acquisition module: Acquires the text description input by the user and outputs a triplet of information including scene entities, entity attributes, and spatial relationships; Fusion module: weightedly fuses the triplet information and the style feature vector to obtain a style-enhanced semantic embedding vector; First module: Input the triplet information into a graph neural network, and generate a dimension of [dimensionality missing] through node embedding and relation aggregation. semantic vectors ; The second module: Maps the style feature vector through a fully connected layer to... The style vector is obtained by concatenating the 3D space. ; Module 3: Calculate the cross-attention weights between the semantic vector and the style vector: ,in, The learnable parameter matrix; Fourth module: Based on the cross-attention weights, the style vector is weighted: ; Module 5: Introducing Learnable Dynamic Weight Parameters The contribution ratio of style vectors to semantic vectors is adjusted through a gating mechanism: ,in, For the Sigmoid function, This is the gate weight matrix; Module 6: Weighted fusion to generate style-enhanced semantic embedding vectors: ; The concatenation module inputs the style-enhanced semantic embedding vector into the generative adversarial network to generate shallow feature maps and deep feature maps. The shallow feature maps and deep feature maps are then concatenated to output an image.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image generation method based on style feature injection as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image generation method based on style feature injection as described in any one of claims 1-6.

Images