Artistic effect generation method based on image feature migration

Abstract

The application discloses an artistic effect generation method based on image feature migration and relates to the technical field of image processing. The method comprises the following steps: obtaining a to-be-processed image and a target style image, dividing the to-be-processed image into multiple image units and extracting content features, simultaneously extracting local features and texture main directions of the style image and mapping the local features and the texture main directions to the image units to obtain local style features and style original brush stroke directions; calculating a spatial level probability based on the content features and extracting narrative information representing visual flow and focus; combining the narrative information and the spatial level probability to calculate a conflict degree and construct a composite role embedding vector, generating a control vector through control mapping and gate constraint; fusing a structure direction, a style brush stroke direction and the narrative information to obtain a brush stroke direction, and performing style transformation and reconstruction on the content features according to the brush stroke direction to output a generated image. The application realizes unified representation of spatial positions and visual importance of image regions, and enables the style migration process to have differentiated regulation and control capabilities.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for generating artistic effects based on image feature transfer. Background Technology

[0002] With the development of computer vision and image processing technologies, image style transfer technology has been widely applied in fields such as digital art creation, image editing, and human-computer interaction. Existing technologies typically extract feature representations from images and utilize statistical matching, feature transformation, or deep neural networks to transfer style information to target images, thereby generating image results with a specific artistic style.

[0003] However, most existing image style transfer methods focus on matching global or local texture statistical features, such as style fusion based on global feature distribution (e.g., Gram matrix) or local feature block matching. These methods can achieve visual style transfer to some extent, but they usually ignore higher-level structural semantic information and visual expression rules in images, especially lacking effective modeling of spatial hierarchy and visual narrative structure.

[0004] Specifically, during style transfer, the lack of a unified modeling and collaborative constraint mechanism for image spatial hierarchy and visual narrative information leads to the application of style information in different regions exhibiting characteristics of indifference or weak constraint. On the one hand, important narrative areas (such as visual focal points) may be over-stylized, losing structural clarity and semantic discernibility; on the other hand, the overall spatial hierarchy of the image is weakened or even destroyed during stylization, resulting in a lack of hierarchy and visual guidance in the generated image, making it difficult to meet the comprehensive requirements of spatial depth and visual narrative in artistic expression. Furthermore, the conflicting relationships between different information (such as the conflict when a distant area serves as a visual focal point) further limit the ability of the generated image to balance artistic expressiveness and structural consistency. Summary of the Invention

[0005] To address the lack of a unified modeling and collaborative constraint mechanism for image spatial hierarchy and visual narrative information in existing technologies, this invention provides an artistic effect generation method based on image feature transfer.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: In a first aspect, this application discloses a method for generating artistic effects based on image feature transfer, comprising the following steps: Obtain the image to be processed and the target style image; The image to be processed is divided into multiple image units, and the content features of each image unit are extracted. At the same time, the local features and main texture direction of the target style image are extracted and mapped to each image unit to obtain the local style features and original brush stroke direction of the style of the image unit. Among them, the content features include at least the texture features and the structural direction generated by the image gradient. Spatial hierarchy probability and confidence are calculated based on content features, and narrative information is also calculated based on content features; the spatial hierarchy includes foreground, midground, and background layers; the narrative information represents the path, direction, and focus of the viewer's eye movement. Calculate the degree of conflict between narrative information and spatial hierarchy probability, and concatenate the narrative information and spatial hierarchy probability into a composite character embedding vector. The composite role embedding vector is used to generate a basic control vector through a preset control mapping function. The basic control vector is then gated and constrained based on confidence and conflict, and then bounded projection is performed to obtain the control vector. The stroke direction is obtained by integrating and optimizing the structural direction, the original stylistic stroke direction, and narrative information. After performing style transformation on content features based on local style features, control vectors, and stroke direction, and then reconstructing the image, the generated image is obtained and output.

[0007] Secondly, this application discloses an art effect generation system based on image feature transfer, which is applied to the aforementioned art effect generation method based on image feature transfer, including: The data acquisition module is used to acquire the image to be processed and the target style image; The feature extraction module is used to divide the image to be processed into multiple image units, extract the content features of each image unit, and extract the local features and main texture direction of the target style image, and map them to each image unit to obtain the local style features and original brush stroke direction of the style of the image unit; wherein, the content features include at least texture features and structural directions generated by image gradients; The feature calculation module is used to calculate the spatial hierarchy probability and confidence based on content features, and to calculate narrative information based on content features; the spatial hierarchy includes foreground, midground, and background layers; the narrative information represents the path, direction, and focus of the viewer's eye movement; The feature embedding module is used to calculate the degree of conflict between narrative information and spatial hierarchy probability, and concatenates the narrative information and spatial hierarchy probability into a composite character embedding vector. The feature mapping module is used to generate a basic control vector from the composite role embedding vector through a preset control mapping function. After gating and constraining the basic control vector based on confidence and conflict, a bounded projection is performed to obtain the control vector. The direction fusion module is used to fuse and optimize the structural direction, the original style of the brushstroke direction, and the narrative information to obtain the brushstroke direction; The image output module is used to perform style transformation and image reconstruction on content features based on local style features, control vectors and stroke direction, and then output the generated image.

[0008] Thirdly, this application discloses a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the aforementioned image feature transfer-based artistic effect generation method.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This application constructs a joint modeling mechanism of spatial hierarchy probability and narrative information, and introduces conflict degree for coupling analysis to achieve a unified representation of the spatial location and visual importance of image regions. This enables the style transfer process to have differentiated control capabilities based on spatial hierarchy and visual focus, thereby effectively maintaining the spatial hierarchy structure of the generated image and highlighting key narrative regions. 2. By mapping the composite role embedding vector to a control vector and performing gating modulation and bounded constraints based on confidence and conflict, adaptive adjustment of style intensity and structure preservation in different image regions is achieved, thereby avoiding over-stylization or structural distortion caused by unreliable or conflicting information and improving the stability and controllability of the style transfer process. Attached Figure Description

[0010] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein: Figure 1 This is a flowchart of the artistic effect generation method based on image feature transfer described in Embodiment 1 of the present invention; Figure 2 Based on Figure 1 The logical flowchart for generating control vectors; Figure 3 Based on Figure 1 The logic flowchart for generating the pen stroke direction; Figure 4 This is a block diagram of the image feature transfer-based artistic effect generation system described in Example 2; Figure 5 Based on Figure 4 Hardware block diagram. Detailed Implementation

[0011] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0012] In existing technologies, image style transfer methods mainly rely on techniques such as global statistical feature matching or local texture mapping to achieve style conversion through alignment of image feature distributions. However, these methods typically focus on style expression at the texture level, lacking effective modeling of the spatial hierarchy and visual narrative information of the image. This leads to a tendency for style application to be uniform across regions during the style transfer process, making it difficult to reflect the hierarchical differences between foreground, midground, and background, and easily causing the visual focus to be weakened or even lost. Furthermore, traditional methods lack the ability to characterize the potential conflict between spatial position and visual importance, and cannot adaptively adjust for special compositions (such as distant focal points), resulting in a difficulty in balancing structural preservation, visual guidance, and artistic expression in the generated results. Therefore, there is an urgent need for an artistic effect generation method that can simultaneously integrate spatial hierarchy information and visual narrative information, and achieve synergistic control between the two, to solve the problem of the lack of structural constraints and semantic guidance in the style transfer process of existing technologies.

[0013] To address the aforementioned issues, this research found that when processing image style generation, a fine-grained correspondence between content structure and style features is first established based on image unit segmentation and content feature extraction. Then, considering that simple feature mapping is insufficient to characterize image spatial relationships and visual expression patterns, spatial hierarchy probability and narrative information are introduced to model the spatial location and visual attention attributes of image regions. Furthermore, conflict degree is used to characterize the inconsistency between spatial location and visual focus. Subsequently, to avoid the unstable influence of the aforementioned multi-source information during style modulation, a gated modulation mechanism is constructed by combining the control mapping function with confidence and conflict degree. This mechanism constrains and optimizes the control vector, allowing the style intensity and structural preservation degree of different regions to be adaptively adjusted. Based on the synergistic effect of the aforementioned multi-layered information, a unified expression of content structure, spatial hierarchy, and visual narrative is achieved during style transformation and image reconstruction. This results in generated images that maintain structural consistency while possessing clearer spatial hierarchy relationships and natural visual flow effects, thereby comprehensively improving the performance capability and visual harmony of style transfer.

[0014] After introducing the basic concept of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0015] Example 1

[0016] like Figure 1 The image illustrates an artistic effect generation method based on image feature transfer, including the following steps: S101. Obtain the image to be processed and the target style image.

[0017] Specifically, the image to be processed is the content image. ,in , These represent the image's height and width, respectively, with the channels using the RGB color space. Target style image. ,in , These represent the height and width of the style image, respectively. The image to be processed and the target style image can be obtained through methods such as reading local files, network transmission, or input via a user interface.

[0018] After acquiring the image, it needs to be preprocessed to unify the content map and style map to a size suitable for network processing, such as a short side of 512 or 768, while maintaining the original aspect ratio. To improve the stability of depth cues, saliency, and structure tensors, mild bilateral filtering or guided filtering, contrast normalization, gradient magnitude pre-calculation, and saliency map pre-calculation can be performed first.

[0019] S102. Divide the image to be processed into multiple image units, extract the content features of each image unit, and extract the local features and main texture direction of the target style image, and map them to each image unit to obtain the local style features and original style brush direction of the image unit; wherein, the content features include at least the texture features and the structural direction generated by the image gradient.

[0020] Regarding image unit partitioning, the image to be processed is divided into segments according to superpixel segmentation or a regular grid method. Each image unit generates an image unit partitioning map. ,in The total number of image units. Represents pixels Belongs to the Image unit The goal is to transform pixel-level complex problems into "region-level graph optimization problems," reducing computational complexity and improving control stability. By dividing the image to be processed into multiple image units, the originally pixel-based high-dimensional unstable computation is transformed into region-level feature modeling. Due to the high consistency within image units, random noise and local anomalies can be suppressed statistically, thus enabling subsequent feature extraction and style mapping to be based on a more stable structure.

[0021] Based on this, an adjacency graph is constructed according to the spatial adjacency relationship between image units. , where vertex set edge set The adjacency weight matrix represents the connection relationship between adjacent image units. , Representing image unit and Similarity weights between them. Adjacency weights. It can be calculated using a Gaussian kernel function based on the difference in color mean and spatial distance between adjacent image units, specifically: ; in, and Image units and The color mean feature vector; and Image units and The geometric center coordinates; This refers to the color similarity bandwidth parameter. This is the spatial distance bandwidth parameter. If the image unit... and If they are not adjacent, then .

[0022] Regarding content feature extraction, for each image unit Extracting content features ,in This refers to the dimension of content features. Content features include at least texture features and structural orientations generated by image gradients, and may also include sharpness, contrast, saliency, edge orientation, scale cues, etc. Structural orientations can be given by the principal feature vector orientations of the local structure tensor within an image unit, denoted as […]. Specifically, for image units Calculate image gradient within pixels Construct the structure tensor ,in For pixels The image gradient vector at that location. Perform eigenvalue decomposition and take the eigenvector corresponding to the largest eigenvalue as the structural direction. By incorporating structural orientations generated from image gradients into content features, each image unit possesses explicit structural dominance information. Since structural orientations reflect edge direction and geometric shape, this information provides a constraint benchmark during style transfer, thus preventing style textures from destroying the original structure. This achieves a balance between style expression and structural preservation, allowing the generated image to maintain structural discernibility while remaining artistic.

[0023] Texture feature extraction specifically involves: performing multi-scale texture analysis on pixels within an image unit, extracting texture responses using filter banks (such as Gabor filters, Laplacian of Gaussian filters, or Local Binary Pattern (LBP)); statistically analyzing the filter responses at different directions and scales to obtain texture description vectors (such as mean, variance, or histogram distributions), which serve as texture features.

[0024] Sharpness feature extraction specifically involves calculating sharpness indices within image units based on image gradient magnitude or Laplacian response; for example, sharpness features can be obtained by statistically analyzing the mean or variance of gradient magnitudes or by calculating the response intensity of the Laplacian operator.

[0025] Contrast feature extraction specifically involves calculating the distribution differences of pixel intensity within an image unit, such as using grayscale variance, local dynamic range, or mean difference; and performing statistical analysis on the RGB or luminance channels to obtain contrast features that reflect the degree of luminance variation.

[0026] The extraction of salient features specifically involves calculating a salient map based on salient detection methods (such as frequency domain residual models or deep network-based salient models); averaging or weighting the salient map within an image unit to obtain the salient features of that image unit.

[0027] Edge orientation feature extraction specifically involves extracting edge information based on the Sobel or Canny operator; statistically analyzing the orientation distribution of edge pixels within an image unit (e.g., constructing an orientation histogram) to obtain edge orientation features.

[0028] Scale cue feature extraction specifically involves extracting scale information based on changes in texture density, edge density, or target size within image units; for example, scale cue features are obtained by statistically analyzing the number of edges or texture frequency per unit area, which can be used to reflect spatial distance relationships.

[0029] When semantic parsing is available, the semantic category response of each image unit is statistically analyzed to enhance the understanding of content such as the main body region, sky region, road region, and building region, thereby improving the stability of subsequent spatial hierarchy judgment and narrative modeling.

[0030] The aforementioned features are concatenated or weighted and fused to form a unified content feature. To avoid the impact of differences in feature dimensions on subsequent spatial hierarchy modeling, narrative path modeling, and control vector generation, the content features of each image unit are normalized to obtain the final content feature representation.

[0031] Regarding style feature mapping, local features and texture principal directions are extracted from the target style image, and then mapped to each image unit using local block matching or feature pyramid methods to obtain the local style features of each image unit. and the original brushstroke direction of the style ,in This refers to the dimension of local stylistic features. The original brushstroke direction reflects the style. From the target style image and image unit The dominant direction of the texture of the best-matching local block is determined, reflecting the original brushstroke direction of the style image in that area.

[0032] To avoid the influence of differences in local style feature magnitude and orientation vector scale on subsequent local style operator and orientation fusion, the following measures are taken: and Normalization is performed to serve as input for subsequent local style transfer and brushstroke direction fusion. Through this processing, the image simultaneously possesses structural, textural, and directional information at the region level, achieving a fine correspondence between content and style. This provides a stable and unified feature foundation for subsequent spatial hierarchy modeling, narrative information extraction, and adaptive style control, thereby improving the structural consistency and expressive accuracy of style transfer at its source.

[0033] S103. Calculate the spatial hierarchy probability and confidence level based on content features, and simultaneously calculate the narrative information based on content features; wherein, the spatial hierarchy includes the foreground layer, the midground layer, and the background layer; the narrative information represents the path, direction, and focus of the viewer's line of sight.

[0034] The specific steps for calculating spatial hierarchy probability and confidence based on content features are as follows: Based on the content features of image units, depth cue feature vectors representing their spatial proximity are extracted, and linearly mapped and normalized to obtain the initial probability of spatial hierarchy. The initial spatial hierarchy probability is globally optimized based on the adjacency relationship between image units to obtain the spatial hierarchy probability and generate the confidence level of the corresponding spatial hierarchy.

[0035] By extracting depth cues representing proximity relationships from content features and obtaining initial probabilities of spatial hierarchy through linear mapping and normalization, the determination of spatial hierarchy shifts from empirical rules to data-driven approaches. Further global optimization, incorporating adjacency relationships, ensures that locally unstable estimates tend to be consistent under overall structural constraints. A confidence level is introduced based on the spatial hierarchy probabilities to quantify the reliability of hierarchy determination for each image unit. Since the sufficiency of depth cues varies across different regions, the confidence level can distinguish between "reliable regions" and "uncertain regions." This provides a basis for subsequent adjustments, preventing uncertain regions from being over-constrained, thereby improving the overall robustness of the computation.

[0036] Specifically, from image units Content features Extracting deep cue feature vectors representing spatial proximity relationships: ; in, For the sharpness component, For contrast components, For texture density components, To obscure the weight of the clues, The scale cue component.

[0037] By performing a linear mapping on the deep cue feature vectors, we obtain the prior logits: ; in, Let be the projection matrix. This is the bias vector. and The training employed publicly available scene analysis and depth estimation datasets. These datasets contain pixel-level depth annotations or scene layer annotations, which can be directly converted into training labels. The label conversion rules are as follows: depth values ​​are normalized to the [0,1] interval, and the image is divided into three layers based on depth: depth ≤ 0.3 for the foreground layer, 0.3 < depth ≤ 0.7 for the midground layer, and depth > 0.7 for the background layer. One-hot labels are generated for each image unit. The loss function is based on cross-entropy loss, combined with L2 regularization to avoid overfitting. The training objective is to ensure that the probability output by the linear mapping highly matches the true layer label, with fixed hyperparameters.

[0038] The training process is as follows: Will and Initialize to fixed values ​​as in Scheme 1 to avoid slow convergence and local optima issues caused by random initialization; for each image unit in the training batch, calculate 5-dimensional depth cue features, and obtain the predicted level probability through linear mapping and softmax; calculate the total loss based on the prediction results and the true labels; use the Adam optimizer to calculate the loss against... and The gradient is calculated and parameters are updated; hyperparameters are fixed: learning rate 1e-3, batch size 32, and training epochs 50; training stops and the final result is saved when the validation set loss decreases by less than 1e-5 for 5 consecutive epochs. and Parameters; the average classification accuracy for foreground / midground / background on the test set should be ≥85%, ensuring the generality of the parameters. A well-trained system is obtained. and .

[0039] Softmax normalization of the prior logits yields the initial probabilities of the spatial hierarchy: ; Right now ,in , , Representing image units respectively The initial probability of belonging to the foreground, midground, and background layers. It is a two-dimensional simplex. This represents the probability that an image unit belongs to the k-th spatial level.

[0040] Relying solely on initial probabilities Susceptible to local noise, graph smoothing regularization is introduced for global optimization. The optimization objective is constructed as follows: ; in, For the first The spatial hierarchy probability of each image unit to be optimized; The initial probability obtained in the previous step; Adjacency weight; The graph smoothing weights control the smoothness of the hierarchical probabilities between adjacent image units. The entropy regularization weight. For the first The probability of the k-th spatial level of an image unit to be optimized.

[0041] In the energy function described above, the first term is a fidelity term, ensuring that the optimized probability does not deviate too far from the initial prior; the second term is the Dirichlet energy on the adjacency graph, which uses adjacency relationships to force the hierarchical probabilities of adjacent image units to remain smooth, avoiding hierarchical jumps caused by noise; the third term is a negative entropy regularization term. Numerically less than or equal to zero, when This term tends to flatten the probability distribution, avoiding overly sharp pseudo-deterministic classifications in regions with insufficient depth cues. The optimization problem is strictly convex when the entropy term is removed, and even with negative entropy added, it can still be stably solved within the simplex using mirror descent or projective Newton's method, ultimately yielding the spatial hierarchy probabilities. .

[0042] After obtaining the spatial hierarchy probabilities, the corresponding spatial hierarchy confidence scores are generated. Specifically, the normalized entropy of the probability distribution is used to measure the degree of certainty in the spatial hierarchy determination. ; in, For probability distribution Information entropy For the maximum entropy of three uniform distributions. When When the distribution is close to uniform, Approaching 0 indicates uncertainty in spatial hierarchy; when When highly concentrated at a certain level, A value close to 1 indicates that the spatial hierarchy height is determined.

[0043] The specific steps for calculating narrative information based on content features are as follows: Based on the content characteristics of the image unit, the narrative focus intensity and narrative termination intensity are calculated; the narrative focus intensity is used to represent the ability of the image unit to attract visual attention. The narrative potential function is constructed based on the adjacency relationship between image units, combined with the narrative focus intensity and narrative termination intensity, and the narrative potential value is obtained after solving it. The narrative direction vector is calculated based on the gradient change of the narrative potential value between image units, and the narrative potential value, narrative focus intensity, and narrative direction vector are used as narrative information.

[0044] Specifically, based on the content features of image units Calculate the intensity of narrative focus and narrative termination intensity Narrative focus intensity It is derived from a combination of factors such as saliency, subject detection, brightness contrast, and semantic priority, and is used to represent image units. The ability to attract visual attention; a higher value indicates that the area is more likely to become a visual focal point. Specifically, one can start by considering content characteristics. Extract the normalized significance score Brightness and contrast scores and semantic priority score Equal components, then weighted summation and sigmoid activation mapping to The interval yields the narrative focus intensity: ; in, , , These are the weighting coefficients for each component. For bias terms, For the sigmoid function. Narrative termination strength. Used to indicate the terminating trend of visual flow, such as the edge of the image, the background fading area, etc. A larger value indicates that the area is more likely to become the converging endpoint of the line of sight. The narrative termination intensity can be calculated based on the normalized distance from the image unit to the image boundary and the degree of local contrast attenuation.

[0045] The narrative path is essentially the direction of the viewer's eye movement on the screen. This application models this flow process by constructing a potential function on the image unit graph, where the focal region acts as the source of the potential, driving its increase, and the terminating region acts as the sink, driving its decrease. The Laplace term ensures the continuity and low deflection of the path. Specifically, the following narrative potential function is constructed with the following optimization objective: ; in, Let be the narrative potential function to be solved; For narrative focus intensity, the second term pulls the potential value of the focus area toward 1; For narrative termination intensity, the third term pulls the potential value of the termination region toward 0; and These are the weights of the focus term and the termination term, respectively; the first term is the Laplacian smoothing term on the image unit graph, which ensures the potential function transitions continuously in space.

[0046] Find the first-order optimal condition for the above energy function, and let it be about With the gradient being zero, we obtain the following linear system: ; in, For the graph Laplace matrix, For degree matrix, This is the adjacency weight matrix; and respectively with and This is a diagonal matrix with diagonal elements. Because... Positive semi-definite, and there exists at least one image unit that satisfies or Under the conditions, It is a positive semi-definite matrix and is related to The sum of these terms forms a positive definite matrix, and the coefficient matrix is ​​invertible, indicating that the linear system has a unique solution. Solving for this matrix yields the narrative potential. .

[0047] In obtaining narrative potential Then, the narrative direction vector on each image unit is calculated. Specifically, the image unit... The narrative gradient vector on is defined as: ; in, For image units The set of neighbors; For image units The geometric center points to the image unit The unit direction vector of the geometric center. This vector represents the potential function in the image unit. The discrete gradient direction in two-dimensional space.

[0048] Normalizing the narrative gradient vector yields the narrative direction vector: ; in, To prevent division by zero of small constants, the narrative potential will ultimately be... Intensity of Narrative Focus Narrative Direction Vector As a narrative information output.

[0049] Through the above steps, image units not only have stable spatial hierarchy division results, but also form a continuous visual narrative structure. The uncertainty is quantified by confidence, thereby achieving collaborative modeling in both spatial relationships and visual attention dimensions, providing a reliable and directional semantic foundation for subsequent style regulation.

[0050] S104. Calculate the degree of conflict between narrative information and spatial hierarchy probability, and concatenate the narrative information and spatial hierarchy probability into a composite character embedding vector.

[0051] Spatial hierarchy probability Describe the spatial position of image units within the frame, and the intensity of focus in narrative information. Describes the visual attention weight of this area. Generally, the foreground area is often the narrative focus, but in certain special compositions, conflict can arise, such as a lighthouse in the background serving as the focal point. To explicitly model this conflict, a degree of conflict is defined: ; in, For the intensity of narrative focus, Foreground layer probability. When Larger and When the size is small, it indicates that although the image unit is located deep within the frame, it is an important narrative focus, and the degree of conflict is high. A value close to 1 indicates significant deep-seated conflicts.

[0052] Based on this, spatial hierarchy probability, narrative information, and conflict degree are concatenated into a composite character embedding vector: ; in, For spatial hierarchy probability, For the intensity of narrative focus, For narrative potential, For the degree of conflict, For spatial hierarchy confidence, This represents the component-wise interaction term between narrative focus intensity and the probability of each level. This embedding vector fuses spatial location information and narrative structure information into a unified feature representation.

[0053] S105. Generate a basic control vector from the composite role embedding vector through a preset control mapping function. After gating and constraining the basic control vector based on confidence and conflict, perform bounded projection to obtain the control vector.

[0054] like Figure 2 As shown, firstly, the composite role is embedded into the vector. Through the preset control mapping function Generate the basic control vector. Control mapping function. The form of using a linear layer with a bounded activation function is adopted: ; in, It is the sigmoid activation function. It is the hyperbolic tangent activation function; These are learnable weight vectors; These are the corresponding bias terms. The five components of the basic control vector correspond to: style intensity. Texture density Color shift coefficient Structural edge retention coefficient Narrative direction fusion coefficient .

[0055] Control mapping function The training method involves constructing a paired dataset of content images and style images, all of which are publicly available. One content image is randomly selected from the content dataset, and three style images of the same type are randomly selected from the style dataset to form a training sample set, creating a 10:1:1 ratio of training, validation, and test samples. Optimal control vector labels are generated for each image unit using prior rules. 1000 representative samples are selected, and digital art professionals annotate the optimal control vectors to verify the effectiveness of the pseudo-labels and to fine-tune the trained model, ensuring the final result meets the requirements of artistic expression.

[0056] The backbone network uses VGG19 (pre-trained on ImageNet with frozen weights) to extract style and content features; the optimizer is AdamW with a learning rate of 1e-4, weight decay of 1e-5, and batch size of 16; the training consists of 30 pre-training epochs (based on a pseudo-labeled dataset) and 5 fine-tuning epochs (based on an expert-labeled dataset); the convergence condition is that the total loss on the validation set decreases by less than 1e-5 for three consecutive epochs, at which point training stops and the final parameters are saved. The resulting trained control mapping function is then obtained.

[0057] The specific steps for obtaining the control vector by gating and constraining the basic control vector based on confidence and conflict levels and then performing bounded projection are as follows: The availability of the narrative direction is determined based on the magnitude of the narrative direction vector, and the basic control vector is gated and modulated by combining the confidence and conflict levels. Apply focus protection upper bound constraints and fusion coupling constraints to the gated modulation base control vector, and project the constrained base control vector onto a preset bounded interval to obtain the control vector corresponding to the image unit.

[0058] Specifically, based on the narrative gradient vector The modulus determines the narrative direction and availability: ; in, To adjust the parameters. When When I was very young, A value approaching 0 indicates that the narrative direction is unavailable at that image unit. The effectiveness of the narrative direction is measured by the magnitude of the narrative direction vector, and it participates in gating modulation, ensuring that directional information only participates in control within stable regions. Since the narrative direction originates from the gradient of the potential function, its stability is related to the consistency of the image structure. This avoids introducing erroneous directional guidance in regions where the narrative is ambiguous, ensuring that directional information complements rather than interferes with the structural direction.

[0059] Combining spatial hierarchy confidence Narrative direction availability and conflict level Constructing gated vectors : ; This is the style intensity gating factor, used to adjust the style injection intensity; This is the texture density gating factor, used to adjust the degree of enhancement of texture details; This is the color shift gate coefficient, used to adjust the magnitude of color style migration; The structure preservation gating factor is used to control the degree to which original structural information is preserved; A gating coefficient is incorporated into the narrative direction to adjust the degree to which the narrative direction affects the direction of the brushstrokes; Each component is calculated in the following form: ; represent Any component in, These are the learnable parameters corresponding to each component.

[0060] Modulating the basic control vector using a gating vector: ; in, This represents element-wise multiplication. The preset baseline control vector has its components set to conservative default values, for example... ,in For medium style intensity value, Medium texture density value Indicates no color shift. For medium structure retention coefficient, This indicates a lack of convergence in narrative direction. The meaning of this gating mechanism is: when the spatial hierarchy is unreliable ( ) or narrative direction is unavailable ( When the gate value approaches 0, the control vector degenerates to the baseline value. Reduce the interference of uncertain information on style generation; when the conflict level When the value is large, style intensity is automatically suppressed using learnable parameters. The increase in the structural retention coefficient This prevents deep focus from being over-stylized.

[0061] Apply a focus protection upper bound constraint to the style intensity component to prevent the narrative focus area from being overwhelmed by excessive style: ; in, Based on the upper bound, This is the adjustment coefficient. Focal intensity. The larger the value, the lower the upper limit of allowed style intensity; if the focus is also in the foreground ( If the value is relatively large, the upper limit can be appropriately relaxed.

[0062] At the same time, the correlation between the control parameters of the coupling regularization term is introduced: ; in, These are the coupling regularization coefficients. The first term penalizes overly strong styles when there is significant background focus conflict; the second term restricts the simultaneous occurrence of high texture density and high directional stretching in the focus area; and the third term penalizes excessively large directional blending coefficients when narrative direction is unavailable.

[0063] Project the constrained control vector onto the preset bounded interval: ; That is, each component must satisfy , , , , Through bounded projection By restricting it to a legal range, the final image unit is obtained. corresponding control vector .

[0064] Through the above steps, multi-source features such as spatial hierarchy probability, confidence level, narrative focus intensity, narrative direction, and conflict level are uniformly mapped and modulated into a stable control vector. This allows the control parameters to be adaptively adjusted according to regional semantics, information reliability, and semantic consistency. At the same time, under the influence of focus protection and fusion constraints, the consistency between structure and narrative expression is maintained, thus providing a stable, controllable, and semantically oriented adjustment basis for subsequent style transformation.

[0065] S106. The stroke direction is obtained by fusing and optimizing the structural direction, the original stroke direction of the style, and the narrative information. For example... Figure 3 As shown, the specific steps are as follows: The narrative direction is extracted from the narrative information, and the structural direction is used as the objective constraint. A fusion objective function is constructed by weighting the original style brushstroke direction and the narrative direction, and the fusion coefficient is obtained by solving it. The fusion direction is obtained by weighted fusion of the original style brushstroke direction and the narrative direction based on the fusion coefficient; After normalizing the fusion direction, it is determined whether the narrative direction is usable and whether the content structure direction is reliable. Otherwise, degradation processing is performed to finally obtain the stroke direction of each image unit.

[0066] Specifically, for each image unit In terms of content structure To constrain the target, construct the original brushstroke direction of the style. With narrative direction Fusion objective function: ; in, The local structural direction is given by the principal direction of the structural tensor in the content features; The original brushstroke direction of the style; The narrative direction vector; For control vector Provided prior fusion coefficients; To stabilize parameters and prevent It deviates too far from the prior knowledge.

[0067] In the objective function described above, the first term requires that the fused direction be as close as possible to the content structure direction. The second constraint ensures that the fusion coefficients do not deviate too far from the prior. Regarding the fusion coefficients... Differentiating and setting it to zero, we obtain a closed-form solution: ; in, Indicates projection onto interval , This represents the vector dot product.

[0068] Based on the optimal fusion coefficient obtained from the solution The original style of brushstrokes and narrative direction are weighted and integrated: ; in, To fuse the direction vectors, The fusion coefficient is... The larger the value, the stronger its influence on the narrative direction.

[0069] After normalizing the fusion direction, it is determined whether the narrative direction is usable and whether the content structure direction is reliable. Otherwise, degradation processing is performed to finally obtain the stroke direction of each image unit.

[0070] The stroke direction is obtained by normalizing the fusion direction: ; in, For image units The final stroke direction vector, To prevent division by zero of small constants.

[0071] At the same time, assess the availability of the narrative direction. If the narrative direction is available... Below the preset threshold ( (This is the minimum threshold for narrative direction availability), indicating that the narrative direction is unreliable at this image unit, in which case degradation processing is performed: ; in, This is an indicator function that indicates when the condition is not met. The value is set to 0, at which point the fusion direction and normalization result are updated accordingly. , This means that the direction of the brushstrokes is completely reduced to the original direction of the style, thereby avoiding the unreliable narrative direction from misleading the brushstrokes.

[0072] When the content structure direction is unreliable or missing ( ≈0), reducing its constraining effect, causing the fusion direction to degenerate into a weighted combination of the original stylistic brushstroke direction and the narrative direction, and serving as the brushstroke direction. That is: Both are unreliable references to the usability of narrative direction, resulting in a degradation of its effectiveness.

[0073] Through the above steps, the structural direction, the original stylistic brushstroke direction, and the narrative direction are adaptively integrated under unified constraints. Combined with reliability judgment and degradation mechanism, a stable and consistent brushstroke direction is obtained. This direction not only conforms to the content structure but also reflects stylistic characteristics and guides visual flow. Thus, it works synergistically with the preceding spatial hierarchy, narrative information, and control vectors, providing a directional foundation with structural consistency and semantic guidance for subsequent style transformations.

[0074] S107. After performing style transformation on the content features based on local style features, control vectors, and stroke direction, perform image reconstruction to obtain the generated image and output it.

[0075] The specific steps are as follows: Construct local style operators to perform region-level style transformation on content features to obtain local style transformation features; Edge protection correction is applied to the local style transformation features to obtain the corrected features. The corrected features of all image units are aggregated, and the generated image is reconstructed through a decoder or inverse operator.

[0076] Before outputting the generated image, the generated image and control vector are updated in a closed loop based on the residual between the generated image and the target reference, until the iteration termination condition is met and the final generated image is output. The target reference quantities include structural reference quantities, focal reference quantities, hierarchical reference quantities, and style loss.

[0077] The structural reference value is obtained by comparing the structural consistency between the generated image and the image to be processed; The focus reference value is obtained by aligning the narrative information recalculated from the generated image with the narrative information of the image to be processed. The hierarchical reference value is obtained by ranking and constraining the visual attributes of different hierarchical regions in the output image based on spatial hierarchical probability; the visual attributes are regional visual intensity indices constructed based on the content features of image units. Style loss is obtained by aligning the content features of the output image with the local features of the style image.

[0078] To maintain the universality of the implementation, a local style operator is adopted. A region-level style transformation is performed on each image unit. This operator can be implemented using methods such as AdaIN, WCT, or local convolutional modulation. Taking AdaIN (Adaptive Instance Normalization) as an example, this operator performs instance normalization on the content features and then performs an affine transformation using the statistics of the style features: ; in, and These are the channel mean and channel standard deviation of the feature, respectively. To prevent division by zero by small constants. Based on this, texture density... A scaling factor that can be used as a style statistic to adjust texture injection intensity and color shift coefficient It can be used to offset the color channel average and brush stroke direction. Anisotropic modulation can be performed on features through directional filtering. For image units... Utilizing its local stylistic features Texture density Color shift coefficient and the direction of the brushstroke Regarding content features Perform a style change: ; in, These are the features after local style transformation. Then, style intensity is used... Interpolation fusion of content features and style transformation features: ; in, As a characteristic of the fusion of styles, The larger the value, the stronger the influence of style transformation.

[0079] By constructing a local style operator, a region-level style transformation is performed on the content features of each image unit. This allows the previously obtained local style features to no longer remain at the representation layer but directly participate in the generation process. Since the local style features have completed the mapping to the content regions, the style effect can be precisely applied at the image unit scale. This transforms style transfer from a globally uniform application to a region-level differentiated effect, making the style expression more closely match the content distribution.

[0080] Using the structural edge preservation coefficient Edge protection correction is applied to the fused features: ; in, This is the final feature after edge protection correction. To extract edge protection features from content features, specifically through content features... It is obtained by applying an edge enhancement filter (such as the Sobel operator or Canny edge detection) while preserving high-frequency structural components. The larger the value, the more likely it is to preserve the edges of the original structure. This involves aggregating the correction features of all image units. Image reconstruction is performed using a decoder or inverse operator: ; in, To reconstruct the generated image, For feature-to-image reconstruction, Mapping is assigned to image units.

[0081] To avoid structural damage, blurred focus, or flattened hierarchy caused by open-loop generation, a total energy functional is constructed to perform closed-loop iterative optimization on the generated image and control vector: ; in, Indicates the middle term; This is a style loss term; For structural reference values; As a key reference quantity; For hierarchical reference values; This is the total variation regularization term; , , , , , These are the weighting coefficients for each item, used to balance the relative importance of different objectives; The initial control vector obtained in step S105; This refers to the coupling regularization terms defined in step S105. The specific definitions of each term are as follows: Structural reference values ​​are defined as: ; in, and The generated image and the image to be processed are respectively in the image unit. Spatial gradient on, For spatial gradient operators; The structure-sensitive weights are derived from the foreground probabilities. Determined by both edge strength and its specific characteristics, it is defined as follows: ,in For image units Average edge strength within, To prevent small constants with zero weight; It is the Frobenius norm.

[0082] To generate a saliency map of an image, it can be calculated by applying a saliency detection algorithm (such as a saliency model based on frequency domain analysis or deep learning) to the generated image; For the intensity of narrative focus The target focus distribution obtained by projecting back to the pixel domain, specifically, for each pixel... ,make This involves mapping the focus intensity at the image unit level to the pixel level and normalizing it into a probability distribution. This represents the focal intensity of the image unit to which pixel (x,y) belongs; This represents the normalized summation of the focal intensity of all pixels in the image.

[0083] The focal reference value is defined as: ; in, Kullback-Leibler divergence measures the difference between the target focus distribution and the actual focus distribution in the generated image.

[0084] Based on spatial hierarchy probability, the average visual intensity indices (e.g., gradient sharpness or local contrast) of the foreground, midground, and background layers in the generated image are denoted as follows: , , Taking gradient sharpness as an example, the average visual intensity index for each level is calculated as follows: ; and respectively and Alternative Perform similar calculations, where To prevent division by zero of small constants, this index is weighted by spatial hierarchy probabilities to ensure that the visual intensity measurement of each hierarchy region is consistent with the probability distribution. The output image is required to satisfy a hierarchy order where foreground visual intensity is greater than midground, and midground is greater than background, using soft-margin rank loss. ; in, The preset minimum interval threshold, By introducing a hierarchical reference and constraining the visual attributes of different hierarchical regions based on spatial hierarchical probabilities, the visual strength relationship between foreground, midground, and background continues to be controlled during the generation stage. Since the spatial hierarchical probabilities originate from the global optimization results mentioned earlier, this reference extends spatial depth information from the representation stage to the output stage. This ensures that the generated image maintains the hierarchical order of foreground, midground, and background after stylization, preventing spatial hierarchy from being diluted by stylistic textures.

[0085] Style loss is defined as: ; in, For feature extraction networks, For local style transformation operators; Indicates the generated image; This represents the image (content) to be processed; This represents the target style image.

[0086] In addition, the total energy also includes the total variation regularization term. Used to suppress noise, it is defined as ,in and These are the difference operators in the horizontal and vertical directions, respectively. For smoothing constants; control parameter penalty term Used to prevent control parameters from deviating too far from their initial values; and for coupling regularization terms. Used to constrain the coupling relationship between control parameters.

[0087] By setting a style loss, the content features of the output image are aligned with the local features of the target style image. Simultaneously, structural reference, focal reference, and hierarchical reference are used for joint optimization, ensuring that style expression is no longer the sole objective, but rather a joint balance is achieved between style consistency, structure preservation, narrative preservation, and hierarchical preservation. This avoids the unidirectional optimization problem of "stronger style, weaker structure" in traditional methods, resulting in outputs that balance artistry and readability.

[0088] During the closed-loop update phase, the image variables and control vectors are updated alternately. Projective gradient descent is performed on the image variables. ; in, For the first Image generated in the next iteration For the first The set of control vectors for each iteration The total energy is related to the image. The gradient.

[0089] Perform projective gradient descent on the control vector: ; in, The total energy is related to the image unit. The partial derivative of the control vector; and The step sizes are the image and control vector step sizes, respectively, and must satisfy... , ( , (These are the Lipschitz constants corresponding to the gradients) to ensure that the energy does not increase monotonically. and These are bounded projection operators for image pixel values ​​and control parameters, respectively. The process iteratively updates the parameters until energy convergence or the preset maximum number of iterations is reached, then outputs the final generated image.

[0090] By performing closed-loop updates on the generated image and control vectors based on the residuals between the generated image and multiple target references, the control vectors are no longer a one-time static result, but can be corrected in reverse according to the current generation effect. Since the control vectors themselves carry spatial hierarchy, narrative conflict, and reliability information, this closed-loop mechanism is equivalent to allowing all the key features mentioned above to participate in the iterative convergence of the final result. This improves the self-correction capability of the overall generation process, enabling the final output to achieve a more stable unity in style, structure, focus, and spatial hierarchy.

[0091] It should be noted that, to ensure numerical stability, the following engineering constraints must be met in the actual implementation: a small constant must be added to all normalization operations. To prevent division by zero; all control parameters are projected onto a bounded interval. The total energy requires standardization of its components, i.e. ,in It is an exponential moving average to avoid scale imbalance between different terms.

[0092] To facilitate understanding of the above embodiments, a specific application scenario of the above embodiments will be used as an example for illustration below: Transform a photograph depicting a harbor at dusk into an artistic image with an oil painting style.

[0093] The image to be processed is one A pixelated photograph of a harbor at dusk, with mooring lines and ships on the pier in the foreground, a tranquil harbor surface in the middle ground, and a lighthouse and sunset on the horizon in the distance (background). The distant lighthouse is the narrative focus of the image. The target style image is an Impressionist oil painting.

[0094] In step S101, the system acquires the aforementioned image to be processed. and target style images .

[0095] In step S102, the system uses a superpixel segmentation method to divide the image to be processed into... For each image unit, an adjacency graph is constructed, and the content features of each image unit are extracted. Simultaneously, local features and the main texture direction of the target style image are extracted and mapped to each image unit to obtain local style features and the original brush stroke direction of the style.

[0096] In step S103, the system calculates spatial hierarchy probabilities based on content features. For example, image units in the dock area have high sharpness and contrast; after linear mapping and global optimization, their foreground layer probability... Approximately 0.95, mid-range probability Approximately 0.03, background layer probability The confidence level is approximately 0.02, representing the spatial hierarchy. The spatial hierarchy probability of the harbor water surface area is approximately 0.79. The lighthouse area is in the distance, and its spatial hierarchy probability is approximately... Meanwhile, after solving the narrative potential function, the system found that the lighthouse region has the highest narrative focus intensity. The narrative direction vector converges from the direction of the dock to the direction of the lighthouse.

[0097] In step S104, for the lighthouse area, its narrative focus intensity Prospect probability Therefore, the degree of conflict This indicates significant deep-seated conflicts. The system concatenates spatial hierarchy probabilities, narrative information, and conflict intensity into a composite character embedding vector.

[0098] In step S105, for the lighthouse area, due to the high degree of conflict, the gating mechanism automatically reduces the style intensity component. (For example, reducing it from 0.8 to 0.4), while simultaneously increasing the structural retention factor. (For example, increasing from 0.3 to 0.7) to ensure that the lighthouse, as the narrative focus, is not overly stylized like an oil painting and loses its structural distinctiveness. At the same time, the upper limit constraint of the focus protection will... Further restrictions were placed within a reasonable range. For the foreground area of ​​the dock, due to the high confidence and lack of conflict in the spatial hierarchy, the control vectors maintained a high style intensity and texture density, giving the foreground a rich oil painting brushstroke effect.

[0099] In step S106, the system optimizes the brushstroke direction for each area. The narrative direction of the harbor water area points towards the lighthouse. After merging with the horizontal style brushstrokes, the brushstroke direction naturally extends from the foreground towards the lighthouse, guiding the viewer's eye towards it. The narrative direction of the dock area is clear, and the merged brushstroke direction is organized along the structural direction of the cables and the hull.

[0100] In step S107, the system performs local style transformation and image reconstruction based on control vectors and brushstroke direction. During the closed-loop update phase, the structural reference ensures that the edge details of the dock area remain clear after stylization; the focal reference ensures that the lighthouse area maintains high saliency in the final image; and the hierarchical reference maintains a spatial hierarchy with a clear foreground, smooth midground, and soft background. After multiple iterations, the final generated image is output. In the final oil painting-style image, the foreground dock features rich, thick brushstrokes, the midground harbor water surface transitions with soft horizontal brushstrokes, and the background lighthouse, although in the distance, retains a clear outline and warm-toned halo due to narrative focus protection. The viewer's gaze naturally flows from the dock, across the water, to the lighthouse along the brushstroke direction. The overall image possesses both the artistic expressiveness of an oil painting and maintains a sense of spatial hierarchy and narrative guidance.

[0101] In summary, this embodiment models the spatial distance relationship and visual narrative path of the scene by constructing a spatial hierarchy probability model and a narrative potential function model. The two are then merged into a composite character embedding vector and an adaptive control vector is generated through a gating modulation mechanism. At the same time, the style brushstroke direction is fused and optimized with the narrative direction to obtain the brushstroke direction. Finally, iterative correction is performed based on a closed-loop feedback mechanism, thereby realizing spatial hierarchy perception, narrative path guidance, and adaptive and controllable artistic effect generation.

[0102] Example 2

[0103] like Figure 4 As shown, the art effect generation system based on image feature transfer includes a data acquisition module, a feature extraction module, a feature calculation module, a feature embedding module, a feature mapping module, a direction fusion module, and an image output module.

[0104] The data acquisition module is used to acquire the image to be processed and the target style image; The feature extraction module is used to divide the image to be processed into multiple image units, extract the content features of each image unit, and extract the local features and main texture direction of the target style image, and map them to each image unit to obtain the local style features and original brush stroke direction of the style of the image unit; wherein, the content features include at least texture features and structural directions generated by image gradients; The feature calculation module is used to calculate the spatial hierarchy probability and confidence based on content features, and to calculate narrative information based on content features; the spatial hierarchy includes foreground, midground, and background layers; the narrative information represents the path, direction, and focus of the viewer's eye movement; The feature embedding module is used to calculate the degree of conflict between narrative information and spatial hierarchy probability, and concatenates the narrative information and spatial hierarchy probability into a composite character embedding vector. The feature mapping module is used to generate a basic control vector from the composite role embedding vector through a preset control mapping function. After gating and constraining the basic control vector based on confidence and conflict, a bounded projection is performed to obtain the control vector. The direction fusion module is used to fuse and optimize the structural direction, the original style of the brushstroke direction, and the narrative information to obtain the brushstroke direction; The image output module is used to perform style transformation on the content features based on local style features, control vectors and stroke direction, and then perform image reconstruction to obtain and output the generated image.

[0105] like Figure 5 As shown, the image feature transfer-based artistic effect generation system can be deployed on servers, workstations, personal computers, edge computing devices, or smart terminals with image processing capabilities, and its hardware includes at least: The processor executes program instructions for image segmentation, feature extraction, probability calculation, vector mapping, orientation fusion, style transformation, and image reconstruction. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a neural network processor (NPU), or a combination thereof. The CPU is used for process scheduling and general-purpose computing, while the GPU or NPU is used for parallel computing tasks such as convolutional feature extraction, saliency detection, style transformation, and iterative optimization.

[0106] The memory is used to store computer programs, model parameters, intermediate features, control vectors, and generation results. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, or solid-state drive (SSD). The RAM is used to cache intermediate data such as the image to be processed, image unit adjacency graph, content features, spatial hierarchy probabilities, narrative information, and stroke direction. The non-volatile memory is used to store the feature extraction model, saliency model, semantic model, style reference image, and historical generation results.

[0107] An image input device is used to acquire the image to be processed and the target style image. The image input device may be a camera, scanner, local file interface, network interface, or user interaction terminal.

[0108] A display device is used to display the generated image, processing status, and parameter configuration interface. The display device may be a monitor, touchscreen, or other graphical output terminal.

[0109] The communication bus is used to connect the processor, memory, input / output interfaces, and accelerator modules to complete data transmission between modules.

[0110] The network communication module is used to obtain the image to be processed, the target style image, and the model parameters from a remote database, cloud storage, or user terminal, and can also send the generated image back to the client.

[0111] In some implementations, the system may also include a dedicated image acceleration card or AI inference card to improve the execution efficiency of saliency detection, semantic priority analysis, local style mapping, and closed-loop optimization.

[0112] This embodiment has the same beneficial effects as Embodiment 1.

[0113] Example 3

[0114] This embodiment provides a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the image feature transfer-based artistic effect generation method of Embodiment 1.

[0115] The image feature transfer-based artistic effect generation method in Example 1 can be applied in software form, such as by designing it as a standalone program and installing it on a computer terminal, which can be a computer, smartphone, etc. Alternatively, it can be designed as an embedded program and installed on a computer terminal, such as on a microcontroller.

[0116] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. An artistic effect generation method based on image feature transfer, characterized in that, Includes the following steps: Obtain the image to be processed and the target style image; The image to be processed is divided into multiple image units, and the content features of each image unit are extracted. At the same time, the local features and main texture direction of the target style image are extracted and mapped to each image unit to obtain the local style features and original brush stroke direction of the style of the image unit. Among them, the content features include at least the texture features and the structural direction generated by the image gradient. Spatial hierarchy probability and confidence are calculated based on content features, and narrative information is also calculated based on content features; the spatial hierarchy includes foreground, midground, and background layers; the narrative information represents the path, direction, and focus of the viewer's eye movement. Calculate the degree of conflict between narrative information and spatial hierarchy probability, and concatenate the narrative information and spatial hierarchy probability into a composite character embedding vector; The composite role embedding vector is used to generate a basic control vector through a preset control mapping function. The basic control vector is then gated and constrained based on confidence and conflict, and then bounded projection is performed to obtain the control vector. The stroke direction is obtained by integrating and optimizing the structural direction, the original stylistic stroke direction, and narrative information. After performing style transformation on the content features based on local style features, control vectors, and stroke direction, image reconstruction is performed to obtain the generated image and output it.

2. The method for generating artistic effects based on image feature transfer according to claim 1, characterized in that, The specific steps for calculating spatial hierarchy probability and confidence based on content features are as follows: Based on the content features of image units, depth cue feature vectors representing their spatial proximity are extracted, and linearly mapped and normalized to obtain the initial probability of spatial hierarchy. The initial spatial hierarchy probability is globally optimized based on the adjacency relationship between image units to obtain the spatial hierarchy probability and generate the confidence level of the corresponding spatial hierarchy.

3. The method for generating artistic effects based on image feature transfer according to claim 1, characterized in that, The specific steps for calculating narrative information based on content features are as follows: Based on the content characteristics of the image unit, the narrative focus intensity and narrative termination intensity are calculated; the narrative focus intensity is used to represent the ability of the image unit to attract visual attention. The narrative potential function is constructed based on the adjacency relationship between image units, combined with the narrative focus intensity and narrative termination intensity, and the narrative potential value is obtained after solving it. The narrative direction vector is calculated based on the gradient change of the narrative potential between image units, and the narrative potential, narrative focus intensity, and narrative direction vector are used as narrative information.

4. The method for generating artistic effects based on image feature transfer according to claim 1, characterized in that, The specific steps for obtaining the control vector by gating and constraining the basic control vector based on confidence and conflict levels and then performing bounded projection are as follows: The availability of the narrative direction is determined based on the magnitude of the narrative direction vector, and the basic control vector is gated and modulated by combining the confidence and conflict levels. Apply focus protection upper bound constraints and fusion coupling constraints to the gated modulation base control vector, and project the constrained base control vector onto a preset bounded interval to obtain the control vector corresponding to the image unit.

5. The method for generating artistic effects based on image feature transfer according to claim 1, characterized in that, The specific steps for fusing and optimizing structural direction, original stylistic brushstroke direction, and narrative information to obtain brushstroke direction are as follows: The narrative direction is extracted from the narrative information, and the structural direction is used as the objective constraint. A fusion objective function is constructed by weighting the original brushstroke direction of the style and the narrative direction, and the fusion coefficient is obtained by solving it. The fusion direction is obtained by weighted fusion of the original style brushstroke direction and the narrative direction based on the fusion coefficient; After normalizing the fusion direction, it is determined whether the narrative direction is usable and whether the content structure direction is reliable. Otherwise, degradation processing is performed to finally obtain the stroke direction of each image unit.

6. The method for generating artistic effects based on image feature transfer according to claim 1, characterized in that, The specific steps for style transformation of content features based on local style features, control vectors, and stroke direction, followed by image reconstruction, are as follows: Construct local style operators to perform region-level style transformation on content features to obtain local style transformation features; Edge protection correction is applied to the local style transformation features to obtain the corrected features. The corrected features of all image units are aggregated, and the generated image is reconstructed through a decoder or inverse operator.

7. The method for generating artistic effects based on image feature transfer according to claim 1, characterized in that, Before outputting the generated image, the generated image and control vector are updated in a closed loop based on the residual between the generated image and the target reference, until the iteration termination condition is met and the final generated image is output. The target reference quantities include structural reference quantities, focal reference quantities, hierarchical reference quantities, and style loss.

8. The method for generating artistic effects based on image feature transfer according to claim 7, characterized in that, The process for setting the target reference value is as follows: The structural reference value is obtained by comparing the structural consistency between the generated image and the image to be processed; The focus reference value is obtained by aligning the narrative information recalculated from the generated image with the narrative information of the image to be processed. The hierarchical reference value is obtained by ranking and constraining the visual attributes of different hierarchical regions in the output image based on spatial hierarchical probability; the visual attributes are regional visual intensity indices constructed based on the content features of image units. Style loss is obtained by aligning the content features of the output image with the local features of the style image.

9. An artistic effect generation system based on image feature transfer, applied to the artistic effect generation method based on image feature transfer as described in any one of claims 1-8, characterized in that the system... include: The data acquisition module is used to acquire the image to be processed and the target style image; The feature extraction module is used to divide the image to be processed into multiple image units, extract the content features of each image unit, and extract the local features and main texture direction of the target style image, and map them to each image unit to obtain the local style features and original brush stroke direction of the style of the image unit; wherein, the content features include at least texture features and structural directions generated by image gradients; The feature calculation module is used to calculate the spatial hierarchy probability and confidence based on content features, and to calculate narrative information based on content features; the spatial hierarchy includes foreground layer, midground layer, and background layer; the narrative information represents the path, direction and focus of the viewer's eye movement; The feature embedding module is used to calculate the degree of conflict between narrative information and spatial hierarchy probability, and concatenate the narrative information and spatial hierarchy probability into a composite character embedding vector. The feature mapping module is used to generate a basic control vector from the composite role embedding vector through a preset control mapping function. After gating and constraining the basic control vector based on confidence and conflict, bounded projection is performed to obtain the control vector. The direction fusion module is used to fuse and optimize the structural direction, the original style of the brushstroke direction, and the narrative information to obtain the brushstroke direction; The image output module is used to perform style transformation on the content features based on local style features, control vectors, and stroke direction, and then perform image reconstruction to obtain and output the generated image.

10. A computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the image feature transfer-based artistic effect generation method as described in any one of claims 1 to 8.

Images