AI comic visual style unification processing method fusing text driving and style transfer

Abstract

The application discloses an AI comic visual style unification processing method fusing text driving and style migration, and particularly relates to the technical field of visual style unification processing, and realizes accurate perception of highlight anchor points and light source directions through construction of a semantic light coupling description set, combines a dynamic range fidelity coefficient and a light and shadow consistency constraint tensor to segmentally constrain and time-series smooth style migration strength, and calculates an adaptive adjustment offset based on a degradation region classification feature set, so that physical light logic and light and shadow consistency are completely retained while visual style unification is realized, and artistic style unification and physical realism are taken into account.

Description

Technical Field

[0001] This invention relates to the field of visual style unification processing technology, and more specifically, to an AI animation visual style unification processing method that integrates text-driven and style transfer. Background Technology

[0002] In existing AI-powered methods for unifying the visual style of animated series, text-driven style transfer techniques typically rely on multimodal models to extract emotional semantics from the script as style guidance, but lack precise modeling of physical lighting parameters. When processing footage with extreme dynamic range, such methods often forcibly compress high-dynamic-range lighting scenes to the mean of style statistical features, resulting in the suppression of physically reasonable local highlights and the enhancement of shadow details, thus causing contrast loss and a grayish appearance.

[0003] More importantly, because the algorithm cannot perceive the physical attribute of light source direction, when fusing source materials from different lighting conditions, the style transfer module ignores the consistency constraints of lighting and shadow logic, only normalizing color and texture, resulting in conflicting lighting and shadow directions on the same character's face. This semantic deficiency in both the intensity and direction of light means that although the final output tends to be uniform in color statistics, it produces a sense of incongruity at the physical visual level that violates the laws of imaging, severely damaging the realism and immersion of the animation scene. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an AI animation visual style unification processing method that integrates text-driven and style transfer to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: An AI-driven approach to unifying the visual style of animated series, combining text-driven and style transfer methods, includes the following steps: The dynamic range distribution features and main light source direction parameters of each frame in the source image set are obtained. Highlight anchor point set and dark anchor point set are constructed by brightness histogram partitioning and gradient field analysis. Semantic lighting coupling description set is obtained by combining script text sentiment semantics. Based on the semantic lighting coupling description set, the source image is divided into a physical highlight protection area and a shadow detail preservation area. The style transfer intensity of the texture layer is segmented and constrained. The highlight suppression factor and shadow enhancement factor are extracted, and the dynamic range fidelity coefficient is calculated. Based on the dynamic range fidelity coefficient and the light source direction vector between adjacent frames, a light and shadow consistency constraint tensor is constructed to obtain the migration step size coefficient. Based on the migration step size coefficient and the local brightness change gradient, the geometric correction prediction of the light and shadow conflict area is calculated to obtain the light and shadow direction uniformity index. Based on the uniformity index of light and shadow direction and the distribution of style transfer intensity, a local light and shadow distortion feature set is extracted, and a semantic illumination joint degradation vector is constructed to obtain a degradation region classification feature set. Based on the degradation region classification feature set, the spatial distribution of overexposed highlight areas and black shadow areas is extracted, and the overexposed highlight areas and black shadow areas are compared with preset physical highlight retention thresholds and shadow detail retention thresholds, respectively, to calculate the adaptive style transfer adjustment offset.

[0006] In a preferred embodiment, the process of obtaining the semantic illumination coupling description set is as follows: The pixel brightness value sequence of each frame in the source image set is collected, and the brightness range is divided into highlight range, midtone range and dark range using an adaptive histogram partitioning method. The pixel percentage in each interval is statistically analyzed, the pixel sets in the highlight area and the pixel sets in the shadow area are extracted, and the average brightness of the highlight area and the average brightness of the shadow area are calculated to construct a dynamic range distribution feature vector. Gradient field analysis is performed on the source image to calculate the pixel gradient magnitude and gradient direction. The top K pixels with the largest pixel gradient magnitude are selected as edge candidate sets. Hough transform is used to detect straight lines in the edge candidate sets and extract the set of straight line segments in each frame. The gradient directions of each straight line segment are clustered, and the direction angle with the highest frequency is selected as the main light source direction parameter. The confidence of the light source direction is calculated in combination with the shadow area distribution of each frame to construct the main light source direction feature vector. The script text's emotional semantics are input into a pre-trained multimodal semantic encoder to extract emotional feature vectors; The dynamic range distribution feature vector, the main light source direction feature vector, and the emotion feature vector are fused in a multimodal manner to construct a semantic-lighting coupled description set.

[0007] In a preferred embodiment, the process of obtaining the dynamic range fidelity coefficient is as follows: Based on the highlight region pixel set, dark region pixel set, sentiment category and sentiment intensity scalar in the semantic lighting coupling description set, the highlight threshold and dark threshold are adaptively corrected for sentiment, and a physical highlight protection area mask and dark detail preservation area mask are generated. Based on the physical highlight protection area mask and shadow detail preservation area mask, image pixels are divided into three categories: highlight protection area, shadow detail preservation area and midtone area, and different style transfer intensity weights are assigned to each category. The highlight suppression factor and shadow enhancement factor were extracted, and the dynamic range fidelity coefficient was calculated.

[0008] In a preferred embodiment, the process of obtaining the migration step size coefficient is as follows: Obtain the main light source direction parameters of adjacent frames, and the corresponding dynamic range fidelity coefficients; Define the light source direction deviation angle; Construct a lighting consistency constraint tensor and calculate the transition step size coefficient based on the lighting consistency constraint tensor.

[0009] In a preferred embodiment, the lighting consistency constraint tensor includes a direction consistency weight, a direction change penalty weight, and a fidelity fusion weight.

[0010] In a preferred embodiment, the process of obtaining the uniformity index of light and shadow direction is as follows: Calculate the local brightness change gradient field based on the pixel brightness value sequence of each frame in the source image set, and extract the gradient direction; define the theoretical shadow direction based on the gradient direction; Obtain the direction of the main light source and the direction of the target light source in the current frame; calculate the deviation angle between the theoretical shadow direction and the current main light source direction in the current frame to obtain the current shadow direction deviation; simultaneously calculate the deviation angle between the theoretical shadow direction and the target light source direction to obtain the target shadow direction deviation. Construct a mask for areas of light and shadow conflict to identify such areas; For each pixel in the area of ​​light and shadow conflict, calculate the geometric correction prediction; Spatially aggregate the geometric correction predictions for all areas of light and shadow conflict to calculate the light and shadow direction uniformity index.

[0011] In a preferred embodiment, the process of obtaining the classification feature set of the degraded region is as follows: Obtain the uniformity index of light and shadow direction and the distribution matrix of style transfer intensity; at the same time, obtain the sentiment category, sentiment intensity, and physical highlight protection area mask and shadow detail preservation area mask in the semantic lighting coupling description set; For each pixel in the image, extract the local lighting distortion feature vector; Construct a semantic illumination joint degradation vector; Cluster analysis is performed on the semantic illumination joint degradation vectors of all pixels in the entire image. The density-based clustering algorithm DBSCAN is used to divide the pixels in the feature space into several clusters. Each cluster is classified according to the mean of the feature vectors within each cluster to obtain the region classification results. Finally, the degradation region classification feature set is obtained.

[0012] In a preferred embodiment, the local lighting distortion feature vector includes style intensity deviation features, shadow direction deviation features, brightness gradient anomaly features, dynamic range local fidelity features, and emotional lighting coupling consistency features.

[0013] In a preferred embodiment, the region classification results include a highlight overexposure degradation region, a dark black degradation region, a shadow direction conflict degradation region, a texture artifact degradation region, and a slight distortion region.

[0014] In a preferred embodiment, the process of obtaining the adaptive style transfer adjustment offset is as follows: From the set of degradation region classification features, regions with degradation type of highlight overexposure degradation area and region with dark black degradation area are selected; For the overexposed and degraded highlight areas, calculate their spatial distribution feature vector, which includes the number of pixels in the area, the area ratio of the area, the average brightness of the area, and the average style intensity deviation of the area. For the dark, degraded black areas, calculate their spatial distribution feature vector, which includes the number of pixels in the region, the area ratio of the region, the average brightness of the region, and the average style intensity deviation of the region. The average brightness of the overexposed and degraded highlight areas is compared with the preset physical highlight retention threshold to calculate the brightness deviation of the highlight areas; the average brightness of the degraded and black shadow areas is compared with the preset shadow detail retention threshold to calculate the brightness deviation of the shadow areas. Calculate the style transfer adjustment offset for the overexposed and degraded highlight areas; Calculate the style transfer adjustment offset for dark, solid black areas; For the mid-tone areas in non-degraded regions, define a basic adjustment offset and do not adjust the style intensity; Generate pixel-level adaptive style transfer adjustment offsets to dynamically correct style transfer intensity weights.

[0015] The technical effects and advantages of this invention are as follows: 1. This invention aims to solve the technical problems of existing AI animation style unification methods, which lack physical lighting parameter modeling, resulting in suppressed highlights and loss of shadow details in extreme dynamic range materials, as well as light and shadow conflicts when materials from different light source directions are blended. By constructing a semantic lighting coupling description set, it achieves accurate perception of highlight anchor points and light source directions. It combines dynamic range fidelity coefficient and light and shadow consistency constraint tensor to perform segmented constraints and temporal smoothing on style transfer intensity. Based on the degradation region classification feature set, it calculates adaptive adjustment of offset. While achieving visual style unification, it fully preserves physical lighting logic and light and shadow consistency, taking into account both artistic style unity and physical realism. Attached Figure Description

[0016] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example: Figure 1 This invention presents a method for unifying the visual style of AI-generated comics by integrating text-driven and style transfer approaches, comprising the following steps: The dynamic range distribution features and main light source direction parameters of each frame in the source image set are obtained. Highlight anchor point set and dark anchor point set are constructed by brightness histogram partitioning and gradient field analysis. Semantic lighting coupling description set is obtained by combining script text sentiment semantics. Based on the semantic lighting coupling description set, the source image is divided into a physical highlight protection area and a shadow detail preservation area. The style transfer intensity of the texture layer is segmented and constrained. The highlight suppression factor and shadow enhancement factor are extracted, and the dynamic range fidelity coefficient is calculated. Based on the dynamic range fidelity coefficient and the light source direction vector between adjacent frames, a light and shadow consistency constraint tensor is constructed to obtain the migration step size coefficient. Based on the migration step size coefficient and the local brightness change gradient, the geometric correction prediction of the light and shadow conflict area is calculated to obtain the light and shadow direction uniformity index. Based on the uniformity index of light and shadow direction and the distribution of style transfer intensity, a local light and shadow distortion feature set is extracted, and a semantic illumination joint degradation vector is constructed to obtain a degradation region classification feature set. Based on the degradation region classification feature set, the spatial distribution of overexposed highlight regions and black shadow regions is extracted. The overexposed highlight regions and black shadow regions are compared with preset physical highlight retention thresholds and shadow detail retention thresholds, respectively. The adaptive style transfer adjustment offset is calculated to dynamically correct the statistical feature mapping curve of the style normalization layer.

[0019] In this embodiment of the invention, the process of obtaining the dynamic range distribution features and main light source direction parameters of each frame in the source image set, constructing a set of highlight anchor points and a set of dark anchor points through brightness histogram partitioning and gradient field analysis, and obtaining a semantic-lighting coupling description set by combining the emotional semantics of the script text is as follows: Pixel brightness value sequence of each frame in the source image set An adaptive histogram partitioning method is used to divide the brightness range. Divided into highlight areas Intermediate adjustment range and dark area High light threshold With dark threshold Based on the cumulative brightness distribution function of each frame The 95th percentile and 5th percentile are defined; The cumulative distribution function of brightness This refers to the following: For a single frame image, after sorting the brightness values ​​L of all its pixels (usually normalized to the range [0,1] or [0,255]) in ascending order, the brightness values ​​are less than or equal to the brightness threshold. The function representing the proportion of pixels in the total number of pixels in the image is as follows: The function is in Monotonically increasing from In this embodiment, using 95th percentile luminance value (i.e., satisfy) (brightness value) and 5th percentile brightness value (i.e., satisfy) The brightness value is used as the dynamic threshold to calibrate the highlight threshold. With dark threshold This enables adaptive partitioning of the dynamic range of images under different exposure conditions; Statistical analysis of pixel percentage within each interval , , Extract the pixel set of the highlight region pixel set in dark areas And calculate the average brightness of the highlight area. Average brightness of dark areas Construct dynamic range distribution feature vector ; Perform gradient field analysis on the source image and calculate the pixel gradient magnitude. With gradient direction The top K pixels with the largest pixel gradient magnitudes are selected as edge candidates. Hough transform is used to perform line detection on the edge candidates, and line segment sets are extracted in each frame. Cluster the gradient directions of each line segment and select the direction angle with the highest frequency. As the parameter of the main light source direction The confidence score of the light source direction is calculated by combining the distribution of shadow regions in each frame (determined by connected components with brightness lower than the local mean). Construct the feature vector of the main light source direction ; The confidence level of the light source direction is calculated by combining the shadow region distribution of each frame. Specifically, the source image is subjected to local brightness mean filtering, and pixels with brightness less than the local mean are identified as the effective shadow region set. For each effective shadow region, calculate its centroid and the region's principal axis direction, compare the consistency of this principal axis direction with the assumed opposite direction of the light source direction, and calculate the difference to obtain the principal axis consistency score; at the same time, extract the gradient direction of the shadow region's edge contour points, calculate the perpendicular deviation between the edge normal and the light source direction, and obtain the edge direction consistency score. The confidence score of the light source direction is obtained by weighting and summing the principal axis consistency score and the edge direction consistency score of each shadow region. ; The script text's emotional semantics are input into a pre-trained multimodal semantic encoder to extract emotional feature vectors. The encoder uses the text encoding branch of the CLIP model to output a scalar of sentiment intensity. With Emotion Category Tags The emotion category label Including but not limited to warm, gloomy, bright, dim, calm, and intense; Dynamic range distribution feature vector , Main light source direction feature vector With sentiment feature vector Perform multimodal fusion to construct a semantic-lighting coupled description set. : .

[0020] In this embodiment of the invention, the process of dividing the source image into a physical highlight protection zone and a shadow detail preservation zone according to the semantic lighting coupling description set, segmenting the style transfer intensity of the texture layer, extracting the highlight suppression factor and shadow enhancement factor, and calculating the dynamic range fidelity coefficient is as follows: Based on the semantic lighting coupling description set Highlight region pixel set Pixel set in dark areas Emotional Category With emotional intensity scalar For high light threshold With dark threshold Perform emotion-adaptive correction and generate a physical specular protection mask. Mask for preserving detail in shadow areas : ,in For high light protection relaxation factor, To preserve the relaxation factor in dark areas, For the normalization function of emotion intensity, and A preset protection strength mapping function for the emotion category (lowering the highlight protection threshold for the warm emotion category and raising the shadow preservation threshold for the gloomy emotion category). According to the physical highlight protection zone mask Mask for preserving detail in shadow areas Image pixels are divided into three categories: highlight protection area, shadow detail preservation area, and midtone area, and each is assigned a different style transfer intensity weight: ,in , , These are the preset baseline style transfer intensity parameters for the highlight protection zone, shadow detail preservation zone, and midtone zone, respectively. This is the specular suppression coefficient. This is the shadow enhancement factor. and These are pre-trained suppression and boosting mapping networks, respectively; The logic behind the division of the highlight protection zone, shadow detail preservation zone, and midtone zone is as follows: High-light protection zone: ; Shadow detail preservation area: ; Intermediate adjustment zone: ,in ,and ; The reference style transfer intensity parameter refers to the initial weight coefficients that are pre-set to control the degree of effect of the style transfer network in different brightness areas of the source image when performing differential stylization processing. These parameters serve as the basic control quantities of the segmented constraint module and determine the magnitude of texture transformation and the degree of preservation of original physical features in each region during the style transfer process.

[0021] The pre-trained suppression and boosting mapping network is specifically as follows: Constructing a suppression-boosting mapping network and Both share the same network architecture, employing a three-layer fully connected network structure with an input layer dimension of [missing information]. (corresponding to the average brightness normalization value, the main light source direction sinusoidal encoding, and the emotion category embedding, respectively), the hidden layer dimension is 128, the output layer dimension is 1, the activation function is ReLU, and the output range is constrained to [0,1] by the Sigmoid function; Preprocessing of input features: Normalizing the average brightness of highlight areas and the average brightness of shadow areas; adjusting the direction of the main light source. Encoded as a two-dimensional direction vector ; Emotional categories The embedding layer maps the data to an emotion embedding vector, which is then concatenated with the aforementioned features to form the input feature vector. The suppression-boosting mapping network was pre-trained using supervised learning. The training dataset consisted of manually annotated comic book image samples. Each sample included the source image, the style-transferred image, and the annotators' ratings on the degree of highlight preservation and shadow detail preservation. Mean squared error was used as the loss function. During training, sentiment category classification loss was introduced as an auxiliary task to enhance the network's ability to understand sentiment semantics. After the network training converges, during the actual inference phase, the network maps the average brightness of the highlight areas, the direction of the main light source, and the sentiment category of the input. Output specular suppression coefficient; a larger value indicates stronger suppression of style transfer intensity in specular regions and more complete preservation of physical specular highlights; mapping network The output shadow enhancement coefficient is larger, indicating a stronger enhancement of details in the shadow area and higher visibility of shadow details; this pre-trained network realizes emotional semantic-driven adaptive highlight suppression and shadow enhancement. Extracting highlight inhibitors With shadow enhancement factor ,in Characterizes the ability to retain brightness in highlight areas. Characterizes the ability to enhance details in dark areas; Calculate the dynamic range fidelity factor : ,in , These represent the average brightness of the highlight and shadow areas before style transfer. , These represent the average brightness of the highlight and shadow areas after style transfer and segmented constraint processing. Contrast retention factor , These represent the pixel percentages of the highlight and shadow areas before style transfer. , These represent the pixel percentages of the corresponding regions after style transfer.

[0022] In this embodiment of the invention, the process of constructing a lighting consistency constraint tensor based on the dynamic range fidelity coefficient and the light source direction vector between adjacent frames, and obtaining the migration step size coefficient, is as follows: In this embodiment of the invention, the process of constructing a lighting consistency constraint tensor based on the dynamic range fidelity coefficient and the light source direction vector between adjacent frames, and obtaining the migration step size coefficient, is as follows: Get adjacent frames and Main light source direction parameters and and the corresponding dynamic range fidelity coefficient and ; Define the light source direction deviation angle That is, the minimum angle between two directions, and its range of values. ; Constructing a light and shadow consistency constraint tensor Its spatial dimension is consistent with the image size H×W, and the third dimension stores the orientation consistency weight, orientation change penalty weight, and fidelity fusion weight, respectively: ,in, For directional consistency weights, For pixels The local gradient direction, The orientation sensitivity coefficient measures the degree of matching between the pixel gradient direction and the direction of the main light source in the current frame, giving pixels located on the edge of the light source direction a higher priority for consistency constraints. The penalty weight for changes in direction, The change sensitivity coefficient is based on the inter-frame light source direction deviation angle. Dynamic adjustment: When the deviation angle is small (lighting direction is stable), the weight approaches 1, and a strong time consistency constraint is applied; when the deviation angle is large (lighting direction changes abruptly), the weight approaches 0, reducing the dependence on the light and shadow of the previous frame, and avoiding the light source direction transition blurring or physical lighting logic confusion caused by forced smoothing. The fidelity fusion weight represents the geometric mean of the dynamic range fidelity of two frames. The lower the fidelity, the smaller the light and shadow consistency constraint weight, thus avoiding the propagation of low-quality light and shadow information to high-quality frames. Calculate the migration step size coefficient based on the lighting consistency constraint tensor: ,in The maximum permissible standard deviation threshold for the direction. The standard deviation of the light source direction. The spatial average product of the directional consistency weight, the directional change penalty weight, and the fidelity fusion weight represents the overall credibility of the current frame.

[0023] In this embodiment of the invention, the process of calculating the geometric correction prediction amount of the light and shadow conflict region and obtaining the light and shadow direction uniformity index based on the migration step size coefficient and the local brightness change gradient is as follows: Based on the pixel brightness value sequence of each frame in the source image set Calculate the gradient field of local brightness variation And extract the gradient direction The theoretical shadow direction is defined based on the gradient direction. That is, the opposite direction of the local gradient is the direction of potential shadow extension; Get the direction of the main light source in the current frame relative to the direction of the target light source (The target direction is the desired uniform lighting direction, which can be obtained by averaging the light source directions of all footage or by manually specifying it.) Calculate the deviation angle between the theoretical shadow direction of the current frame and the current main light source direction to obtain the current shadow direction deviation. Simultaneously calculate the deviation angle between the theoretical shadow direction and the target light source direction to obtain the target shadow direction deviation. ; Constructing a mask for areas of light and shadow conflict The determination condition is: if the difference between the current shadow direction deviation and the target shadow direction deviation is greater than a preset deviation threshold. If the pixel is located outside the highlight protection zone or the shadow detail preservation zone (i.e., the area where style transfer may cause obvious lighting distortion), then the current area is determined to be a light and shadow conflict area. For each pixel in the area of ​​light and shadow conflict Calculate the geometrically corrected prediction. This prediction represents the magnitude of the local brightness gradient adjustment that needs to be applied to bring the actual shadow direction closer to the target light source direction: ,in The migration step size factor controls the intensity of the geometric correction. This represents the magnitude of the local brightness gradient. The item value is This is used to control the correction direction (positive values ​​enhance the current gradient, negative values ​​suppress the current gradient); this prediction represents the intensity of the gradient field change required to make the shadow direction consistent with the target light source direction, positive values ​​indicate that the brightness change in this direction needs to be increased, negative values ​​indicate that it needs to be reduced; Spatially aggregate the geometric correction predictions for all areas of light and shadow conflict, and calculate the light and shadow direction uniformity index. : ,in This is a collection of pixels representing areas of light and shadow conflict. This represents the total number of pixels in the area of ​​light and shadow conflict. The Euclidean distance from the current pixel to the nearest highlight protection zone or shadow detail preservation zone. The distance decay constant (0.1 times the length of the image diagonal) makes conflict areas far from the protected area have a greater impact on the uniformity index; This is to prevent division by zero by a very small constant (generally taken as...). ).

[0024] In this embodiment of the invention, the process of extracting local lighting distortion feature sets based on the uniformity index of lighting direction and style transfer intensity distribution, and constructing a semantic lighting joint degradation vector to obtain a degradation region classification feature set is as follows: Obtain the uniformity index of light and shadow direction With style transfer intensity distribution matrix (The style transfer intensity weights calculated in the preceding steps) constitute); Simultaneously acquire the semantic lighting coupling description set Emotional categories Emotional intensity and physical highlight protection zone mask Mask for preserving detail in shadow areas ; For each pixel in the image Extracting local light and shadow distortion feature vectors It includes the following five dimensions: Style intensity deviation characteristics ,in Reference style intensity parameters for the intermediate control zone This feature characterizes the degree of deviation of the current pixel stylization intensity relative to the midtone region. The greater the deviation, the stronger the style transfer's intervention on the lighting structure. Shadow direction deviation characteristics ,in The target shadow direction deviation is a feature that characterizes the difference between the pixel shadow direction and the uniform target direction. The larger the value, the more inconsistent the shadow direction is. Brightness gradient anomaly characteristics , which is the Manhattan distance of the change in the brightness gradient field before and after style transfer, reflects the degree of distortion of lighting details during the transfer process; Dynamic range local fidelity features ,in, For pixels The original brightness value, Physical highlight protection mask before style transfer The average brightness value of all pixels within the range. Dark detail preservation area before style transfer The average brightness value of all pixels within the protected area. This feature measures the degree to which the brightness deviates from the original typical value of highlights or shadows within the protected area. A positive value indicates that highlights are suppressed or shadows are brightened, while a negative value indicates the opposite trend. Emotional illumination coupling consistency features ,in The direction of the main light source in the current frame. The target light source direction is determined by the degree of matching between the emotional category and the light source direction adjustment amount. This measure measures the coordination between the current lighting state and the emotional semantics. For warm and bright emotions, it is desirable for the light source direction to have small differences, while for gloomy and dim emotions, it is desirable for the light source direction to have large differences to enhance the dramatic effect. Constructing a semantic illumination joint degradation vector The local lighting distortion feature vector is concatenated with the global features in the semantic lighting coupling description set: ; Semantic illumination joint degradation vector for all pixels in the entire image Cluster analysis was performed using the density-based clustering algorithm DBSCAN (neighborhood radius). Minimum number of samples The feature space is divided into several clusters; based on the mean of the feature vectors within each cluster, each cluster is classified to obtain the region classification result. If the average style intensity deviation of the cluster is greater than or equal to the preset highlight overexposure style threshold, and the average dynamic range local fidelity of the cluster is greater than or equal to the preset highlight overexposure brightness threshold, and the intersection ratio of the cluster pixels with the highlight protection area is greater than 50%, then it is determined to be a highlight overexposure degradation area. Otherwise, if the average style intensity deviation of the cluster is greater than or equal to the preset dark black style threshold, and the average dynamic range local fidelity of the cluster is less than or equal to the negative dark black brightness threshold (i.e. the average value is negative and the absolute value exceeds the threshold), and the intersection ratio of the cluster pixels with the dark detail preservation area is greater than 50%, then it is determined to be a dark black degradation area. Otherwise, if the average shadow direction deviation of the cluster is greater than or equal to the preset shadow direction conflict threshold, and (when the global emotion category is "warm" or "bright", the average emotion lighting coupling consistency of the cluster is less than or equal to the preset warm emotion consistency threshold, or when the global emotion category is "gloomy" or "dark", the average emotion lighting coupling consistency of the cluster is greater than or equal to the preset gloomy emotion consistency threshold), then it is determined to be a shadow direction conflict degradation zone; Otherwise, if the average value of the brightness gradient anomaly of the cluster is greater than or equal to the preset texture artifact threshold, and the average value of the style intensity deviation of the cluster is less than the highlight overexposure style threshold, and the average value of the shadow direction deviation of the cluster is less than the shadow direction conflict threshold, then it is determined to be a texture artifact degradation area. Otherwise, it is judged as a slightly distorted area; Finally, the classification feature set of degraded regions is obtained. ,in For the first A set of spatial pixels in a degradation region. Label the degradation type of this area (choose one of the five categories mentioned above). This is the mean of the feature vectors of this region.

[0025] In this embodiment of the invention, the spatial distribution of overexposed highlight areas and completely black shadow areas is extracted based on the degradation region classification feature set. The overexposed highlight areas and completely black shadow areas are then compared with preset physical highlight retention thresholds and shadow detail retention thresholds, respectively. The adaptive style transfer adjustment offset is calculated to dynamically correct the statistical feature mapping curve of the style normalization layer. Classification feature set of degraded regions Screening for degradation types A collection of regions where highlights are overexposed and degraded. The collection of areas with dark, degraded regions Extract the pixel spatial distribution of each region; For the overexposed and degraded highlight areas, calculate their spatial distribution feature vector. Number of pixels in the included area Regional area ratio Average brightness of the area and regional average style intensity deviation ; For the dark, degraded areas, calculate their spatial distribution feature vector. This includes the number of pixels in a region, the area ratio of the region, the average brightness of the region, and the average style intensity deviation of the region. The average brightness of the overexposed and degraded highlight areas Compared with the preset physical highlight retention threshold Compare and calculate the brightness deviation in the highlight area. The average brightness of the dark, degraded areas will be reduced. Compared to the preset threshold for preserving shadow details Compare and calculate the brightness deviation in the dark areas. ; Calculate the style transfer adjustment offset for the overexposed and degraded highlight region. Used to reduce the style intensity in highlight areas to restore the physical highlight texture: ,in The base step size for highlight adjustment is set to 0.25 (based on empirical values). The larger this offset is, the more severe the highlight overexposure degradation is, and the stronger the style transfer intensity in this area needs to be suppressed so that the pixel value returns to the original highlight value. Calculate the style transfer adjustment offset for dark and solid black areas. Used to reduce the style intensity in dark areas to restore detail in dark areas: ,in Set the base step size for shadow adjustment to 0.3 (shadow detail recovery usually requires stronger adjustment than highlights); the larger this offset, the more severe the degradation of black shadows, and the stronger the suppression of style transfer intensity in this area is needed to make the pixel value return to the original shadow value; Define the base adjustment offset for the intermediate adjustment range in the non-degenerate region. No adjustment is made to the style intensity; The final output is a pixel-level adaptive style transfer adjustment offset. Used to dynamically adjust style transfer strength weights : ; The corrected style transfer strength weight is .

[0026] This invention constructs a semantic-lighting coupling description set, integrating the dynamic range distribution features of images and the direction parameters of the main light source into the text's emotional semantics. This allows style transfer to move beyond relying solely on abstract emotional tags, establishing a precise perception of highlight anchor sets, shadow anchor sets, and light source direction. This fundamentally solves the technical deficiency of traditional methods in distinguishing between physical lighting and style texture. Based on this, the invention divides high dynamic range regions into physical highlight protection zones and shadow detail preservation zones according to the semantic-lighting coupling description set. A segmented constraint mechanism is introduced to extract highlight suppression factors and shadow enhancement factors. By calculating the dynamic range fidelity coefficient, proactive control over highlight overexposure and shadow blacking is achieved, effectively avoiding contrast loss and graying issues caused by forcibly compressing extreme dynamic range materials to the mean. Utilizing the light and shadow consistency constraint tensor and the transfer step size coefficient, a temporal correlation of light source direction is established between adjacent frames. This allows source materials under different lighting conditions to obtain explicit light and shadow logic constraints during the fusion process, overcoming the limitations of traditional methods that only perform color and texture mapping. This invention addresses the fundamental flaw of neglecting the consistency of light source direction in a one-size-fits-all approach. By calculating the geometric correction prediction quantity and the light and shadow direction uniformity index, it couples local brightness gradient changes with the light source direction migration function, quantifies and locates areas of light and shadow conflict, and unifies the light and shadow direction of the same character's face across different materials, eliminating the sense of incongruity that violates the laws of physical imaging. Based on the semantic lighting joint degradation vector constructed from the local lighting distortion feature set, it further realizes the refined classification and spatial positioning of various degradation types such as overexposed highlight areas, completely black shadow areas, and shadow direction conflict areas. Finally, by calculating the adaptive style transfer adjustment offset, it implements differentiated style intensity correction for different degradation types, so that the physical texture of the highlight area and the detail of the shadow area can be completely preserved. This invention starts from the dual dimensions of light intensity and direction, constructs a complete technical closed loop of perception-constraint-quantification-location-correction, and achieves the unification of the visual style of comics and TV series while fundamentally ensuring the authenticity and consistency of physical lighting logic, significantly improving the immersion and visual quality of the generated content.

[0027] This invention aims to address the technical problems of existing AI animation style unification methods, which suffer from suppressed highlights and loss of shadow details in extreme dynamic range materials due to the lack of physical lighting parameter modeling, as well as the resulting light and shadow conflicts when materials from different light source directions are blended. By constructing a semantic lighting coupling description set, it achieves accurate perception of highlight anchor points and light source directions. It combines dynamic range fidelity coefficients and lighting consistency constraint tensors to perform segmented constraints and temporal smoothing on style transfer intensity, and calculates adaptive adjustment offsets based on the degradation region classification feature set. This achieves visual style unification while fully preserving physical lighting logic and lighting consistency, thus balancing artistic style unity with physical realism.

[0028] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0029] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0030] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An AI-driven method for unifying the visual style of animated series by integrating text-driven and style transfer approaches, characterized by: Includes the following steps: The dynamic range distribution features and main light source direction parameters of each frame in the source image set are obtained. Highlight anchor point set and dark anchor point set are constructed by brightness histogram partitioning and gradient field analysis. Semantic lighting coupling description set is obtained by combining script text sentiment semantics. Based on the semantic lighting coupling description set, the source image is divided into a physical highlight protection area and a shadow detail preservation area. The style transfer intensity of the texture layer is segmented and constrained. The highlight suppression factor and shadow enhancement factor are extracted, and the dynamic range fidelity coefficient is calculated. Based on the dynamic range fidelity coefficient and the light source direction vector between adjacent frames, a light and shadow consistency constraint tensor is constructed to obtain the migration step size coefficient. Based on the migration step size coefficient and the local brightness change gradient, the geometric correction prediction of the light and shadow conflict area is calculated to obtain the light and shadow direction uniformity index. Based on the uniformity index of light and shadow direction and the distribution of style transfer intensity, a local light and shadow distortion feature set is extracted, and a semantic illumination joint degradation vector is constructed to obtain a degradation region classification feature set. Based on the degradation region classification feature set, the spatial distribution of overexposed highlight areas and black shadow areas is extracted, and the overexposed highlight areas and black shadow areas are compared with preset physical highlight retention thresholds and shadow detail retention thresholds, respectively, to calculate the adaptive style transfer adjustment offset.

2. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 1, characterized in that: The process of obtaining the semantic lighting coupling description set is as follows: The pixel brightness value sequence of each frame in the source image set is collected, and the brightness range is divided into highlight range, midtone range and dark range using an adaptive histogram partitioning method. The pixel percentage in each interval is statistically analyzed, the pixel sets in the highlight area and the pixel sets in the shadow area are extracted, and the average brightness of the highlight area and the average brightness of the shadow area are calculated to construct a dynamic range distribution feature vector. Gradient field analysis is performed on the source image to calculate the pixel gradient magnitude and gradient direction. The top K pixels with the largest pixel gradient magnitude are selected as edge candidate sets. Hough transform is used to detect straight lines in the edge candidate sets and extract the set of straight line segments in each frame. The gradient directions of each straight line segment are clustered, and the direction angle with the highest frequency is selected as the main light source direction parameter. The confidence of the light source direction is calculated in combination with the shadow area distribution of each frame to construct the main light source direction feature vector. The script text's emotional semantics are input into a pre-trained multimodal semantic encoder to extract emotional feature vectors; The dynamic range distribution feature vector, the main light source direction feature vector, and the emotion feature vector are fused in a multimodal manner to construct a semantic-lighting coupled description set.

3. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 1, characterized in that: The process of obtaining the dynamic range fidelity coefficient is as follows: Based on the highlight region pixel set, dark region pixel set, sentiment category and sentiment intensity scalar in the semantic lighting coupling description set, the highlight threshold and dark threshold are adaptively corrected for sentiment, and a physical highlight protection area mask and dark detail preservation area mask are generated. Based on the physical highlight protection area mask and shadow detail preservation area mask, image pixels are divided into three categories: highlight protection area, shadow detail preservation area and midtone area, and different style transfer intensity weights are assigned to each category. The highlight suppression factor and shadow enhancement factor were extracted, and the dynamic range fidelity coefficient was calculated.

4. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 1, characterized in that: The process of obtaining the migration step size coefficient is as follows: Obtain the main light source direction parameters of adjacent frames, and the corresponding dynamic range fidelity coefficients; Define the light source direction deviation angle; Construct a lighting consistency constraint tensor and calculate the transition step size coefficient based on the lighting consistency constraint tensor.

5. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 4, characterized in that: The light and shadow consistency constraint tensor includes directional consistency weight, directional change penalty weight, and fidelity fusion weight.

6. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 1, characterized in that: The process of obtaining the uniformity index of light and shadow direction is as follows: Calculate the local brightness change gradient field based on the pixel brightness value sequence of each frame in the source image set, and extract the gradient direction; define the theoretical shadow direction based on the gradient direction; Obtain the direction of the main light source and the direction of the target light source in the current frame; calculate the deviation angle between the theoretical shadow direction and the current main light source direction in the current frame to obtain the current shadow direction deviation; simultaneously calculate the deviation angle between the theoretical shadow direction and the target light source direction to obtain the target shadow direction deviation. Construct a mask for areas of light and shadow conflict to identify such areas; For each pixel in the area of ​​light and shadow conflict, calculate the geometric correction prediction; Spatially aggregate the geometric correction predictions for all areas of light and shadow conflict to calculate the light and shadow direction uniformity index.

7. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 1, characterized in that: The process of obtaining the classification feature set of the degraded region is as follows: Obtain the uniformity index of light and shadow direction and the distribution matrix of style transfer intensity; at the same time, obtain the sentiment category, sentiment intensity, and physical highlight protection area mask and shadow detail preservation area mask in the semantic lighting coupling description set; For each pixel in the image, extract the local lighting distortion feature vector; Construct a semantic illumination joint degradation vector; Cluster analysis is performed on the semantic illumination joint degradation vectors of all pixels in the entire image. The density-based clustering algorithm DBSCAN is used to divide the pixels in the feature space into several clusters. Each cluster is classified according to the mean of the feature vectors within each cluster to obtain the region classification results. Finally, the degradation region classification feature set is obtained.

8. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 7, characterized in that: The local lighting distortion feature vector includes style intensity deviation features, shadow direction deviation features, brightness gradient anomaly features, dynamic range local fidelity features, and emotional lighting coupling consistency features.

9. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 7, characterized in that: The region classification results include overexposed highlight degradation areas, completely black shadow degradation areas, shadow direction conflict degradation areas, texture artifact degradation areas, and slightly distorted areas.

10. The AI ​​animation visual style unification processing method integrating text-driven and style transfer as described in claim 1, characterized in that: The process of obtaining the adaptive style transfer adjustment offset is as follows: From the set of degradation region classification features, regions with degradation type of highlight overexposure degradation area and region with dark black degradation area are selected; For the overexposed and degraded highlight areas, calculate their spatial distribution feature vector, which includes the number of pixels in the area, the area ratio of the area, the average brightness of the area, and the average style intensity deviation of the area. For the dark, degraded black areas, calculate their spatial distribution feature vector, which includes the number of pixels in the region, the area ratio of the region, the average brightness of the region, and the average style intensity deviation of the region. The average brightness of the overexposed and degraded highlight areas is compared with the preset physical highlight retention threshold to calculate the brightness deviation of the highlight areas; the average brightness of the completely black and degraded shadow areas is compared with the preset shadow detail retention threshold to calculate the brightness deviation of the shadow areas. Calculate the style transfer adjustment offset for the overexposed and degraded highlight areas; Calculate the style transfer adjustment offset for dark, solid black areas; For the mid-tone areas in non-degraded regions, define a basic adjustment offset and do not adjust the style intensity; Generate pixel-level adaptive style transfer adjustment offsets to dynamically correct style transfer intensity weights.

Images