A method and system for forming a new image based on batch switching background of a convolutional neural network

Abstract

This invention relates to the field of image processing technology and discloses a method and system for batch switching of backgrounds to form new images based on convolutional neural networks. The method includes: analyzing a cartoon image to obtain a foreground segmentation mask and a foreground material feature tensor; analyzing a target background image to obtain a background illumination feature vector; simulating the dynamic light and shadow interaction between the cartoon foreground and the new background by constructing a dual-medium light interaction field; and achieving fine-grained fusion of edge regions by combining an edge transmission matrix. This solves the visual fragmentation problem caused by the lack of light and shadow interaction in traditional convolutional neural network background switching. At the same time, the illumination consistency is further optimized through a light and shadow residual map correction mechanism, significantly improving the fusion and adaptability of cartoon images with various real or virtual scenes. Furthermore, the batch processing method can efficiently adapt to diverse marketing scenarios, meeting batch replacement needs while ensuring visual naturalness.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and system for generating new images by batch switching of backgrounds based on convolutional neural networks. Background Technology

[0002] In the advertising industry, to adapt to diverse marketing scenarios, it is often necessary to replace the backgrounds of brand cartoon characters in multiple scenarios (such as parks, cities, homes, etc.), and to achieve integration and adaptation with various real or virtual scenarios through systematic batch replacement.

[0003] Currently, background switching technology has formed a core technology system based on deep learning convolutional neural networks, and is gradually integrating generative adversarial networks, attention mechanisms, and Transformer architecture. The mainstream technical path is usually divided into three stages: first, using semantic segmentation networks to achieve pixel-level separation of the image foreground and background; second, generating a background region that matches the style of the foreground based on the target background features; and finally, completing the fine-grained edge alignment and illumination consistency correction of the foreground and new background through a fusion network, outputting a naturally fused target image, thereby achieving background replacement.

[0004] However, current convolutional neural network-based solutions for replacing backgrounds of cartoon characters still have significant technical flaws. Specifically, when switching backgrounds, convolutional neural network technology often assumes that the cartoon character and the ambient lighting of the new background maintain a static match. However, in reality, many cartoon characters and backgrounds have a light and shadow blending effect. But when switching backgrounds, this light and shadow interaction is often overlooked, resulting in a noticeable visual disconnect between the cartoon character and the new background, leading to a poor visual effect of the new image. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for batch switching of backgrounds to form new images based on convolutional neural networks, thus solving the aforementioned problems.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0007] A method for batch switching of backgrounds to form new images based on convolutional neural networks includes:

[0008] S1: Analyze the cartoon image to obtain the foreground segmentation mask and foreground material feature tensor; analyze the target background image to obtain the background illumination feature vector.

[0009] S2: The foreground material feature tensor and the background illumination feature vector are fused and calculated to generate a dual-medium light interaction field;

[0010] S3: Combine the analysis of the dual-medium light interaction field and the foreground segmentation mask to obtain the edge transmission matrix;

[0011] S4: Perform initial background switching between the target background image and the foreground segmentation mask to obtain the initial switching image;

[0012] S5: Based on the dual-medium light interaction field, the foreground region of the initial switching image is illuminated and then fused with the background region of the initial switching image using the edge transmission matrix to generate a primary composite image.

[0013] S6: Calculate the gradient difference map between the primary composite image and the target background image, and correct the gradient difference map to obtain the light and shadow residual map;

[0014] S7: Overlay the light and shadow residual map with the primary composite map to generate the final new map.

[0015] Furthermore, the cartoon image is analyzed to obtain the foreground segmentation mask and the foreground material feature tensor, including:

[0016] The cartoon images were analyzed to obtain multi-scale feature maps;

[0017] Calculate the foreground attribution probability based on the multi-scale feature map and generate a foreground probability distribution map;

[0018] Material properties are aggregated from multi-scale feature maps to obtain the original tensor of the foreground material;

[0019] The foreground probability distribution map is processed to obtain the foreground segmentation mask;

[0020] The original tensor of the foreground material is analyzed to obtain the feature tensor of the foreground material.

[0021] Furthermore, the target background image is analyzed to obtain the background illumination feature vector, including:

[0022] The target background image is analyzed to obtain a shallow feature map of background illumination.

[0023] The shallow feature map of background illumination is calculated to determine the illumination direction weight matrix and the regional illumination intensity coefficient;

[0024] The background illumination shallow feature map, illumination direction weight matrix and regional illumination intensity coefficient are fused and dimensionally reduced to generate a background illumination feature vector.

[0025] Furthermore, the foreground material feature tensor and the background illumination feature vector are fused and calculated to generate a dual-medium light interaction field, including:

[0026] Align the foreground material feature tensor with the background lighting feature vector to generate a feature matching matrix;

[0027] Based on the feature matching matrix, the interaction parameters between the foreground material feature tensor and the background illumination feature vector are calculated to obtain the light-material interaction tensor.

[0028] The interaction tensor of optical materials is analyzed to generate a two-medium optical interaction field.

[0029] Furthermore, by combining the analysis of the dual-medium optical interaction field and the foreground segmentation mask, the edge transmission matrix is ​​obtained, including:

[0030] The foreground segmentation mask is analyzed to determine the boundary region between the foreground and the background, and an edge region mask is generated.

[0031] Based on edge region masking, optical interaction features at the edge are extracted from the dual-medium optical interaction field to obtain the edge optical interaction subfield;

[0032] Based on the material properties of the edge region in the foreground material feature tensor, the transmission attenuation of the edge light interaction subfield is calculated to obtain the edge transmission attenuation coefficient.

[0033] The edge transmission attenuation coefficient and the edge light interaction subfield are spatially matched based on the edge region mask to generate the edge transmission matrix.

[0034] Furthermore, an initial background switching is performed on the target background image and the foreground segmentation mask to obtain an initial switching map, including:

[0035] Edge analysis is performed on the edge region mask to obtain an edge-optimized mask;

[0036] The target background image is weighted based on an edge optimization mask, and background region adaptation weights are generated according to the position distribution of the foreground in the image.

[0037] Based on the edge optimization mask and the background region adaptation weight, the target background image is filtered to obtain the adapted background image;

[0038] The foreground area of ​​the cartoon image is replaced and blended with the adapted background image using an edge-optimized mask to generate the initial transition image.

[0039] Furthermore, based on the dual-medium light interaction field, illumination modulation is applied to the foreground region of the initial switching map, and the background region of the initial switching map is fused with the edge transmission matrix to generate a primary composite map, including:

[0040] The foreground illumination modulation matrix is ​​obtained by extracting the dual-medium light interaction field.

[0041] The foreground region of the initial switching map is illuminated according to the foreground illumination modulation matrix to generate an illuminated foreground region map;

[0042] The background fusion weights are determined based on the edge transmission matrix;

[0043] The background region of the initial switching image is optimized based on the background fusion weights to obtain the optimized background region image;

[0044] The boundary transition coefficient is calculated by analyzing the foreground and background region maps.

[0045] Based on the boundary transition coefficient, the foreground region map and the background region map are stitched together to generate a primary composite image.

[0046] Furthermore, the gradient difference map between the primary composite image and the target background image is calculated, and the gradient difference map is corrected to obtain the lighting residual map, including:

[0047] Calculate the pixel gradient difference between the initial composite image and the target background image to generate an initial gradient difference map;

[0048] The initial gradient difference map is extracted to obtain the gradient anomaly weight map;

[0049] The initial gradient difference map is corrected based on the gradient anomaly weight map, generating a corrected gradient difference map, and the error compensation coefficients in the correction process are obtained at the same time.

[0050] The modified gradient difference map is subjected to residual enhancement processing based on the error compensation coefficient to generate a light and shadow residual map.

[0051] Furthermore, the residual image is overlaid with the primary composite image to generate the final new image, including:

[0052] Perform a region fit analysis on the light and shadow residual map and the primary composite map to generate residual fit coefficients;

[0053] Based on the residual adaptation coefficient, the light and shadow residual map is dynamically adjusted in a region to generate a weighted residual map;

[0054] The weighted residual map is merged with the primary composite map while maintaining color consistency between the two to generate the final new map.

[0055] Furthermore, a system for batch switching of backgrounds to form new images based on convolutional neural networks, applied to any of the above-described methods for batch switching of backgrounds to form new images based on convolutional neural networks, includes:

[0056] The bidirectional analysis unit is used to analyze cartoon images to obtain foreground segmentation masks and foreground material feature tensors, and to analyze target background images to obtain background illumination feature vectors.

[0057] The fusion unit is used to fuse and calculate the foreground material feature tensor and the background illumination feature vector to generate a dual-medium light interaction field.

[0058] Combined analysis unit is used to combine the dual-medium light interaction field and the foreground segmentation mask for analysis to obtain the edge transmission matrix;

[0059] The switching unit is used to perform initial background switching between the target background image and the foreground segmentation mask to obtain an initial switching image.

[0060] The modulation unit is used to modulate the illumination of the foreground region of the initial switching image according to the dual-medium light interaction field, and to fuse the background region of the initial switching image with the edge transmission matrix to generate a primary composite image.

[0061] The correction unit is used to calculate the gradient difference map between the primary composite image and the target background image, and to correct the gradient difference map to obtain the light and shadow residual map.

[0062] The overlay unit is used to overlay the light and shadow residual map with the primary composite map to generate the final new map.

[0063] In summary, the present invention has the following main beneficial effects:

[0064] By constructing a dual-medium light interaction field, the core problem of insufficient fusion of cartoon characters and new background lighting is effectively solved. This solution extracts the foreground material feature tensor and the background lighting feature vector to simulate the dynamic light and shadow interaction between the two at different medium interfaces. This allows the highlights and shadows at the edges of the cartoon character to change naturally with the lighting conditions of the new background. For example, when the target background is a backlit scene, the system will automatically calculate the transmission attenuation coefficient of the cartoon character's edge to generate a semi-transparent light and shadow effect that conforms to physical laws. This avoids the visual disjointedness caused by switching in traditional technologies. This dynamic light and shadow interaction method improves the fusion of cartoon characters with various real or virtual scenes and enhances the visual realism of the new image.

[0065] By calculating the gradient difference between the initial composite image and the target background and performing targeted optimization, a deep adaptation of the foreground and background at the level of detail is achieved. Traditional convolutional neural networks are prone to problems such as "hard edges" or "excessive blurring" when processing edges. However, this solution uses dual correction of edge transmission matrix and light and shadow residual map to preserve the line features of the cartoon character and make the edge area transition naturally with the background lighting. This refined processing enables the batch-generated advertising materials to reach a new level of visual consistency, reduces the cost of manual post-production adjustments, and improves the efficiency of systematic batch production.

[0066] By combining multi-scale feature analysis and material attribute aggregation, this solution achieves adaptive matching of cartoon characters in diverse scenarios, significantly expanding the application boundaries of batch background replacement. Traditional solutions often suffer from style inconsistencies due to incomplete extraction of lighting features when facing complex backgrounds (such as urban night scenes and soft indoor lighting). However, this solution, through dimensionality reduction and fusion of background lighting feature vectors, can accurately capture the core lighting features of different scenarios and generate an adaptation scheme by combining the reflectivity and transparency of foreground materials. When switching backgrounds, it can automatically adjust the lighting intensity and color tendency of the cartoon character, ensuring that it maintains brand recognition in various marketing scenarios while forming a harmonious and unified visual style with the background. This enables rapid batch replacement of scenes, improves the production efficiency of marketing materials, and ensures that the visual effect of the new image is consistent with reality. Attached Figure Description

[0067] Figure 1 This is a flowchart of the method steps for forming new images by batch switching of backgrounds based on convolutional neural networks according to the present invention;

[0068] Figure 2 This is a schematic diagram of the system for forming new images by batch switching of backgrounds based on convolutional neural networks according to the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] refer to Figure 1 and Figure 2 A method for generating new images by batch switching of backgrounds based on convolutional neural networks includes:

[0071] The cartoon image is analyzed to obtain the foreground segmentation mask and the foreground material feature tensor. The target background image is analyzed to obtain the background lighting feature vector. The cartoon image represents the original image of the cartoon-style subject that needs to be retained, and the target background image represents the new background image that is planned to be replaced in the cartoon image.

[0072] The foreground material feature tensor and the background illumination feature vector are fused and calculated to generate a dual-medium light interaction field;

[0073] By combining the analysis of the dual-medium optical interaction field and the foreground segmentation mask, the edge transmission matrix is ​​obtained;

[0074] An initial background switching is performed on the target background image and the foreground segmentation mask to obtain an initial switching image;

[0075] Based on the dual-medium light interaction field, the foreground region of the initial switching image is illuminated and then fused with the background region of the initial switching image using the edge transmission matrix to generate a primary composite image.

[0076] Calculate the gradient difference map between the primary composite image and the target background image, and correct the gradient difference map to obtain the light and shadow residual map;

[0077] The residual image of light and shadow is overlaid with the primary composite image to generate the final new image.

[0078] By generating a dual-media light interaction field, edge transmission matrix, and light and shadow residual map, the foreground lighting modulation and background area are naturally integrated, avoiding the visual disjointedness caused by ignoring light and shadow interaction. At the same time, it can efficiently adapt to the diverse marketing scenarios in the advertising industry, quickly generate new images that are integrated and adapted to various real or virtual scenes, significantly improve the visual effect and efficiency of background switching, and meet the application needs of brand cartoon characters in multiple scenarios.

[0079] In one embodiment, the cartoon image is analyzed to obtain a foreground segmentation mask and a foreground material feature tensor, including:

[0080] The analysis of cartoon images yields multi-scale feature maps. Specifically, the process involves: inputting the cartoon image into a convolutional neural network (CNN). First, the image passes through a first convolutional layer using multiple 3×3 kernels for sliding convolution. This convolution operation element-wise multiplies the cartoon image with the kernels and sums the results, generating multiple feature response values. These response values ​​are arranged in a two-dimensional array, and the values ​​at different positions in the array are mapped to different pixel brightness (or color), ultimately forming a shallow feature map. Next, max pooling is performed on the shallow feature map, dividing it into non-overlapping 2×2 regions and retaining the maximum value within each region, resulting in a pooled feature map. Finally, the pooled feature map is input into a middle convolutional layer, where mixed-size kernels (3×3 and 5×5) are used for parallel processing. Different kernel sizes capture different scales of data. The deep convolutional layer uses a multi-scale feature map, where small convolutional kernels (3×3) focus on local details in the pooled feature map, such as texture and color variations, while large convolutional kernels (5×5) capture broader contextual information, such as object outlines and relationships. Small and large convolutional kernels generate independent feature maps, which are then concatenated to create a mid-level feature map. This mid-level feature map is input into a deep convolutional layer using larger convolutional kernels (7×7 or global average pooling) to extract highly abstract global features, such as character poses and scene layouts. Then, 1×1 convolutions compress the feature dimensions, reducing the number of parameters and integrating cross-channel information. The features are then combined by concatenating them along the channel dimension to obtain the deep feature map. Finally, the shallow, mid-level, and deep feature maps are concatenated along the channel dimension to form a multi-scale feature map.

[0081] The process involves calculating the foreground classification probability based on the multi-scale feature map and generating a foreground probability distribution map. Specifically, the multi-scale feature map is input into a decoder network based on a U-Net architecture. The decoder network gradually restores the spatial resolution of the multi-scale feature map to the original cartoon image size through deconvolution layers and upsampling operations. Then, a 1×1 convolution kernel is used to convolve the restored multi-scale feature map, mapping its channels to two, corresponding to the foreground and background categories respectively. At this point, each spatial location in the multi-scale feature map generates an array containing two values. This array is then processed as follows: the exponent values ​​of these two values ​​are calculated, and then each exponent... Divide the value by the sum of the two exponential values ​​to obtain two values ​​between 0 and 1. The sum of these two values ​​is 1, which represents the probability that the pixel belongs to the foreground and the background, respectively. Finally, based on the preset category mapping relationship of the two values ​​in the channel dimension (setting the first channel as the background probability and the second channel as the foreground probability), the value representing the foreground probability at each spatial location is extracted. These values ​​are arranged one by one according to their spatial location in the original cartoon image to form a single-channel two-dimensional array with the same size as the original cartoon image. Then, by mapping the foreground probability values ​​at different positions in the array to the grayscale (or color) values ​​of the pixels, the foreground probability distribution map is finally generated.

[0082] Material attribute aggregation is performed on multi-scale feature maps to obtain the original foreground material tensor. Specifically, this involves: first, determining the foreground region as the pixel range with a probability value greater than 0.5 based on the foreground probability distribution map; then, cropping the features of the foreground region in the multi-scale feature map, retaining only the multi-scale features within the foreground region (including the foreground portions of shallow, medium, and deep feature maps); next, separating three levels of feature maps from the cropped multi-scale features: shallow feature maps containing texture details; medium feature maps containing color distribution; and deep feature maps containing material reflectivity. Regions are divided according to material attributes, where color attributes correspond to the foreground region of the medium feature map, texture attributes to the foreground region of the shallow feature map, and reflectivity attributes to the foreground region of the deep feature map. Finally, corresponding attribute features are extracted from each region, and RGB channel values ​​are extracted from the color attribute region. The local variance is calculated from the texture attribute region, and the high-frequency components are analyzed from the reflective attribute region. Specifically, the part corresponding to the foreground region in the deep feature map is selected, and the feature map is blurred using Gaussian filtering to obtain a smoothed feature map. Then, the smoothed feature map is subtracted from the foreground region part of the deep feature map, and the difference is the high-frequency component. Then, the probability value of each pixel in the foreground probability distribution map is used as a weight to perform a weighted average of the feature values ​​in each attribute region (for example, if the foreground probability of a pixel is 0.8, its feature value is 0.8 (weight), and the probability of background pixels is 0 and they are not included in the calculation). This operation is performed on the color, texture, and reflective attributes respectively to obtain the color aggregation vector, texture aggregation vector, and reflective aggregation vector. The color aggregation vector, texture aggregation vector, and reflective aggregation vector are concatenated according to the channel dimension to form a three-dimensional array containing complete material information of the foreground region, that is, the original tensor of the foreground material.

[0083] The foreground probability distribution map is processed to obtain a foreground segmentation mask. Specifically, the foreground probability distribution map is binarized pixel by pixel. Pixels with a probability value greater than 0.5 are identified as foreground and assigned a value of 255 (white), while pixels with a probability value less than or equal to 0.5 are identified as background and assigned a value of 0 (black), forming an initial binary mask. The initial binary mask is then subjected to an erosion operation, specifically: a structuring element (a 3×3 square pixel block) is used to traverse the initial binary mask. If the structuring element completely overlaps with the foreground region, the pixel is retained as foreground; otherwise, it is set as background. A dilation operation is then performed, specifically: a structuring element is used to traverse the initial binary mask. If the structuring element intersects with the foreground region, the pixel is set as foreground. Gaussian filtering is then applied to the initial binary mask after the erosion and dilation operations to blur the edge regions, making the transition between the foreground and background more natural, thus obtaining the final foreground segmentation mask.

[0084] The original foreground material tensor is analyzed to obtain the foreground material feature tensor. Specifically, the original foreground material tensor is separated into three attribute channels: color, texture, and reflection, resulting in three independent feature tensors. Each separated feature tensor is standardized by calculating the average of all feature values ​​in that channel, subtracting the average from each feature value to obtain the difference, calculating the standard deviation of these differences, and finally dividing each difference by the standard deviation to eliminate the dimensional differences between different attributes. A multilayer perceptron is used to process the standardized feature tensors. A fully connected layer performs nonlinear transformation on the features. In this process, neurons accumulate the input feature tensors according to trained weights, and then perform nonlinear transformation according to set rules (outputting 0 when the input value is less than 0; retaining the value directly when the input value is greater than 0), thereby extracting core material attribute features such as color saturation, texture coarseness, and reflection intensity. Finally, the features of each channel processed by the multilayer perceptron are concatenated together according to the original channel dimensions to form a new tensor, which is the foreground material feature tensor.

[0085] By accurately extracting foreground segmentation masks and material features through multi-scale feature maps, and combining foreground probability distribution to achieve material attribute aggregation and standardization, the texture, color, reflection and other features of the cartoon character can be accurately preserved. At the same time, based on the probability division of foreground and background, after erosion, dilation and Gaussian filtering, the edges of the foreground segmentation mask are made more natural, effectively avoiding the fusion and fragmentation caused by the rough foreground extraction, and improving the initial fit between the cartoon character and the new background.

[0086] By generating a dual-medium light interaction field and an edge transmission matrix, the dynamic fusion of foreground lighting modulation and background region is achieved. Then, by combining the light and shadow residual map obtained by gradient difference map correction, the synthesis effect is further optimized. The light and shadow interaction between the cartoon character and the new background is fully considered, which solves the visual disjointness problem caused by static lighting matching. Moreover, the batch processing capability is adapted to the needs of multiple scenarios in the advertising industry, which can efficiently generate naturally blended new images and significantly improve the visual effect and efficiency of background replacement.

[0087] In one embodiment, the target background image is analyzed to obtain a background illumination feature vector, including:

[0088] The target background image is analyzed to obtain a shallow feature map of background illumination. Specifically, this involves: inputting the target background image into a convolutional neural network; generating a shallow feature map through sliding convolution using 3×3 kernels; reducing the dimensionality of the shallow feature map using 2×2 max pooling to enhance robustness to illumination changes; processing the shallow feature map in parallel using 5×5 and 7×7 kernels to extract local illumination patterns and global illumination distribution, respectively, resulting in feature maps capturing local illumination patterns (e.g., point light projection) and global illumination distribution (e.g., ambient light direction); concatenating the two feature maps and normalizing them; uniformly dividing the normalized feature map into multiple 8×8 pixel cells; calculating the gradient (i.e., the rate of change of pixel brightness in the horizontal and vertical directions) of all pixels in each cell; and then summing the gradient values ​​in each of the eight directions of each pixel's gradient as the weight for that direction, forming an 8-dimensional orientation histogram for that cell; and then... Using a 16×16 pixel "block" as the unit, the histogram of all cells within the block is normalized to weaken the impact of overall brightness changes on features, enhance contrast under different lighting conditions, and ultimately generate a direction-sensitive lighting feature map that reflects the direction of illumination. Simultaneously, the color space of the target background image is converted from RGB to HSV (i.e., decomposed into three components: hue (H), saturation (S), and brightness (V)). For the brightness component V, a 1×1 convolution kernel is used: the 1×1 convolution kernel integrates the channel information of the brightness component V pixel by pixel to generate the corresponding feature map. This feature map is used as the main illumination intensity feature map. A Fourier transform is performed on the hue component H: the spatial distribution of the hue component H (e.g., the distribution of red and blue areas in the image) is converted into frequency information to identify periodic lighting patterns, resulting in a periodic lighting pattern feature map. The direction-sensitive lighting feature map, the main illumination intensity feature map, and the periodic lighting pattern feature map are then concatenated to obtain the shallow background illumination feature map.

[0089] The background illumination shallow feature map is calculated to determine the illumination direction weight matrix and regional illumination intensity coefficients. The illumination direction weight matrix represents the directional distribution of light sources in the background, and the regional illumination intensity coefficients reflect the differences in illumination intensity in different regions. Specifically, this involves: extracting all normalized 16×16 pixel blocks from the direction-sensitive illumination feature map of the background illumination shallow feature map. Each block contains four 8×8 cells. For each 8×8 cell's 8D orientation histogram, a weight is assigned to each direction in the 8D orientation histogram, and the weights of each direction are distributed... Multiply the value of the direction by the angle corresponding to that direction, and then add the results of these 8 products to get the direction value of this cell. Finally, arrange the direction values ​​of all cells in order according to their positions in the image to form the illumination direction weight matrix. Use the sliding window method for the main illumination intensity feature map. The window size is 16×16 pixels. Let the window start from the upper left corner of the image and slide to the right and down pixel by pixel in order until it covers the entire image. For all pixels in each window, calculate their average brightness. This average value is the regional illumination intensity coefficient of the area corresponding to the window.

[0090] The background illumination shallow feature map, illumination direction weight matrix, and regional illumination intensity coefficients are fused and dimensionality reduced to generate a background illumination feature vector. Specifically, this involves converting the direction-sensitive illumination feature map, main illumination intensity feature map, and periodic illumination pattern feature map from the background illumination shallow feature map into one-dimensional vectors with only one row each. These three one-dimensional vectors, along with the illumination direction weight matrix and regional illumination intensity coefficients, are then concatenated in sequence to form a high-dimensional vector. The correlation between every two dimensions in the high-dimensional vector is calculated (e.g., whether changes in dimension A and dimension B are synchronous, or whether changes in dimension C and dimension D are opposite). These correlations are then arranged into a square matrix according to a fixed rule, where the number of rows and columns of the matrix equals the number of dimensions of the high-dimensional vector. This square matrix is ​​the covariance matrix. The eigenvalues ​​and corresponding eigenvectors of the covariance matrix are then calculated. Eigenvectors are "directions" in a high-dimensional space, while eigenvalues ​​represent the fluctuation range of data in the corresponding eigenvector direction (the larger the eigenvalue, the more obvious the change in the data in that direction and the more information it contains). These eigenvectors are the principal components of the covariance matrix. The eigenvector corresponding to the largest eigenvalue is the first principal component (the direction of the most significant data change), the one corresponding to the second largest eigenvalue is the second principal component (the direction of the second most significant data change), and so on. Then, the eigenvalues ​​corresponding to these principal components are counted, sorted from largest to smallest, and then accumulated one by one. The proportion of the sum of the accumulated eigenvalues ​​to the total sum of all eigenvalues ​​is calculated until this proportion exceeds 95% (meaning that these selected principal components have contained more than 95% of the key information in the original high-dimensional vector). Finally, these selected principal components are combined to obtain the background illumination eigenvector.

[0091] By employing multi-scale convolution and orientation histogram analysis, the illumination features of the target background are accurately extracted. By combining multiple convolutional kernels with max pooling, the distribution of local point light sources and global ambient light is effectively captured. After HSV color space conversion and Fourier transform, the illumination intensity, direction, and periodicity patterns are separated. After normalization and gradient statistics processing, the generated background illumination feature vector can comprehensively reflect key information such as the direction and intensity of the light source, thus solving the adaptation deviation problem caused by the coarse background illumination feature extraction in traditional solutions.

[0092] By fusing shallow background lighting features, directional weight matrix, and intensity coefficients, the resulting dimensionality-reduced lighting feature vector can be dynamically matched with foreground material features. Combined with dual-medium light interaction field calculation, it enables coordinated lighting adjustment between the cartoon character and the new background, compensating for the shortcomings of static lighting matching. It can also adapt to various real and virtual scenes, providing stable lighting parameters for batch generation of naturally blended advertising images, significantly reducing the visual disconnect between the cartoon character and the new background, and improving the production quality and efficiency of multi-scene marketing materials.

[0093] In one embodiment, the foreground material feature tensor and the background illumination feature vector are fused and calculated to generate a dual-medium light interaction field, including:

[0094] Aligning the foreground material feature tensor with the background lighting feature vector to generate a feature matching matrix involves: flattening the foreground material feature tensor into a one-dimensional vector, making it consistent with the dimension of the background lighting feature vector; calculating the similarity of each corresponding element in the two vectors: first, calculating the product of the corresponding elements of the foreground material feature tensor and the background lighting feature vector, adding the products together to obtain the sum; then, calculating the square root of the sum of squares of each element of the foreground material feature tensor and the background lighting feature vector, dividing the sum by the product of the two square roots to obtain the matching value of each element pair; and arranging all the matching values ​​in the original element order to form a feature matching matrix.

[0095] Based on the feature matching matrix, the interaction parameters between the foreground material feature tensor and the background lighting feature vector are calculated to obtain the light material interaction tensor. Specifically, the matching value at each position in the feature matching matrix is ​​used as a weight, and multiplied by the corresponding elements in the foreground material feature tensor and the background lighting feature vector respectively. The two product results are then added together to obtain the interaction parameters. All parameters are arranged in their original positions to form the light material interaction tensor.

[0096] The optical-material interaction tensor is analyzed to generate a two-medium optical interaction field. Specifically, the optical-material interaction tensor is decomposed according to the spatial dimension to obtain the interaction parameters corresponding to each pixel position. For each interaction parameter, the value of the interaction parameter is multiplied by the attenuation coefficient of the adjacent pixel parameter (determined by the spatial distance between the adjacent pixel and the current pixel; the closer the distance, the stronger the association, and the larger the coefficient; the farther the distance, the weaker the association, and the smaller the coefficient). These multiplication results are then added together to obtain the energy value at this position. After calculating the energy values ​​of all pixel positions in the same way, these energy values ​​are arranged in the original spatial position order to obtain the two-medium optical interaction field.

[0097] By using feature alignment and interactive computation, the foreground material tensor and background lighting vector are flattened and matched. The feature matching matrix generated by similarity calculation accurately quantifies the degree of fit between the two. The light-material interaction tensor generated with the matching value as weight effectively captures the interaction between material and lighting. The dual-medium light interaction field generated by spatial dimension splitting and attenuation coefficient weighting can realistically simulate the light and shadow energy transfer at different pixel positions, breaking through the limitations of static lighting matching. This makes the light and shadow interaction between cartoon characters and new backgrounds more in line with physical laws, significantly reducing the visual disharmony caused by light and shadow fragmentation, and ensuring the naturalness and realism of batch synthesized images.

[0098] In one embodiment, the dual-medium light interaction field and the foreground segmentation mask are combined for analysis to obtain the edge transmission matrix, including:

[0099] The foreground segmentation mask is analyzed to determine the boundary region between the foreground and background, generating an edge region mask. This edge region mask accurately marks the pixel positions of the foreground edges. Specifically, this involves: smoothing the foreground segmentation mask; calculating the gradient of the smoothed foreground segmentation mask; and performing convolutions using the Sobel operator in both the horizontal and vertical directions. The horizontal convolution kernel is... The vertical convolution kernel is The horizontal and vertical convolution results are squared separately, then the two squared results are added together, and finally the square root of the sum is taken. After the calculation, each pixel will have a corresponding value. These values ​​are arranged in order of their original positions to form a complete gradient magnitude map. The gradient magnitude map is used to visually present the strength of gradient changes at each pixel in the image. The gradient direction angle of each pixel in the gradient magnitude map is calculated. After the gradient direction angles of all pixels in the gradient magnitude map have been calculated, these angle values ​​are arranged in order of their original positions to obtain... In the gradient direction map, a gradient magnitude threshold is set. Pixels with gradient magnitudes greater than the threshold are marked as candidate edge points. These candidate edge points are then arranged sequentially according to their original positions to generate an initial binary edge map. The gradient magnitudes of the current pixel and its two adjacent pixels are compared along the gradient direction. If the gradient magnitude of the current pixel is greater than the gradient magnitudes of its two adjacent pixels, it is considered a local maximum and is retained (marked as 1 in the refined binary map). If the gradient magnitude of the current pixel is less than or equal to the gradient magnitude of any adjacent pixel, then... If the local maximum condition is not met, the pixel is suppressed to 0 (marked as 0 in the thinned binary image). After all candidate edge points in the initial edge binary image have been calculated, the potentially wide edge regions in the initial edge binary image are compressed, retaining only the center pixel (local maximum point) of the gradient direction on each edge. The resulting binary image is the thinned edge binary image. High and low thresholds are set. Pixels in the thinned edge binary image with gradient magnitudes greater than the high threshold are marked as strong edge points, pixels with gradient magnitudes between the low and high thresholds are marked as weak edge points, and the remaining pixels are marked as background points. Strong edges... Edge connections are made between weak edge points and edge points. The pixels of all weak edge points are traversed. If there is a strong edge point in the 8-neighborhood of the pixel (i.e., the 8 adjacent pixels in the top, bottom, left, right and four diagonal directions around the pixel), it is marked as a valid edge point. Otherwise, it is suppressed as a background point. These valid edge points are marked as 1 (representing edge pixels) in the new binary image. The remaining pixels (weak edge points and background points that are not marked as valid edge points) are marked as 0. The resulting binary image is the edge region mask. The edge region mask only retains the real foreground edge pixels, and the edge width is at the single pixel level, which accurately corresponds to the boundary area between the foreground and the background.

[0100] Based on the edge region mask, the optical interaction features at the edge are extracted from the dual-medium optical interaction field to obtain the edge optical interaction subfield. Specifically, this includes: recording the row and column positions of the pixel coordinates of all pixels marked as valid edge points in the edge region mask; traversing each pixel in the dual-medium optical interaction field and checking whether the coordinates of the pixel are in the recorded valid edge point coordinates; if so, the energy value of the pixel is retained; if not, the energy value of the pixel is set to 0; and arranging all retained and set energy values ​​in order according to their original spatial positions in the dual-medium optical interaction field to form the edge optical interaction subfield.

[0101] Based on the material properties of the edge region in the foreground material feature tensor, the transmission attenuation of the edge light interaction subfield is calculated to obtain the edge transmission attenuation coefficient. Specifically, this includes: extracting the color saturation, texture density, and reflectivity attribute values ​​of the edge region from the foreground material feature tensor; multiplying the color saturation by 0.3, the texture density by 0.2, and the reflectivity by 0.5 for each pixel in the edge light interaction subfield; and then adding the three product results together. The sum is the edge transmission attenuation coefficient of that pixel.

[0102] The edge transmission attenuation coefficient and the edge light interaction subfield are spatially matched based on the edge region mask to generate an edge transmission matrix. Specifically, the process involves: using the edge region mask as a spatial reference, traversing each pixel in the image; for pixels marked as valid edge points in the mask, finding their corresponding attenuation coefficient in the edge transmission attenuation coefficient matrix and their corresponding energy value in the edge light interaction subfield; multiplying the corresponding attenuation coefficient and energy value to obtain the transmission value of the pixel; for pixels not marked as valid edge points in the edge region mask, their transmission values ​​are directly set to 0; and arranging the transmission values ​​of all pixels in their original spatial positions to form the edge transmission matrix.

[0103] By using Sobel operator gradient calculation and dual-threshold edge detection, a single-pixel-level edge region mask is generated, which can accurately determine the boundary region between the foreground and background. Compared with the wide edges and false edges that are prone to occur in traditional edge extraction, this application effectively preserves real edge pixels through smoothing, local maximum suppression and edge connection, providing accurate spatial positioning for subsequent light and shadow fusion. At the same time, the edge light interaction subfield extracted from the dual-medium light interaction field based on the mask can focus on the light and shadow energy transfer characteristics at the edge, reducing the fusion deviation caused by edge positioning ambiguity.

[0104] By generating an edge transmission matrix through spatial matching with the edge light interaction subfield, we can understand the light and shadow attenuation in the edge region, so that the light transmission between the edge of the cartoon character and the new background conforms to the material characteristics. This solves the problems of harsh edges and abrupt changes in light and shadow in traditional technologies. When processing in batches, it can quickly adapt to the edge blending requirements of different scenes, significantly reduce the visual disjointness, and ensure a natural transition effect between the cartoon character and the background of multiple scenes in the advertising image.

[0105] In one embodiment, an initial background switching is performed on the target background image and the foreground segmentation mask to obtain an initial switching image, including:

[0106] Edge analysis is performed on the edge region mask to obtain an edge optimization mask. Specifically, this involves: combining all candidate edge points marked in the edge region mask to obtain an initial edge point set; traversing the initial edge point set, checking each candidate edge point's eight adjacent pixels (top, bottom, left, right, and four corners); if adjacent pixels are candidate edge points, connecting them to form a continuous edge segment; candidate edge points are marked as 255 (white), and non-candidate edge points are marked as 0 (black); a 3×3 Gaussian kernel is used, with weights distributed Gaussianly at each position within the kernel. It is determined that the center pixel has the highest weight, which gradually decreases towards the periphery, and the sum of all weights is 1. The center of the Gaussian kernel is aligned with each pixel on the continuous edge segment, so that the kernel covers the pixel and its 8 neighboring pixels. The original value of each covered pixel is multiplied by the corresponding Gaussian kernel weight, and then all products are added together to obtain the smoothed value of the pixel. Each pixel on the continuous edge segment is processed in this way to make the value transition between the edge pixel and its neighboring pixels smoother. All the processed pixels are combined together to form an edge optimization mask.

[0107] Based on edge optimization mask, region weights are assigned to the target background image. According to the position distribution of the foreground in the image, background region adaptation weights are generated. Specifically, the continuous edge segments marked by the edge optimization mask are used as the foreground boundary. For each pixel in the target background image, the straight-line distance between it and all pixels on the foreground boundary is calculated. The minimum distance value is taken as the distance threshold. When the pixel distance is less than the threshold, the weight increases linearly from 0 to 1 as the distance increases. When the distance is greater than or equal to the threshold, the weight is fixed at 1. Thus, a weight between 0 and 1 is assigned to each pixel to form the background region adaptation weights.

[0108] Based on the edge optimization mask and background region adaptation weights, the target background image is filtered to obtain an adapted background image. Specifically, this involves: marking pixels with a value of 255 (white) in the edge optimization mask as foreground regions and pixels with a value of 0 (black) as background regions. Smoothed edge pixels in the edge optimization mask, with values ​​between 0 and 255, are still classified as foreground regions. For foreground regions, pixels at the corresponding positions in the target background image are directly discarded. For background regions, each pixel is processed one by one, and the background region adaptation weight (between 0 and 1) of that pixel is multiplied by the value of the corresponding pixel in the target background image. The result is the retained pixel value. These retained pixels are arranged and combined in order according to their original positions in the image to form the adapted background image.

[0109] The foreground region of the cartoon image and the adapted background image are replaced and merged according to the edge optimization mask to generate an initial switching image. Specifically, the pixels with a value of 255 (white) in the edge optimization mask correspond to the foreground interior region of the cartoon image, pixels with a value between 0 and 255 correspond to the foreground edge region of the cartoon image, and pixels with a value of 0 (black) correspond to the background region of the cartoon image. For all pixel positions with a value ≥1 in the edge optimization mask (i.e., the foreground interior region and the edge region), the original pixel information of the corresponding position in the cartoon image is directly retained. For pixel positions with a value of 0 in the edge optimization mask (i.e., the background region), the pixel information of the corresponding position in the adapted background image is replaced at that position. Finally, all the retained (foreground region) and replaced (background region) pixels are arranged in order according to their original spatial positions to form the initial switching image.

[0110] By using Gaussian smoothing to optimize the edge mask, the transition between edge pixels and neighboring pixels of the cartoon character is smoother, solving the problem of harsh edge processing in traditional methods. Based on the regional weight allocation of the edge optimization mask, the weight is dynamically adjusted according to the distance between the pixel and the foreground boundary, so that the pixels of the target background image near the foreground edge retain features that fit the foreground better. This reduces visual discontinuities caused by rough edge processing during the initial background switching and improves the initial fit between the background and foreground in batch replacement.

[0111] By using edge-optimized masks to clearly define the foreground and background areas, the original pixels in the foreground area are preserved, while the background area is replaced with pixels from the adapted background image. This achieves precise switching between the cartoon character and the new background. The adapted background image is generated by combining background area adaptation weights to ensure that the background is presented at the edges in accordance with distance rules. This allows the initial switching image to retain the original appearance of the cartoon character while allowing the new background to blend naturally. It is suitable for batch processing in multiple scenes, providing high-quality initial images for subsequent lighting modulation and edge blending. This effectively reduces the sense of disjointedness between the cartoon character and the new background and improves the visual harmony of the advertising image.

[0112] In one embodiment, illumination modulation is performed on the foreground region of the initial switching map based on the dual-medium light interaction field, and the background region of the initial switching map is fused with the edge transmission matrix to generate a primary composite map, including:

[0113] The foreground illumination modulation matrix is ​​obtained by extracting the dual-medium light interaction field. Specifically, the foreground region is determined by taking the continuous edge segments marked by the edge optimization mask as the foreground boundary, extracting the energy value of each pixel in the dual-medium light interaction field, multiplying the energy value of each pixel by the reflection intensity at the corresponding position in the foreground material feature tensor, and then dividing it by the square root of the sum of squares of the elements of the background illumination feature vector to obtain the modulation coefficient of the pixel, and arranging all the modulation coefficients according to their original spatial positions to form the foreground illumination modulation matrix.

[0114] The foreground region of the initial switching image is illuminated according to the foreground illumination modulation matrix to generate an illumination-modulated foreground region image. Specifically, this includes: determining the foreground region divided according to the edge optimization mask in the initial switching image; traversing all pixels in the foreground region; extracting the original color value of each pixel in the initial switching image; finding the modulation coefficient corresponding to the pixel in the foreground illumination modulation matrix; multiplying the two to obtain the illumination-adjusted pixel value; and arranging all pixels according to their original spatial coordinates after calculation to form the illumination-modulated foreground region image.

[0115] Based on the edge transmission matrix, the background fusion weight is determined. Specifically, this includes: traversing each pixel of the edge transmission matrix one by one, checking the value of the corresponding position of the pixel in the edge transmission matrix. If the value is not 0, the value is directly used as the background fusion weight of the pixel. If the value is 0, the background fusion weights of the 8 neighboring pixels around the pixel are found. The sum of these background fusion weights is calculated and divided by 8 to obtain the average value. The average value is used as the background fusion weight of the pixel.

[0116] The background region of the initial switching image is optimized based on the background fusion weight to obtain an optimized background region image. Specifically, the background region of the initial switching image is divided into multiple continuous pixel blocks of a fixed size. For each pixel block, the average value of the background fusion weight corresponding to all pixels in the block is calculated as the overall fusion weight of the block. The average value of all pixel values ​​of the corresponding pixel block in the adapted background image is calculated to obtain the average value of the adapted block. The overall fusion weight of the block is subtracted from 1 and then multiplied by the pixel value of each pixel block in the initial switching image. The average value of the adapted block is then multiplied by the overall fusion weight to obtain each optimized pixel. The optimized pixels of all pixel blocks are combined in their original positions to form the optimized background region image.

[0117] The transition coefficient is calculated by examining the foreground and background regions. Specifically, the transition coefficient is determined by using the continuous edge segments of the edge optimization mask as the boundary between the foreground and background, and extending 5 pixels to each side of the boundary as the transition zone. For each pixel within the transition zone, its straight-line distance to the boundary is calculated. The transition coefficient for pixels on the boundary is set to 1, and for pixels 5 pixels from the boundary, it is 0. For pixels in between, the transition coefficient is calculated by linearly decreasing the distance proportionally (first, the actual distance from the pixel to the boundary is calculated, then this distance is divided by 5 (the maximum width of the transition zone) to obtain a ratio between 0 and 1. Subtracting this ratio from 1 gives the transition coefficient for that pixel. For example, if a pixel is 2 pixels from the boundary, the ratio is 2 / 5 = 0.4, and the transition coefficient is 1 - 0.4 = 0.6). This process yields the transition coefficient for each pixel.

[0118] Based on the boundary transition coefficient, the foreground region map and the background region map are stitched together to generate a primary composite image. Specifically, for each pixel in the foreground region map and the background region map, if the pixel is in the transition zone, the boundary transition coefficient of the pixel is multiplied by the pixel value of the corresponding position in the foreground region map to obtain a first value. Then, the boundary transition coefficient is subtracted from 1 and multiplied by the pixel value of the corresponding position in the background region map to obtain a second value. The two values ​​are added together to obtain the value of the pixel. All pixels are arranged according to their original coordinates and combined into a complete image to form the primary composite image.

[0119] By generating a foreground lighting modulation matrix through a dual-medium light interaction field, and combining the reflectivity of the foreground material with the background lighting characteristics, the foreground area of ​​the initial switching image is dynamically adjusted, enabling the lighting effects of the cartoon character to interact with the new background. Compared to traditional static lighting matching, this modulation method makes the foreground brightness and reflection more consistent with the background ambient light characteristics, solving the visual disjointedness caused by lighting inconsistencies. Furthermore, it can quickly adapt to different scene lighting parameters during batch processing, providing cartoon characters in various marketing scenarios with scene-logical lighting presentations, ensuring consistent lighting between the foreground and background.

[0120] In one embodiment, the gradient difference map between the primary composite image and the target background image is calculated, and the gradient difference map is corrected to obtain a lighting residual map, including:

[0121] Calculate the pixel gradient difference between the primary composite image and the target background image to generate an initial gradient difference map. Specifically, this includes applying the Sobel operator (with a horizontal kernel of 1) to each pixel of both the primary composite image and the target background image. Vertical core is Convolution is performed in the horizontal and vertical directions respectively to obtain the horizontal and vertical gradient values. For the same pixel in the primary composite image and the target background image, the difference between the horizontal gradient value and the difference between the vertical gradient value are calculated. The two differences are squared and added together, and the square root is taken to obtain the gradient difference at that position. The gradient differences at all positions are arranged according to the original image position to form the initial gradient difference map.

[0122] The initial gradient difference map is extracted to obtain a gradient anomaly weight map. Specifically, this involves: calculating the average gradient difference of all pixels in the initial gradient difference map; setting 1.5 times the average as the gradient difference threshold; iterating through each pixel in the initial gradient difference map; if the gradient difference is greater than the threshold, marking it as an anomaly; otherwise, marking it as a normal point; for anomaly points, calculating the average gradient difference between it and its eight neighboring pixels (top, bottom, left, right, and four diagonal directions); dividing the average by the maximum gradient difference of the entire map to obtain the anomaly weight of the anomaly point; for pixels marked as normal points, setting their anomaly weight to 0; and arranging the anomaly weights of all pixels according to their original positions in the initial gradient difference map to obtain the gradient anomaly weight map.

[0123] The initial gradient difference map is corrected based on the gradient anomaly weight map to generate a corrected gradient difference map. At the same time, the error compensation coefficient in the correction process is obtained. Specifically, it includes: traversing each pixel in the initial gradient difference map; if it is an anomaly, multiply the gradient difference of the anomaly by (1 minus its anomaly weight) to obtain the corrected value; if it is a normal point, use its corresponding gradient difference as the corrected value; arrange the corrected values ​​of all pixels in their original positions to form the corrected gradient difference map; calculate the absolute difference of the gradient difference of the corresponding pixels in the corrected gradient difference map and the initial gradient difference map; add all the absolute differences to get the sum; divide the sum by the total number of pixels in the initial gradient difference map to obtain the error compensation coefficient.

[0124] The modified gradient difference map is subjected to residual enhancement processing based on the error compensation coefficient to generate a light and shadow residual map. Specifically, the process involves: traversing each pixel of the modified gradient difference map, multiplying the gradient difference value corresponding to the pixel by (1 + error compensation coefficient) to obtain the enhanced residual value. If the enhanced residual value exceeds 255, it is set to 255; if the enhanced residual value is lower than 0, it is set to 0; if the enhanced residual value is between 0 and 255, it remains unchanged. Then, all pixels are arranged in their original positions to form the light and shadow residual map.

[0125] By calculating and correcting gradient differences, the system accurately captures the light and shadow deviation between the initial composite image and the target background. The Sobel operator is used to extract gradient differences, and outlier points are corrected by combining outlier weights. Then, residual enhancement is used to generate a light and shadow residual map. This preserves the real light and shadow differences while avoiding distortion caused by over-correction. The residual enhancement is more in line with the light and shadow rules of the scene, effectively making up for the subtle light and shadow interactions that were not captured in the initial composite image. This further reduces the visual disjointedness and improves the fineness of the integration between the cartoon character and the background.

[0126] In one embodiment, the residual image of light and shadow is overlaid with the primary composite image to generate a final new image, including:

[0127] A region fit analysis is performed on the lighting residual map and the primary composite map to generate residual fit coefficients. Specifically, this involves: dividing the primary composite map into multiple non-overlapping regular sub-regions of a fixed size; calculating the standard deviation of pixel values ​​within each sub-region, using the standard deviation as the texture complexity index of that region; simultaneously calculating the average gradient magnitude of the corresponding position in the lighting residual map for that sub-region, using the average gradient magnitude as the lighting intensity index of that region; constructing color histograms for each sub-region of the primary composite map and its corresponding region in the target background image; obtaining an index reflecting the similarity of color distribution between the sub-region and the corresponding background region, i.e., the color matching index, by calculating the intersection of the two histograms; and weighting the texture complexity index, lighting intensity index, and color matching index by summing them with weight coefficients of 0.4, 0.4, and 0.2, respectively, and normalizing the summation result to a value between 0 and 1 to obtain the residual fit coefficients.

[0128] Among them, the texture complexity index and the light and shadow change intensity index directly determine the coordination between the light and shadow residual map and the primary composite map in terms of color distribution and light and shadow change trend. They are the core indicators of adaptability, so both indicators are given high weight (0.4). Color is significantly affected by light, and simple color matching cannot guarantee the realism of the composite. Color differences can be appropriately corrected through subsequent color adjustment. However, the mismatch between texture and light and shadow is difficult to be easily adjusted and compensated for, so the color matching index is given low weight (0.2).

[0129] The lighting residual map is dynamically adjusted based on the residual adaptation coefficient to generate a weighted residual map. Specifically, the lighting residual map is divided into corresponding sub-regions with the same fixed size as the primary composite map. Each sub-region is traversed, and the residual values ​​of all pixels in the sub-region are multiplied by the residual adaptation coefficient of the sub-region to obtain the weighted residual values ​​of each pixel. The weighted residual values ​​of all pixels are arranged in order according to their original positions in the lighting residual map to form a weighted residual map.

[0130] The weighted residual image is fused with the primary composite image while maintaining color consistency to generate the final new image. Specifically, this involves: traversing each pixel in both the weighted residual image and the primary composite image; first, extracting the RGB values ​​of the corresponding pixel from the primary composite image and the weighted residual value from the corresponding pixel in the weighted residual image; calculating the proportion of the pixel's RGB value to the total sum of the three colors; then, splitting the weighted residual value according to this proportion and adding it to the RGB values ​​to obtain new RGB values, which are the fused pixel values ​​at that location; calculating the overall RGB mean of the primary composite image; then calculating the overall RGB mean of all fused pixel values; subtracting the two means to obtain the difference; and finally, subtracting the corresponding difference from the RGB value of each fused pixel to obtain the adjusted pixel value, ensuring that the color mean matches the primary composite image. Figure 1Then, arrange all the adjusted pixel values ​​in their original positions to form the final new image.

[0131] By dynamically adjusting the residual image of light and shadow using the residual adaptation coefficient, and combining it with key indicators such as texture complexity and intensity of light and shadow changes, the residual image is accurately adapted to the primary composite image. During fusion, the residual values ​​are split according to the RGB ratio while maintaining color consistency. This not only enhances the details of light and shadow but also avoids color imbalance, making up for the subtle light and shadow deviations in the primary composite image. This makes the light and shadow interaction between the cartoon character and the new background more natural, completely eliminating the sense of visual disjointedness, and obtaining high-quality images that meet the marketing needs of multiple scenarios, thereby enhancing the visual expressiveness of the brand's cartoon character in diverse scenarios.

[0132] In one embodiment, a system for batch switching of backgrounds to form new images based on a convolutional neural network is applied to any of the above-described methods for batch switching of backgrounds to form new images based on a convolutional neural network, comprising:

[0133] The bidirectional analysis unit is used to analyze cartoon images to obtain foreground segmentation masks and foreground material feature tensors, and to analyze target background images to obtain background illumination feature vectors.

[0134] The fusion unit is used to fuse and calculate the foreground material feature tensor and the background illumination feature vector to generate a dual-medium light interaction field.

[0135] Combined analysis unit is used to combine the dual-medium light interaction field and the foreground segmentation mask for analysis to obtain the edge transmission matrix;

[0136] The switching unit is used to perform initial background switching between the target background image and the foreground segmentation mask to obtain an initial switching image.

[0137] The modulation unit is used to modulate the illumination of the foreground region of the initial switching image according to the dual-medium light interaction field, and to fuse the background region of the initial switching image with the edge transmission matrix to generate a primary composite image.

[0138] The correction unit is used to calculate the gradient difference map between the primary composite image and the target background image, and to correct the gradient difference map to obtain the light and shadow residual map.

[0139] The overlay unit is used to overlay the light and shadow residual map with the primary composite map to generate the final new map.

[0140] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for forming a new image based on batch switching background of a convolutional neural network, characterized in that, Comprise: S1: analyze the cartoon image to obtain foreground segmentation mask and foreground material feature tensor, and analyze the target background image to obtain background lighting feature vector; S2: fuse and calculate the foreground material feature tensor and the background lighting feature vector to generate a double-medium light interaction field, comprising: align the foreground material feature tensor and the background lighting feature vector to generate a feature matching matrix; based on the feature matching matrix, calculate the interaction parameters of the foreground material feature tensor and the background lighting feature vector to obtain a light material interaction tensor; analyze the light material interaction tensor to generate a double-medium light interaction field; S3: combine and analyze the double-medium light interaction field and the foreground segmentation mask to obtain an edge transmission matrix, comprising: analyze the foreground segmentation mask to determine the junction area of the foreground and the background, and generate an edge region mask; based on the edge region mask, extract the light interaction features at the edge from the double-medium light interaction field to obtain an edge light interaction subfield; according to the material properties of the edge region in the foreground material feature tensor, perform transmission attenuation calculation on the edge light interaction subfield to obtain an edge transmission attenuation coefficient; according to the edge region mask, perform spatial matching on the edge transmission attenuation coefficient and the edge light interaction subfield to generate an edge transmission matrix; S4: perform initial background switching on the target background image and the foreground segmentation mask to obtain an initial switching image; S5: according to the double-medium light interaction field, perform lighting modulation on the foreground region of the initial switching image, and combine the edge transmission matrix to fuse the background region of the initial switching image to generate a primary synthesis image; S6: calculate the gradient difference map of the primary synthesis image and the target background image, and correct the gradient difference map to obtain a light shadow residual map; S7: superimpose the light shadow residual map and the primary synthesis image to generate a final new image. 2.The method of claim 1, wherein, analyze the cartoon image to obtain foreground segmentation mask and foreground material feature tensor, comprising: analyze the cartoon image to obtain a multi-scale feature graph; calculate the foreground attribution probability based on the multi-scale feature graph to generate a foreground probability distribution map; aggregate the material properties of the multi-scale feature graph to obtain a foreground material original tensor; process the foreground probability distribution map to obtain the foreground segmentation mask; analyze the foreground material original tensor to obtain the foreground material feature tensor.

3. The method of claim 2, wherein, analyze the target background image to obtain the background lighting feature vector, comprising: analyze the target background image to obtain a background lighting shallow feature graph; calculate the background lighting shallow feature graph to determine the lighting direction weight matrix and the regional lighting intensity coefficient; fuse and reduce the dimension of the background lighting shallow feature graph, the lighting direction weight matrix and the regional lighting intensity coefficient to generate the background lighting feature vector.

4. The method of claim 1, wherein, perform initial background switching on the target background image and the foreground segmentation mask to obtain an initial switching image, comprising: perform edge analysis on the edge region mask to obtain an edge optimization mask; based on the edge optimization mask, perform regional weight distribution on the target background image to generate a background region adaptive weight according to the position distribution of the foreground in the image; According to the edge optimization mask and the background area adaptive weight, the target background image is screened to obtain an adaptive background image; The foreground area of the cartoon image is replaced and fused with the adaptive background image according to the edge optimization mask to generate an initial switching image.

5. The method of claim 4, wherein, According to the double-medium light interaction field, the foreground area of the initial switching image is light modulated, and the background area of the initial switching image is fused in combination with the edge transmission matrix to generate a primary synthesis image, including: The double-medium light interaction field is extracted to obtain a foreground light modulation matrix; According to the foreground light modulation matrix, the foreground area of the initial switching image is light modulated to generate a foreground area image after light modulation; Based on the edge transmission matrix, a background fusion weight is determined; According to the background fusion weight, the background area of the initial switching image is optimized to obtain a background area image after optimization; The foreground area image and the background area image are calculated to obtain a boundary transition coefficient; Based on the boundary transition coefficient, the foreground area image and the background area image are spliced and fused to generate a primary synthesis image.

6. The method of claim 5, wherein, The gradient difference image of the primary synthesis image and the target background image is calculated, and the gradient difference image is corrected to obtain a light and shadow residual image, including: The pixel gradient difference value of the primary synthesis image and the target background image is calculated to generate an initial gradient difference image; The initial gradient difference image is extracted to obtain a gradient anomaly weight image; Based on the gradient anomaly weight image, the initial gradient difference image is corrected to generate a corrected gradient difference image, and an error compensation coefficient in the correction process is obtained; Based on the error compensation coefficient, the corrected gradient difference image is residual reinforced to generate a light and shadow residual image.

7. The method of claim 6, wherein, The light and shadow residual image and the primary synthesis image are superimposed to generate a final new image, including: The light and shadow residual image and the primary synthesis image are regionally adapted to generate a residual adaptation coefficient; Based on the residual adaptation coefficient, the light and shadow residual image is regionally dynamically adjusted to generate a weighted residual image; The weighted residual image and the primary synthesis image are fused while maintaining color consistency between the two to generate a final new image.

8. A system for forming a new image based on batch switching of a background of a convolutional neural network, applied to the method for forming a new image based on batch switching of a background of a convolutional neural network according to any one of claims 1-7, characterized in that, including: The bidirectional analysis unit is used for analyzing the cartoon image to obtain a foreground segmentation mask and a foreground material feature tensor, and analyzing the target background image to obtain a background light feature vector; The fusion unit is used for fusing and calculating the foreground material feature tensor and the background light feature vector to generate a double-medium light interaction field; The combination analysis unit is used for combining and analyzing the double-medium light interaction field and the foreground segmentation mask to obtain an edge transmission matrix; The switching unit is used for performing initial background switching on the target background image and the foreground segmentation mask to obtain an initial switching image; The modulation unit is used for light modulating the foreground area of the initial switching image according to the double-medium light interaction field, and fusing the background area of the initial switching image in combination with the edge transmission matrix to generate a primary synthesis image; The correction unit is used for calculating the gradient difference image of the primary synthesis image and the target background image, and correcting the gradient difference image to obtain a light and shadow residual image; The superposition unit is used for superimposing the light and shadow residual image and the primary synthesis image to generate a final new image.

Images