A style transfer method for content-image object separation

Abstract

This invention discloses a style transfer method for separating content images from objects, comprising: extracting style images and content images of a target image; processing the style image using a salient object detection module to distinguish between the main style object and background objects; constructing a depth estimation mechanism and processing the content image using a depth estimation module to obtain the depth information of the content image; extracting features of the main style object based on the style separation attributes of the convolutional feature channels and designing an opacity-sensitive loss function to obtain a stylized main style object transfer result; combining the characteristics of complex objects in the content background, adjusting its sharpness and style information based on defocusing and instance normalization to obtain a stylized background transfer result; and finally combining the object transfer results. This invention effectively solves the problem of messy style transfer results in complex scenes by guiding the style transfer module to achieve style transfer through a depth estimation mechanism when dealing with content images with cluttered main content.

Description

Technical Field

[0001] This invention belongs to the fields of digital image processing and machine vision, and particularly relates to a style transfer method for separating content image objects. Background Technology

[0002] Excellent works of art often provide information different from real photographs. However, without extensive professional training, ordinary people may not be able to independently create a work of art that satisfies themselves and others. Furthermore, the time and cost required to train a true artist is immeasurable. Moreover, even artists skilled in stylistic composition require a considerable period to complete a single artwork. To efficiently convert real photographs into artistic images, image style transfer tasks have emerged. Image style transfer aims to combine the content of a real photograph with the style of an artwork, creating a stylized photograph that simultaneously possesses both photographic content and artistic style. In style transfer tasks, the image providing the content is called the content image, the image providing the style is called the style image, and the resulting image is called the stylized image. Style transfer specifically refers to image style transfer. The development of style transfer can be mainly divided into two stages: the first stage, from the mid-1990s to 2016, was characterized by the use of mathematical models to simulate textures; the second stage, from 2016 to the present, is characterized by the use of deep learning and neural networks for style transfer. Compared to traditional methods, the former is more conventional, while the latter incorporates new approaches. Therefore, the first stage is called "conventional style transfer," and the second stage is called "neural style transfer." Conventional style transfer typically uses mathematical and signal processing techniques, such as texture synthesis, histogram matching, and filtering. These methods involve manipulating pixels to simulate the desired style. For example, frequency domain filtering can enhance or weaken certain frequency components of an image, thereby altering its appearance. The advantage of conventional style transfer lies in its higher computational speed and lower resource consumption, but it may not be able to capture more sophisticated artistic styles and textures.

[0003] Neural network-based style transfer methods are more flexible and efficient. These methods use deep learning techniques to learn and apply image styles. They train neural networks to capture features of different artistic styles, and then apply these features to the input image to generate a new image with the desired style. The advantage of neural style transfer lies in its ability to better capture the details and complexity of artistic styles; however, the time and resource requirements vary significantly depending on the network structure. Traditional style transfer and neural style transfer should not be seen as replacements of each other; rather, some current neural style transfer achievements are based on ideas from traditional style transfer and digital image processing. Image style transfer has numerous applications in real life, such as in environmental atmosphere rendering, font generation, font recognition, portrait editing, computer-aided design, photo restoration, virtual reality (VR), and augmented reality (AR). Furthermore, as a fundamental task in computer vision, style transfer can assist in research on other tasks such as adversarial example research, image generation research, and domain adaptation. Style transfer has wide applications from both practical and scientific research perspectives. However, current neural style transfer methods still have some shortcomings. An excellent artwork should have a sense of hierarchy in its scene description and retain a certain amount of negative space. However, current neural style transfer methods often overlook the clear hierarchy and the combination of whole and scattered elements in artworks, resulting in unclear hierarchy in the stylization results when transferring styles from images with cluttered content. This paper addresses these issues and also overcomes the shortcoming of style transfer methods in maintaining consistency with the background characteristics of the original style image when transferring styles from certain images. Summary of the Invention

[0004] Purpose of the Invention: The purpose of this invention is to provide a style transfer method for separating content image objects. When faced with content images with cluttered main subjects, a depth estimation mechanism guides the style transfer module to perform style transfer on specific regions, effectively solving the problem of messy style transfer results in complex scenes, and ensuring that the background of the style transfer result maintains a high degree of consistency with the background style of the original style image.

[0005] Technical solution: The present invention provides a style transfer method for content image object separation, comprising the following steps:

[0006] Step 1: Extract the style image and content image of the target image, and input the style image into the salient object detection module to distinguish the main object and background object in the style image;

[0007] Step 2: Based on the characteristics of the content image, which has many scenes and subjects and the same subjects are at the same depth, a depth estimation mechanism is constructed. The content image is input into the depth estimation module. Then, based on the ambiguity of the photographic image being taken from different camera models, a standard camera space transformation is constructed. The depth information in the content image is estimated by using the standard camera transformation and denormalization operation, so as to separate the content image into region of interest images and non-region of interest images.

[0008] Step 3: For the main object of the style image, based on its obvious texture features, the style features of the main object of the style image are extracted based on the style separation attributes of the Fourier transform and deep convolution feature channels. An opacity-sensitive loss function is designed to train the transfer results of the main object and obtain the stylized main object transfer results.

[0009] Step 4: For the background object of the style image, based on its single color and the fact that the non-interesting content contains many complex objects, the sharpness and style information of the non-interesting content are adjusted based on background defocus and adaptive instance normalization to obtain the stylized background object migration result.

[0010] Step 5: Merge the stylized subject object transfer results and the stylized background object transfer results to achieve object separation style transfer of the target image.

[0011] Furthermore, step 1 specifically includes the following steps:

[0012] Step 101: Use a saliency detection network Style images Processing is performed to obtain a saliency map. , is used to represent the probability that each pixel value belongs to the subject, and the formula is as follows:

[0013]

[0014] Step 102: Based on the saliency map obtained in Step 101 The style image S is separated into main parts. With background The formula is as follows:

[0015]

[0016] in, This indicates element-wise multiplication.

[0017] Furthermore, step 2 specifically includes the following steps:

[0018] Step 201: Process the input content image Wavelet decomposition yields low-frequency components. With high frequency components The formula is as follows:

[0019]

[0020] in This represents wavelet transform, applied to high-frequency components. The nonlocal means method is applied for noise reduction to obtain the denoised high-frequency components. The formula is as follows:

[0021]

[0022] Among them, weight Calculated based on the similarity of pixel blocks;

[0023] Low frequency components With the high-frequency components after noise reduction Perform inverse wavelet transform to obtain the enhanced image. :

[0024]

[0025] Step 202: The enhanced image obtained in step 101 To eliminate metric ambiguity caused by different camera models, the standard camera transform is expressed as:

[0026]

[0027] in, This represents a standard camera space transformation operation. Represents a standard camera model image;

[0028] Step 203: In the standard camera space, process the standard camera model image obtained in step 202. Depth estimation is performed to obtain a standardized depth map. The formula is as follows:

[0029]

[0030] in, This represents the depth estimation model. Content image Depth information;

[0031] Step 204: Perform a normalization operation to reduce errors, using the following formula:

[0032]

[0033] in, This indicates a normalization operation. and Representing depth information The minimum and maximum values ​​in;

[0034] Step 205: Based on the normalized depth information from step 204 Combined with user input, the original content image is... Divide the region into regions of interest and regions of non-interest, and create a mask. With the corresponding Channel mask ;

[0035] Step 206: Apply the four masks obtained in step 205 to the enhanced content image to obtain a content interest image. Images that are not of interest to the content ;

[0036] Furthermore, in step 205, the channel mask The formula is as follows:

[0037] by This indicates the number of regions in the content image. Indicates the region of interest mask:

[0038]

[0039] Represents the mask of the corresponding region of interest. Channel mask:

[0040]

[0041] Indicates the mask for non-interest regions:

[0042]

[0043] Indicates the mask corresponding to the non-interest region. Channel mask:

[0044]

[0045] Furthermore, in step 206, the content interest image Images that are not of interest to the content The formula is as follows:

[0046]

[0047] in, This indicates element-wise multiplication. This indicates a splicing operation.

[0048] Furthermore, step 3 specifically includes the following steps:

[0049] Step 301: For the style subject object obtained in Step 1 Compared with the content interest image obtained in step 2 Perform an octave convolution to decompose it into high-frequency and low-frequency components, as shown in the following formula:

[0050]

[0051] in, and These represent the high-frequency and low-frequency components of the content interest image, respectively. and These represent the high-frequency and low-frequency components of the main style object, respectively.

[0052] Step 302: Use the style feature extractor E_{s} to extract the style features of the main style object obtained in step 301, and use the kernel prediction network to predict the corresponding convolution kernel and bias, as shown in the following formula:

[0053]

[0054] in, and For convolutional and bias prediction networks, This represents the convolution kernel and bias obtained from predictions of high-frequency style main objects. This represents the convolution kernel and bias obtained from the prediction of low-frequency style main objects;

[0055] Step 303: The high-frequency and low-frequency components of the content interest image are combined with the convolutional kernel and bias predicted in step 302, and then processed in the generator. Stylized subject object obtained by performing style transfer The formula is as follows:

[0056]

[0057] Step 304: Based on the stylized subject object obtained in step 303, calculate the loss function. Then, backpropagation is performed to update the convolutional and bias prediction networks. and style transfer generator The formula is as follows:

[0058]

[0059] in, These represent the content loss function, style loss function, and opacity loss function, respectively. These are hyperparameters used to represent the proportion of the three loss functions in the total loss function.

[0060] Furthermore, in step 304, the content loss function The formula used to ensure that the stylized image retains a similar structure and detail to the original content image is as follows:

[0061]

[0062] in, This represents a pre-trained VGG feature extraction network;

[0063] The style loss function The formula used to ensure that the stylized image can capture and reproduce the texture and stylistic features of the stylized image is as follows:

[0064]

[0065] in, Represents the Gram matrix. Indicates the first Layer feature map;

[0066] The opacity loss function To ensure the style transfer network's sensitivity to opacity information in the image, and to enhance the aesthetic effect of style transfer using positive and negative samples, the formula is as follows:

[0067]

[0068] in, Indicates the use of generators The The high-frequency components of the layer output feature map. Indicates the use of generators The The low-frequency portion of the layer output feature map. Indicates the first generation generated by the generator Zhang's stylized main subject, express of aisle, Indicates the first One positive sample, Indicates the first One negative sample.

[0069] Furthermore, step 4 specifically includes the following steps:

[0070] Step 401: Apply Gaussian blur to non-interesting objects to defocus them, making them better match the single color characteristics of the background object. The formula is as follows:

[0071]

[0072] in, The blurred content is not the object of interest. It is the original content, not an object of interest. It is the standard deviation of Gaussian blur, which controls the degree of blur.

[0073] Step 402: The pre-trained neural network VGG 19 extracts style and background object features from the blurred content non-interest objects in step 401. These features are used to guide the style transfer of content non-interest objects, as shown in the following formula:

[0074]

[0075] in, From the style background object Features extracted from This represents the pre-trained feature extraction network VGG19;

[0076] Step 403: Combining the characteristics of the content non-interest object and the style background object, use Adaptive Instance Normalization (AdaIN) to adjust the style of the content non-interest object obtained in Step 402;

[0077]

[0078] in, and These represent the calculations of the mean and standard deviation, respectively. This represents the characteristics of the style background object. Represents a stylized background object.

[0079] Furthermore, step 5 specifically involves performing a final merging operation between the stylized subject object and the stylized background object, using the following formula:

[0080]

[0081] in, A mask representing the main object. Represents the identity matrix. This represents the final style transfer result.

[0082] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0083] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0084] 1. When faced with images containing a chaotic main content, this invention uses a depth estimation mechanism to guide the style transfer module to perform style transfer on specific regions, effectively solving the problem of messy style transfer results in complex scenes.

[0085] 2. In the training of the style transfer network, a redesigned style training loss function that incorporates image alpha channel information was implemented. This not only guides the model to converge quickly but also allows the model to consider the opacity information in the image, effectively solving the problem of transferring PNG format images. During network training, the difference between positive and negative samples is also used to train the network, achieving more effective style transfer.

[0086] 3. In terms of content image decomposition, the characteristic that the same subject is basically at the same depth is utilized to decompose the content image correlation matrix, and then extract the subject object region and the background object region. This can better distinguish the boundary between the subject and the background. Attached Figure Description

[0087] Figure 1 This is a diagram of the overall framework of the model proposed in this invention;

[0088] Figure 2 This is the multi-scale information extraction module of the present invention;

[0089] Figure 3 This invention proposes a cross-scale information interaction method.

[0090] Figure 4 This is a schematic diagram of the depthwise convolutional gated forward propagation layer of the present invention;

[0091] Figure 5 This is the final style transfer flowchart of the present invention;

[0092] Figure 6 This is a flowchart of the present invention. Detailed Implementation

[0093] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0094] The style transfer method for content image object separation described in this embodiment of the present invention has the following overall model framework: Figure 1 As shown, the following steps are included:

[0095] Step 1: Based on the characteristic that the background and main content of traditional Chinese artworks are quite different, the style image is input into the salient object detection module to distinguish the main part and background part of the style image, thereby achieving the purpose of separating different style parts in the style image.

[0096] Furthermore, step 1 of the above statement specifically includes:

[0097] Step 101, using a saliency detection network Style images Processing is performed to obtain a saliency map. , is used to represent the probability that each pixel value belongs to the subject, and the formula is as follows:

[0098]

[0099] Step 102, based on the saliency map obtained in step 101 The style image S is separated into main parts. With background The formula is as follows:

[0100]

[0101] in, This indicates element-wise multiplication.

[0102] Step 2: Based on the characteristics of numerous and complex scenes and subjects in the content image, and the fact that similar subjects are generally at the same depth, a depth estimation mechanism is constructed. The content image is input into the depth estimation module, and then, considering the ambiguity of the photographic images taken from different camera models, a standard camera space transformation is constructed. The standard camera transformation and denormalization operation are used to estimate the depth information in the content image, thereby achieving the goal of separating the content image into regions of interest (ROI) and non-ROI images. The process is as follows: Figure 1 As shown.

[0103] Furthermore, step 2 specifically includes:

[0104] Step 201, process the input content image. Wavelet decomposition yields low-frequency components. With high frequency components The formula is as follows:

[0105]

[0106] in This represents wavelet transform. For high-frequency components... The nonlocal means method is applied for noise reduction to obtain the denoised high-frequency components. The formula is as follows:

[0107]

[0108] Among them, weight Calculated based on the similarity of pixel blocks.

[0109] Low frequency components With the high-frequency components after noise reduction Perform inverse wavelet transform to obtain the enhanced image.

[0110]

[0111] Step 202, the enhanced image obtained in step 101 The transformation is performed to a standard camera space to eliminate metric ambiguity caused by different camera models. The standard camera transform can be expressed as:

[0112]

[0113] in, This represents a standard camera space transformation operation. This represents a standard camera model image.

[0114] Step 203: In the standard camera space, process the standard camera model image obtained in step 202. Depth estimation is performed to obtain a standardized depth map. The formula is as follows:

[0115]

[0116] in, This represents the depth estimation model. Content image Depth information.

[0117] Step 204: Perform a normalization operation to reduce error. The formula is as follows:

[0118]

[0119] in, This indicates a normalization operation. and Representing depth information The minimum and maximum values ​​in.

[0120] Step 205, based on the normalized depth information from step 204 Combined with user input, the original content image is... Divide the region into regions of interest and regions of non-interest, and create a mask. With the corresponding Channel mask The formula is as follows:

[0121] by This indicates the number of regions in the content image. Indicates the region of interest mask:

[0122]

[0123] Represents the mask of the corresponding region of interest. Channel mask:

[0124]

[0125] Indicates the mask for non-interest regions:

[0126]

[0127] Indicates the mask corresponding to the non-interest region. Channel mask:

[0128]

[0129] Step 206: Apply the four masks obtained in step 205 to the enhanced content image to obtain a content interest image. Images that are not of interest to the content The formula is as follows.

[0130]

[0131] in, This indicates element-wise multiplication. This indicates a splicing operation, the process is as follows: Figure 2 As shown.

[0132] Step 3: For the main style object, based on its obvious texture features, the style features of the main style object are extracted based on the style separation attributes of the Fourier transform and deep convolution feature channels. An opacity-sensitive loss function is designed to train the transfer results of the main object to obtain the stylized main style object transfer results.

[0133] Furthermore, step 3 specifically includes:

[0134] Step 301, for the style subject object obtained in Step 1 Compared with the content interest image obtained in step 2 Perform an octave convolution to decompose it into high-frequency and low-frequency components, as shown in the following formula:

[0135]

[0136] in, and These represent the high-frequency and low-frequency components of the content interest image, respectively. and These represent the high-frequency and low-frequency components of the main style object, respectively.

[0137] Step 302, using a style feature extractor Extract the style features of the main style object obtained in step 301, and predict the corresponding convolutional kernels and biases using a kernel prediction network, as shown in the following formula:

[0138]

[0139] in, and For convolutional and bias prediction networks, This represents the convolution kernel and bias obtained from predictions of high-frequency style main objects. This represents the convolution kernel and bias obtained from the prediction of the low-frequency style subject object.

[0140] Step 303: The high-frequency and low-frequency components of the content interest image are combined with the predicted convolutional kernel and bias from step 302, and then processed in the generator. Stylized subject object obtained by performing style transfer The formula is as follows:

[0141]

[0142] Step 304: Based on the stylized subject object obtained in step 303, calculate the loss function. Then, backpropagation is performed to update the convolutional and bias prediction networks. and style transfer generator The formula is as follows:

[0143]

[0144] in, These represent the content loss function, style loss function, and opacity loss function, respectively. These are hyperparameters used to represent the proportion of the three loss functions in the total loss function.

[0145] Content loss The formula used to ensure that the stylized image retains a similar structure and detail to the original content image is as follows:

[0146]

[0147] in, This represents the pre-trained VGG feature extraction network.

[0148] Style loss The formula used to ensure that the stylized image can capture and reproduce the texture and stylistic features of the stylized image is as follows:

[0149]

[0150] in, Represents the Gram matrix. Indicates the first Layer feature map.

[0151] Opacity loss To ensure the style transfer network's sensitivity to opacity information in the image, and to enhance the aesthetic effect of style transfer using positive and negative samples, the formula is as follows:

[0152]

[0153] in, Indicates the use of generators The The high-frequency components of the layer output feature map. Indicates the use of generators The The low-frequency portion of the layer output feature map. Indicates the first generation generated by the generator Zhang's stylized main subject, express of aisle, Indicates the first One positive sample, Indicates the first One negative sample, the process is as follows: Figure 3 As shown.

[0154] Step 4: For the style background object, based on its single color and the fact that the content non-interest object contains many complex objects, adjust the sharpness and style information of the content non-interest object based on background defocus and adaptive instance normalization to obtain the stylized background object migration result.

[0155] Furthermore, step 4 specifically involves:

[0156] Step 401: Apply Gaussian blur to the non-interesting objects in the content to defocus them, making them better match the single color characteristics of the background object. The formula is as follows:

[0157]

[0158] in, The blurred content is not the object of interest. It is the original content, not an object of interest. It is the standard deviation of Gaussian blur, which controls the degree of blur.

[0159] Step 402: The pre-trained neural network VGG 19 extracts style and background object features from the blurred content non-interest objects in step 401. These features are used to guide the style transfer of content non-interest objects, as shown in the following formula:

[0160]

[0161] in, From the style background object Features extracted from This represents the pre-trained feature extraction network VGG19.

[0162] Step 403: Combining the characteristics of the content non-interest objects and the style background objects, use Adaptive Instance Normalization (AdaIN) to adjust the style of the content non-interest objects obtained in step 402.

[0163]

[0164] in, and These represent the calculations of the mean and standard deviation, respectively. This represents the characteristics of the style background object. Represents a stylized background object.

[0165] Step 4: For the style background object, based on its single color and the fact that the content non-interest object contains many complex objects, adjust the sharpness and style information of the content non-interest object based on background defocus and adaptive instance normalization to obtain the stylized background object migration result.

[0166] Step 5: After obtaining the stylized subject object and the stylized background object, using the original content image as a reference, normalize the stylized subject object migration results and the stylized background object migration results to reduce the style difference between them, and then perform a final merging operation. The process is as follows: Figure 4 As shown.

[0167] Furthermore, step 5 specifically includes:

[0168] Step 501: Perform the final merging operation between the stylized subject object and the stylized background object, using the following formula:

[0169]

[0170] in, A mask representing the main object. Represents the identity matrix. This represents the final style transfer result, and the process is as follows: Figure 5 As shown.

Claims

1. A style transfer method for separating content-image objects, characterized in that, The steps include: Step 1: Extract the style image and content image of the target image, and input the style image into the salient object detection module to distinguish the main object and background object in the style image; Step 2: Based on the characteristics of the content image, which has many scenes and subjects and the same subjects are at the same depth, a depth estimation mechanism is constructed. The content image is input into the depth estimation module. Then, based on the ambiguity of the photographic image being taken from different camera models, a standard camera space transformation is constructed. The depth information in the content image is estimated by using the standard camera transformation and denormalization operation, so as to separate the content image into region of interest images and non-region of interest images. Step 3: For the main object of the style image, based on its obvious texture features, the style features of the main object of the style image are extracted based on the style separation attributes of the Fourier transform and deep convolution feature channels. An opacity-sensitive loss function is designed to train the transfer results of the main object and obtain the stylized main object transfer results. Step 4: For the background object of the style image, based on its single color and the fact that the non-interesting content contains complex objects, adjust the sharpness and style information of the non-interesting content based on background defocus and adaptive instance normalization to obtain the stylized background object migration result. Step 5: Merge the stylized subject object transfer results and the stylized background object transfer results to achieve object separation style transfer of the target image.

2. The style transfer method for content image object separation according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 101: Use a saliency detection network Style images Processing is performed to obtain a saliency map. , is used to represent the probability that each pixel value belongs to the subject, and the formula is as follows: Step 102: Based on the saliency map obtained in Step 101 The style image S is separated into main parts. With background , the formula is as follows: in, This indicates element-wise multiplication.

3. The style transfer method for content image object separation according to claim 1, characterized in that, Step 2 specifically includes the following steps: Step 201: Process the input content image Wavelet decomposition yields low-frequency components. With high frequency components , the formula is as follows: in This represents wavelet transform, applied to high-frequency components. The nonlocal means method is applied for noise reduction to obtain the denoised high-frequency components. , the formula is as follows: Among them, weight Calculated based on the similarity of pixel blocks; Low frequency components With the high-frequency components after noise reduction Perform inverse wavelet transform to obtain the enhanced image. : Step 202: The enhanced image obtained in step 101 To eliminate metric ambiguity caused by different camera models, the standard camera transform is expressed as: in, This represents a standard camera space transformation operation. Represents a standard camera model image; Step 203: In the standard camera space, process the standard camera model image obtained in step 202. Depth estimation is performed to obtain a standardized depth map. , the formula is as follows: in, This represents the depth estimation model. Content image Depth information; Step 204: Perform a normalization operation to reduce errors, using the following formula: in, This indicates a normalization operation. and Representing depth information The minimum and maximum values ​​in; Step 205: Based on the normalized depth information from step 204 Combined with user input, the original content image is... Divide the region into regions of interest and regions of non-interest, and create a mask. With the corresponding Channel mask ; Step 206: Apply the four masks obtained in step 205 to the enhanced content image to obtain a content interest image. Images that are not of interest to the content .

4. The style transfer method for content image object separation according to claim 3, characterized in that, In step 205, the channel mask The formula is as follows: by This indicates the number of regions in the content image. Indicates the region of interest mask: Represents the mask of the corresponding region of interest. Channel mask: Indicates the mask for non-interest regions: Indicates the mask corresponding to the non-interest region. Channel mask: 。 5. The style transfer method for content image object separation according to claim 3, characterized in that, In step 206, the content interest image Images that are not of interest to the content , the formula is as follows: in, This indicates element-wise multiplication. This indicates a splicing operation.

6. The style transfer method for content image object separation according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 301: For the style subject object obtained in Step 1 Compared with the content interest image obtained in step 2 Perform an octave convolution to decompose it into high-frequency and low-frequency components, as shown in the following formula: in, and These represent the high-frequency and low-frequency components of the content interest image, respectively. and These represent the high-frequency and low-frequency components of the main style object, respectively. Step 302: Use the style feature extractor Extract the style features of the main style object obtained in step 301, and predict the corresponding convolutional kernels and biases using a kernel prediction network, as shown in the following formula: in, and For convolutional and bias prediction networks, This represents the convolution kernel and bias obtained from predictions of high-frequency style main objects. This represents the convolution kernel and bias obtained from the prediction of low-frequency style main objects; Step 303: The high-frequency and low-frequency components of the content interest image are combined with the convolutional kernel and bias predicted in step 302, and then processed in the generator. Stylized subject object obtained by performing style transfer , the formula is as follows: Step 304: Based on the stylized subject object obtained in step 303, calculate the loss function. Then, backpropagation is performed to update the convolutional and bias prediction networks. and style transfer generator , the formula is as follows: in, These represent the content loss function, style loss function, and opacity loss function, respectively. These are hyperparameters used to represent the proportion of the three loss functions in the total loss function.

7. The style transfer method for content image object separation according to claim 6, characterized in that, In step 304, the content loss function The formula used to ensure that the stylized image retains a similar structure and detail to the original content image is as follows: in, This represents a pre-trained VGG feature extraction network; The style loss function The formula used to ensure that the stylized image can capture and reproduce the texture and stylistic features of the stylized image is as follows: in, Represents the Gram matrix. Indicates the first Layer feature map; The opacity loss function To ensure the style transfer network's sensitivity to opacity information in the image, and to enhance the aesthetic effect of style transfer using positive and negative samples, the formula is as follows: in, Indicates the use of generators The The high-frequency components of the layer output feature map. Indicates the use of generators The The low-frequency portion of the layer output feature map. Indicates the first generation generated by the generator Zhang's stylized main subject, express of aisle, Indicates the first One positive sample, Indicates the first One negative sample.

8. The style transfer method for content image object separation according to claim 1, characterized in that, Step 4 specifically includes the following steps: Step 401: Apply Gaussian blur to non-interesting objects to defocus them, making them better match the single color characteristics of the background object. The formula is as follows: in, The blurred content is not the object of interest. It is the original content, not an object of interest. It is the standard deviation of Gaussian blur, which controls the degree of blur. Step 402: The pre-trained neural network VGG 19 extracts style and background object features from the blurred content non-interest objects in step 401. These features are used to guide the style transfer of content non-interest objects, as shown in the following formula: in, From the style background object Features extracted from This represents the pre-trained feature extraction network VGG19; Step 403: Combining the characteristics of the content non-interest object and the style background object, use Adaptive Instance Normalization (AdaIN) to adjust the style of the content non-interest object obtained in Step 402; in, and These represent the calculations of the mean and standard deviation, respectively. This represents the characteristics of the style background object. Represents a stylized background object.

9. The style transfer method for content image object separation according to claim 8, characterized in that, Step 5 specifically involves performing a final merging operation between the stylized subject object and the stylized background object, using the following formula: in, A mask representing the main object. Represents the identity matrix. This represents the final style transfer result.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

Images