Image processing method and device, equipment, storage medium and computer program product

Abstract

The invention relates to the technical field of image processing, and discloses an image processing method, device and equipment, a storage medium and a computer program product.The method comprises the steps that in response to an image processing request, a to-be-processed image is input into a preset image processing model, and multi-scale features of the to-be-processed image are extracted through the preset image processing model; carrying out the fusion of the multi-scale features, obtaining a target region, superposing the target region with a new background template, and obtaining a background-replaced image; according to the method, the multi-scale features of the to-be-processed image are extracted through the model, and background replacement is performed after the multi-scale features are fused, so that large-scale global information and small-scale detail information in the image can be captured at the same time, a target with rich details can be processed more accurately, and the naturalness of a background replacement effect can be remarkably improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] Image background replacement is a common image processing task, typically used to remove or replace the background in an image to highlight the main object or to transplant the object into a new scene. This is of great significance in applications such as advertising production, film and television special effects, intelligent image editing, virtual reality, and augmented reality.

[0003] However, most image background replacement methods are based on single-scale feature extraction, which often results in low segmentation accuracy in complex backgrounds or when dealing with details (such as hair or semi-transparent objects), leading to unnatural background replacement effects. Summary of the Invention

[0004] The main objective of this application is to provide an image processing method, apparatus, device, storage medium, and computer program product, which aims to solve the technical problem that existing image background replacement methods are usually based on single-scale feature extraction, resulting in low segmentation accuracy and unnatural background replacement effects when dealing with complex backgrounds or detailed processing.

[0005] To achieve the above objectives, this application provides an image processing method, the image processing method comprising:

[0006] In response to an image processing request, the image to be processed is input into a preset image processing model;

[0007] The preset image processing model is used to extract multi-scale features from the image to be processed, and the multi-scale features are fused to obtain the target region.

[0008] The target area is overlaid with the new background template to obtain the image after background replacement.

[0009] Optionally, the preset image processing model includes: an encoder and a decoder;

[0010] The step of extracting multi-scale features from the image to be processed using the preset image processing model and fusing the multi-scale features to obtain the target region includes:

[0011] The encoder extracts shallow, mid, and high-level features from the image to be processed.

[0012] The target region is obtained by fusing the shallow features, the mid-level features, and the high-level features using the decoder.

[0013] Optionally, the encoder includes: a backbone network and a depth processing module;

[0014] The step of extracting shallow, mid-level, and high-level features of the image to be processed using the encoder includes:

[0015] The shallow features, mid-level features, and high-level feature maps of the image to be processed are extracted through the backbone network.

[0016] The high-level feature map is processed by the depth processing module to obtain the deep features of the image to be processed.

[0017] Optionally, the deep processing module includes an attention module and a DS-ASPP module, wherein the attention module and the DS-ASPP module are connected in parallel.

[0018] The step of processing the high-level feature map through the depth processing module to obtain the deep features of the image to be processed includes:

[0019] The attention module is used to enhance the features of the high-level feature map to obtain a first feature map.

[0020] The standard convolution in the high-level feature map is decomposed using the DS-ASPP module to obtain the second feature map;

[0021] The deep features of the image to be processed are obtained by summing the first feature map and the second feature map element by element.

[0022] Optionally, the attention module includes: a position attention module and a channel attention module;

[0023] The step of performing feature enhancement on the high-level feature map through the attention module to obtain the first feature map includes:

[0024] The positional attention module enhances the same features at different locations in the high-level feature map to obtain a third feature map.

[0025] The channel attention module enhances the feature information of different channels in the high-level feature map to obtain a fourth feature map;

[0026] The first feature map is obtained by summing the third and fourth feature maps element by element.

[0027] Optionally, the step of enhancing the same features at different locations in the high-level feature map through the positional attention module to obtain a third feature map includes:

[0028] Construct a positional attention matrix corresponding to the high-level feature map, wherein the positional matrix includes multiple first influence factors, and the first influence factors are the influence factors of one position on another position in the high-level feature map;

[0029] The same features at different locations in the high-level feature map are enhanced based on the first influence factor to obtain the third feature map.

[0030] Optionally, the step of enhancing the feature information of different channels in the high-level feature map through the channel attention module to obtain a fourth feature map includes:

[0031] Construct the channel attention matrix corresponding to the high-level feature map, wherein the channel attention matrix includes multiple second influence factors, and the second influence factors are the influence factors of one channel on another channel in the high-level feature map;

[0032] The feature information of different channels in the advanced feature map is enhanced based on the second influence factor to obtain the fourth feature map.

[0033] Optionally, the step of decomposing the standard convolution in the high-level feature map using the DS-ASPP module to obtain the second feature map includes:

[0034] The standard convolution in the high-level feature map is decomposed into depthwise separable convolutions using the DS-ASPP module.

[0035] The second feature map is obtained by parallel sampling with different dilation rates through the depthwise separable convolution.

[0036] Optionally, the backbone network includes: an initial convolutional layer, a pooling layer, and a residual block group, wherein the residual block group includes multiple residual blocks, and each residual block includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. The first convolutional layer is used for dimensionality reduction, the second convolutional layer is used for feature extraction, and the third convolutional layer is used for dimensionality increase.

[0037] The step of extracting shallow features, mid-level features, and high-level feature maps of the image to be processed through the backbone network includes:

[0038] The shallow features of the image to be processed are extracted through the initial convolutional layer;

[0039] The shallow features are downsampled using the pooling layer;

[0040] The residual block group is used to resize the downsampled features to obtain the mid-level and high-level feature maps of the image to be processed.

[0041] Optionally, the step of fusing the shallow features, the mid-level features, and the high-level features through the decoder to obtain the target region includes:

[0042] The high-level features are upsampled, and the upsampled high-level features are fused with the mid-level features to obtain a first fused feature;

[0043] The first fused feature is subjected to convolution and upsampling processing, and the processed first fused feature is fused with the shallow feature to obtain the second fused feature;

[0044] The second fused feature is subjected to convolution and upsampling processing to obtain the target region.

[0045] Optionally, before inputting the image to be processed into the preset image processing model in response to the image processing request, the method further includes:

[0046] Scene images of various scenarios are acquired and labeled to obtain a binary mask image, wherein the target region pixels are labeled with a first value and the background pixels are labeled with a second value;

[0047] Adjust the scene image and the binary mask image to the same size;

[0048] Data augmentation is performed on the resized scene image and the resized binary mask image;

[0049] The training set, validation set, and test set are constructed based on the data-augmented scene image and the data-augmented binary mask image.

[0050] The initial image processing model is trained using the training set, the validation set, and the test set to obtain a preset image processing model.

[0051] Furthermore, to achieve the above objectives, this application also proposes an image processing apparatus, the image processing apparatus comprising:

[0052] The model input module is used to input the image to be processed into a preset image processing model in response to an image processing request;

[0053] The model processing module is used to extract multi-scale features of the image to be processed through the preset image processing model, and to fuse the multi-scale features to obtain the target region;

[0054] The background replacement module is used to overlay the target area with a new background template to obtain an image after background replacement.

[0055] Optionally, the preset image processing model includes an encoder and a decoder;

[0056] The model processing module is further configured to extract shallow, mid-level, and high-level features of the image to be processed through the encoder; and to fuse the shallow, mid-level, and high-level features through the decoder to obtain the target region.

[0057] Optionally, the encoder includes: a backbone network and a depth processing module;

[0058] The model processing module is further configured to extract shallow features, mid-level features, and high-level feature maps of the image to be processed through the backbone network; and to perform deep processing on the high-level feature maps through the deep processing module to obtain the deep features of the image to be processed.

[0059] Optionally, the deep processing module includes an attention module and a DS-ASPP module, wherein the attention module and the DS-ASPP module are connected in parallel.

[0060] The model processing module is further configured to enhance the high-level feature map through the attention module to obtain a first feature map; decompose the standard convolution in the high-level feature map through the DS-ASPP module to obtain a second feature map; and sum the first feature map and the second feature map element by element to obtain the deep features of the image to be processed.

[0061] Optionally, the attention module includes: a position attention module and a channel attention module;

[0062] The model processing module is further configured to enhance the same features at different positions in the high-level feature map through the position attention module to obtain a third feature map; enhance the feature information of different channels in the high-level feature map through the channel attention module to obtain a fourth feature map; and sum the third feature map and the fourth feature map element by element to obtain a first feature map.

[0063] Optionally, the model processing module is further configured to construct a position attention matrix corresponding to the high-level feature map, wherein the position matrix includes multiple first influence factors, the first influence factors being the influence factors of one position on another position in the high-level feature map; and to enhance the same features at different positions in the high-level feature map according to the first influence factors to obtain a third feature map.

[0064] In addition, to achieve the above objectives, this application also proposes an image processing apparatus, which includes a memory, a processor, and an image processing program stored in the memory and executable on the processor, the image processing program being configured to implement the image processing method as described above.

[0065] In addition, to achieve the above objectives, this application also proposes a storage medium storing an image processing program, which, when executed by a processor, implements the image processing method as described above.

[0066] In addition, to achieve the above objectives, this application also provides a computer program product, which includes an image processing program that, when executed by a processor, implements the image processing method as described above.

[0067] One or more technical solutions proposed in this application have at least the following technical effects:

[0068] In this application, a method is disclosed that, in response to an image processing request, an image to be processed is input into a preset image processing model, multi-scale features of the image to be processed are extracted by the preset image processing model, and the multi-scale features are fused to obtain a target region. The target region is then superimposed with a new background template to obtain an image after background replacement. Since this application extracts multi-scale features of the image to be processed through a model and performs background replacement after fusing the multi-scale features, it can simultaneously capture large-scale global information and small-scale detail information in the image, more accurately process targets with rich details, and thus significantly improve the naturalness of the background replacement effect. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0070] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a flowchart illustrating the first embodiment of the image processing method of this application;

[0072] Figure 2 This is a flowchart illustrating the second embodiment of the image processing method of this application;

[0073] Figure 3 This is a flowchart illustrating the third embodiment of the image processing method of this application;

[0074] Figure 4 This is a diagram of the overall architecture of MSCP-Net, representing an embodiment of the image processing method of this application.

[0075] Figure 5This is a diagram illustrating the architecture of the position attention module in one embodiment of the image processing method of this application.

[0076] Figure 6 This is a diagram illustrating the channel attention module architecture of an embodiment of the image processing method of this application.

[0077] Figure 7 This is a flowchart illustrating the fourth embodiment of the image processing method of this application;

[0078] Figure 8 This is a schematic diagram of the module structure of the image processing device according to an embodiment of this application;

[0079] Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the image processing method in the embodiments of this application.

[0080] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0081] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0082] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0083] Image background replacement is a common image processing task, typically used to remove or replace the background in an image to highlight the main object or transplant the object into a new scene. This is of great significance in applications such as advertising production, film and television special effects, intelligent image editing, virtual reality, and augmented reality. Traditional background replacement methods are mainly based on image matting techniques, such as: using the image's alpha channel (transparency information) to separate the foreground and background, manually or automatically generating foreground and background masks in the image, and replacing the background by processing the masks; using a blue or green background during shooting and using chroma key technology to separate the foreground and background, which is widely used in film and television production. However, these methods have significant limitations in practical applications. For complex backgrounds or foregrounds with rich details (such as hair, semi-transparent objects, etc.), the matting algorithm may have difficulty separating them accurately, resulting in blurred segmentation edges or missed detections; manual intervention is required to generate masks or adjust parameters, increasing the user's workload; and for non-standardized backgrounds (non-solid color backgrounds) or complex scenes, chroma key-based matting does not perform well. With the rapid development of computer vision and deep learning technologies, semantic segmentation has become an important research direction in the field of image processing. Using deep learning models for semantic segmentation classifies pixels in an image into different semantic categories, such as foreground objects and background, thereby achieving foreground extraction. Furthermore, it separates different instances within the same category, achieving more refined instance segmentation and further enhancing the practicality of deep learning techniques for background replacement tasks.

[0084] However, existing image background replacement methods are mostly based on single-scale feature extraction in practical applications of instance segmentation and background replacement. Often, the segmentation accuracy is low in complex backgrounds or when dealing with details (such as hair or semi-transparent objects), resulting in unnatural background replacement effects.

[0085] Therefore, in order to overcome the above-mentioned defects, this application provides a solution, which includes: in response to an image processing request, inputting the image to be processed into a preset image processing model, extracting multi-scale features of the image to be processed through the preset image processing model, fusing the multi-scale features to obtain a target region, and superimposing the target region with a new background template to obtain a background-replaced image; since this solution extracts multi-scale features of the image to be processed through a model and performs background replacement after fusing the multi-scale features, it can simultaneously capture large-scale global information and small-scale detail information in the image, more accurately process targets with rich details, and thus significantly improve the naturalness of the background replacement effect.

[0086] It should be noted that the executing entity in this embodiment may be an image processing device with data processing, network communication and program running functions, such as a computer, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0087] Based on this, embodiments of this application provide an image processing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image processing method of this application.

[0088] In a first embodiment, the image processing method includes:

[0089] Step S10: In response to the image processing request, input the image to be processed into the preset image processing model.

[0090] It should be understood that the application scenario of this application embodiment can be image background replacement. In specific implementation, it can be to select a target area (i.e., the area that needs to be retained in the image to be processed, such as a person, an object, etc.) from the image to be processed and replace it on a new background template. This embodiment does not limit this.

[0091] It is understood that users can initiate an image processing request by inputting the image to be processed, or they can initiate an image processing request first and then input the image to be processed into a preset image processing model. The image to be processed can be any color image with the target to be processed, and this embodiment does not limit this.

[0092] It should be noted that the preset image processing model can be any color image with the target to be processed as input and output a model that segments the target region from the background. This embodiment does not limit this.

[0093] Step S20: Extract multi-scale features of the image to be processed using the preset image processing model, and fuse the multi-scale features to obtain the target region.

[0094] It should be understood that multi-scale features can be feature maps of multiple image sizes, such as feature maps of the original image size, 1 / 4 image size, 1 / 8 image size, and 1 / 16 image size. This embodiment does not limit this.

[0095] Understandably, fusing multi-scale features can be done by gradually integrating multi-scale features to reconstruct the details and edges of the target region, allowing the target region to naturally blend into the new background image.

[0096] Step S30: Overlay the target area with the new background template to obtain the image after background replacement.

[0097] It is understood that the new background template can be preset or uploaded by the user, and this embodiment does not impose any restrictions on this. In a specific implementation, the target area can also be overlaid with the new background template in the preset image processing model to obtain the image after background replacement.

[0098] This embodiment extracts multi-scale features from the image to be processed through a model, and performs background replacement after fusing the multi-scale features. This allows for the simultaneous capture of large-scale global information and small-scale detail information in the image, enabling more accurate processing of detailed targets and significantly improving the naturalness of the background replacement effect.

[0099] Reference Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the image processing method of this application, based on the above. Figure 1 The first embodiment shown illustrates a second embodiment of the image processing method of this application.

[0100] In the second embodiment, the preset image processing model includes an encoder and a decoder; step S20 includes:

[0101] Step S201: Extract the shallow, mid-level and high-level features of the image to be processed using the encoder.

[0102] It should be understood that, in order to improve the accuracy of the target region and achieve high-precision instance segmentation and background replacement in complex backgrounds and scenes with rich details, in this embodiment, shallow features, mid-level features and high-level features of the image to be processed are extracted by an encoder, and the shallow features, mid-level features and high-level features are fused by a decoder to obtain the target region.

[0103] In its implementation, the pre-defined image processing model consists of an encoder and a decoder. The encoder is responsible for extracting multi-scale high-level features of the image. Its structure includes a backbone network ResNet-101, a depthwise separable convolutional extended spatial pyramid pooling module (DS-ASPP), and a dual attention mechanism module (DAMM) that combines positional and channel attention modules. Through these modules, the encoder effectively captures key information at different scales and enhances the feature representation of foreground objects.

[0104] Step S202: The shallow features, the middle features, and the high-level features are fused by the decoder to obtain the target region.

[0105] Understandably, the decoder is responsible for restoring the spatial resolution and location information of an image, and its structure includes multi-scale feature fusion and upsampling modules. The decoder reconstructs the details and edges of the target region by progressively fusing the multi-scale features extracted by the encoder, allowing it to blend naturally into the new background image.

[0106] This embodiment extracts shallow, mid, and high-level features from the image to be processed through an encoder, and then fuses these features through a decoder to obtain the target region. This improves the accuracy of the target region and enables high-precision instance segmentation and background replacement in complex backgrounds and scenes with rich details.

[0107] Reference Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the image processing method of this application, based on the above. Figure 2 The second embodiment shown presents a third embodiment of the image processing method of this application.

[0108] In the third embodiment, the encoder includes: a backbone network and a deep processing module; step S201 includes:

[0109] Step S2011: Extract shallow features, mid-level features, and high-level feature maps of the image to be processed through the backbone network.

[0110] It should be understood that, in order to enhance features and / or reduce computational complexity and improve the clarity of the target region, in this embodiment, in addition to extracting shallow features, mid-level features and high-level feature maps of the image to be processed through the backbone network, a deep processing module is also used to perform deep processing on the high-level feature maps to obtain deep features of the image to be processed.

[0111] Understandably, the encoder's backbone network module can use the deep residual network ResNet-101, which consists of 101 trainable layers. Its core structure constructs residual blocks by introducing skip connections. Each residual block includes two 3x3 convolutional layers, followed by batch normalization and ReLU activation functions. Skip connections directly add the input to the output of the convolutional layer, thus forming residual learning. This connection can be identity mapping, i.e., directly adding the input to the output, or adjusting the number of channels through 1x1 convolutions to adapt to different dimensions of input and output, significantly mitigating the gradient vanishing problem.

[0112] Furthermore, to retain more detailed information and extract denser features, the backbone network includes: an initial convolutional layer, a pooling layer, and a residual block group. The residual block group includes multiple residual blocks, each of which includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. The first convolutional layer is used for dimensionality reduction, the second convolutional layer is used for feature extraction, and the third convolutional layer is used for dimensionality increase. Step S2011 includes: extracting shallow features of the image to be processed through the initial convolutional layer; downsampling the shallow features through the pooling layer; and resizing the downsampled features through the residual block group to obtain the mid-level features and high-level feature maps of the image to be processed.

[0113] It should be understood that, in this embodiment, the preset image processing model can be a Multi-Scale Channel and Position Attention Network (MSCP-Net) model. For ease of understanding, refer to... Figure 4 Please provide an explanation. Figure 4 This is a diagram illustrating the overall architecture of MSCP-Net, an embodiment of the image processing method of this application. Figure 4 In summary, the architecture of ResNet-101 can be divided into the following stages:

[0114] 1. Initial convolution (Layer 0) and pooling (Maxpool):

[0115] Layer0: A 7x7 convolutional layer with a stride of 2, followed by batch normalization and ReLU activation (the features obtained by Layer0 are the shallow features of the image to be processed).

[0116] Maxpool: A 3x3 max pooling layer with a stride of 2, used for downsampling.

[0117] 2. Residual Block Groups: ResNet-101 contains four residual block groups, each with several residual blocks. A variant where each residual block contains three convolutional layers is called the Bottleneck structure. Each Bottleneck residual block contains one 1x1 convolutional layer (for dimensionality reduction), one 3x3 convolutional layer, and one 1x1 convolutional layer (for dimensionality increase). The specific number of layers is as follows:

[0118] Layer 1: Contains 3 residual blocks, and the image size is reduced by 1 / 4 (the features obtained by Layer 1 processing are the mid-layer features of the image to be processed).

[0119] Layer 2: Contains 4 residual blocks, with the image size reduced by 1 / 8.

[0120] Layer 3: Contains 22 residual blocks, with the image size reduced by 1 / 16, and is the deepest part of the network.

[0121] Layer 4 contains 3 residual blocks. Adjusting the stride of the convolutional layers in Layer 4 to 1 helps preserve more detail and extract denser features. The final result is a high-level feature map that is 1 / 16 the size of the original image.

[0122] Step S2012: Perform depth processing on the high-level feature map through the depth processing module to obtain the deep features of the image to be processed.

[0123] It should be understood that the deep processing module may include an attention module and / or a DS-ASPP module, wherein the attention module is used to enhance features and the DS-ASPP module is used to reduce computational complexity.

[0124] In this embodiment, shallow features, mid-level features, and high-level feature maps of the image to be processed are extracted through a backbone network. Furthermore, a deep processing module is used to perform deep processing on the high-level feature maps to obtain deep features of the image to be processed, thereby enhancing features and / or reducing computational complexity.

[0125] Furthermore, in order to retain the excellent performance of the DS-ASPP module while also taking into account the excellent feature enhancement capabilities of the attention module, this embodiment connects the attention module and the DS-ASPP module in parallel. The deep processing module includes: an attention module and a DS-ASPP module, which are connected in parallel. Step S2012 includes: performing feature enhancement on the high-level feature map through the attention module to obtain a first feature map; decomposing the standard convolution in the high-level feature map through the DS-ASPP module to obtain a second feature map; and summing the first feature map and the second feature map element-wise to obtain the deep features of the image to be processed.

[0126] Understandably, existing image background replacement methods cannot effectively enhance the focus on important features while suppressing interference from unimportant features during multi-scale feature extraction, resulting in deficiencies in extracting the sharpness of foreground target edges. Therefore, to overcome these shortcomings, this embodiment introduces an attention module to enhance the focus on important features and effectively suppress interference from unimportant features.

[0127] In a specific implementation, the attention module can include a positional attention module and / or a channel attention module. The positional attention module can focus on the spatial relationships in the image and enhance the extraction of edge information of the foreground target, while the channel attention module can focus on the importance of different feature channels and improve the response to key features, thereby effectively improving the edge clarity of the foreground target. In particular, it performs well in the process of multi-scale feature extraction and solves the problem of insufficient foreground edge extraction in existing methods.

[0128] It should be understood that most existing image background replacement methods rely on deep learning algorithms with high computational complexity, requiring significant computational resources and time, making them difficult to deploy in real-time applications. Therefore, to overcome these shortcomings, this embodiment introduces a lightweight attention module, reducing computational load while maintaining high-precision segmentation performance. Simultaneously, it uses depthwise separable convolution to decompose standard convolution into two steps: depthwise convolution and pointwise convolution, thereby reducing the computational load and number of parameters in convolution operations. This allows the model to run efficiently even in resource-constrained environments, meeting the needs of real-time applications. This improvement enables the technology to be widely applied in scenarios requiring efficient segmentation and background replacement, such as augmented reality (AR), virtual reality (VR), and real-time video processing.

[0129] To retain the superior performance of the DS-ASPP module while also leveraging the excellent feature enhancement capabilities of the DAMM, a parallel connection between the DAMM structure and the DS-ASPP module was designed and implemented. In this parallel structure, the backbone network extracts image features, and the high-level feature maps are processed through two branches: the upper branch uses the DAMM structure, and the lower branch uses the DS-ASPP structure. The processed high-level feature maps are then summed element-wise, and finally, the number of channels in the fused feature map is reduced to 64 before being input into the decoder to obtain the prediction result. This application effectively addresses many challenges in instance segmentation, providing a more accurate solution for target background replacement tasks in complex scenes.

[0130] Furthermore, this embodiment introduces a dual attention module (DAMM) composed of a position attention module (PAM) and a channel attention module (CAM), which helps the model better preserve spatial and channel details when extracting images of people. This effectively solves problems such as slow fitting speed, inaccurate edge target segmentation, inconsistent segmentation of large-scale targets, and omissions in target extraction, thereby improving the robustness and accuracy of the model. The attention module includes a position attention module and a channel attention module. The step of enhancing the high-level feature map through the attention module to obtain a first feature map includes: enhancing the same features at different positions in the high-level feature map through the position attention module to obtain a third feature map; enhancing the feature information of different channels in the high-level feature map through the channel attention module to obtain a fourth feature map; and summing the third and fourth feature maps element-wise to obtain the first feature map.

[0131] Further, the step of enhancing the same features at different positions in the high-level feature map through the position attention module to obtain a third feature map includes: constructing a position attention matrix corresponding to the high-level feature map, wherein the position matrix includes multiple first influence factors, the first influence factors being the influence factors of one position in the high-level feature map on another position; enhancing the same features at different positions in the high-level feature map according to the first influence factors to obtain a third feature map.

[0132] Further, the step of enhancing the feature information of different channels in the high-level feature map through the channel attention module to obtain the fourth feature map includes: constructing a channel attention matrix corresponding to the high-level feature map, wherein the channel attention matrix includes multiple second influence factors, the second influence factors being the influence factors of one channel on another channel in the high-level feature map; and enhancing the feature information of different channels in the high-level feature map according to the second influence factors to obtain the fourth feature map.

[0133] For ease of understanding, please refer to Figure 4 Please provide an explanation. Figure 4 This is a diagram illustrating the overall architecture of MSCP-Net, an embodiment of the image processing method of this application. Figure 4This paper introduces a dual attention module (DAMM) consisting of a Position Attention Module (PAM) and a Channel Attention Module (CAM). The PAM focuses on the spatial relationships within the image, enhancing attention to the target region and enabling the network to more accurately capture spatial information of foreground objects. This is particularly important when dealing with complex backgrounds or objects with rich details, significantly improving edge sharpness and segmentation accuracy. The CAM weights feature channels, highlighting key channels and suppressing interference from unimportant channels. This helps improve the network's response to important features, especially during multi-scale feature fusion, better capturing and enhancing key features. DAMM effectively combines PAM and CAM, integrating both spatial and channel information. The Position Attention Module accurately locates the target region, while the Channel Attention Module optimizes feature representation. This dual attention module enables the network to perform well in instance segmentation and background replacement tasks, handling more complex scenes and details, thereby improving overall segmentation accuracy and the naturalness of background replacement.

[0134] For ease of understanding, please refer to Figure 5 Please provide an explanation. Figure 5 This is a diagram illustrating the architecture of the position attention module in one embodiment of the image processing method of this application. Figure 5 PAM selectively enhances the correlation between any two points in an image based on semantic relevance. Each pixel can collect effective contextual information from semantically relevant pixels throughout the image, allowing identical features at different locations to mutually enhance each other and improve semantic segmentation capabilities. Local features can be obtained through the backbone network. The matrix is ​​generated by performing a 1×1 convolution operation on feature A to obtain vectors B and C with C / 8 channels, and vector D with C channels. Then, the dimensions of matrix B are transformed into B′. N = H × W, and simultaneously transpose matrix B' to B T Transform dimension C into dimension C′ where N = H × W, connect C′ and B T Perform matrix multiplication to obtain a (H×W)×(H×W) matrix. Finally, use SoftMax to calculate the positional attention matrix. The resulting matrix S can be viewed as the weights obtained through the position attention module. The calculation result of each row represents the dependency relationship between all pixels and a certain pixel. The larger the SoftMax value, the stronger the relative dependency. The calculation formula is as follows:

[0135]

[0136] In the formula: B i Representative matrix The i-th element in C j Representative matrix The j-th element in the current channel, where N represents the number of elements in the current channel, and S... ji This represents the influence factor of position i on position j (i.e., the first influence factor).

[0137] Matrix D is multiplied by the positional attention matrix after dimensionality transformation, and then summed element-wise with the original feature map A to obtain the final result E (i.e., the third feature map). The calculation formula is as follows:

[0138]

[0139] In the formula: D i Representation matrix The i-th element in S; ji Let be the i-th element of matrix S; α is a learnable parameter, initially set to 0.

[0140] As can be seen from the above formula, the final feature E of each position is the weighted sum of all position features and the original feature. Therefore, the position attention module has a global context view and attempts to selectively aggregate the context according to the position, so that similar semantic features can promote each other and maintain semantic consistency.

[0141] For ease of understanding, please refer to Figure 6 Please provide an explanation. Figure 6 This is a diagram illustrating the channel attention module architecture of an embodiment of the image processing method of this application. Figure 6 In this model, because there are different relationships between feature maps in different channels, CAM extracts semantic information between different channels and highlights related feature maps by using different weights. CAM directly... Calculate the channel attention matrix Transform the dimension of feature matrix A into Where N = H × W, then A′ and A′ T Matrix multiplication is performed, and X is calculated using the SoftMax function. The maximum similarity value in each row of X is subtracted to increase attention to other similarities. The attention weights are then multiplied by A′, and the result is multiplied by a learnable coefficient β. Finally, the summation of this summation with the feature map A yields the final result E. The channel attention map X is calculated using A. ji The calculation is as follows:

[0142]

[0143] In the formula: X ji A represents the influence factor (i.e., the second influence factor) of the i-th channel on the j-th channel. i Represents the value of the i-th element. This represents the value of the j-th element of the transpose matrix. The channel attention matrix is ​​multiplied by the input feature map, multiplied by a learnable parameter β, where β is initially 0. Then, it is summed element-wise with the input feature map to obtain the final output E (i.e., the fourth feature map), calculated as follows:

[0144]

[0145] As shown in the above formula, the final feature of each channel is a weighted sum of the features of all channels and the original channel features. Therefore, CAM can enhance the feature information of different channels, enabling the neural network to exert a more efficient capability.

[0146] Furthermore, in the ResNet-101 backbone network extraction process, the overall receptive field grows slowly with the increase of the number of convolutional layers, at the cost of image information loss and size reduction. The ASPP module introduced in this application increases the network's receptive field without downsampling. The step of decomposing the standard convolution in the high-level feature map using the DS-ASPP module to obtain the second feature map includes: decomposing the standard convolution in the high-level feature map into depthwise separable convolutions using the DS-ASPP module; and performing parallel sampling with different dilation rates on the depthwise separable convolutions to obtain the second feature map.

[0147] For ease of understanding, please refer to Figure 4 Please provide an explanation. Figure 4 This is a diagram illustrating the overall architecture of MSCP-Net, an embodiment of the image processing method of this application. Figure 4In the ResNet-101 backbone network extraction process, the overall receptive field grows slowly with the increase of the number of convolutional layers, at the cost of image information loss and size reduction. This application introduces the ASPP module, which increases the network's receptive field without downsampling. As the network structure deepens and a dual attention module is introduced, the number of network parameters and computational cost continuously increase, leading to larger storage space requirements and longer training time. Therefore, this application introduces Depthwise Separable Convolution (DSC) into the ASPP structure. This structure not only directly reduces computational cost and improves training speed but also avoids the gradient explosion problem. The proposed DS-ASPP module includes a 1×1 convolution, three 3×3 depthwise separable dilated convolutions to expand the receptive field, with dilation rates of (6, 12, 18) and a global pooling operation. DS-ASPP achieves parallel sampling with different dilation rates, thereby capturing multi-scale contextual information and reducing information loss. During the convolution operation, if P... c For DSC parameters, P n For the standard convolution parameter count, C c For DSC computation, C n The computational cost is that of standard convolution. The ratio of parameters and computational cost between depthwise separable convolution and standard convolution is shown below:

[0148]

[0149] In the formula: C is the number of input channels, M is the number of output channels, k is the kernel size, and H×W is the output feature size. It can be seen that the computational complexity of depthwise separable convolution is significantly reduced compared to standard convolution. Its computational cost is only 1 / k of that of standard convolution. 2 It sacrifices extremely low precision but gains an order-of-magnitude reduction in computational load.

[0150] In the third embodiment, step S202 includes:

[0151] Step S2021: Upsample the high-level features and fuse the upsampled high-level features with the mid-level features to obtain the first fused feature.

[0152] Step S2022: Perform convolution and upsampling processing on the first fused feature, and then fuse the processed first fused feature with the shallow feature to obtain the second fused feature.

[0153] Step S2023: Perform convolution and upsampling processing on the second fused feature to obtain the target region.

[0154] For ease of understanding, please refer to Figure 4 Please provide an explanation. Figure 4 This is a diagram illustrating the overall architecture of MSCP-Net, an embodiment of the image processing method of this application. Figure 4 In this architecture, the decoder's task is to restore the spatial resolution and location information of the image. During the decoding stage, the Feature Fusion (MS-FF) module fuses the mid-level feature map from the ResNet-101 with the high-level feature map output from the decoding stage after a 4x upsampling. Then, it fuses this with the shallow feature map from the ResNet-101 after a 3×3 convolution and a 2x upsampling. Finally, it performs a 3×3 convolution and a 2x upsampling to obtain the target segmentation and recognition result. First, the feature map generated in the encoding stage is upsampled by 4x bilinear interpolation and fused with the mid-level feature map generated by Layer 1 of the encoder backbone network. Second, the result is upsampled by 2x bilinear interpolation after a 3×3 convolution and fused with the feature map generated by Layer 0 of the backbone network after a 1×1 convolution, thereby enhancing the network's ability to acquire multi-scale contextual features. Finally, the feature map is convolved by 3×3 and then upsampled by 2x bilinear interpolation to generate a more refined instance segmentation and background replacement result. This design improves segmentation accuracy while optimizing the network's spatial information recovery performance.

[0155] This embodiment improves segmentation accuracy and robustness while enhancing model training and inference efficiency by introducing an innovative encoder-decoder structure, multi-scale feature fusion, depthwise separable dilated convolution, and dual attention modules. This provides a more accurate and efficient solution for instance segmentation and target background replacement in complex scenarios.

[0156] Reference Figure 7 , Figure 7 This is a flowchart illustrating the fourth embodiment of the image processing method of this application. Based on the above embodiments, the fourth embodiment of the image processing method of this application is proposed.

[0157] In the fourth embodiment, before step S10, the method further includes:

[0158] Step S01: Obtain scene images of various scenarios and annotate the scene images to obtain a binary mask image, wherein the target region pixels are annotated with the first value and the background pixels are annotated with the second value.

[0159] It should be understood that the MSCP-Net network is trained using any color image containing the target to be processed, outputting the target region segmented from the background, and then replacing it with any desired background. The encoding structure is mainly used for multi-scale feature extraction, while the decoding structure is used for feature fusion and recovery of spatial detail information. Corresponding segmentation models are built for different targets; that is, for each type of target task scenario, a specific model needs to be trained to handle that situation. First, the MSCP-Net model parameters are initialized, fixing the cropped size of the training set, the number of inputs per batch, and the initial learning rate. Training is performed on the training set using the PyTorch framework. The loss value between the true label and the probability map is calculated based on the segmentation loss function, which consists of cross-entropy loss and mean squared loss. The loss value during model training is monitored, and predictions are made on the test set. Evaluation metrics (Acc, Precision, Recall, F1-Score) are analyzed. Parameter optimization is performed by adjusting the learning rate and the number of iterations. The network parameters are continuously updated through backpropagation until the network converges.

[0160] Step S02: Adjust the scene image and the binary mask image to the same size.

[0161] Step S03: Perform data augmentation on the resized scene image and the resized binary mask image.

[0162] Step S04: Construct training set, validation set and test set based on the data augmented scene image and the data augmented binary mask image.

[0163] Step S05: Train the initial image processing model based on the training set, the validation set, and the test set to obtain a preset image processing model.

[0164] For ease of understanding, the following examples are provided, but are not intended to limit this application. As an example, the model training steps include:

[0165] 1. Data preparation and experimental environment.

[0166] The selection of datasets during model training is very flexible. This invention can effectively extract various targets from various images and then replace the background as needed. Therefore, the ground truth labels of the training set are the background region and the target region.

[0167] Image data: Collect images of various scenes containing the targets to be processed by the task, ensuring image diversity, including different backgrounds, lighting conditions, target categories, etc.

[0168] Annotation data: Image annotation tools (such as LabelMe, VGG Image Annotator, etc.) are used to annotate the targets in each image at the pixel level. The annotation result is usually a binary mask of the same size as the original image, where target area pixels are labeled as 1 and background pixels are labeled as 0.

[0169] Unify image and mask dimensions: Adjust all images and their corresponding masks to the same size, such as 256x256 or 512x512. Image scaling can be performed using OpenCV or Pillow.

[0170] Data augmentation: To increase the generalization ability of the model, data augmentation operations are performed on the image and mask, such as rotation, flipping, cropping, brightness adjustment, etc., to ensure that the image and the corresponding mask are operated synchronously.

[0171] Image classification: The sample database is divided into training set, validation set and test set in a ratio of 8:1:1.

[0172] 2. Evaluation indicators.

[0173] To provide a comprehensive evaluation, six evaluation metrics can be used on the test set, including accuracy, precision, recall, intersection-over-union (IoU), F1 score, and Bayesian error rate (BER). Portrait matting is a segmentation problem; pixels in the portrait region are positive examples, and pixels in the background region are negative examples. Therefore, all predictions can be categorized into a confusion matrix, including true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The formulas for calculating the six metrics are as follows:

[0174]

[0175] This embodiment acquires scene images from various scenarios, annotates these images to obtain binary mask images, adjusts the scene images and binary mask images to the same size, performs data augmentation on the resized scene images and binary mask images, constructs training, validation, and test sets based on the data-augmented scene images and binary mask images, and trains an initial image processing model using the training, validation, and test sets to obtain a preset image processing model, thereby improving the accuracy of model training.

[0176] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the image processing method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0177] This application also provides an image processing apparatus, please refer to... Figure 8 The image processing apparatus includes:

[0178] The model input module 10 is used to input the image to be processed into a preset image processing model in response to an image processing request;

[0179] The model processing module 20 is used to extract multi-scale features of the image to be processed through the preset image processing model, and to fuse the multi-scale features to obtain the target region;

[0180] Background replacement module 30 is used to overlay the target area with a new background template to obtain a background-replaced image.

[0181] The image processing apparatus provided in this application, employing the image processing method described in the above embodiments, can solve the technical problem that existing image background replacement methods, typically based on single-scale feature extraction, suffer from low segmentation accuracy and unnatural background replacement effects when dealing with complex backgrounds or detailed processing. Compared with the prior art, the beneficial effects of the image processing apparatus provided in this application are the same as those of the image processing method provided in the above embodiments, and other technical features in the image processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0182] This application provides an image processing apparatus, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image processing method in Embodiment 1 above.

[0183] The following is for reference. Figure 9 This document illustrates a structural schematic diagram of an image processing device suitable for implementing embodiments of this application. The image processing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The image processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0184] like Figure 9As shown, the image processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the image processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the image processing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows an image processing device with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0185] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0186] The image processing device provided in this application, employing the image processing method described in the above embodiments, can solve the technical problem that existing image background replacement methods, which are typically based on single-scale feature extraction, suffer from low segmentation accuracy and unnatural background replacement effects when dealing with complex backgrounds or detailed processing. Compared with the prior art, the beneficial effects of the image processing device provided in this application are the same as those of the image processing method provided in the above embodiments, and other technical features in this image processing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0187] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0188] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0189] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the image processing method described in the above embodiments.

[0190] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), or flash memory, optical fiber, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0191] The aforementioned computer-readable storage medium may be included in an image processing device or may exist independently without being assembled into an image processing device.

[0192] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an image processing device, cause the image processing device to perform the aforementioned image processing method.

[0193] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0194] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0195] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0196] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image processing method. This solves the technical problem that existing image background replacement methods, which are typically based on single-scale feature extraction, suffer from low segmentation accuracy and unnatural background replacement effects when dealing with complex backgrounds or detailed processing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image processing method provided in the above embodiments, and will not be repeated here.

[0197] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method described above.

[0198] The computer program product provided in this application can solve the technical problem that existing image background replacement methods are usually based on single-scale feature extraction, resulting in low segmentation accuracy and unnatural background replacement effects when dealing with complex backgrounds or detailed processing. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the image processing methods provided in the above embodiments, and will not be repeated here.

[0199] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

[0200] This application discloses A1, an image processing method, the image processing method comprising:

[0201] In response to an image processing request, the image to be processed is input into a preset image processing model;

[0202] The preset image processing model is used to extract multi-scale features from the image to be processed, and the multi-scale features are fused to obtain the target region.

[0203] The target area is overlaid with the new background template to obtain the image after background replacement.

[0204] A2. The image processing method as described in A1, wherein the preset image processing model includes: an encoder and a decoder;

[0205] The step of extracting multi-scale features from the image to be processed using the preset image processing model and fusing the multi-scale features to obtain the target region includes:

[0206] The encoder extracts shallow, mid, and high-level features from the image to be processed.

[0207] The target region is obtained by fusing the shallow features, the mid-level features, and the high-level features using the decoder.

[0208] A3. The image processing method as described in A2, wherein the encoder comprises: a backbone network and a depth processing module;

[0209] The step of extracting shallow, mid-level, and high-level features of the image to be processed using the encoder includes:

[0210] The shallow features, mid-level features, and high-level feature maps of the image to be processed are extracted through the backbone network.

[0211] The high-level feature map is processed by the depth processing module to obtain the deep features of the image to be processed.

[0212] A4. The image processing method as described in A3, wherein the depth processing module includes: an attention module and a DS-ASPP module, wherein the attention module and the DS-ASPP module are connected in parallel;

[0213] The step of processing the high-level feature map through the depth processing module to obtain the deep features of the image to be processed includes:

[0214] The attention module is used to enhance the features of the high-level feature map to obtain a first feature map.

[0215] The standard convolution in the high-level feature map is decomposed using the DS-ASPP module to obtain the second feature map;

[0216] The deep features of the image to be processed are obtained by summing the first feature map and the second feature map element by element.

[0217] A5. The image processing method as described in A4, wherein the attention module includes: a position attention module and a channel attention module;

[0218] The step of performing feature enhancement on the high-level feature map through the attention module to obtain the first feature map includes:

[0219] The positional attention module enhances the same features at different locations in the high-level feature map to obtain a third feature map.

[0220] The channel attention module enhances the feature information of different channels in the high-level feature map to obtain a fourth feature map;

[0221] The first feature map is obtained by summing the third and fourth feature maps element by element.

[0222] A6. The image processing method as described in A5, wherein the step of enhancing the same features at different locations in the high-level feature map through the positional attention module to obtain a third feature map includes:

[0223] Construct a positional attention matrix corresponding to the high-level feature map, wherein the positional matrix includes multiple first influence factors, and the first influence factors are the influence factors of one position on another position in the high-level feature map;

[0224] The same features at different locations in the high-level feature map are enhanced based on the first influence factor to obtain the third feature map.

[0225] A7. The image processing method as described in A5, wherein the enhancement of feature information of different channels in the high-level feature map by the channel attention module to obtain a fourth feature map includes:

[0226] Construct the channel attention matrix corresponding to the high-level feature map, wherein the channel attention matrix includes multiple second influence factors, and the second influence factors are the influence factors of one channel on another channel in the high-level feature map;

[0227] The feature information of different channels in the advanced feature map is enhanced based on the second influence factor to obtain the fourth feature map.

[0228] A8. The image processing method as described in A4, wherein the step of decomposing the standard convolution in the high-level feature map using the DS-ASPP module to obtain the second feature map includes:

[0229] The standard convolution in the high-level feature map is decomposed into depthwise separable convolutions using the DS-ASPP module.

[0230] The second feature map is obtained by parallel sampling with different dilation rates through the depthwise separable convolution.

[0231] A9. The image processing method as described in A3, wherein the backbone network includes: an initial convolutional layer, a pooling layer, and a residual block group, wherein the residual block group includes multiple residual blocks, and the residual block includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein the first convolutional layer is used for dimensionality reduction, the second convolutional layer is used for feature extraction, and the third convolutional layer is used for dimensionality increase.

[0232] The step of extracting shallow features, mid-level features, and high-level feature maps of the image to be processed through the backbone network includes:

[0233] The shallow features of the image to be processed are extracted through the initial convolutional layer;

[0234] The shallow features are downsampled using the pooling layer;

[0235] The residual block group is used to resize the downsampled features to obtain the mid-level and high-level feature maps of the image to be processed.

[0236] A10. The image processing method as described in A2, wherein fusing the shallow features, the mid-level features, and the high-level features through the decoder to obtain the target region includes:

[0237] The high-level features are upsampled, and the upsampled high-level features are fused with the mid-level features to obtain a first fused feature;

[0238] The first fused feature is subjected to convolution and upsampling processing, and the processed first fused feature is fused with the shallow feature to obtain the second fused feature;

[0239] The second fused feature is subjected to convolution and upsampling processing to obtain the target region.

[0240] A11. The image processing method according to any one of A1 to A10, further comprising, before inputting the image to be processed into a preset image processing model in response to an image processing request:

[0241] Scene images of various scenarios are acquired and labeled to obtain a binary mask image, wherein the target region pixels are labeled with a first value and the background pixels are labeled with a second value;

[0242] Adjust the scene image and the binary mask image to the same size;

[0243] Data augmentation is performed on the resized scene image and the resized binary mask image;

[0244] The training set, validation set, and test set are constructed based on the data-augmented scene image and the data-augmented binary mask image.

[0245] The initial image processing model is trained using the training set, the validation set, and the test set to obtain a preset image processing model.

[0246] This application also discloses B12, an image processing apparatus, the image processing apparatus comprising:

[0247] The model input module is used to input the image to be processed into a preset image processing model in response to an image processing request;

[0248] The model processing module is used to extract multi-scale features of the image to be processed through the preset image processing model, and to fuse the multi-scale features to obtain the target region;

[0249] The background replacement module is used to overlay the target area with a new background template to obtain an image after background replacement.

[0250] B13. The image processing apparatus as described in B12, wherein the preset image processing model includes an encoder and a decoder;

[0251] The model processing module is further configured to extract shallow, mid-level, and high-level features of the image to be processed through the encoder; and to fuse the shallow, mid-level, and high-level features through the decoder to obtain the target region.

[0252] B14. The image processing apparatus as described in B13, wherein the encoder comprises: a backbone network and a depth processing module;

[0253] The model processing module is further configured to extract shallow features, mid-level features, and high-level feature maps of the image to be processed through the backbone network; and to perform deep processing on the high-level feature maps through the deep processing module to obtain the deep features of the image to be processed.

[0254] B15. The image processing apparatus as described in B14, wherein the depth processing module includes: an attention module and a DS-ASPP module, wherein the attention module and the DS-ASPP module are connected in parallel;

[0255] The model processing module is further configured to enhance the high-level feature map through the attention module to obtain a first feature map; decompose the standard convolution in the high-level feature map through the DS-ASPP module to obtain a second feature map; and sum the first feature map and the second feature map element by element to obtain the deep features of the image to be processed.

[0256] B16. The image processing apparatus as described in B15, wherein the attention module includes: a position attention module and a channel attention module;

[0257] The model processing module is further configured to enhance the same features at different positions in the high-level feature map through the position attention module to obtain a third feature map; enhance the feature information of different channels in the high-level feature map through the channel attention module to obtain a fourth feature map; and sum the third feature map and the fourth feature map element by element to obtain a first feature map.

[0258] B17. The image processing apparatus as described in B16, wherein the model processing module is further configured to construct a position attention matrix corresponding to the high-level feature map, wherein the position matrix includes a plurality of first influence factors, the first influence factors being the influence factors of a certain position on another position in the high-level feature map; and to enhance the same features at different positions in the high-level feature map according to the first influence factors to obtain a third feature map.

[0259] This application also discloses C18, an image processing apparatus, the image processing apparatus comprising: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program, when executed by the processor, implements the image processing method described above.

[0260] This application also discloses D19, a storage medium storing an image processing program, which, when executed by a processor, implements the image processing method described above.

[0261] This application also discloses E20, a computer program product including an image processing program that, when executed by a processor, implements the image processing method as described above.

Claims

1. An image processing method, characterized by, The image processing method includes: In response to an image processing request, the image to be processed is input into a preset image processing model; The preset image processing model is used to extract multi-scale features from the image to be processed, and the multi-scale features are fused to obtain the target region. The target area is overlaid with the new background template to obtain the image after background replacement.

2. The image processing method of claim 1, wherein, The preset image processing model includes: an encoder and a decoder; The step of extracting multi-scale features from the image to be processed using the preset image processing model and fusing the multi-scale features to obtain the target region includes: The encoder extracts shallow, mid, and high-level features from the image to be processed. The target region is obtained by fusing the shallow features, the mid-level features, and the high-level features using the decoder.

3. The image processing method of claim 2, wherein, The encoder includes: a backbone network and a depth processing module; The step of extracting shallow, mid-level, and high-level features of the image to be processed using the encoder includes: The shallow features, mid-level features, and high-level feature maps of the image to be processed are extracted through the backbone network. The high-level feature map is processed by the depth processing module to obtain the deep features of the image to be processed.

4. The image processing method of claim 3, wherein, The deep processing module includes an attention module and a DS-ASPP module, wherein the attention module and the DS-ASPP module are connected in parallel. The step of processing the high-level feature map through the depth processing module to obtain the deep features of the image to be processed includes: The attention module is used to enhance the features of the high-level feature map to obtain a first feature map. The standard convolution in the high-level feature map is decomposed using the DS-ASPP module to obtain the second feature map; The deep features of the image to be processed are obtained by summing the first feature map and the second feature map element by element.

5. The image processing method of claim 4, wherein, The attention module includes: a position attention module and a channel attention module; The step of performing feature enhancement on the high-level feature map through the attention module to obtain the first feature map includes: The positional attention module enhances the same features at different locations in the high-level feature map to obtain a third feature map. The channel attention module enhances the feature information of different channels in the high-level feature map to obtain a fourth feature map; The first feature map is obtained by summing the third and fourth feature maps element by element.

6. The image processing method as described in claim 5, characterized in that, The step of enhancing the same features at different locations in the high-level feature map through the positional attention module to obtain a third feature map includes: Construct a positional attention matrix corresponding to the high-level feature map, wherein the positional matrix includes multiple first influence factors, and the first influence factors are the influence factors of one position on another position in the high-level feature map; The same features at different locations in the high-level feature map are enhanced based on the first influence factor to obtain the third feature map.

7. An image processing apparatus, characterized in that, The image processing device includes: The model input module is used to input the image to be processed into a preset image processing model in response to an image processing request; The model processing module is used to extract multi-scale features of the image to be processed through the preset image processing model, and to fuse the multi-scale features to obtain the target region; The background replacement module is used to overlay the target area with a new background template to obtain an image after background replacement.

8. An image processing device, characterized in that, The image processing device includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program, when executed by the processor, implements the image processing method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores an image processing program, which, when executed by a processor, implements the image processing method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes an image processing program that, when executed by a processor, implements the image processing method as described in any one of claims 1 to 6.

Images