An animal cutout method and device, computer equipment and storage medium

Abstract

The application discloses an animal matting method and device, computer equipment and a storage medium, the method comprising: inputting an animal image into a MobileNetv3 network to extract a first feature map; performing cross-scale feature fusion on an edge mask map through edge enhancement to obtain a second feature map; performing pixel information correlation processing on the first feature map by using context attention to obtain a third feature map; splicing the second feature map and the third feature map to obtain a first spliced feature map, inputting the first spliced feature map into multi-scale hollow convolution to perform feature fusion and obtain a fourth feature map; splicing the first spliced feature map and the fourth feature map to obtain a second spliced feature map, and performing convolution on the second spliced feature map to obtain a matting result. The application uses edge enhancement to perform cross-scale feature fusion, improves the detail processing capability for animal hair, uses context attention to perform pixel information correlation processing, improves the capability of obtaining the relationship between pixels in animal matting, and thus improves the accuracy of animal matting.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an animal image cutout method, apparatus, computer device, and storage medium. Background Technology

[0002] In the field of image processing, an image can be viewed as consisting of a foreground and a background. Image matting is the process of separating the foreground and background of a given image. Animal matting, on the other hand, uses the location of the animal in the image as the foreground and the location of the non-animal in the image as the background. As people's living standards improve, pets occupy an increasingly important place in people's lives, and the demand for pet image editing is growing. At the same time, due to the development of the internet, animal matting technology has been increasingly widely used in many fields, such as pet grooming, animal photography, and game development. Furthermore, with the continuous advancement and development of animal matting technology, its application scenarios are constantly expanding, such as in advertising design, image compositing, virtual reality and augmented reality, image editing, and film special effects.

[0003] In animal matting technology, existing models generally suffer from several shortcomings that significantly impact the accuracy of the results. For instance, as a pixel-level computer vision task, animal matting requires models to identify each pixel in an animal image and predict its probability of being in the foreground. Each pixel in an animal image has a strong correlation with its surrounding pixels, but current models generally have a weak ability to capture these inter-pixel relationships, resulting in poor foreground prediction. Furthermore, most animals are characterized by abundant fur, requiring animal matting models to possess stronger detail processing capabilities compared to ordinary matting models. This is a challenging issue in animal matting, and current models generally lack techniques to address the issue of animal fur, leading to subpar matting results.

[0004] Therefore, how to improve the ability to obtain the relationship between pixels and the ability to process the details of animal hair in animal matting, thereby improving the accuracy of animal matting, is a problem to be solved by those skilled in the art. Summary of the Invention

[0005] This invention provides an animal cutout method, apparatus, computer device, and storage medium, aiming to improve the ability to obtain pixel relationships and process details of animal fur in animal cutout, thereby improving the accuracy of animal cutout.

[0006] In a first aspect, embodiments of the present invention provide an animal image cutout method, comprising:

[0007] The process involves acquiring an animal image to be cut out, inputting the animal image to be cut out into a MobileNetv3 network for feature extraction to obtain a first feature map, and then using an edge detection operator to process the animal image to be cut out to obtain a corresponding edge mask map. Finally, the edge mask map is fused across scales using an edge enhancement module to obtain a second feature map.

[0008] The first feature map is processed by using a context attention module to associate pixel information, resulting in a third feature map.

[0009] The second and third feature maps are concatenated to obtain a first concatenated feature map, which is then input into a multi-scale dilated convolution module for multi-scale feature fusion to obtain a fourth feature map.

[0010] The first and fourth feature maps are spliced ​​together to obtain a second spliced ​​feature map, which is then convolved to obtain the final animal image matting result.

[0011] Secondly, embodiments of the present invention provide an animal image cutout device, comprising:

[0012] The feature extraction unit is used to acquire the animal image to be cut out, input the animal image to be cut out into the MobileNetv3 network for feature extraction to obtain a first feature map; and use an edge detection operator to perform image processing on the animal image to be cut out to obtain a corresponding edge mask map, and use an edge enhancement module to perform cross-scale feature fusion on the edge mask map to obtain a second feature map.

[0013] The pixel information association unit is used to perform pixel information association processing on the first feature map using the context attention module to obtain the third feature map;

[0014] The first feature concatenation unit is used to concatenate the second feature map and the third feature map to obtain the first concatenated feature map, and input the first concatenated feature map into the multi-scale dilated convolution module for multi-scale feature fusion to obtain the fourth feature map;

[0015] The second stitching feature unit is used to stitch the first stitching feature map and the fourth feature map to obtain the second stitching feature map, and to perform convolution processing on the second stitching feature map to obtain the final animal image matting result.

[0016] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the animal cutout method as described in the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the animal cutout method as described in the first aspect.

[0018] This invention discloses an animal image matting method, apparatus, and related medium. The method includes: acquiring an animal image to be matted; inputting the animal image to be matted into a MobileNetv3 network for feature extraction to obtain a first feature map; performing image processing on the animal image to be matted using an edge detection operator to obtain a corresponding edge mask map, and performing cross-scale feature fusion on the edge mask map using an edge enhancement module to obtain a second feature map; performing pixel information association processing on the first feature map using a context attention module to obtain a third feature map; performing feature concatenation on the second and third feature maps to obtain a first concatenated feature map, and inputting the first concatenated feature map into a multi-scale dilated convolution module for multi-scale feature fusion to obtain a fourth feature map; performing feature concatenation on the first and fourth feature maps to obtain a second concatenated feature map, and performing convolution processing on the second concatenated feature map to obtain the final animal image matting result. This invention utilizes an edge enhancement module for cross-scale feature fusion, improving the processing capability for animal fur. It also employs a contextual attention module for pixel information association, enabling better acquisition of relationships between pixels in animal images and enhancing the prediction ability for animal foregrounds. Furthermore, it utilizes a multi-scale dilated convolution module for multi-scale feature fusion, obtaining a larger receptive field and thus acquiring more pixel information, thereby improving the animal matting effect. Therefore, this invention, by combining the MobileNetv3 network, edge enhancement module, contextual attention module, and multi-scale dilated convolution module for animal image matting, can effectively improve the accuracy of animal matting. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is a flowchart illustrating an image matting method provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the network structure of an image matting method provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of a sub-process of an image matting method provided in an embodiment of the present invention;

[0023] Figure 4 This is a network diagram of the edge enhancement module in an image matting method provided in an embodiment of the present invention;

[0024] Figure 5 This is a network diagram of the context attention module in an image matting method provided in an embodiment of the present invention;

[0025] Figure 6 An example image showing the experimental results of a data matting method provided in an embodiment of the present invention;

[0026] Figure 7 A schematic block diagram of an image cutout device provided in an embodiment of the present invention;

[0027] Figure 8 This is a schematic block diagram of a cutout device provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0030] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0031] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] Please see below. Figure 1 , Figure 1 The flowchart of an animal image cutout method provided in an embodiment of the present invention is shown, specifically including steps S101 to S104.

[0033] S101. Obtain the animal image to be cut out, input the animal image to be cut out into the MobileNet v3 network for feature extraction to obtain a first feature map; and use an edge detection operator to perform image processing on the animal image to be cut out to obtain a corresponding edge mask map, and perform cross-scale feature fusion on the edge mask map through an edge enhancement module to obtain a second feature map.

[0034] S102. The first feature map is processed by pixel information association using the context attention module to obtain the third feature map;

[0035] S103. The second feature map and the third feature map are concatenated to obtain a first concatenated feature map, and the first concatenated feature map is input into the multi-scale dilated convolution module for multi-scale feature fusion to obtain a fourth feature map.

[0036] S104. Perform feature stitching on the first stitched feature map and the fourth feature map to obtain a second stitched feature map, and perform convolution processing on the second stitched feature map to obtain the final animal image matting result.

[0037] In this embodiment, combined with Figure 2As shown, firstly, the MobileNetv3 network is used to extract features from the animal image to be matted, thus obtaining the first feature map. Simultaneously, an edge detection operator is used to process the animal image to obtain a corresponding edge mask map. Then, an edge enhancement module is used to perform cross-scale feature fusion on the edge mask map, resulting in the second feature map. Next, the first feature map is input into a contextual attention module (CAM) for pixel information association processing, thus obtaining the third feature map. In practical applications, the contextual attention module can be embedded within the MobileNetv3 network. Then, the second and third feature maps are concatenated to obtain the corresponding first concatenated feature map. This first concatenated feature map is then input into a multi-scale dilated convolution module, which performs multi-scale feature fusion on the first concatenated feature map, resulting in the fourth feature map. Finally, the first and fourth feature maps are concatenated to form the second concatenated feature map, which is then convolved to obtain the final animal image matting result.

[0038] This embodiment utilizes an edge enhancement module for cross-scale feature fusion to enhance the perception of objects of different sizes in animal matting, thereby improving the processing capability for animal fur. Simultaneously, a contextual attention module performs pixel information association processing on the feature maps corresponding to the animal image to better obtain the relationships between pixels in the animal image, thus improving the prediction capability for the animal foreground in animal matting. Furthermore, a multi-scale dilated convolution module performs multi-scale feature fusion on the stitched feature maps to obtain a larger receptive field, thereby improving the predictive performance of animal matting. This embodiment effectively improves the accuracy of animal matting by combining the different processing capabilities of each module to perform animal image matting.

[0039] Combination Figure 3 As shown, in one embodiment, step S101 includes steps S301 to S303.

[0040] S301. Use the convolutional layers in the MobileNetv3 network to extract image features from the animal image to be cut out, and obtain the first image features;

[0041] S302. The first image features are sequentially subjected to dimensionality upscaling, enhanced representation processing, and dimensionality reduction processing using the inverted residual module in the MobileNetv3 network to obtain the second image features.

[0042] S303. Input the second image features into the Squeeze-and-Excitation module in the MobileNetv3 network, and use the Squeeze-and-Excitation module to perform feature enhancement processing on the second image features to obtain the first feature map.

[0043] In this embodiment, the animal image to be cut out is first input into the MobileNetv3 network, and image features are extracted using convolutional layers. The first image features are then output to the Inverted Residual Block. Next, the Inverted Residual Block is used to sequentially perform dimensionality upscaling, representation enhancement, and dimensionality reduction on the first image features to output the second image features. Finally, the second image features are input into the Square-and-Excitation Block for feature enhancement to obtain the first feature map.

[0044] In this embodiment, the MobileNetv3 network extracts image features using a series of convolutional layers to locate the approximate position of the animal in the image to be cut out. These convolutional layers typically employ lightweight convolutional operations, such as depthwise separable convolution, to reduce computational cost and parameter count, thereby improving the processing speed and performance of the MobileNetv3 network. Furthermore, the input to the MobileNetv3 network (the animal image to be cut out) is usually an RGB image, and RGB images generally require preprocessing before being input into the MobileNetv3 network to adapt the input image to the network's requirements. Specifically, RGB image preprocessing may include scaling, normalization, and other procedures.

[0045] In one specific embodiment, step S302 includes:

[0046] The first image features are up-dimensionalized using the dilated convolutional layer in the inverted residual module to obtain the first intermediate features;

[0047] The first intermediate feature is enhanced by using the depthwise separable convolutional layer in the inverted residual module to obtain the second intermediate feature;

[0048] The second intermediate feature is reduced in dimensionality using the linear residual connection in the inverted residual module to obtain the second image feature.

[0049] In this embodiment, the inverted residual module includes an expanded convolutional layer, a depthwise separable convolutional layer, and linear residual connections. After the first image feature is input into the inverted residual module, it first undergoes dimensionality upscaling in the expanded convolutional layer, and the output of the expanded convolutional layer yields the first intermediate feature. The purpose of this step is to complete the channel expansion of the depthwise separable convolutional layer, so that the first intermediate feature is suitable for subsequent depthwise separable convolutional layers for further processing. Secondly, the depthwise separable convolutional layer is used to enhance the representation of the first intermediate feature, and the output yields the second intermediate feature. Finally, the second intermediate feature is dimensionality reduced through linear residual connections to obtain the second image feature. This embodiment, by using the inverted residual module for image processing, can increase the expressive power of the MobileNetv3 network while maintaining its lightweight structure, thereby ensuring the processing speed and performance of animal matting.

[0050] In a specific embodiment, the dilated convolutional layer performs dimensionality-upgrading processing on the first image features using 1×1 convolution, thereby increasing the number of feature channels; the depthwise separable convolutional layer performs enhanced representation processing on the first intermediate features using 3×3 convolution, in which the feature dimension remains unchanged; and the linear residual connection performs dimensionality-downgrading processing on the second intermediate features using 1×1 convolution.

[0051] In another specific embodiment, step S303 includes:

[0052] The second image features are compressed using the global average pooling layer in the Squeeze-and-Excitation module to obtain a feature vector;

[0053] The feature vector is obtained by learning feature channel weights and performing nonlinear transformation on the feature vector using the fully connected layer and activation function in the Squeeze-and-Excitation module;

[0054] The weight vector is multiplied element-wise with the second image feature to obtain the first feature map.

[0055] In this embodiment, the Squeeze-and-Excitation module includes a global average pooling layer, a fully connected layer, and an activation function. Specifically, the Squeeze-and-Excitation module is divided into two parts: Squeeze and Excitation. The Squeeze part includes a global average pooling layer, and the Excitation part includes a fully connected layer and an activation function. Therefore, in this embodiment, when using the Squeeze-and-Excitation module to perform feature enhancement processing on the second image features, the second image features are first compressed using a global average pooling layer, compressing the spatial dimension of each feature channel into a scalar. This global average pooling operation compresses the second image features into corresponding feature vectors. Secondly, the channel weights of this feature vector are learned through a fully connected layer and an activation function (typically ReLU activation function), and a weight vector is obtained through a nonlinear transformation. Finally, the weight vector is element-wise multiplied with the second image features to obtain a first feature map, which is then applied to the image features. This enhances the high-importance feature channels in the image features while suppressing the low-importance feature channels, thereby enhancing the representational power of important features in the image features. In specific application scenarios, when the number of channels in the feature map input to the Squeeze-and-Excitation module is C, the lengths of the corresponding feature vector and weight vector are also C.

[0056] In one embodiment, step S101 further includes:

[0057] The edge mask image of the animal image to be cut out is obtained using the Canny edge detection operator;

[0058] The size of the edge mask image is scaled to obtain multiple input images of different sizes;

[0059] Each input image is sequentially input into the first CBL structure and the second CBL structure, and the output result of the second CBL structure is amplified by the sampling operator to make the output results of all second CBL structures have the same size, and then multiple output images of the same size are obtained.

[0060] Multiple output images of the same size are added together and fused to obtain the second feature map.

[0061] In this embodiment, combined with Figure 4As shown, firstly, the Canny edge detection operator is used to perform edge detection on the animal image to be cut out, obtaining an edge mask image corresponding to the animal image to be cut out. Secondly, the edge mask image is resized to adjust its size, resulting in multiple input images of different sizes. Thirdly, each input image is sequentially input into the first CBL structure and the second CBL structure for feature extraction. The extraction result is output by the second CBL structure, and then the sampling operator upsamples the result of the second CBL structure to make the output results of all second CBL structures the same size, thus obtaining multiple output images of the same size. Finally, the multiple output images of the same size are combined into a second feature map by addition and fusion. This embodiment enhances the perception ability of objects of different sizes in animal cutout by inputting multiple input images of different sizes into the edge enhancement module for cross-scale feature fusion, thereby improving the processing ability of animal fur in animal cutout and solving the tricky problem of animal fur with many furs and high requirements for detail processing of fur at the edges in animal cutout.

[0062] For example, in specific application scenarios, the edge mask image can be scaled to obtain input images of sizes 128×128, 256×256, and 512×512. These three input images of different sizes are sequentially input into the first CBL structure and the second CBL structure for feature extraction. Then, the sampling operator performs amplification processing: the 128×128 feature image is magnified to four times its original size, the 256×256 feature image is magnified to twice its original size, and the 512×512 feature image is magnified to one time its original size (i.e., the size remains unchanged). Finally, the three feature images obtained from the three input images are all 512×512 in size. Of course, the scaling size and magnification factor in this embodiment are not limited to fixed values ​​and can be adjusted according to actual application needs in other application scenarios.

[0063] It should be noted that the first CBL structure and the second CBL structure in this embodiment have the same structure. The distinction between the two is only for the convenience of representing the processing steps. Specifically, both the first CBL structure and the second CBL structure include convolution (Conv), batch normalization (BN), and activation function (Leaky ReLU).

[0064] Combination Figure 5 As shown, in one embodiment, step S102 includes:

[0065] The first feature map is convolved using the convolutional layer in the context attention module to obtain the key map, query map, and value map corresponding to the first feature map.

[0066] The key graph is convolved with k×k, and the convolved key graph is combined with the query graph to obtain a first combined feature map; where k is an odd multiple of 1.

[0067] The first combined feature map is convolved with 1×1, and the first combined feature map after 1×1 convolution is then processed by convolution, size scaling, averaging and normalization in sequence to obtain the second combined feature map.

[0068] Perform a 1×1 convolution on the value map, and add the elements in the value map after the 1×1 convolution to the elements in the second combined feature map to obtain a third combined feature map;

[0069] The third combined feature map and the key map after k×k convolution are combined to obtain the fourth combined feature map, and the fourth combined feature map is convolved to obtain the third feature map.

[0070] In this embodiment, based on the first feature map corresponding to the animal image, a series of operations such as convolution and stitching in the context attention module are used to perform pixel information association processing on the first feature map, so as to better obtain the relationship between pixels in the animal image and improve the prediction ability of animal foreground in animal matting. Specifically, in this embodiment, firstly, convolutional layers are used to process the key map, query map, and value map corresponding to the first feature map. Secondly, the key map is subjected to k×k convolution, and the key map after k×k convolution is used to merge with the query map to obtain the first merged feature map. Thirdly, the first merged feature map is subjected to 1×1 convolution, and then the first merged feature map is subjected to convolution, size scaling, averaging, and normalization in sequence to obtain the second merged feature map. Next, the value map is subjected to 1×1 convolution, and the value map after 1×1 convolution and the second merged feature map are added element-wise to obtain the third merged feature map. Finally, the third merged feature map and the key map after k×k convolution are merged, and the fourth merged feature map obtained by the merge is subjected to convolution to obtain the third feature map, which is used as the final output of the context attention module. Here, for ease of representation, the operations performed sequentially on the first combined feature map, such as convolution (Conv), reshape, mean, and softmax, are collectively referred to as CRMS processing.

[0071] In a specific embodiment, if the size of the first feature map of the input context attention module is H×W×C (where H represents height, W represents width, and C represents the number of channels), then the sizes of its corresponding key map, query map, and value map are also H×W×C; the size of the key map after k×k convolution is H×W×C; the first combined feature map is obtained by combining the key map and query map after k×k convolution, so the size of the first combined feature map is H×W×2C, and the size of the first combined feature map after 1×1 convolution is H×W×D; in addition, the sizes of the second combined feature map, the value map after 1×1 convolution, and the third combined feature map obtained by CRMS processing are all H×W×C, the size of the fourth combined feature map obtained by combining the third combined feature map and the key map after k×k convolution is H×W×2C, and the fourth combined feature map is processed by convolution to obtain the third feature map with a size of H×W×C. Here, D and C both represent the number of channels in the feature map, but the values ​​of D and C are different. k represents the size of the convolution kernel. Both D and k are variables that can be changed. For example, D can be set to C / 2 and k can be set to 3. Of course, in other embodiments, different sizes of D and k can also be set according to the actual situation.

[0072] In one embodiment, step S103 includes:

[0073] The first stitched feature map is input into multiple dilated convolutions with different sampling rates, and the outputs of the multiple dilated convolutions with different sampling rates are convolved to obtain multiple first intermediate images.

[0074] The size of the first stitched feature map is obtained, and each of the first intermediate images is sampled according to the size of the first stitched feature map so that the size of each of the first intermediate images is the same as the size of the first stitched feature map, and then multiple second intermediate images of the same size are obtained.

[0075] All the second intermediate images are combined with the first spliced ​​feature image to obtain the fourth feature image.

[0076] In this embodiment, the first stitched feature map is first processed using multiple dilated convolutions with different sampling rates. Then, the outputs of these multiple dilated convolutions are convolved separately to obtain multiple first intermediate images. Next, the size of the first stitched feature map is used as a standard to sample each first intermediate image, ensuring that the size of each first intermediate image is the same as that of the first stitched feature map, thus obtaining multiple second intermediate images of the same size. Finally, all the second intermediate images and the first stitched feature map are stitched together to form a fourth feature map. This embodiment performs multi-scale feature fusion by combining the outputs of multiple dilated convolutions with different sampling rates and the first stitched feature map, resulting in a larger receptive field and thus acquiring more pixel information, thereby improving the effect of animal matting.

[0077] It should be noted that the parameters in the multi-scale dilated convolution module can be adjusted according to actual application requirements. For example, four dilated convolutions with different sampling rates can be set (corresponding to...). Figure 2 The first spliced ​​feature map is processed by DC1 to DC4 in the sample map. These four sampling rates can be set to 4, 8, 12 and 18 respectively.

[0078] In specific application scenarios, multiple animal images can be used as a training dataset, and then trained according to the animal matting method provided in this embodiment of the invention to obtain a corresponding animal matting model. This model can then be used for animal matting, thereby improving the accuracy of animal matting. For example... Figure 6 As shown, Figure 6 From left to right, the image shows the animal image to be cut out, the alpha image output by the animal cutting-out model (i.e., the final animal image cutting-out result in the above animal cutting-out method), and the green screen background display image. The animal cutting-out model described above is used to... Figure 6 By performing image cutout processing on the animal image on the left side, we can obtain... Figure 6 The alpha image in the middle is used as a base to add a green screen background. Figure 6 The green screen background shown on the right. The experimental results demonstrate that the animal matting model can accurately locate animals in animal images, producing good edge effects and more accurate processing of details such as animal fur.

[0079] Figure 7 This is a schematic block diagram of an animal cutout device 700 provided in an embodiment of the present invention. The device 700 includes:

[0080] The feature extraction unit 701 is used to acquire the animal image to be cut out, input the animal image to be cut out into the MobileNetv3 network for feature extraction to obtain a first feature map; and use an edge detection operator to perform image processing on the animal image to be cut out to obtain a corresponding edge mask map, and perform cross-scale feature fusion on the edge mask map through an edge enhancement module to obtain a second feature map.

[0081] The pixel information association unit 702 is used to perform pixel information association processing on the first feature map using the context attention module to obtain the third feature map;

[0082] The first feature stitching unit 703 is used to stitch the second feature map and the third feature map to obtain a first stitched feature map, and input the first stitched feature map into the multi-scale dilated convolution module for multi-scale feature fusion to obtain a fourth feature map;

[0083] The second stitching feature unit 704 is used to stitch the first stitching feature map and the fourth feature map to obtain a second stitching feature map, and to perform convolution processing on the second stitching feature map to obtain the final animal image matting result.

[0084] In one embodiment, the feature extraction unit 701 includes:

[0085] Sub-feature extraction unit 801 is used to extract image features from the animal image to be cut out using the convolutional layer in the MobileNetv3 network to obtain the first image features;

[0086] The enhanced representation unit 802 is used to perform dimensionality-upgrading, enhanced representation processing, and dimensionality-reduction processing on the first image features in sequence using the inverted residual module in the MobileNetv3 network to obtain the second image features.

[0087] The feature enhancement unit 803 is used to input the second image features into the Squeeze-and-Excitation module in the MobileNetv3 network, and use the Squeeze-and-Excitation module to perform feature enhancement processing on the second image features to obtain the first feature map.

[0088] In one embodiment, the feature extraction unit 701 further includes:

[0089] The edge detection unit is used to obtain the edge mask image of the animal image to be cut out using the Canny edge detection operator;

[0090] A scaling processing unit is used to scale the size of the edge mask image to obtain multiple input images of different sizes;

[0091] The magnification processing unit is used to sequentially input each of the input images into the first CBL structure and the second CBL structure, and to magnify the output result of the second CBL structure through a sampling operator so that the output results of all the second CBL structures have the same size, and then obtain multiple output images of the same size.

[0092] Multiple output images of the same size are added together and fused to obtain the second feature map.

[0093] In one embodiment, the pixel information association unit 702 includes:

[0094] The convolution processing unit is used to perform convolution processing on the first feature map using the convolutional layer in the context attention module to obtain the key map, query map and value map corresponding to the first feature map;

[0095] The first merging unit is used to perform a k×k convolution on the key graph and merge the k×k convolutioned key graph with the query graph to obtain a first merged feature map; wherein, k takes the value of an odd multiple of 1;

[0096] The second stitching unit is used to perform a 1×1 convolution on the first stitched feature map, and then perform convolution processing, size scaling processing, mean averaging processing and normalization processing on the first stitched feature map after the 1×1 convolution to obtain the second stitched feature map.

[0097] The third merging unit is used to perform a 1×1 convolution on the value map, and add the elements in the value map after the 1×1 convolution to the elements in the second merging feature map to obtain the third merging feature map;

[0098] The fourth stitching unit stitches the third stitched feature map and the key map after k×k convolution to obtain the fourth stitched feature map, and then performs convolution on the fourth stitched feature map to obtain the third feature map.

[0099] In one embodiment, the first splicing feature unit 703 includes:

[0100] The dilated convolution unit is used to input the first stitched feature map into multiple dilated convolutions with different sampling rates, and to perform convolution processing on the output results of the multiple dilated convolutions with different sampling rates to obtain multiple first intermediate images.

[0101] A size sampling unit is used to obtain the size of the first stitched feature map, and to sample each of the first intermediate maps according to the size of the first stitched feature map so that the size of each of the first intermediate maps is the same as the size of the first stitched feature map, and then obtain multiple second intermediate maps with the same size.

[0102] The fifth stitching unit is used to stitch all the second intermediate images with the first stitched feature image to obtain the fourth feature image.

[0103] In one embodiment, the enhanced representation unit 802 includes:

[0104] The feature upscaling unit is used to upscale the first image features using the dilated convolutional layer in the inverted residual module to obtain the first intermediate features.

[0105] The feature enhancement unit is used to enhance the representation of the first intermediate feature by utilizing the depthwise separable convolutional layer in the inverted residual module to obtain the second intermediate feature.

[0106] The feature dimensionality reduction unit is used to perform dimensionality reduction processing on the second intermediate feature using the linear residual connection in the inverted residual module to obtain the second image feature.

[0107] In one embodiment, the feature enhancement unit 803 includes:

[0108] The feature compression unit is used to compress the second image features using the global average pooling layer in the Squeeze-and-Excitation module to obtain a feature vector;

[0109] The weight learning unit is used to perform feature channel weight learning and nonlinear transformation on the feature vector using the fully connected layer and activation function in the Squeeze-and-Excitation module to obtain a weight vector.

[0110] The element-wise multiplication unit is used to multiply the weight vector with the second image feature element by element to obtain the first feature map.

[0111] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0112] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0113] This invention also provides a computer device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the computer device may also include various network interfaces, power supplies, and other components.

[0114] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0115] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for cutting out animal images, characterized in that, include: Obtain the animal image to be cut out, and input the animal image to be cut out into the MobileNetv3 network for feature extraction to obtain the first feature map; The image of the animal to be cut out is processed using an edge detection operator to obtain a corresponding edge mask image, and the edge mask image is fused across scales using an edge enhancement module to obtain a second feature image. The first feature map is processed by using a context attention module to associate pixel information, resulting in a third feature map. The second and third feature maps are concatenated to obtain a first concatenated feature map, which is then input into a multi-scale dilated convolution module for multi-scale feature fusion to obtain a fourth feature map. The first and fourth feature maps are spliced ​​together to obtain the second spliced ​​feature map, and the second spliced ​​feature map is convolved to obtain the final animal image matting result. The step of using a context attention module to perform pixel information association processing on the first feature map to obtain a third feature map includes: The first feature map is convolved using the convolutional layer in the context attention module to obtain the key map, query map, and value map corresponding to the first feature map. The key graph is convolved with k×k, and the convolved key graph is combined with the query graph to obtain a first combined feature map; where k is an odd multiple of 1. The first combined feature map is convolved with 1×1, and the first combined feature map after 1×1 convolution is then processed by convolution, size scaling, averaging and normalization in sequence to obtain the second combined feature map. Perform a 1×1 convolution on the value map, and add the elements of the value map after the 1×1 convolution to the elements of the second combined feature map to obtain a third combined feature map; The third combined feature map and the key map after k×k convolution are combined to obtain the fourth combined feature map, and the fourth combined feature map is convolved to obtain the third feature map.

2. The animal image cutout method according to claim 1, characterized in that, The step of inputting the animal image to be cut out into the MobileNetv3 network for feature extraction to obtain a first feature map includes: The first image feature is obtained by using the convolutional layers in the MobileNetv3 network to extract image features from the animal image to be cut out. The first image features are sequentially subjected to dimensionality upscaling, enhanced representation processing, and dimensionality reduction processing using the inverted residual module in the MobileNetv3 network to obtain the second image features. The second image features are input into the Squeeze-and-Excitation module in the MobileNetv3 network, and the Squeeze-and-Excitation module is used to perform feature enhancement processing on the second image features to obtain the first feature map.

3. The animal image cutout method according to claim 1, characterized in that, The image processing of the animal image to be cut out is performed using an edge detection operator to obtain a corresponding edge mask image. Then, a second feature map is obtained by cross-scale feature fusion of the edge mask image using an edge enhancement module. This includes: The edge mask image of the animal image to be cut out is obtained using the Canny edge detection operator; The size of the edge mask image is scaled to obtain multiple input images of different sizes; Each input image is sequentially input into the first CBL structure and the second CBL structure, and the output result of the second CBL structure is amplified by the sampling operator to make the output results of all second CBL structures have the same size, and then multiple output images of the same size are obtained. Multiple output images of the same size are added together and fused to obtain the second feature map.

4. The animal image cutout method according to claim 1, characterized in that, The step of inputting the first stitched feature map into a multi-scale dilated convolution module for multi-scale feature fusion to obtain a fourth feature map includes: The first stitched feature map is input into multiple dilated convolutions with different sampling rates, and the outputs of the multiple dilated convolutions with different sampling rates are convolved to obtain multiple first intermediate images. The size of the first stitched feature map is obtained, and each of the first intermediate images is sampled according to the size of the first stitched feature map so that the size of each of the first intermediate images is the same as the size of the first stitched feature map, and then multiple second intermediate images of the same size are obtained. All the second intermediate images are combined with the first spliced ​​feature image to obtain the fourth feature image.

5. The animal image cutout method according to claim 2, characterized in that, The second image features are obtained by sequentially performing dimensionality upscaling, enhanced representation processing, and dimensionality reduction processing on the first image features using the inverted residual module in the MobileNetv3 network, including: The first image features are up-dimensionalized using the dilated convolutional layer in the inverted residual module to obtain the first intermediate features; The first intermediate feature is enhanced by using the depthwise separable convolutional layer in the inverted residual module to obtain the second intermediate feature; The second intermediate feature is reduced in dimensionality using the linear residual connection in the inverted residual module to obtain the second image feature.

6. The animal image cutout method according to claim 2, characterized in that, The step of using the Squeeze-and-Excitation module to perform feature enhancement processing on the second image features to obtain the first feature map includes: The second image features are compressed using the global average pooling layer in the Squeeze-and-Excitation module to obtain a feature vector; The feature vector is obtained by learning feature channel weights and performing nonlinear transformation on the feature vector using the fully connected layer and activation function in the Squeeze-and-Excitation module; The weight vector is multiplied element-wise with the second image feature to obtain the first feature map.

7. An animal cutout device, characterized in that, include: The feature extraction unit is used to acquire the animal image to be cut out, input the animal image to be cut out into the MobileNetv3 network for feature extraction to obtain a first feature map; and use an edge detection operator to perform image processing on the animal image to be cut out to obtain a corresponding edge mask map, and use an edge enhancement module to perform cross-scale feature fusion on the edge mask map to obtain a second feature map. The pixel information association unit is used to perform pixel information association processing on the first feature map using the context attention module to obtain the third feature map; The first feature concatenation unit is used to concatenate the second feature map and the third feature map to obtain the first concatenated feature map, and input the first concatenated feature map into the multi-scale dilated convolution module for multi-scale feature fusion to obtain the fourth feature map; The second stitching feature unit is used to stitch the first stitching feature map and the fourth feature map to obtain the second stitching feature map, and to perform convolution processing on the second stitching feature map to obtain the final animal image matting result; The pixel information association unit includes: The convolution processing unit is used to perform convolution processing on the first feature map using the convolutional layer in the context attention module to obtain the key map, query map and value map corresponding to the first feature map; The first merging unit is used to perform a k×k convolution on the key graph and merge the k×k convolutioned key graph with the query graph to obtain a first merged feature map; wherein, k takes the value of an odd multiple of 1; The second stitching unit is used to perform a 1×1 convolution on the first stitched feature map, and then perform convolution processing, size scaling processing, mean averaging processing and normalization processing on the first stitched feature map after the 1×1 convolution to obtain the second stitched feature map. The third merging unit is used to perform a 1×1 convolution on the value map, and add the elements in the value map after the 1×1 convolution to the elements in the second merging feature map to obtain the third merging feature map; The fourth stitching unit stitches the third stitched feature map and the key map after k×k convolution to obtain the fourth stitched feature map, and then performs convolution on the fourth stitched feature map to obtain the third feature map.

8. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the animal cutout method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the animal cutout method as described in any one of claims 1 to 6.

Images