An automatic animal image cutout method

Abstract

The present invention proposes an automatic animal image cutout method, firstly, the animal picture is input into the moving window encoding module to obtain features of different scales; the obtained encoding features are passed through the adaptive feature fusion module, combined with the detail feature branch module, and multi-scale information is fused in the upsampling process to grasp the richer detail information of the animal image and obtain the detail feature map; in the third part, the obtained encoding features are input into the global semantic branch module, and a multi-dimensional attention enhancement module is added in the middle to guide the network to focus on the target area, suppress irrelevant information, obtain global semantic features, and finally combine the global semantic features and the detail semantic features through the branch fusion module to obtain the final cutout result. The present invention uses the adaptive feature fusion module to enrich the details of the animal texture, strengthen the fine processing capability, and uses the multi-dimensional attention to suppress irrelevant information, enhance the background transition, and improve the recognition of the animal target subject, thereby achieving the precise effect of animal cutout.

Description

Technical Field

[0001] The invention belongs to an image processing method, and in particular relates to an automatic animal image cutout method. Background Art

[0002] Animal cutout background technology, as an important field in image processing, refers to accurately separating the main body of the animal from the picture and removing or replacing the background. In modern society, animals are deeply loved by people. Whether it is a cute pet at home or a bird of prey in the zoo, it brings real meaning to people. At the same time, people's demand for editing animal images is also increasing. Animal image cutout technology is not only a tool for image editing and synthesis, but also has broad application prospects in biological research, environmental protection, cultural communication and artificial intelligence applications. Especially in the study of ecology and animal behavior, through the accurate cutout of animal images, non-invasive monitoring can be effectively carried out, providing important data support for protecting endangered species, studying animal behavior and conducting ecological environmental assessments.

[0003] However, due to the similarity between animals and backgrounds in natural environments, the diversity of their morphology, and the complex lighting and occlusion in images, traditional image cutout techniques often face great challenges. Many animals have similar colors to the background and complex edge transitions, which results in poor results for traditional cutout methods based on color, edge, or region segmentation. In particular, for animals with detailed textures such as hair and feathers, traditional algorithms often fail to accurately extract detailed edge information. Through in-depth research on animal image cutout techniques, for example, deep learning methods such as convolutional neural networks (CNNs) have made significant progress in image segmentation. Although methods based on convolutional neural networks (CNNs) can achieve good cutout results, the prerequisite is to have trimmed or scribbled triplicate images, which makes them unlikely to be used in automatic applications. With the rapid development of the times, automatic cutout models have emerged. However, most of the existing automatic cutout models are aimed at portrait cutout. Portrait cutouts mostly focus on details such as hair, face, clothing, etc., and the contrast between the background and the target in most scenes is higher and easier to process. However, they are not good at processing the complex environmental background of animals, resulting in poor overall recognition of the animals. They are also difficult to handle the natural detailed textures of animals, such as skin, hair, scales, claws, ears, etc., resulting in unsatisfactory automatic cutout effects for animals.

[0004] Therefore, we need to further develop automatic animal image cutout technology to solve the similarities between animals and backgrounds in natural environments, the diversity of their morphology to enhance overall animal recognition, and how to more finely cut out animal detail textures to improve the accuracy of animal cutouts. Summary of the invention

[0005] In view of the above problems, the present invention proposes an automatic animal image cutout method. The method adopts a shared encoding and multi-decoding structure, and the user only needs to input the animal image to realize end-to-end automatic animal cutout.

[0006] An automatic animal image cutout method comprises the following steps:

[0007] Step S1: construct an animal image dataset, where the animal image dataset includes a number of animal images;

[0008] Step S2: constructing an animal image cutout model, the model comprising a moving window encoding module, a global semantic branch module, a detail feature branch module and a branch fusion module; importing the animal image of the data set in step S1 into the moving window encoding module to obtain encoding features of different scales; wherein the encoding features comprise the first feature, the second feature, the third feature, the fourth feature and the fifth feature extracted in sequence;

[0009] Step S3: import the fifth feature in step S2 into the global semantic branch module to obtain the global semantic feature;

[0010] Step S4: importing the second feature, the third feature, the fourth feature and the fifth feature of the encoding feature in step S2 into the detail feature branch module to obtain the detail semantic feature;

[0011] Step S5: importing the global semantic features and the detail semantic features into the branch fusion module, performing feature fusion to obtain fusion features, that is, the predicted animal transparency mask map;

[0012] Step S6: Construct a loss function and optimize the model by minimizing the loss.

[0013] Furthermore, step S1 is specifically as follows:

[0014] The image of the AM-2K dataset is obtained as the foreground and the image of the BG-20K dataset is obtained as the background, and the foreground and background are synthesized into animal images to construct an animal image dataset; it means:

[0015] ;

[0016] in, represents an animal image, r represents a pixel index, F represents foreground, B represents background, Indicates the probability that the pixel belongs to the foreground, ranging from 0 to 1; , , , They represent the composite pixel value, foreground pixel value, background pixel value and foreground pixel transparency value of pixel index r respectively.

[0017] Furthermore, the moving window encoding module in step S2 is composed of a Swin Transformer network, including five components connected in series in sequence, wherein the first component is an image block embedding module, and the second, third, fourth and fifth components have the same basic structure, and are all stacked by at least one set of Swin Transformer blocks and image block merging modules, and the five components are labeled A1, A2, A3, A4 and A5 in sequence;

[0018] Among them, A2, A3 and A5 are composed of 1 Swin Transformer block and 1 image block merging module stacked once, while the fourth component A4 is composed of 1 Swin Transformer block and 1 image block merging module stacked 3 times;

[0019] Step S2 is specifically as follows:

[0020] Step S21: uniformly cropping the input animal images to obtain animal images of a fixed size, and then performing a data enhancement operation on the animal images of the fixed size, wherein the data enhancement operation includes cropping and / or flipping, thereby completing the preprocessing of the animal images;

[0021] Step S22: Input the preprocessed animal image into the first component A1 of the moving window encoding module to preliminarily extract the first feature of the animal image; specifically:

[0022] Three consecutive 3×3 convolutional layers and ReLU activation functions are used to increase the number of channels and gradually extract complex features. Then two layers of maximum pooling layers are used to reduce the spatial resolution and the amount of calculation while retaining the global characteristics. Finally, the features are normalized by the LayerNorm operation to ensure the stability of subsequent processing. Finally, the first feature is obtained. ; The specific formula is:

[0023] ;

[0024] ;

[0025] in, represents the preprocessed animal image, is a collection of multi-layer convolutions, that is, three consecutive 3×3 convolutions, represents the number of convolutional layers, is the set of biases for multi-layer convolutions, is the number of bias layers of multi-layer convolution, is a two-dimensional convolution operation, is a nonlinear activation function, is the maximum pooling operation, It is the layer normalization operation;

[0026] Step S23: The first feature Input to the second component A2, the second component A2 outputs the second feature , then the output of the second component A2 is used as the input of the third component A3, and so on to extract features;

[0027] The process of processing input features by the Swin Transformer block and the image block merging module is as follows:

[0028] The i-th feature of the input first passes through the Swin Transformer block, where the input feature is layer-normalized, then window division is performed to obtain the window feature, and then the attention feature is obtained through the window multi-head self-attention. After that, the attention feature is added to the i-th feature of the input through the residual connection and layer-normalized through the multi-layer perceptron. The output of the multi-layer perceptron is added to the layer-normalized result to obtain the result feature, and finally the result feature is downsampled through the image block merging module to finally obtain the i+1th feature. The specific formula is:

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] in, , is the i-th feature, B is the batch size, H is the height of the feature map, W is the width of the feature map, C is the number of channels, represents a real number set, P is the window feature, Represents the attention feature, Indicates the result characteristics; represents the query vector, represents the transpose of the key vector, represents the dimension, b represents the bias term, represents the image segmentation operation, represents the normalization function, represents the window multi-head attention mechanism, For self-attention, Representation layer normalization, represents a multi-layer perceptron, represents the downsampling operation, represents the i+1th feature, , , , and Together they constitute the coding features.

[0035] Further, the global semantic branch module in step S3 includes several global semantic decoding blocks connected in series. Specifically, the global semantic decoding block includes four semantic multi-convolution decoding blocks T1, T2, T3, T4 and a semantic single decoding block T5 connected in series, wherein a multi-dimensional attention enhancement module is connected in series after the semantic multi-convolution decoding block; the semantic multi-convolution decoding block is composed of three 3×3 convolution layers, three normalization layers, three ReLU layers and an upsampling layer connected in series; the multi-dimensional attention enhancement module is composed of two branches, the first branch includes a convolution block and a channel attention block connected in series, and the second branch includes a feature segmentation block and a spatial attention block connected in series; the semantic single decoding block T5 is composed of one 3×3 convolution layer and an upsampling layer connected in series;

[0036] Step S3 is specifically as follows:

[0037] The global semantic branch module inputs the fifth feature into the semantic multi-convolution decoding block T1 to obtain the decoding feature; the decoding feature is input into the two branches of the multi-dimensional attention enhancement module, the first branch extracts the global feature, and the second branch divides the decoding feature into multiple sub-regions, then weights the sub-regions through spatial attention, and then re-splices the weighted spatial features to obtain the splicing output with a dimension of 3, and performs dimensionality reduction splicing on the splicing output to obtain the local enhanced feature map, and finally adds the global feature and the local enhanced feature map element by element in different dimensions to obtain the output of the multi-dimensional attention enhancement module;

[0038] The output of the multi-dimensional attention enhancement module corresponding to the semantic multi-convolution decoding block T1 is imported into the semantic multi-convolution decoding block T2 for decoding to obtain the decoding features; the output of the multi-dimensional attention enhancement module is enhanced to obtain the corresponding output of the multi-dimensional attention enhancement module;

[0039] Similarly, the semantic multi-convolution decoding block T4 outputs the final global semantic features through the semantic single decoding block after obtaining the output of the corresponding multi-dimensional attention enhancement module;

[0040] The specific calculation formula of the global semantic decoding block T4 and the corresponding multi-dimensional attention enhancement module is as follows:

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] The specific calculation formula of the semantic single decoding block is as follows:

[0050] ;

[0051] in, Represents the fifth characteristic, Represents global features, represents the sub-region with height index i and width index j, Z represents the decoding feature, represents batch normalization, represents the attention mechanism, represents a nonlinear activation function, represents the convolution operation, represents a 3×3 convolution operation, represents the upsampling operation, represents spatial attention; i represents the segmentation index along the height direction, j represents the segmentation index along the width direction, Represents the tensor concatenation dimension, represents an element-wise multiplication operation, Represents a splicing operation; Represents a spatial feature with height index i and width index j; Representing spatial features and spatial characteristics At the output of the splicing with dimension 3, Representing spatial features and spatial characteristics The output of the splicing is in dimension 3; Feature map representing local enhancement; represents the final output of the multi-dimensional attention enhancement module, Represents global semantic features.

[0052] Further, the detail feature branch module in step S5 includes an adaptive feature fusion module and a detail feature decoding block, the detail feature decoding block includes four sequentially connected detail multi-convolution decoding blocks F5, F4, F3, F2 and a detail single decoding block F1, wherein the detail multi-convolution decoding block includes two sequentially connected depth-separable convolution layers, one 3×3 convolution layer, one batch normalization layer, one ReLU layer and an upsampling layer; the adaptive feature fusion module includes an efficient regional attention module, an efficient channel weighting module and a spatial attention module sequentially connected; the detail single decoding block F5 is composed of a 3×3 convolution layer and an upsampling layer connected in series;

[0053] Step S5 is specifically as follows:

[0054] The adaptive feature fusion module then passes the input coding features through the efficient regional attention module to enhance the important areas of the coding features, highlight the location of key information in the image, and obtain an enhanced feature map. Secondly, through the efficient channel weighting module, the channels of the encoding features are weighted to enhance the features of important channels to obtain the channel feature map ; Then the channel feature map Weighted, highlighting the important areas in space, and obtaining the spatial feature map ;Finally, the feature map is enhanced by splicing and spatial feature map , get adaptive fusion features ; The detail multi-convolution decoding block upsamples the encoded features to obtain the corresponding upsampled features ; Adaptively fusion features And the corresponding upsampled features Fusion is performed and the fused features are input into the detail multi-convolution decoding block After concatenating with the encoded features, upsampling is performed; and so on, until the upsampling features of the detail multi-convolution decoding block F2 are obtained. And the corresponding adaptive fusion features ; Detail single decoding block F1 pair and adaptive fusion features splicing and upsampling to obtain detailed semantic features; among them, ;

[0055] The upsampling process of the detail multi-convolutional decoding block is expressed as: ;

[0056] The splicing process of the adaptive feature fusion module is expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] The process of obtaining detailed semantic features is expressed as:

[0062] ;

[0063] in, represents the up-sampled features, represents the upsampling operation, represents a nonlinear activation function, represents batch normalization operation, represents a 1×1 convolution operation, represents a two-dimensional convolution operation, Represents a two-dimensional convolution operation, a batch normalization operation, and a nonlinear activation operation. Represents a point-by-point convolution operation, that is, a 3×3 convolution, represents a depth-wise separable convolution operation, represents the index of the channel, The total number of channel indices. Represents a splicing operation, represents the activation function, Represents a weighted operation.

[0064] Further, step S5 is specifically as follows: using the activation function to process the global semantic features and the detailed semantic features to generate a global semantic probability map and detail feature probability map ; Then from the global semantic probability map Extract the category index of the maximum value, generate the background, tri-image mask and foreground based on the category index; finally, convert the detail feature probability map Multiply it element-by-element with the three-part image mask, and then perform element-by-element addition with the foreground to get the transparency mask, which is expressed as:

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] ;

[0072] in, represents the dimension of the tensor, Represents the category index, Indicates an indicator function, which returns 1 when the condition is met, otherwise it returns 0. Indicates the background, represents the tri-map mask, Indicates prospects, represents element-wise multiplication, represents the maximum value function, represents the activation function, Represents the fusion feature, which is the final output animal image transparency mask.

[0073] Beneficial effects of the present invention:

[0074] (1) The automatic animal cutout method proposed in the present invention does not require user annotation or manual adjustment, which greatly reduces the usage threshold. By sharing the encoding and multi-decoding structure and adopting an end-to-end trained deep learning model, the input image can quickly and accurately separate the animal from the background, thereby reducing the time and complexity of manual operation.

[0075] (2) The present invention can capture local detail features, focus more on local characteristics in a small range, enhance the high-frequency information of animal detail textures, and help the model process complex textures.

[0076] (3) The present invention enhances the network's ability to perceive information at different scales, thereby improving the overall recognition of animals when processing complex environmental backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 The present invention is a flowchart of the steps of an automatic animal image cutout method. DETAILED DESCRIPTION

[0078] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0079] Reference Figure 1 , an automatic animal image cutout method, comprising the following steps:

[0080] Step S1: construct an animal image dataset, where the animal image dataset includes a number of animal images;

[0081] Step S2: constructing an animal image cutout model, the model comprising a moving window encoding module, a global semantic branch module, a detail feature branch module and a branch fusion module; importing the animal image of the data set in step S1 into the moving window encoding module to obtain encoding features of different scales; wherein the encoding features comprise the first feature, the second feature, the third feature, the fourth feature and the fifth feature extracted in sequence;

[0082] Step S3: import the fifth feature in step S2 into the global semantic branch module to obtain the global semantic feature;

[0083] Step S4: importing the second feature, the third feature, the fourth feature and the fifth feature of the encoding feature in step S2 into the detail feature branch module to obtain the detail semantic feature;

[0084] Step S5: importing the global semantic features and the detail semantic features into the branch fusion module, performing feature fusion to obtain fusion features, that is, the predicted animal transparency mask map;

[0085] Step S6: Construct a loss function and optimize the model by minimizing the loss.

[0086] Furthermore, step S1 is specifically as follows:

[0087] The image of the AM-2K dataset is obtained as the foreground and the image of the BG-20K dataset is obtained as the background, and the foreground and background are synthesized into animal images to construct an animal image dataset; it means:

[0088] ;

[0089] in, represents an animal image, r represents a pixel index, F represents foreground, B represents background, Indicates the probability that the pixel belongs to the foreground, ranging from 0 to 1; , , , They represent the composite pixel value, foreground pixel value, background pixel value and foreground pixel transparency value of pixel index r respectively.

[0090] Furthermore, the moving window encoding module in step S2 is composed of a Swin Transformer network, including five components connected in series in sequence, wherein the first component is an image block embedding module, and the second, third, fourth and fifth components have the same basic structure, and are all stacked by at least one set of Swin Transformer blocks and image block merging modules, and the five components are labeled A1, A2, A3, A4 and A5 in sequence;

[0091] Among them, A2, A3 and A5 are composed of 1 Swin Transformer block and 1 image block merging module stacked once, while the fourth component A4 is composed of 1 Swin Transformer block and 1 image block merging module stacked 3 times;

[0092] Step S2 is specifically as follows:

[0093] Step S21: uniformly cropping the input animal images to obtain animal images of a fixed size, and then performing a data enhancement operation on the animal images of the fixed size, wherein the data enhancement operation includes cropping and / or flipping, thereby completing the preprocessing of the animal images;

[0094] Step S22: Input the preprocessed animal image into the first component A1 of the moving window encoding module to preliminarily extract the first feature of the animal image; specifically:

[0095] Three consecutive 3×3 convolutional layers and ReLU activation functions are used to increase the number of channels and gradually extract complex features. Then two layers of maximum pooling layers are used to reduce the spatial resolution and the amount of calculation while retaining the global characteristics. Finally, the features are normalized by the LayerNorm operation to ensure the stability of subsequent processing. Finally, the first feature is obtained. ; The specific formula is:

[0096] ;

[0097] ;

[0098] in, represents the preprocessed animal image, is a collection of multi-layer convolutions, that is, three consecutive 3×3 convolutions, represents the number of convolutional layers, is the set of biases for multi-layer convolutions, is the number of bias layers of multi-layer convolution, is a two-dimensional convolution operation, is a nonlinear activation function, is the maximum pooling operation, It is the layer normalization operation;

[0099] Step S23: The first feature Input to the second component A2, the second component A2 outputs the second feature , then the output of the second component A2 is used as the input of the third component A3, and so on to extract features;

[0100] The process of processing input features by the Swin Transformer block and the image block merging module is as follows:

[0101] The i-th feature of the input first passes through the Swin Transformer block, where the input feature is layer-normalized, then window division is performed to obtain the window feature, and then the attention feature is obtained through the window multi-head self-attention. After that, the attention feature is added to the i-th feature of the input through the residual connection and layer-normalized through the multi-layer perceptron. The output of the multi-layer perceptron is added to the layer-normalized result to obtain the result feature, and finally the result feature is downsampled through the image block merging module to finally obtain the i+1th feature. The specific formula is:

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] in, , is the i-th feature, B is the batch size, H is the height of the feature map, W is the width of the feature map, C is the number of channels, represents a real number set, P is the window feature, Represents the attention feature, Indicates the result characteristics; represents the query vector, represents the transpose of the key vector, represents the dimension, b represents the bias term, represents the image segmentation operation, represents the normalization function, represents the window multi-head attention mechanism, For self-attention, Representation layer normalization, represents a multi-layer perceptron, represents the downsampling operation, represents the i+1th feature, , , , and Together they constitute the coding features.

[0108] Further, the global semantic branch module in step S3 includes several global semantic decoding blocks connected in series. Specifically, the global semantic decoding block includes four semantic multi-convolution decoding blocks T1, T2, T3, T4 and a semantic single decoding block T5 connected in series, wherein a multi-dimensional attention enhancement module is connected in series after the semantic multi-convolution decoding block; the semantic multi-convolution decoding block is composed of three 3×3 convolution layers, three normalization layers, three ReLU layers and an upsampling layer connected in series; the multi-dimensional attention enhancement module is composed of two branches, the first branch includes a convolution block and a channel attention block connected in series, and the second branch includes a feature segmentation block and a spatial attention block connected in series; the semantic single decoding block T5 is composed of one 3×3 convolution layer and an upsampling layer connected in series;

[0109] Step S3 is specifically as follows:

[0110] The global semantic branch module inputs the fifth feature into the semantic multi-convolution decoding block T1 to obtain the decoding feature; the decoding feature is input into the two branches of the multi-dimensional attention enhancement module, the first branch extracts the global feature, and the second branch divides the decoding feature into multiple sub-regions, then weights the sub-regions through spatial attention, and then re-splices the weighted spatial features to obtain the splicing output with a dimension of 3, and performs dimensionality reduction splicing on the splicing output to obtain the local enhanced feature map, and finally adds the global feature and the local enhanced feature map element by element in different dimensions to obtain the output of the multi-dimensional attention enhancement module;

[0111] The output of the multi-dimensional attention enhancement module corresponding to the semantic multi-convolution decoding block T1 is imported into the semantic multi-convolution decoding block T2 for decoding to obtain the decoding features; the output of the multi-dimensional attention enhancement module is enhanced to obtain the corresponding output of the multi-dimensional attention enhancement module;

[0112] Similarly, the semantic multi-convolution decoding block T4 outputs the final global semantic features through the semantic single decoding block after obtaining the output of the corresponding multi-dimensional attention enhancement module;

[0113] The specific calculation formula of the global semantic decoding block T4 and the corresponding multi-dimensional attention enhancement module is as follows:

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] ;

[0121] ;

[0122] The specific calculation formula of the semantic single decoding block is as follows:

[0123] ;

[0124] in, Represents the fifth characteristic, Represents global features, represents the sub-region with height index i and width index j, Z represents the decoding feature, represents batch normalization, represents the attention mechanism, represents a nonlinear activation function, represents the convolution operation, represents a 3×3 convolution operation, represents the upsampling operation, represents spatial attention; i represents the segmentation index along the height direction, j represents the segmentation index along the width direction, Represents the tensor concatenation dimension, represents an element-wise multiplication operation, Represents a splicing operation; Represents a spatial feature with height index i and width index j; Representing spatial features and spatial characteristics At the output of the splicing with dimension 3, Representing spatial features and spatial characteristics The output of the splicing is in dimension 3; Feature map representing local enhancement; represents the final output of the multi-dimensional attention enhancement module, Represents global semantic features.

[0125] Further, the detail feature branch module in step S5 includes an adaptive feature fusion module and a detail feature decoding block, the detail feature decoding block includes four sequentially connected detail multi-convolution decoding blocks F5, F4, F3, F2 and a detail single decoding block F1, wherein the detail multi-convolution decoding block includes two sequentially connected depth-separable convolution layers, one 3×3 convolution layer, one batch normalization layer, one ReLU layer and an upsampling layer; the adaptive feature fusion module includes an efficient regional attention module, an efficient channel weighting module and a spatial attention module sequentially connected; the detail single decoding block F5 is composed of a 3×3 convolution layer and an upsampling layer connected in series;

[0126] Step S5 is specifically as follows:

[0127] The adaptive feature fusion module then passes the input coding features through the efficient regional attention module to enhance the important areas of the coding features, highlight the location of key information in the image, and obtain an enhanced feature map. Secondly, through the efficient channel weighting module, the channels of the encoding features are weighted to enhance the features of important channels to obtain the channel feature map ; Then the channel feature map Weighted, highlighting the important areas in space, and obtaining the spatial feature map ;Finally, the feature map is enhanced by splicing and spatial feature map , get adaptive fusion features ; The detail multi-convolution decoding block upsamples the encoded features to obtain the corresponding upsampled features ; Adaptively fusion features And the corresponding upsampled features Fusion is performed and the fused features are input into the detail multi-convolution decoding block After concatenating with the encoded features, upsampling is performed; and so on, until the upsampling features of the detail multi-convolution decoding block F2 are obtained. And the corresponding adaptive fusion features ; Detail single decoding block F1 pair and adaptive fusion features splicing and upsampling to obtain detailed semantic features; among them, ;

[0128] The upsampling process of the detail multi-convolutional decoding block is expressed as: ;

[0129] The splicing process of the adaptive feature fusion module is expressed as:

[0130] ;

[0131] ;

[0132] ;

[0133] ;

[0134] The process of obtaining detailed semantic features is expressed as:

[0135] ;

[0136] in, represents the up-sampled features, represents the upsampling operation, represents a nonlinear activation function, represents batch normalization operation, represents a 1×1 convolution operation, represents a two-dimensional convolution operation, Represents a two-dimensional convolution operation, a batch normalization operation, and a nonlinear activation operation. Represents a point-by-point convolution operation, that is, a 3×3 convolution, represents a depth-wise separable convolution operation, represents the index of the channel, The total number of channel indices. Represents a splicing operation, represents the activation function, Represents a weighted operation.

[0137] Further, step S5 is specifically as follows: using the activation function to process the global semantic features and the detailed semantic features to generate a global semantic probability map and detail feature probability map ; Then from the global semantic probability map Extract the category index of the maximum value, generate the background, tri-image mask and foreground based on the category index; finally, convert the detail feature probability map Multiply it element-by-element with the three-part image mask, and then perform element-by-element addition with the foreground to get the transparency mask, which is expressed as:

[0138] ;

[0139] ;

[0140] ;

[0141] ;

[0142] ;

[0143] ;

[0144] ;

[0145] in, represents the dimension of the tensor, Represents the category index, Indicates an indicator function, which returns 1 when the condition is met, otherwise it returns 0. Indicates the background, represents the tri-map mask, Indicates prospects, represents element-wise multiplication, represents the maximum value function, represents the activation function, Represents the fusion feature, which is the final output animal image transparency mask.

[0146] The present invention is described in detail above in conjunction with the embodiments of the accompanying drawings. A person skilled in the art can make various variations of the present invention according to the above description. Therefore, some details in the embodiments should not be construed as limiting the present invention, and the present invention shall be protected by the scope defined by the attached claims.

Claims

1. An automatic animal image cutout method, characterized in that: The following steps are involved: Step S1: construct an animal image dataset, where the animal image dataset includes a number of animal images; Step S2: constructing an animal image cutout model, the model comprising a moving window encoding module, a global semantic branch module, a detail feature branch module and a branch fusion module; importing the animal image of the data set in step S1 into the moving window encoding module to obtain encoding features of different scales; wherein the encoding features comprise the first feature, the second feature, the third feature, the fourth feature and the fifth feature extracted in sequence; Step S3: import the fifth feature in step S2 into the global semantic branch module to obtain the global semantic feature; Step S4: importing the second feature, the third feature, the fourth feature and the fifth feature of the encoding feature in step S2 into the detail feature branch module to obtain the detail semantic feature; Step S5: importing the global semantic features and the detail semantic features into the branch fusion module, performing feature fusion to obtain fusion features, that is, the predicted animal transparency mask map; Step S6: Construct a loss function and optimize the model by minimizing the loss.

2. The automatic animal image cutout method according to claim 1, characterized in that: Step S1 is specifically as follows: The image of the AM-2K dataset is obtained as the foreground and the image of the BG-20K dataset is obtained as the background, and the foreground and background are synthesized into animal images to construct an animal image dataset; it means: ； in, represents an animal image, r represents a pixel index, F represents foreground, B represents background, Indicates the probability that the pixel belongs to the foreground, ranging from 0 to 1; , , , They represent the composite pixel value, foreground pixel value, background pixel value and foreground pixel transparency value of pixel index r respectively.

3. The automatic animal image cutout method according to claim 2, characterized in that: The moving window encoding module in step S2 is composed of a Swin Transformer network, including five components connected in series. The first component is an image block embedding module. The second, third, fourth and fifth components have the same basic structure and are all stacked by at least one set of Swin Transformer blocks and image block merging modules. The five components are labeled A1, A2, A3, A4 and A5 in sequence. Among them, A2, A3 and A5 are composed of 1 Swin Transformer block and 1 image block merging module stacked once, while the fourth component A4 is composed of 1 Swin Transformer block and 1 image block merging module stacked 3 times; Step S2 is specifically as follows: Step S21: uniformly cropping the input animal images to obtain animal images of a fixed size, and then performing a data enhancement operation on the animal images of the fixed size, wherein the data enhancement operation includes cropping and / or flipping, thereby completing the preprocessing of the animal images; Step S22: Input the preprocessed animal image into the first component A1 of the moving window encoding module to preliminarily extract the first feature of the animal image; specifically: Three consecutive 3×3 convolutional layers and ReLU activation functions are used to increase the number of channels and gradually extract complex features. Then two layers of maximum pooling layers are used to reduce the spatial resolution and the amount of calculation while retaining the global characteristics. Finally, the features are normalized by the LayerNorm operation to ensure the stability of subsequent processing. Finally, the first feature is obtained. ; The specific formula is: ； ； in, represents the preprocessed animal image, is a collection of multi-layer convolutions, that is, three consecutive 3×3 convolutions, represents the number of convolutional layers, is the set of biases for multi-layer convolutions, is the number of bias layers of multi-layer convolution, is a two-dimensional convolution operation, is a nonlinear activation function, is the maximum pooling operation, It is the layer normalization operation; Step S23: The first feature Input to the second component A2, the second component A2 outputs the second feature , then the output of the second component A2 is used as the input of the third component A3, and so on to extract features; The process of processing input features by the Swin Transformer block and the image block merging module is as follows: The i-th feature of the input first passes through the Swin Transformer block, where the input feature is layer-normalized, then window division is performed to obtain the window feature, and then the attention feature is obtained through the window multi-head self-attention. After that, the attention feature is added to the i-th feature of the input through the residual connection and layer-normalized through the multi-layer perceptron. The output of the multi-layer perceptron is added to the layer-normalized result to obtain the result feature, and finally the result feature is downsampled through the image block merging module to finally obtain the i+1th feature. The specific formula is: ； ； ； ； ； in, , is the i-th feature, B is the batch size, H is the height of the feature map, W is the width of the feature map, C is the number of channels, represents a real number set, P is the window feature, Represents the attention feature, Indicates the result characteristics; represents the query vector, represents the transpose of the key vector, represents the dimension, b represents the bias term, represents the image segmentation operation, represents the normalization function, represents the window multi-head attention mechanism, For self-attention, Representation layer normalization, represents a multi-layer perceptron, represents the downsampling operation, represents the i+1th feature, , , , and Together they constitute the coding features.

4. The automatic animal image cutout method according to claim 3, characterized in that: The global semantic branch module in step S3 includes several global semantic decoding blocks connected in series. Specifically, the global semantic decoding block includes four semantic multi-convolution decoding blocks T1, T2, T3, T4 and a semantic single decoding block T5 connected in series, wherein a multi-dimensional attention enhancement module is connected in series after the semantic multi-convolution decoding block; the semantic multi-convolution decoding block is composed of three 3×3 convolution layers, three normalization layers, three ReLU layers and an upsampling layer connected in series; the multi-dimensional attention enhancement module is composed of two branches, the first branch includes a convolution block and a channel attention block connected in series, and the second branch includes a feature segmentation block and a spatial attention block connected in series; the semantic single decoding block T5 is composed of one 3×3 convolution layer and an upsampling layer connected in series; Step S3 is specifically as follows: The global semantic branch module inputs the fifth feature into the semantic multi-convolution decoding block T1 to obtain the decoding feature; the decoding feature is input into the two branches of the multi-dimensional attention enhancement module, the first branch extracts the global feature, and the second branch divides the decoding feature into multiple sub-regions, then weights the sub-regions through spatial attention, and then re-splices the weighted spatial features to obtain the splicing output with a dimension of 3, and performs dimensionality reduction splicing on the splicing output to obtain the local enhanced feature map, and finally adds the global feature and the local enhanced feature map element by element in different dimensions to obtain the output of the multi-dimensional attention enhancement module; The output of the multi-dimensional attention enhancement module corresponding to the semantic multi-convolution decoding block T1 is imported into the semantic multi-convolution decoding block T2 for decoding to obtain the decoding features; the output of the multi-dimensional attention enhancement module is enhanced to obtain the corresponding output of the multi-dimensional attention enhancement module; Similarly, the semantic multi-convolution decoding block T4 outputs the final global semantic features through the semantic single decoding block after obtaining the output of the corresponding multi-dimensional attention enhancement module; The specific calculation formula of the global semantic decoding block T4 and the corresponding multi-dimensional attention enhancement module is as follows: ； ； ； ； ； ； ； ； The specific calculation formula of the semantic single decoding block is as follows: ； in, Represents the fifth characteristic, Represents global features, represents the sub-region with height index i and width index j, Z represents the decoding feature, represents batch normalization, represents the attention mechanism, represents a nonlinear activation function, represents the convolution operation, represents a 3×3 convolution operation, represents the upsampling operation, represents spatial attention; i represents the segmentation index along the height direction, j represents the segmentation index along the width direction, Represents the tensor concatenation dimension, represents an element-wise multiplication operation, Represents a splicing operation; Represents a spatial feature with height index i and width index j; Representing spatial features and spatial characteristics At the output of the splicing with dimension 3, Representing spatial features and spatial characteristics The output of the splicing is in dimension 3; Feature map representing local enhancement; represents the final output of the multi-dimensional attention enhancement module, Represents global semantic features.

5. The automatic animal image cutout method according to claim 4, characterized in that: The detail feature branch module in step S5 includes an adaptive feature fusion module and a detail feature decoding block, the detail feature decoding block includes four sequentially connected detail multi-convolution decoding blocks F5, F4, F3, F2 and a detail single decoding block F1, wherein the detail multi-convolution decoding block includes two sequentially connected depth-separable convolution layers, one 3×3 convolution layer, one batch normalization layer, one ReLU layer and an upsampling layer; the adaptive feature fusion module includes an efficient regional attention module, an efficient channel weighting module and a spatial attention module sequentially connected; the detail single decoding block F5 is composed of a 3×3 convolution layer and an upsampling layer connected in series; Step S5 is specifically as follows: The adaptive feature fusion module then passes the input coding features through the efficient regional attention module to enhance the important areas of the coding features, highlight the location of key information in the image, and obtain an enhanced feature map. Secondly, through the efficient channel weighting module, the channels of the encoding features are weighted to enhance the features of important channels to obtain the channel feature map ; Then the channel feature map Weighted, highlighting the important areas in space, and obtaining the spatial feature map ;Finally, the feature map is enhanced by splicing and spatial feature map , get adaptive fusion features ; The detail multi-convolution decoding block upsamples the encoded features to obtain the corresponding upsampled features ; Adaptively fusion features And the corresponding upsampled features Fusion is performed and the fused features are input into the detail multi-convolution decoding block After concatenating with the encoded features, upsampling is performed; and so on, until the upsampling features of the detail multi-convolution decoding block F2 are obtained. And the corresponding adaptive fusion features ; Detail single decoding block F1 pair and adaptive fusion features splicing and upsampling to obtain detailed semantic features; among them, ; The upsampling process of the detail multi-convolutional decoding block is expressed as: ； The splicing process of the adaptive feature fusion module is expressed as: ； ； ； ； The process of obtaining detailed semantic features is expressed as: ； in, represents the up-sampled features, represents the upsampling operation, represents a nonlinear activation function, represents batch normalization operation, represents a 1×1 convolution operation, represents a two-dimensional convolution operation, Represents a two-dimensional convolution operation, a batch normalization operation, and a nonlinear activation operation. Represents a point-by-point convolution operation, that is, a 3×3 convolution, represents a depth-wise separable convolution operation, represents the index of the channel, The total number of channel indices. Represents a splicing operation, represents the activation function, Represents a weighted operation.

6. The automatic animal image cutout method according to claim 4, characterized in that: Step S5 is specifically as follows: using the activation function to process the global semantic features and the detailed semantic features to generate a global semantic probability map and detail feature probability map ; Then from the global semantic probability map Extract the category index of the maximum value, generate the background, tri-image mask and foreground based on the category index; finally, convert the detail feature probability map Multiply it element-by-element with the three-part image mask, and then perform element-by-element addition with the foreground to get the transparency mask, which is expressed as: ； ； ； ； ； ； ； in, represents the dimension of the tensor, Represents the category index, Indicates an indicator function, which returns 1 when the condition is met, otherwise it returns 0. Indicates the background, represents the tri-map mask, Indicates prospects, represents element-wise multiplication, represents the maximum value function, represents the activation function, Represents the fusion feature, which is the final output animal image transparency mask.

Images