Segmentation method of quadratic element head portrait segmentation model based on MobileNetV2

Abstract

The invention discloses a segmentation method of a quadratic element head portrait segmentation model based on MobileNetV2, and relates to the technical field of computer vision and image processing, and the method employs MobileNetV2 as a backbone network of the segmentation model, employs the unique inverse residual structure and linear bottleneck of MobileNetV2 to adapt to the image features of a quadratic element head portrait, and achieves the segmentation of the quadratic element head portrait based on the MobileNetV2. According to the depth separable convolution of the MobileNetV2, homogeneous color features of a quadratic element head portrait are efficiently captured through depth-by-depth convolution, and then the features are flexibly combined through point-by-point convolution, so that model parameters and floating point operand are greatly reduced, meanwhile, a linear bottleneck layer avoids damage of a nonlinear activation function to low-dimensional feature information, and therefore, the method is suitable for large-scale popularization and application. According to the invention, the method effectively retains the key line and contour information for determining the accuracy of the segmentation boundary, can precisely analyze each semantic component of the head portrait, achieves the real-time and high-precision semantic segmentation of the quadratic element head portrait, and provides a technical support for the image editing and application of a mobile terminal.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, specifically to a segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2. Background Technology

[0002] With the popularization of anime and manga culture, the demand for anime and manga avatars in social media platforms, games, anime merchandise and other fields is increasing. In scenarios such as personalized editing, style transfer and feature analysis of anime and manga avatars, it is necessary to accurately segment the avatar image into different regions such as background, face, hair, eyes, and mouth. This process is known as anime and manga avatar segmentation.

[0003] Traditional image segmentation methods, such as threshold-based segmentation, edge detection-based segmentation, and traditional machine learning methods, are ill-suited to the characteristics of anime portrait images, which are rich in color, diverse in style, and complex in feature boundaries. As a result, they have low segmentation accuracy and cannot meet the needs of practical applications.

[0004] Currently, deep learning technology has been widely used in image segmentation, and segmentation models based on convolutional neural networks have become mainstream. However, existing deep learning-based segmentation models have the following problems:

[0005] Imbalance between model size and efficiency: Most high-precision segmentation models have a large number of parameters and high computational complexity, making them difficult to deploy in resource-constrained scenarios such as mobile devices and mini-programs;

[0006] Poor adaptability to anime avatars: Existing segmentation models are mostly designed for natural images and do not fully consider the unique style of anime avatars, resulting in insufficient segmentation accuracy for anime avatars, especially in the segmentation effect of detailed areas such as hair and eyes.

[0007] Insufficient dataset adaptability: There are very few anime head image samples in the existing publicly available image segmentation datasets, and there is a lack of fine-grained category annotations for anime head images, making it difficult to support the training of segmentation models specifically for anime head images.

[0008] Therefore, there is an urgent need for a segmentation method that balances lightweight model, high segmentation accuracy, and is specifically adapted to anime avatars, in order to overcome the shortcomings of existing technologies. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a segmentation method for anime avatars based on the MobileNetV2 model. This method utilizes MobileNetV2 as the backbone network of the segmentation model, leveraging its unique inverse residual structure and linear bottleneck to adapt to the image features of anime avatars. MobileNetV2's depthwise separable convolutions efficiently capture the homogeneous color features of anime avatars through depthwise convolutions, and then flexibly combine these features through pointwise convolutions, greatly reducing model parameters and floating-point computation. Simultaneously, the linear bottleneck layer avoids the destruction of low-dimensional feature information by nonlinear activation functions, effectively preserving the key lines and contour information that determine the accuracy of the segmentation boundaries. This collaborative mechanism can accurately parse the various semantic components of the avatar, achieving real-time, high-precision semantic segmentation of anime avatars, providing technical support for image editing and applications on mobile devices.

[0010] To solve the above-mentioned technical problems, this invention provides the following technical solution: a segmentation method for two-dimensional avatars based on the MobileNetV2 segmentation model, the specific steps of which are as follows:

[0011] S100. Construct a segmentation dataset of two-dimensional avatar images, wherein each sample in the segmentation dataset contains the original two-dimensional avatar image and the corresponding pixel-level segmentation label;

[0012] S200. Construct an encoder-decoder segmentation model with MobileNetV2 network as the backbone, where the decoder adopts a symmetrical structure similar to U-Net, and the feature maps of the corresponding stages of the encoder and decoder are fused through skip connections.

[0013] S300. The segmentation model is trained using the segmentation dataset to obtain a pre-trained model that can output a multi-channel segmentation probability map.

[0014] S400. Input the two-dimensional head image to be segmented into the pre-trained model to obtain the corresponding multi-channel segmentation probability map, and parse the multi-channel segmentation probability map to generate the final segmentation mask.

[0015] S500: Based on the segmentation mask and downstream application requirements, extract pixel regions of specified semantic parts from the original avatar image, and perform attribute analysis on the pixel regions.

[0016] Furthermore, in S100, the segmentation dataset specifically includes the original dataset DP and the optimized dataset ODP, wherein the DP dataset is labeled with 19 segmentation categories and the ODP dataset is labeled with 11 segmentation categories;

[0017] The segmentation label is a single-channel image with the same resolution as the original image, where the integer value of each pixel represents its category index.

[0018] Furthermore, the 19 segmentation categories of the DP dataset include background, facial skin, eyebrows, eyes, pupils, mouth, teeth, tongue, ears, hair, hat, glasses, collar, necklace, scarf, gloves, top, skirt, and shoes;

[0019] The ODP dataset was obtained by merging semantically similar categories from the DP dataset. The 11 segmentation categories include background, face, eyes, mouth, ears, hair, head ornaments, face ornaments, top, bottom, and accessories.

[0020] Furthermore, in S200, the specific architecture of the encoder-decoder segmentation model with the MobileNetV2 network as its backbone includes:

[0021] The encoder consists of a pre-trained MobileNetV2 network for feature extraction. The MobileNetV2 backbone network includes an initial fully convolutional layer, multiple inverse residual bottleneck layer sequences, and downsampling layers to progressively convert the input image into feature maps with different spatial scales and semantic abstraction levels. During the encoding process, the model records intermediate feature maps from different stages for subsequent skip connections.

[0022] Decoder: It consists of multiple upsampling blocks symmetrically connected in series. Each upsampling block contains a transposed convolutional layer or a bilinear interpolation upsampling operation, followed by at least one standard convolutional layer. The output of each upsampling block is concatenated with the intermediate feature map of the corresponding stage of the encoder in the channel dimension to fuse high-resolution spatial detail information and deep semantic information.

[0023] Output Header: At the end of the decoder, a 1x1 convolutional layer is connected. The number of kernels in this convolutional layer is equal to the number of segmentation categories C. After the 1x1 convolutional layer, a Softmax activation function is connected to normalize the output to a probability vector of each pixel in all C categories, thereby generating the multi-channel segmentation probability map.

[0024] Furthermore, the specific steps of S300 in training the segmentation model using the segmentation dataset are as follows:

[0025] S301. Divide the segmented dataset in S100 into a training set, a validation set, and a test set;

[0026] S302. Preprocess the pre-trained two-dimensional avatar images, input the preprocessed images into the segmentation model constructed in S200, perform forward propagation, and obtain the predicted probability map.

[0027] S303. Calculate the loss function, which is used to quantify the difference between the probability map predicted by the model and the true segmentation label;

[0028] S304. Calculate the gradient of the loss function with respect to all parameters of the model through backpropagation, and update the parameters accordingly;

[0029] S305. Repeat S302-S304 until the model's performance on the validation set is stable, and a converged pre-trained model is obtained.

[0030] Furthermore, in S400, the process of parsing the multi-channel segmentation probability map is as follows:

[0031] For each spatial location (i, j) in the multi-channel segmentation probability map, perform the ArgMax operation on the probability vectors of the C categories.

[0032] The ArgMax operation finds the category index k corresponding to the maximum probability value in the probability vector, i.e., k = argmax(Pij), where Pij is the probability vector at position (i, j).

[0033] Assign the category index k to the pixel value at position (i, j) of the output segmentation mask to generate a single-channel segmentation mask, where each pixel value directly represents its predicted semantic category.

[0034] The segmentation mask is presented in two formats: a merged view and a multi-channel binary mask.

[0035] Furthermore, in the segmentation mask:

[0036] For the merged view, a predefined color lookup table is applied to the generated segmentation mask. Each category index value in the mask is mapped to a specific color, ultimately generating a color image where different color regions intuitively represent different semantic parts.

[0037] For multi-channel binary masks, the single-channel segmentation mask and the one-hot encoding of C categories are decomposed into C independent single-channel binary images of the same size as the original image, generating C corresponding binary masks. For the M-th channel binary mask, all pixels with a value of M in the original single-channel segmentation mask have a value of 1 in this binary mask, and all other pixels have a value of 0. In each binary mask, the region with a pixel value of 1 represents the semantic part corresponding to that category.

[0038] Furthermore, in step S500, the process of performing attribute analysis on the extracted pixel region is as follows:

[0039] Based on the target semantic category requirements specified by the downstream application, obtain its corresponding binary mask;

[0040] The binary mask is used as a mask and multiplied pixel by pixel with the original two-dimensional head image to extract the pixel region image containing only the target semantic part. The pixel values ​​outside the region are set to zero.

[0041] Perform statistical analysis on the color values ​​of all non-zero pixels in the pixel region: calculate the arithmetic mean of each channel of all non-zero pixels in the RGB color space to obtain the average color value, and use the average color value as the representative color of the semantic part.

[0042] Compared with existing technologies, this segmentation method based on the MobileNetV2-based two-dimensional avatar segmentation model has the following advantages:

[0043] I. This invention employs MobileNetV2 as the backbone network of its segmentation model. Leveraging MobileNetV2's unique inverse residual structure and linear bottleneck, it adapts to the image features of anime avatars. MobileNetV2's depthwise separable convolutions efficiently capture the homogeneous color features of anime avatars through depthwise convolutions, and then flexibly combine these features through pointwise convolutions, greatly reducing model parameters and floating-point computation. At the same time, the linear bottleneck layer avoids the destruction of low-dimensional feature information by nonlinear activation functions, effectively preserving the key lines and contour information that determine the accuracy of the segmentation boundary. This collaborative working mechanism can accurately parse the various semantic components of the avatar, achieving real-time, high-precision semantic segmentation of anime avatars, and providing technical support for image editing and applications on mobile devices.

[0044] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0046] Figure 1 The flowchart shows the segmentation method of the two-dimensional avatar segmentation model based on MobileNetV2;

[0047] Figure 2 This is a flowchart illustrating the steps of a segmentation method for a two-dimensional avatar based on the MobileNetV2-based segmentation model.

[0048] Figure 3 This is a diagram showing the original image and the sample set corresponding to the pixel-level segmentation labels in Example 1;

[0049] Figure 4 This is a diagram showing the composition of segmentation labels in the segmentation task dataset DP in Example 1;

[0050] Figure 5 This is a diagram showing the composition of the ODP dataset samples in Example 1;

[0051] Figure 6 The image shows the segmentation results of different channels in MobileNetV2 in Example 1.

[0052] Figure 7 This is a merged view of the MobileNetV2 segmentation results in Example 1. Detailed Implementation

[0053] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0054] Example 1

[0055] This embodiment provides the working principle of a segmentation method based on the MobileNetV2-based 2D avatar segmentation model. It aims to address issues such as insufficient accuracy of traditional segmentation methods in 2D avatar segmentation, the imbalance between size and efficiency in existing deep learning segmentation models, and poor adaptability to 2D avatars. Figure 2 As shown, this method constructs a dedicated two-dimensional avatar segmentation dataset, builds an encoder-decoder segmentation model with MobileNetV2 as the backbone, completes model training using the dataset, inputs the image to be segmented into the trained model to generate a segmentation mask, and finally extracts the specified semantic region based on the mask and performs attribute analysis. This achieves two-dimensional avatar semantic segmentation that balances lightweightness and high accuracy, providing technical support for downstream applications such as personalized editing and style transfer of two-dimensional avatars on mobile devices.

[0056] First, the segmentation dataset construction phase (S100) is initiated, providing a sample set containing the original images and corresponding pixel-level segmentation labels, such as... Figure 3As shown, to ensure the model can learn the features of different semantic categories of anime avatars, the segmentation dataset specifically includes the original Danbooru-Parsing (DP) dataset and the optimized Danbooru-Parsing (ODP) dataset. The DP dataset focuses on the fine-grained semantic category division, allowing the model to learn more detailed features of anime avatar components. The ODP dataset simplifies the category system by merging semantically similar categories, reducing model training complexity and improving model efficiency in scenarios where high category fine-grainedness is not required. The DP dataset labels 19 segmentation categories, covering background, facial skin, eyebrows, eyes, pupils, mouth, teeth, tongue, ears, hair, hat, glasses, collar, necklace, scarf, gloves, top, skirt, and shoes. Figure 5 As shown, the ODP dataset was obtained by merging semantically similar categories from the DP dataset, and it is labeled with 11 segmentation categories, including background, face, eyes, mouth, ears, hair, head ornaments, face ornaments, top, bottom, and accessories. This preserves key semantic information, reduces the number of categories, and lowers the learning difficulty for the model. The segmentation labels are in single-channel image format with the same resolution as the original images, such as... Figure 4 As shown, the integer value of each pixel represents its category index.

[0057] Then, the segmentation model construction stage (S200) is entered, where an encoder-decoder segmentation model with MobileNetV2 network as the backbone is built to achieve effective feature extraction and semantic segmentation of anime head images. The encoder consists of a pre-trained MobileNetV2 network, which extracts features from the input anime head image, progressively converting the image into feature maps with different spatial scales and semantic abstraction levels. MobileNetV2's unique inverse residual structure and linear bottleneck can adapt to the image features of anime head images. The inverse residual structure effectively extracts detailed features from the image through a process of first increasing and then decreasing dimensionality. The linear bottleneck avoids the destruction of low-dimensional feature information by non-linear activation functions, which helps to preserve the key lines and contour information that determine the accuracy of segmentation boundaries in anime portraits. Meanwhile, MobileNetV2 uses depthwise separable convolutions, which efficiently capture homogeneous color features of anime portraits through depthwise convolutions and then flexibly combine these features through pointwise convolutions, greatly reducing model parameters and floating-point computation, meeting the deployment requirements of the model in resource-constrained scenarios such as mobile devices. Specifically, the MobileNetV2 backbone network includes an initial fully convolutional layer, multiple inverse residual bottleneck layer sequences, and downsampling layers. The initial fully convolutional layer's role is to... The input image undergoes preliminary feature extraction and dimensionality adjustment, transforming it from the original RGB three-channel format to the number of feature channels required by the model, laying the foundation for subsequent feature extraction. Multiple inverse residual bottleneck layer sequences are the core of feature extraction. Each inverse residual bottleneck layer uses a 1×1 convolution to increase the dimensionality of the input feature map, acquiring richer information. Then, a 3×3 depthwise separable convolution is used to convolve each channel individually, extracting the features of that channel. Finally, a 1×1 convolution reduces the dimensionality of the feature map, compressing the number of parameters. Multiple such bottleneck layers are arranged in a specific order to form a sequence, progressively extracting deep semantic features of the image. The downsampling layer then... Operations with a stride greater than 1 reduce the spatial resolution of the feature maps, increase the receptive field, and enable the model to capture a wider range of image information while reducing subsequent computation. During the encoding process, the model records intermediate feature maps from different stages. These intermediate feature maps contain information at different spatial scales. Shallow intermediate feature maps have high spatial resolution and contain more detailed information, such as the edges and textures of the avatar. Deep intermediate feature maps have low spatial resolution but high semantic abstraction and contain more global semantic information, such as the overall structure of the avatar and the categories of parts. These intermediate feature maps will be used for subsequent skip connections to fuse feature information at different scales.The decoder consists of multiple symmetrically concatenated upsampling blocks. It progressively upsamples the low-resolution, highly semantically abstract feature maps output by the encoder, restoring them to a resolution close to the original image. It also fuses intermediate feature maps from the corresponding encoder stage to supplement detailed information and improve segmentation accuracy. Each upsampling block contains a transposed convolutional layer or a bilinear interpolation upsampling operation, followed by at least one standard convolutional layer. This adjusts and optimizes the upsampled feature maps, reducing noise introduced during upsampling and further extracting effective features to prepare for subsequent feature fusion and segmentation. The output of each upsampling block is concatenated with the intermediate feature maps from the corresponding encoder stage along the channel dimension. This effectively combines the detailed information from the encoder's intermediate feature maps with the semantic information from the upsampling block output, enabling the model to determine component categories based on deep semantic information and accurately locate component boundaries based on detailed information during segmentation, thereby improving segmentation accuracy. At the end of the decoder, a 1×1 convolutional layer is connected. The number of kernels in this convolutional layer is equal to the number of segmentation categories C. This layer converts the multi-channel feature map output by the decoder into a number of channels corresponding to the number of segmentation categories, with each channel corresponding to one segmentation category. This prepares for subsequent calculation of the probability of each category. The 1×1 convolutional layer only changes the number of channels in the feature map, without changing its spatial resolution. It can achieve the transformation of category dimensions while preserving the spatial information of the feature map, and the computational cost is small, without significantly increasing the complexity of the model. After the 1×1 convolutional layer, a Softmax activation function is connected to normalize the output to a probability vector of each pixel in all C categories, thereby generating a multi-channel segmentation probability map. The function of the Softmax activation function is to convert the output value of each pixel in each category channel into a probability value between 0 and 1, and the sum of the probability values ​​of all categories is 1. Each channel corresponds to the probability distribution of one category, and the pixel value in each channel is the probability that the pixel belongs to the corresponding category.

[0058] Next, the model training phase (S300) begins. This phase uses the segmentation dataset built in S100 to train the segmentation model constructed in S200. By continuously adjusting the model parameters, the model can accurately learn the features of different semantic categories of anime avatars, thereby obtaining a pre-trained model with stable performance and high segmentation accuracy. Specific steps include:

[0059] Dataset partitioning (S301): The segmentation dataset in S100 is divided into a training set, a validation set, and a test set. The training set is used for the model to learn data features and adjust model parameters. The validation set is used to monitor the training progress of the model during training and to determine whether the model has overfitting or underfitting. The test set is used to evaluate the generalization ability of the model after the model training is completed and to test the model's segmentation performance on unseen data.

[0060] Image preprocessing and forward propagation (S302): The two-dimensional head images in the training set are preprocessed to eliminate the differences between images and provide the model with input data of uniform format and feature distribution, thereby improving the training efficiency and stability of the model. The preprocessed images are then input into the segmentation model constructed in S200 for forward propagation. During the forward propagation process, the images are sequentially processed by the encoder for feature extraction, the decoder for feature upsampling and fusion, and the output head for probability calculation, finally obtaining the predicted multi-channel segmentation probability map. This probability map reflects the model's prediction result for the category to which each pixel in the input image belongs.

[0061] Loss function calculation (S303): The loss function is used to quantify the difference between the probability map predicted by the model and the true segmentation label, and is the basis for updating the model parameters.

[0062] Parameter Update (S304): The gradient of the loss function with respect to all parameters of the model is calculated through backpropagation, and the model parameters are updated based on the gradient information. According to the chain rule, starting from the loss function, the partial derivatives of the loss function with respect to the parameters of each layer of the model are calculated step by step, i.e., the gradients. These gradients reflect the degree of influence of parameter changes on the loss function. A positive gradient of a parameter means that increasing the parameter will increase the loss function, and a negative gradient of a parameter means that increasing the parameter will decrease the loss function. After obtaining the gradients of each parameter, the optimizer is used to update the model parameters, adaptively adjusting the learning rate of each parameter to improve the efficiency and stability of model training.

[0063] Model convergence judgment (S305): Repeat steps S302-S304, that is, continuously preprocess the training set images, calculate the prediction probability map through forward propagation, calculate the loss function, and update the model parameters through backpropagation until the model's performance on the validation set is stable, and a converged pre-trained model is obtained.

[0064] Next, the segmentation mask generation stage (S400) is entered. In this stage, the two-dimensional head image to be segmented is input into the pre-trained model to obtain a multi-channel segmentation probability map. The probability map is then analyzed to generate a segmentation mask that clearly reflects the semantic regions of the image, laying the foundation for the subsequent extraction of specified semantic pixel regions. The two-dimensional head image to be segmented is processed in the same way as in S302 to ensure that the format and feature distribution of the input image are consistent with the input during model training, avoiding inaccurate model segmentation results due to differences in input data. The preprocessed image is input into the pre-trained model obtained in S305. After processing by the model's encoder, decoder, and output head, a corresponding multi-channel segmentation probability map is generated. This probability map has the same structure as the predicted probability map output by the model in S302. Each channel corresponds to a segmentation category, and each pixel value within a channel represents the probability that the pixel belongs to the corresponding category. The multi-channel segmentation probability map is parsed, that is, for the probability vectors of C categories at each spatial position (i, j) in the multi-channel segmentation probability map, an ArgMax operation is performed. The purpose of the ArgMax operation is to find the category index corresponding to the maximum probability value from the probability vector of each pixel. This category index is the semantic category to which the model predicts the pixel belongs. The category index k obtained through the ArgMax operation is assigned to the pixel value at position (i, j) of the output segmentation mask, thereby generating a single-channel segmentation mask. In this segmentation mask, as shown... Figure 6 As shown, each pixel value directly represents its predicted semantic category. The segmentation mask can be presented in two formats: a merged view and a multi-channel binary mask. These two formats are suitable for different application scenarios. For the merged view, as shown... Figure 7 As shown, a predefined color lookup table is applied to the generated segmentation mask. Each category index value in the mask is mapped to a specific color, ultimately generating a color image. Different color regions intuitively represent different semantic parts. For multi-channel binary masks, the single-channel segmentation mask is decomposed into C independent single-channel binary images of the same size as the original image by decomposing the one-hot encoding of C categories. C corresponding binary masks are generated. For the binary mask of the Mth channel, all pixels with a value of M in the original single-channel segmentation mask have a value of 1 in this binary mask, and all other pixels have a value of 0. In each binary mask, the region with a pixel value of 1 represents the semantic part corresponding to that category.

[0065] Finally, the process moves to the designated semantic region extraction and attribute analysis stage (S500). Based on the segmentation mask generated in S400 and the requirements of downstream applications, pixel regions of the designated semantic parts are accurately extracted from the original anime head image. Attribute analysis is then performed on these regions to provide specific semantic region data and attribute information for downstream applications. According to the target semantic category requirements specified by the downstream application, the corresponding semantic category is determined. Based on the determined target semantic category, the corresponding binary mask is obtained from the multi-channel binary mask generated in S400. The corresponding channel in the multi-channel binary mask is selected because each channel in the multi-channel binary mask specifically corresponds to... A semantic category can accurately identify the pixel region corresponding to that category, avoiding color recognition errors that may occur when extracting regions from merged views, thus improving the accuracy of region extraction. The obtained corresponding binary mask is used as a mask, and a pixel-by-pixel multiplication operation is performed with the original anime head image. For each pixel in the original image, its pixel value is multiplied by the corresponding pixel value in the binary mask. If the pixel value at that position in the binary mask is 1, the pixel value in the original image remains unchanged; if the pixel value at that position in the binary mask is 0, the pixel value in the original image is set to zero. This operation... This method can extract pixel regions containing only the target semantic part of the image, with pixel values ​​outside this region set to zero, thus achieving precise separation of the target semantic region. Statistical analysis is then performed on the color values ​​of all non-zero pixels within the extracted target semantic region to calculate the representative color of that semantic part. The core of this color value statistical analysis is calculating the arithmetic mean of all non-zero pixels in each channel of the RGB color space, obtaining the average color value, which is then used as the representative color of that semantic part. The RGB color space includes three channels: Red (R), Green (G), and Blue (B), with each channel ranging from 0 to 255. Each pixel... Color is determined by the values ​​of these three channels. By iterating through all pixels in the extracted pixel region image, non-zero pixels are filtered out, and the sum of the pixel values ​​of these non-zero pixels in the R, G, and B channels is calculated. Then, the total number of non-zero pixels is calculated, and the sum of the pixel values ​​in the R channel is divided by the total number of non-zero pixels to obtain the arithmetic mean of the R channel. Similarly, the arithmetic mean of the G and B channels is calculated. The RGB value composed of these three averages is the average color value of the semantic part, which is the representative color of the semantic part. By calculating the representative color of the target semantic part, important attribute information can be provided to downstream applications.

[0066] In summary, this embodiment details the complete implementation process of the segmentation method for anime avatars based on the MobileNetV2 segmentation model, from the construction of the segmentation dataset, the establishment of the segmentation model, model training, to the generation of the segmentation mask, and then to the extraction of specified semantic regions and attribute analysis. This method can be widely applied to anime avatar-related applications in social media platforms, games, anime merchandise, and other fields.

[0067] Example 2

[0068] Based on Example 1, this example provides a segmentation method for two-dimensional avatars using a MobileNetV2-based two-dimensional avatar segmentation model, detailing the specific steps involved in two-dimensional avatar segmentation. Figure 1 As shown, the specific steps are as follows:

[0069] (1) Data preparation and model building

[0070] Obtain a dataset containing original two-dimensional avatar images and their corresponding pixel-level segmentation labels.

[0071] The segmentation labels are single-channel images, and the integer value of each pixel represents its semantic category index.

[0072] The complete dataset is randomly divided into three mutually exclusive subsets: training set, validation set, and test set.

[0073] (2) Constructing a segmentation model

[0074] Encoder: Loads a MobileNetV2 model pre-trained on ImageNet as the backbone for feature extraction.

[0075] Decoder: Construct an upsampling path symmetrical to the encoder. Each decoder block contains upsampling operations and convolutional layers.

[0076] Skip connection: The feature maps output by each stage of the encoder are concatenated with the feature maps of the corresponding stages of the decoder along the channel dimension.

[0077] Output Header: A 1x1 convolutional layer is added at the end of the decoder, with the number of output channels equal to the number of segmentation categories C. It is followed by a Softmax activation function to output the category probability distribution of each pixel.

[0078] (3) Model training

[0079] Read a batch of anime avatar images from the training set.

[0080] The image is preprocessed, and the preprocessed image is then input into the constructed segmentation model.

[0081] The model performs forward computation and ultimately outputs a multi-channel probability map.

[0082] The probability map output by the model is compared with the true segmentation labels, and the error of the current prediction is calculated using the combined loss function.

[0083] The backpropagation algorithm is used to calculate the gradient of the loss function with respect to all parameters of the model. The optimizer is then used to update the model parameters based on the gradient to reduce the loss, and this pre-trained model is saved.

[0084] (4) Image segmentation reasoning

[0085] Receive a two-dimensional head image to be segmented, and perform the same preprocessing operations on the image as during training.

[0086] The data is then input into a pre-trained model and forward propagated to obtain a multi-channel probability map.

[0087] (5) Generate a mask by analyzing the probability graph

[0088] Perform the ArgMax operation on each pixel position of the probability map and select the category index with the highest probability.

[0089] Combine the category indices of all pixels into a single-channel segmentation mask.

[0090] (6) Application of results and attribute analysis

[0091] Based on application requirements, obtain the binary mask corresponding to the target category from the segmentation mask.

[0092] Using this binary mask as a template, perform pixel-by-pixel multiplication with the original image to extract the clean pixel region containing only the target semantic part.

[0093] (7) Calculate representative attributes

[0094] Statistical analysis was performed on the extracted pixel regions.

[0095] Calculate the average or mode of all non-zero pixels in the RGB color space, and use that value as the representative color for this semantic part.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A segmentation method for two-dimensional avatars based on the MobileNetV2 segmentation model, characterized in that, The specific steps of this method are as follows: S100. Construct a segmentation dataset of two-dimensional avatar images, wherein each sample in the segmentation dataset contains the original two-dimensional avatar image and the corresponding pixel-level segmentation label; S200. Construct an encoder-decoder segmentation model with MobileNetV2 network as the backbone, where the decoder adopts a symmetrical structure similar to U-Net, and the feature maps of the corresponding stages of the encoder and decoder are fused through skip connections. S300. The segmentation model is trained using the segmentation dataset to obtain a pre-trained model that can output a multi-channel segmentation probability map. S400. Input the two-dimensional head image to be segmented into the pre-trained model to obtain the corresponding multi-channel segmentation probability map, and parse the multi-channel segmentation probability map to generate the final segmentation mask. S500: Based on the segmentation mask and downstream application requirements, extract pixel regions of specified semantic parts from the original avatar image, and perform attribute analysis on the pixel regions.

2. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 1, characterized in that, In S100, the segmentation dataset specifically includes the original dataset DP and the optimized dataset ODP, wherein the DP dataset is labeled with 19 segmentation categories and the ODP dataset is labeled with 11 segmentation categories; The segmentation label is a single-channel image with the same resolution as the original image, where the integer value of each pixel represents its category index.

3. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 2, characterized in that, The DP dataset has 19 segmentation categories, including background, facial skin, eyebrows, eyes, pupils, mouth, teeth, tongue, ears, hair, hat, glasses, collar, necklace, scarf, gloves, top, skirt, and shoes. The ODP dataset was obtained by merging semantically similar categories from the DP dataset. The 11 segmentation categories include background, face, eyes, mouth, ears, hair, head ornaments, face ornaments, top, bottom, and accessories.

4. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 1, characterized in that, In S200, the specific architecture of the encoder-decoder segmentation model with MobileNetV2 network as the backbone includes: The encoder consists of a pre-trained MobileNetV2 network for feature extraction. The MobileNetV2 backbone network includes an initial fully convolutional layer, multiple inverse residual bottleneck layer sequences, and downsampling layers to progressively convert the input image into feature maps with different spatial scales and semantic abstraction levels. During the encoding process, the model records intermediate feature maps from different stages for subsequent skip connections. Decoder: It consists of multiple upsampling blocks symmetrically connected in series. Each upsampling block contains a transposed convolutional layer or a bilinear interpolation upsampling operation, followed by at least one standard convolutional layer. The output of each upsampling block is concatenated with the intermediate feature map of the corresponding stage of the encoder in the channel dimension to fuse high-resolution spatial detail information and deep semantic information. Output Header: At the end of the decoder, a 1x1 convolutional layer is connected. The number of kernels in this convolutional layer is equal to the number of segmentation categories C. After the 1x1 convolutional layer, a Softmax activation function is connected to normalize the output to a probability vector of each pixel in all C categories, thereby generating the multi-channel segmentation probability map.

5. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 1, characterized in that, The specific steps of S300 in training the segmentation model using the segmentation dataset are as follows: S301. Divide the segmented dataset in S100 into a training set, a validation set, and a test set; S302. Preprocess the pre-trained two-dimensional avatar images, input the preprocessed images into the segmentation model constructed in S200, perform forward propagation, and obtain the predicted probability map. S303. Calculate the loss function, which is used to quantify the difference between the probability map predicted by the model and the true segmentation label; S304. Calculate the gradient of the loss function with respect to all parameters of the model through backpropagation, and update the parameters accordingly; S305. Repeat S302-S304 until the model's performance on the validation set is stable, and a converged pre-trained model is obtained.

6. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 1, characterized in that, In S400, the process of parsing the multi-channel segmentation probability map is as follows: For each spatial location (i, j) in the multi-channel segmentation probability map, perform the ArgMax operation on the probability vectors of the C categories. The ArgMax operation finds the class index k corresponding to the maximum probability value in the probability vector, i.e., k = argmax(P ij ), where P ij Let be the probability vector at position (i, j); Assign the category index k to the pixel value at position (i, j) of the output segmentation mask to generate a single-channel segmentation mask, where each pixel value directly represents its predicted semantic category. The segmentation mask is presented in two formats: a merged view and a multi-channel binary mask.

7. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 6, characterized in that, In the segmentation mask: For the merged view, a predefined color lookup table is applied to the generated segmentation mask. Each category index value in the mask is mapped to a specific color, ultimately generating a color image where different color regions intuitively represent different semantic parts. For multi-channel binary masks, the single-channel segmentation mask and the one-hot encoding of C categories are decomposed into C independent single-channel binary images of the same size as the original image, generating C corresponding binary masks. For the M-th channel binary mask, all pixels with a value of M in the original single-channel segmentation mask have a value of 1 in this binary mask, and all other pixels have a value of 0. In each binary mask, the region with a pixel value of 1 represents the semantic part corresponding to that category.

8. The segmentation method for a two-dimensional avatar segmentation model based on MobileNetV2 according to claim 1, characterized in that, In step S500, the process of performing attribute analysis on the extracted pixel region is as follows: Based on the target semantic category requirements specified by the downstream application, obtain its corresponding binary mask; The binary mask is used as a mask and multiplied pixel by pixel with the original two-dimensional head image to extract the pixel region image containing only the target semantic part. The pixel values ​​outside the region are set to zero. Perform statistical analysis on the color values ​​of all non-zero pixels in the pixel region: calculate the arithmetic mean of each channel of all non-zero pixels in the RGB color space to obtain the average color value, and use the average color value as the representative color of the semantic part.

Images