Anchoring-free example portrait semantic analysis method

Abstract

The application discloses an anchor-free instance portrait semantic analysis method. The application trains an instance portrait semantic analysis model by using a training set, wherein the instance portrait semantic analysis model comprises a feature extraction module, a character instance detection branch and a character instance fine-grained perception branch; the character instance fine-grained perception branch comprises a detail keeping module, a human body part context coding module, a character instance analysis module and an instance analysis result refining module; the application adopts a one-stage anchor-free detector based on center point prediction for character instance boundary box position prediction and an edge-guided character instance semantic analysis module for portrait semantic part identification; the anchor-free character detector not only inherits the advantages of pixel-level design, but also can effectively avoid the super parameter sensitivity problem caused by generating candidate boundary boxes; the edge-guided portrait semantic analysis module can effectively distinguish different character instance positions and adjacent portrait semantic categories.

Description

Technical Field

[0001] This invention belongs to the field of computer vision instance portrait semantic segmentation, and specifically relates to an anchor-box-free instance portrait semantic parsing method. Background Technology

[0002] Instance-based semantic parsing of human images is a fundamental problem in computer vision and multimedia, focusing on pixel-level content analysis centered on humans in real-world scenarios. The goal of instance-based semantic parsing is not only to effectively distinguish regions of different human instances, but also to accurately parse the semantic category of the human image within each instance's region.

[0003] Current mainstream instance-based human image semantic parsing models, such as Parsing R-CNN, RP R-CNN, and M-CE2P, all follow the Mask R-CNN framework. Their two-stage person instance detectors propose anchor box candidate regions and perform fine-grained human image part segmentation. However, two-stage anchor box-based person instance detectors have two drawbacks. First, their performance is highly sensitive to predefined hyperparameters in anchor box generation, such as aspect ratio, anchor box area, and scale. When encountering new object detection tasks or processing new datasets, careful fine-tuning or redesigning of anchor box generation hyperparameters is necessary to achieve better performance. Furthermore, selecting bounding box samples with high recall and controlling the imbalance ratio of positive and negative samples during training both incur additional computational and storage costs. Second, the two-stage anchor box-based object detection method operates on a non-pixel-level prediction model, which is inconsistent with the pixel-level prediction model required for instance-based human image semantic parsing tasks. Therefore, we explore a fully convolutionally solvable, anchor-box-free, one-stage detector that avoids hyperparameter sensitivity and is more suitable for instance-based human image semantic parsing tasks.

[0004] For the semantic parsing task of person instances, simply having a powerful and elegant fully convolutional person detector is insufficient; the ultimate goal is to accurately parse pixel-level predictions of person instance regions. Given a person instance region, the objective is to perform fine-grained segmentation of this region and parse the category of each pixel. Typically, a person instance region contains multiple body part categories; therefore, edge information is introduced as a meaningful clue to distinguish the boundaries between different person instances and the edges of adjacent body part category regions within a single person instance region. A simple edge prediction branch and a person instance semantic parsing branch are executed in parallel. The motivation for this design strategy is to enable the person instance detector to provide relatively accurate bounding box predictions, allowing the person instance semantic parsing module to focus on fine-grained recognition.

[0005] Furthermore, since instance-level image semantic parsing tasks require not only instance-level detection but also part-level segmentation, simply using pixel-level cross-entropy loss functions for optimization cannot yield ideal image semantic parsing results. Because instance-level image semantic segmentation is performed based on predicted candidate bounding boxes, it often results in some low-quality predicted bounding boxes and inaccurate predicted person instance regions. Simultaneously, within the candidate bounding boxes, part-level image semantic category prediction results also exhibit unsatisfactory performance. Summary of the Invention

[0006] To address the technical problems existing in the prior art, the purpose of this invention is to provide an anchor-free method for semantic parsing of instance portraits. This invention solves the problems of hyperparameter sensitivity in current instance portrait semantic parsing models, the difficulty in effectively distinguishing the boundaries of adjacent person instances and the boundaries of adjacent portrait semantic category regions, and the low prediction confidence based on bounding boxes, thus resulting in unsatisfactory instance portrait parsing performance.

[0007] The technical solution of this invention is as follows:

[0008] An anchor-box-free method for semantic parsing of instance portraits, comprising the following steps:

[0009] During the model training phase, a training set is acquired or generated, in which each person image is labeled with the person's spatial location information and the semantic category information of each pixel. The training set is then used to train an instance-based portrait semantic parsing model, which includes a feature extraction module, a person instance detection branch, and a person instance fine-grained perception branch. The person instance fine-grained perception branch includes a detail preservation module, a human body part context encoding module, a person instance parsing module, and an instance parsing result refinement module.

[0010] The feature extraction module is used to extract features of the person image at multiple different scales, with each scale corresponding to a feature map; wherein, the i-th scale corresponds to feature map P. i , i = 1 to n; n is the set number of scales;

[0011] For each feature map P i The data are then input into the parallel person instance detection branch and the person instance fine-grained perception branch, respectively. The feature map P is then predicted by the person instance detection branch. i The category to which the candidate bounding box belongs for each pixel; the feature map P is obtained through fine-grained perception branch prediction of person instances. i The perceptual probability of the center of the bounding box and the offset of the bounding box position for each pixel;

[0012] For candidate bounding boxes categorized as "person," the detail preservation module performs region-of-interest (ROI) pooling on a feature map at a set scale to obtain the features of the corresponding person instance bounding box, which is then input into the human body part context encoding module. The human body part context encoding module includes a human body part context pyramid module and a non-local mining module. The human body part context pyramid module extracts multi-scale contextual information of the person instance from the input features and sends it to the non-local mining module. The non-local mining module mines the spatial location relationships in the input multi-scale contextual information and inputs them into the person instance parsing module. The person instance parsing module predicts the pixel-level semantic parsing map and edge information map of the person instance based on the input features.

[0013] The instance parsing result refinement module is used to evaluate the quality of the parsing results based on the pixel-level semantic parsing graph obtained from the character instance parsing module.

[0014] During the model training phase, each module is optimized using a different loss function. The loss function for the person instance detection branch is: in, The loss used to calculate the regression of the candidate bounding box offsets for person instances. Used to calculate the loss for classifying candidate bounding boxes of person instances. Used to calculate the center perception probability loss corresponding to the bounding box.

[0015] The loss function for the character instance parsing module is: in Used to calculate the loss of the pixel-level semantic parsing graph of person instances. Used to calculate the loss of the edge information graph of person instances.

[0016] The loss function for the refining module is: Where L miou L is used to calculate the quality loss of the pixel-level semantic parsing graph of person instances. miou-score The confidence probability loss is used to calculate the pixel-level semantic parsing graph of a person instance; α, β, θ, and γ are the weight coefficients, respectively.

[0017] The overall total loss function is the sum of the losses of all modules:

[0018] In the model application stage, for a person image to be parsed, the feature extraction module extracts various feature representations of different sizes from the input image. These features are then input into the person instance detection branch to obtain the detection results of the person image in the input image. That is, a rectangular box is used to represent the region where the person instance is located. Then, the homogeneous pooling operation is used on feature P3. Based on the obtained rectangular box region, the person instance parsing module is used to obtain the final pixel-level parsing result of each person instance.

[0019] To address the aforementioned technical issues, an anchor-free method for semantic parsing of instance portraits is proposed, comprising the following steps:

[0020] (1) Person Instance Detection Branch

[0021] Given an input image of a person, a multilayer perceptual backbone network extracts multi-level perceptual features from the input image. Specifically, five different levels of features, P3, P4, P5, P6, and P7, have sizes of 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 of the input image, respectively. Multi-level features P3 to P7 share the person instance detection sub-model, where feature map P... i Each spatial location (x, y) has three predicted outputs: a four-dimensional bounding box offset vector t. * A classifier for a single person and a center perception score. A four-dimensional bounding box regression vector t. * Represented as (l * ,t * ,r * ,b * ), where l * ,t * ,r * ,b * These represent the offset distances from the spatial position (x, y) to the left, top, right, and bottom edges of the bounding box, respectively.

[0022] (2) Edge-guided fine-grained perception branch of character instances

[0023] The edge-guided fine-grained perception branch for human instances comprises four main parts: detail preservation module, human body part context encoding module, human portrait parsing module, and refinement module.

[0024] a) Detail Preservation Module

[0025] The detail-preserving module is implemented on feature P3 through region-of-interest (ROI) aligned pooling, where P3 is the finest-grained output feature in the feature pyramid, i.e., the one with the largest spatial dimension. Based on the characteristics of the person instance semantic parsing task, there are two main reasons for adopting the detail-preserving mechanism. One reason is the need for a sufficiently large number of labeled person instance samples to train a robust model, as small-scale person instances possess less appearance information. Because coarse-resolution features, i.e., those with smaller spatial dimensions, provide limited appearance information, especially for smaller-scale human body categories within the person region, performing RIO pooling on coarser-resolution features (such as P7) can lead to the neglect of small-scale human body parts, such as gloves, the left hand, or glasses, due to downsampling operations at the coarse-resolution layer. Therefore, fine-grained resolution is more suitable for fine-grained image semantic parsing tasks, as it provides more detailed appearance information, thereby assisting in better person instance segmentation. Inspired by the high-resolution preservation strategy employed in the semantic segmentation model DeepLab V3+, which leverages detailed information from lower-level feature maps to aid in recovering fine-grained information, this invention selects the finest-grained feature P3 from the feature pyramid to perform region-of-interest (ROI) pooling. Intuitively, performing RIO pooling solely on feature map P3 might not be sufficient, as it could overlook some higher-level semantic information. However, in reality, the P3 feature already incorporates higher-level semantic features.

[0026] b) Human Parts Context Encoding Module

[0027] The human body part context encoding module is executed on the output features of the detail-preserving module. Contextual information is a highly valuable clue, and its effectiveness has been proven in tasks such as semantic segmentation and human image semantic parsing. This invention employs two methods to mine contextual information: scale and category relationships. Scale factor contextual information is used to address the issue of scale inconsistencies. Human body part categories typically encompass multiple scales; therefore, employing an effective method to capture global and local information to form multi-scale contextual feature representations greatly aids subsequent fine-grained human image semantic parsing steps. In fact, feature pyramids are a popular and effective architecture capable of fusing multi-scale information. Specifically, a human body part context pyramid module is used to probe multi-scale contextual information of human instances, where multiple parallel aggregation-excitation units employ different spatial amplitude ratios to explore different scale information. Another method is to mine category relationships to extract contextual information. This approach provides valuable correlations between multiple human body categories, and the adjacency matrix of these relationships improves the contextual feature representation. The contextual information of a spatial location is generally related to a series of spatial locations; a spatial location (or pixel) in an image corresponds to a real human body category. This leads to the conclusion that constructing the relationship between spatial location and its contextual location can reflect the relationships between different human body categories. Inspired by self-attention mechanisms, self-supervised attention mechanisms can capture spatial location dependencies over longer distances, ultimately selecting non-local operations to mine spatial location correlations in the input multi-scale contextual information. Finally, the two methods of exploring contextual information are combined to form the human body part context encoding module, which can provide rich contextual feature representations to help identify fine-grained human body categories.

[0028] c) Character Instance Analysis Module

[0029] The person instance parsing module includes two parallel outputs: person instance parsing and edge prediction. One motivation for introducing the edge prediction branch is to help distinguish different human body categories within a single person instance region. Typically, multiple different human semantic categories exist within a person instance region; accurately distinguishing adjacent human part categories greatly helps in correctly parsing all image semantic category regions. Another motivation is that within candidate bounding box regions, due to the existence of overlapping person instances, there will definitely be multiple different person instances; therefore, effectively distinguishing overlapping person instances is also crucial. Since edge information has already been validated in image semantic parsing, it is an effective clue to help distinguish the boundaries of human body parts. This invention borrows from edge information and extends its role, not only distinguishing different person instances but also helping to accurately parse person instance regions. The image parsing module is implemented after the enhanced contextual feature representation. Specifically, four consecutive convolutional layers are used to provide features for the image parsing module, which are then input into the two prediction branches: image semantic category and edge prediction.

[0030] d) Instance Analysis Result Refinement Module

[0031] The semantic parsing results for instance-level human images suffer from two main problems due to predicted bounding boxes: low-quality global bounding boxes and incorrect part-level semantic parsing maps. Low-quality bounding boxes, with inaccurate predicted regions, lead to lower Intersection over Union (IoU) scores and negatively impact the instance-level average accuracy score. Simultaneously, inaccurate predictions in the human image parsing map result in lower IoU scores for some human image semantic categories, resulting in unsatisfactory global semantic parsing results. The improvement module aims to employ a complementary mechanism: global bounding boxes focus on improving the quality of predicted bounding boxes, while part-level human image parsing maps focus on improving the quality of each human image semantic category. Specifically, the global bounding boxes use a sub-network that evaluates IoU scores to estimate the quality of the predicted instance-level human image within the detection edit box. This sub-network comprises five convolutional layers, and the final output is a one-dimensional IoU score. The part-level parsing results utilize an optimized, solvable surrogate loss based on structured awareness.

[0032] The features and beneficial effects of this invention are:

[0033] The anchor-free instance portrait semantic parsing method proposed in this invention achieves a running efficiency of 13.3 frames per second (fps) on a machine configured with a CPU i7-3770 CPU@3.40GHz and an 11G NVIDIA 2080Ti GPU. Attached Figure Description

[0034] Figure 1 An overall flowchart for anchor-free instance portrait semantic parsing.

[0035] Figure 2 This is a structural diagram of the context encoding module for human portrait components.

[0036] Figure 3 For the context pyramid module of human figure parts.

[0037] Figure 4 This is a schematic diagram of a non-local mining module.

[0038] Figure 5 A refined module structure diagram based on the example analysis results. Detailed Implementation

[0039] This invention addresses two major problems in existing two-stage anchor-box-based instance portrait semantic parsing models: heuristic bounding box design and non-pixel-level design characteristics. It proposes an anchor-box-free instance portrait semantic parsing method. The invention and specific embodiments are further described below with reference to the accompanying drawings.

[0040] This invention addresses the task of semantic parsing of people in complex real-world scenarios by proposing an anchor-free instance-based semantic parsing method to effectively resolve the positional information of each person in an image and the semantic category of each pixel in the image. The model is a pixel-level solvable, fully convolutional design. Specifically, it employs a one-stage anchor-free detector based on center point prediction for predicting the bounding box position of person instances and an edge-guided semantic parsing module for recognizing semantic parts of the image. The anchor-free person detector not only inherits the advantages of pixel-level design but also effectively avoids the hyperparameter sensitivity problem caused by generating candidate bounding boxes. The edge-guided semantic parsing module can effectively distinguish different person instance positions and adjacent semantic categories. The proposed anchor-free instance-based semantic parsing process is as follows: Figure 1 As shown, the specific steps are as follows:

[0041] (1) Given an input image of a person, a multilayer perceptron backbone network is first used to extract features of the input image at various scales. Specifically, the input image first undergoes multiple convolution operations to obtain features of five different spatial dimensions, namely C1, C2, C3, C4, and C5, with sizes of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. Figure 1The illustrations for C1 and C2 are omitted. Specifically, C1 to C5 each employ multiple convolutional and downsampling operations to progressively obtain features of different spatial sizes. Using the three features C3, C4, and C5, five different levels of features P3, P4, P5, P6, and P7 are obtained through five different strides, with sizes of 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 of the input image, respectively. P3, P4, and P5 are obtained by performing a 1×1 convolution operation on C3, C4, and C5 in a top-down connection pattern within the convolutional neural network. (See attached diagram.) Figure 1 As shown, P3, P4, and P5 are obtained by connecting features C3, C4, and C5 of the convolutional network from the top down through a convolutional layer with a kernel size of 1×1. P6 and P7 are obtained by applying a convolutional layer with a kernel size of 3×3 and a stride of 2 to P5 and P6, respectively.

[0042] (2) For the five levels of feature maps {P3, P4, P5, P6, P7} obtained in step (1), for each feature map P i The algorithm is processed through two parallel prediction branches to obtain the category of the candidate box corresponding to each pixel in the feature map, the offset of the bounding box position, and the center receptivity prediction of that pixel. One prediction branch consists of four consecutive convolutional layers and a class prediction convolution operation to obtain the class prediction of the target box corresponding to each pixel, i.e., person or background. The other prediction branch also extracts the feature representation through four consecutive convolutional layers, passes through a center prediction convolution operation to obtain the center receptivity probability of the bounding box corresponding to each pixel, and passes through another regression prediction convolution operation to obtain the prediction of the four displacement coordinates of the bounding box. Offset vector t * =(l * ,t * ,r * ,b * The bounding box is composed of the offset distances from the spatial position (m,n) to the left, top, right, and bottom edges of the bounding box. in and Let t represent the coordinates of the top-left and bottom-right vertices of the bounding box, respectively. Then, the four-dimensional offset vector t... * It can be represented as:

[0043]

[0044]

[0045] The loss function for two parallel prediction branches is written as:

[0046]

[0047] in, This represents the focal loss function. Indicates the loss by intersection and union. This represents the binary cross-entropy loss. and These represent the losses for the three outputs: classification, offset regression, and center perception, respectively.

[0048] (3) The person instance detection boxes obtained in step (2) (candidate boxes predicted as people in the previous step) are processed by region of interest aligned pooling (RoIAlign) on feature P3 to obtain the feature representation of each person instance bounding box. The output size of the aligned pooling is 32×32, so each person instance candidate bounding box obtains a feature representation of size 32×32×256. This representation is then processed by the human body part context encoding module (e.g., Figure 2 (As shown) further enhances feature representation, resulting in a new feature of size 32×32×512. The human body part context encoding module consists of two parts: one is the human body part context pyramid module (as shown) Figure 3 As shown), the other part is the non-local mining module (such as...). Figure 4 As shown). Figure 3 As shown, the input features of the human body part context pyramid module are the pooled features aligned with the candidate bounding boxes of human instances (size 32×32×256). These features are processed through five parallel convolutional branches, yielding feature representations of different sizes: 32×32×256, 8×8×256, 4×4×256, 2×2×256, and 1×1×256. The first parallel branch consists of a 3×3 kernel convolutional layer and a 1×1 convolution. The second to fifth parallel branches consist of 2, 3, 4, and 5 depthwise separable convolutional layers with 3×3 kernels, respectively, to obtain encoded features. These features are then upsampled by 4x, 8x, 16x, and 32x to obtain new activation features, which are then multiplied element-wise with the input features before a final 1×1 convolution. Finally, the output features of the five parallel branches are concatenated to obtain a new candidate box feature of size 32×32×1280. This is then subjected to a new 3×3 convolution operation to obtain the output feature of this module, with a size of 32×32×512. The output features of the human body part context pyramid module are subsequently used as input to the non-local mining module, such as... Figure 4As shown, the input features are processed through two branches. The first branch undergoes a 1×1 convolution operation to obtain new features (32×32×512 in size). The second branch undergoes max pooling and two 1×1 convolution operations to obtain two sets of features of size 16×16×512. One set of features and the output features of the first branch are then reshaped to calculate the similarity between the two features, i.e., the similarity matrix between their spatial locations, with a size of (32×32)×(16×16). The similarity matrix is ​​then normalized using Softmax, multiplied by the other set of features from the second branch, and reshaped to obtain a new feature representation of size 32×32×512. This new feature representation is then processed by a 1×1 convolution and group normalization (GN) operation, and added element-wise to the input features to obtain the new output features of the nonlocal mining module (32×32×512 in size).

[0049] The output features of the non-local mining module are input into the person instance parsing module. The person instance parsing module first processes the input features through four consecutive convolutional layers and one group standardization layer. Then, it uses two parallel branches to predict the pixel-level semantic parsing map (size 256×256×C, where C represents the number of body part categories) and edge information map (size 256×256×2) of the person instance. Both parallel prediction branches are implemented using deconvolution operations. The deconvolution operation used for the person image parsing map has a kernel size of 4×4 and a stride of 2; the edge prediction uses a deconvolution operation with a kernel size of 2×2 and a stride of 2.

[0050] The loss function of the character instance parsing module can be expressed as:

[0051]

[0052] in This represents the standard cross-entropy loss function. It is a weighted cross-entropy loss function, α = β = 2.

[0053] (4) The prediction results obtained in step (3) are further optimized through the refinement module. First, for the pixel-level semantic parsing results of the person instance, the loss function of uniform intersection and union is used for further optimization. Second, a portrait semantic parsing quality prediction model is used to predict the quality of the semantic parsing results, with the specific structure as follows: Figure 5As shown, the semantic parsing result of the human image (256×256×C) is first subjected to max pooling to obtain a feature of size 32×32×C. Then, it is concatenated with the last layer feature in step (3), i.e., 32×32×512, to obtain a new feature representation of 32×32×(512+C). This is then subjected to two convolutional operations with a kernel size of 3×3 and a kernel number of 128, one average pooling operation, and three convolutional layers (each with a kernel size of 1×1, and output sizes of 256, 256, and 1, respectively). Finally, the semantic parsing quality evaluation score of the bounding box of the human image instance is obtained.

[0054] The loss function of the refining module is expressed as:

[0055]

[0056] Where L miou L represents the average intersection and ratio loss function. miou-score It is the mean square error function, θ = 2, γ = 1.

[0057] The overall total loss function formula is shown below:

[0058]

[0059] During model training, the input image needs to be labeled with the spatial location of each person in the image, represented by the center point coordinates, length, and width. Additionally, pixel-level semantic category information for each person needs to be labeled, i.e., the semantic category of the human body to which each pixel belongs, such as face, left arm, right arm, hat, etc. The shorter side length of the output image is randomly selected from the range [640, 800] as the shorter side length, and then adjusted to 800 pixels while maintaining the aspect ratio, while the longer side becomes less than or equal to 1333 pixels. The model is trained using the Stochastic Gradient Descent (SGD) optimization method for a total of 75 epochs, with a batch size of 8 images. The initial learning rate is set to 0.005, which is reduced to one-tenth of the initial learning rate at epochs 50 and 65. The weight decay and momentum coefficients are set to 0.0001 and 0.9, respectively. The backbone network is initialized using model parameters pre-trained on ImageNet classification data. During the test inference phase, the person instance detection branch outputs 50 bounding boxes of person instances with the highest confidence scores. These candidate boxes are then input into the subsequent edge-guided instance portrait parsing branch to predict the result of portrait semantic parsing.

[0060] The above description is merely an embodiment of the present invention and does not limit the scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection claimed by the present invention.

Claims

1. An anchor-free instance portrait semantic parsing method, characterized in that, The method comprises: A model training stage: obtaining or generating a training set, each image of a person in the training set being labeled with spatial position information of the person and semantic category information of each pixel; training an instance portrait semantic parsing model using the training set, the instance portrait semantic parsing model comprising a feature extraction module, a person instance detection branch, and a person instance fine-grained perception branch; the person instance fine-grained perception branch comprising a detail preservation module, a human body part context encoding module, a person instance parsing module, and an instance parsing result refining module; wherein The feature extraction module is configured to extract a plurality of different scale features of the person image, each scale corresponding to a feature map; wherein the i-th scale corresponds to a feature map P i , i = 1 ~ n; n is a set number of scales. For each feature map P i , which are respectively input into a parallel human instance detection branch and a human instance fine-grained perception branch, a class to which each candidate box corresponding to each pixel point in the feature map P i predicted by the human instance fine-grained perception branch; and a center perception probability of a bounding box corresponding to each pixel point in the feature map P i predicted by the human instance detection branch, and an offset of the position of the bounding box. For a candidate box of the category of a person, the detail preservation module performs a region of interest alignment pooling operation on the candidate box on a set scale feature map to obtain features of a corresponding person instance bounding box and input the features to the human body part context encoding module; the human body part context encoding module comprises a human body part context pyramid module and a non-local mining module; the human body part context pyramid module is configured to obtain multi-scale context information of the person instance from the input features and send the multi-scale context information to the non-local mining module; the non-local mining module is configured to mine spatial position correlation in the input multi-scale context information and input the spatial position correlation to the person instance parsing module; the person instance parsing module is configured to predict a pixel-level semantic parsing map and an edge information map of the person instance according to the input features; and the instance parsing result refining module is configured to evaluate the quality of the parsing result according to the pixel-level semantic parsing map obtained in the person instance parsing module. The loss function used in training the instance portrait semantic parsing model is The loss function for the person instance detection branch is used to calculate the loss of the person instance candidate box offset regression. The loss function for the person instance parsing module is The loss for calculating the pixel-level semantic parsing graph of the person instance is used, The loss for calculating the edge information graph of the person instance is used. refine the loss function of the instance parsing result refinement module, L miou a quality loss L for calculating the pixel-level semantic parsing map of the person instance, miou-score a confidence probability loss for calculating the pixel-level semantic parsing map of the person instance; α, β, θ, γ are weight coefficients, respectively. A model application stage: for a person image to be parsed, the feature extraction module is used to extract multiple different scale features of the person image to be parsed and input the multiple different scale features to the person instance detection branch to obtain a pixel-level parsing result of each person instance in the person image to be parsed.

2. The method of claim 1, wherein, The loss function of the person instance detection branch wherein, The loss for calculating the person instance candidate box classification, The loss for calculating the center perception probability corresponding to the bounding box, The intersection over union loss.

3. The method of claim 2, wherein, is a focal loss function, is a binary cross-entropy loss function.

4. The method of claim 1, wherein, L miou is the average cross-entropy loss function, L miou-score is the mean squared error function.

5. The method of claim 1, wherein, The semantic category information comprises a face, a left arm, a right arm, and a hat.

6. The method of claim 1, wherein, The set scale feature map is a feature map corresponding to a maximum scale.

7. A server, characterized by The computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step of the method of any one of claims 1 to 6.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step of the method of any one of claims 1 to 6. The computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step of the method of any one of claims 1 to 6.

Images