Moving human body segmentation method and system based on deep learning

Abstract

This invention relates to human body segmentation technology, and discloses a deep learning-based method and system for segmenting moving human bodies. The deep learning-based method for segmenting moving human bodies includes: acquiring multi-scale human feature information; inputting a human image into an encoder network model to acquire multi-scale human feature information; constraining the human feature information of S1 to output predicted human keypoint information; inputting the human feature information into a decoder to acquire the segmentation result, the output of which includes a binary mask; calculating a loss function and training a network model using the predicted human keypoint information and the predicted binary mask; and updating the network model parameters based on the calculated loss function. This invention enables non-interactive automatic processing of moving human bodies under complex background conditions, generating high-precision edge segmentation results.

Description

Technical Field

[0001] The present invention relates to human body segmentation technology, and in particular to a moving human body segmentation method and system based on deep learning. Background Art

[0002] Image segmentation technology is a common image processing method, which is widely used in movie special effects, image synthesis and other tasks. It is the basic operation of special effects production and image editing. The purpose of image segmentation is to separate the foreground area or target of interest from the image, so as to further edit and process the foreground. Traditional image segmentation methods are usually based on color sampling or smoothing characteristics of local areas of the image. In addition, additional human-computer interaction is required, such as specifying the foreground and background. It is difficult to handle segmentation tasks under complex background conditions.

[0003] For example, in the prior art, the patent application number is: CN201910600394.7; the patent name is, training method and device for human segmentation neural network, patent application date: 2019-07-04. The patent application number is: CN202010728063.4, the patent name is, a foreground target extraction method and device, patent application date: 2020-07-23. In the segmentation process, most of them are aimed at static scenes, and the processing of non-static scenes is poor. In addition, the segmentation accuracy of fast-moving human bodies is not high. Summary of the invention

[0004] The present invention aims at the problems that the human body segmentation technology used in the prior art is mostly aimed at static scenes, but has poor processing for non-static scenes and low segmentation accuracy for fast-moving human bodies, and provides a moving human body segmentation method and system based on deep learning.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0006] The moving human body segmentation method based on deep learning includes:

[0007] S1, obtaining multi-scale human feature information, inputting a human image into an encoder network model to obtain multi-scale human feature information, the multi-scale human feature information including high-resolution human feature information and low-resolution human feature information, wherein the high-resolution human feature information includes high-precision edge information, and the low-resolution human feature information includes clear semantic information;

[0008] S2, human body feature information constraint, constrains the human body feature information of S1, and outputs the predicted human body key point information;

[0009] S3, segmentation of human feature information, inputting the human feature information of S1 into the decoder to obtain the segmentation result of the human feature information, and the output result includes a binary mask image;

[0010] S4, training of the network model, based on the human body key point information predicted by S2 and the binary mask image predicted by S3, calculates the loss function and trains the network model;

[0011] S5, updating the network model, updating the network model parameters according to the loss function calculated in S4.

[0012] Preferably, S4, training the network model, the training of the network model includes calculating the loss function, the calculation of the loss function includes: spatial transformation content consistency loss and edge pixel loss;

[0013] Edge pixel loss, the labeled binary mask image is eroded by an erosion operation O E and the expansion operation O D To obtain the edge area B of the human body,

[0014] B=~(O E (I)|~O D (I));

[0015] Among them, ~ is the negation operation, | is the OR operation;

[0016] In the training stage, the edge region of the image is obtained through the mask map of the edge region, and then supervised learning is performed on the edge region of the image;

[0017] Spatial transformation content consistency loss,

[0018] Perform pixel transformation on the input image I, and then perform spatial transformation T, and the network model outputs the predicted result M;

[0019] The image I without spatial transformation is input into the network model, and the result M' is predicted, and the loss constraint M'=T'(M), where T' is the inverse transformation of T.

[0020] Preferably, S2, the calculation of the loss function also includes key point feature loss, constraining the human feature map F, reducing the interference of background information, and predicting the human key point information, using MSE loss to calculate the difference between the predicted human key points and the real human key points.

[0021] Preferably, the training of the network model also includes data enhancement and optimization of the network model, and the data enhancement includes:

[0022] Generation of new data enhancement images, data enhancement based on pixel conversion and data enhancement based on spatial conversion methods to generate two new images I A and IB ;

[0023] The first weighted average, for I A and I B The two images are weighted averaged to generate I AUG1 , I AUG1 =W*I A +(1-W)*I B , where W is between 0 and 1;

[0024] The second weighted average is to generate I by weighted average of the input original image and AUG1 AUG2 , I AUG2 =M*I+(1-M)*I AUG2 , where M is between 0 and 1;

[0025] Model prediction, input original image I, I AUG1 and I AUG2 Three pictures, predict these three pictures and output M, M1 and M2;

[0026] Model constraints are constrained through KL divergence, so that the distribution of the binary mask images output after the three images are predicted remains consistent;

[0027] The background replacement method is used to enrich the training data set. When replacing the background, the annotated binary mask image is slightly eroded at the edges to generate a synthetic image. A new synthetic image I'=M*I+(1-M)*B is generated. g , where B g is the background image;

[0028] The optimization of the network model records the weight information during the model training process through the optimizer, calculates the weight information on average every K times, and updates the network model parameters, where K is the number of recorded iterations.

[0029] Preferably, the segmentation of the human feature information includes enlarging the low-resolution feature information to the resolution size of the feature information F through bilinear interpolation, and inputting the enlarged feature map into the low-resolution branch, wherein the low-resolution branch includes a channel attention module and a convolution layer, and outputs a predicted binary mask map MASK;

[0030] The predicted binary mask map MASK and the input image are input into the high-resolution branch, and a binary mask map with high-precision edge information is output; wherein the high-resolution branch contains 6 convolutional layers.

[0031] Preferably, the training of the network model further includes preventing the network model from overfitting;

[0032] Preventing the network model from overfitting includes calculating the loss value for back-propagation to update the parameters when the loss value of the loss function is greater than the threshold h, and calculating the loss value and using the negative value of the loss for back-propagation to update the parameters when the loss value is less than the threshold h.

[0033] In order to solve the above technical problems, the present invention also provides a moving human body segmentation system based on deep learning, which includes a multi-scale human body feature information acquisition module, a human body feature information constraint module, a human body feature information segmentation module, a network model training module and a network model update module;

[0034] A module for acquiring multi-scale human feature information, wherein the module is used to input a human image into an encoder network model to acquire multi-scale human feature information, wherein the multi-scale human feature information includes high-resolution human feature information and low-resolution human feature information, wherein the high-resolution human feature information includes high-precision edge information, and the low-resolution human feature information includes distinct semantic information;

[0035] A human feature information constraint module, which is used to constrain the human feature information of the multi-scale human feature information acquisition module and output predicted human key point information;

[0036] A human feature information segmentation module, which is used to input the human feature information of the multi-scale human feature information acquisition module into the decoder to obtain the segmentation result of the human feature information, and the output result includes a binary mask image;

[0037] The training module of the network model calculates the loss function and trains the network model based on the human key point information predicted by the constraint module of the human feature information and the binary mask image predicted by the segmentation module of the human feature information;

[0038] The updating module of the network model updates the parameters of the network model according to the loss function calculated by the training module of the network model.

[0039] Preferably, the training module of the network model includes a loss function calculation module, a data enhancement module and a network model optimization module. The loss function calculation module is used to evaluate the degree of difference between the network model prediction results and the actual annotation results. The data enhancement module is used to perform data augmentation on the input of the network model. The network model optimization module is used to improve the robustness and generalization of the network model.

[0040] In order to solve the above technical problems, the present invention further provides an electronic device, characterized in that it includes: at least one processor and a memory;

[0041] The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs a moving human body segmentation method such as deep learning.

[0042] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, a deep learning method for segmenting a moving human body is implemented.

[0043] The present invention has significant technical effects due to the adoption of the above technical solution:

[0044] The present invention first inputs the image to be segmented into the encoder network, performs feature extraction and outputs multi-scale human feature information, and then inputs the multi-scale human feature information into the decoder network. The decoder outputs a predicted binary mask image according to the multi-scale human feature information, and then obtains the loss value by weighted calculation of key point feature loss, mask pixel loss, edge pixel loss, content consistency loss and distribution consistency loss according to the prediction result and the labeling result, and then performs back propagation to update the model parameters. The optimizer saves the model parameters, averages the model parameters every several times, and uses the averaged model parameters as the initial value of the next iteration. The above process is repeated and iterated until the model converges.

[0045] The human body segmentation technology designed by the present invention can realize non-interaction automatic processing of moving human bodies under complex background conditions and generate edge high-precision segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the present invention.

[0047] Figure 2 It is a network model diagram of the present invention.

[0048] Figure 3 It is a schematic diagram of key points of the human body of the present invention.

[0049] Figure 4 It is a training flow chart of the network model of the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0051] Example 1

[0052] The moving human body segmentation method based on deep learning includes:

[0053] S1, obtaining multi-scale human feature information, inputting a human image into an encoder network model to obtain multi-scale human feature information, the multi-scale human feature information including high-resolution human feature information and low-resolution human feature information, wherein the high-resolution human feature information includes high-precision edge information, and the low-resolution human feature information includes clear semantic information;

[0054] S2, human body feature information constraint, constrains the human body feature information of S1, and outputs the predicted human body key point information;

[0055] S3, segmentation of human feature information, inputting the human feature information of S1 into the decoder to obtain the segmentation result of the human feature information, and the output result includes a binary mask image;

[0056] S4, training of the network model, based on the human body key point information predicted by S2 and the binary mask image predicted by S3, calculates the loss function and trains the network model;

[0057] S5, updating the network model, updating the network model parameters according to the loss function calculated in S4.

[0058] S4, training of the network model. The training of the network model includes the calculation of the loss function. The calculation of the loss function includes: spatial transformation content consistency loss and edge pixel loss;

[0059] Edge pixel loss, the labeled binary mask image is eroded by an erosion operation O E and the expansion operation O D To obtain the edge area B of the human body,

[0060] B=~(O E (I)|~O D (I))

[0061] Among them, ~ is the negation operation, | is the OR operation;

[0062] In the training stage, the edge region of the image is obtained through the mask map of the edge region, and then supervised learning is performed on the edge region of the image;

[0063] Spatial transformation content consistency loss,

[0064] Perform pixel transformation on the input image I, and then perform spatial transformation T, and the network model outputs the predicted result M;

[0065] The image I without spatial transformation is input into the network model, and the result M' is predicted, and the loss constraint M'=T'(M), where T' is the inverse transformation of T.

[0066] The training of the network model also includes data enhancement and optimization of the network model. Data enhancement includes:

[0067] Generation of new data enhancement images, data enhancement based on pixel conversion and data enhancement based on spatial conversion methods to generate two new images I A and I B ;

[0068] The first weighted average, for I A and I B The two images are weighted averaged to generate I AUG1 , I AUG1 =W*I A +(1-W)*I B , where W is between 0 and 1;

[0069] The second weighted average is to generate I by weighted average of the input original image and AUG1 AUG2 , I AUG2 =M*I+(1-M)*I AUG2 , where M is between 0 and 1;

[0070] Model prediction, input original image I, I AUG1 and I AUG2 Three pictures, predict these three pictures and output M, M1 and M2;

[0071] Model constraints are constrained through KL divergence, so that the distribution of the binary mask images output after the three images are predicted remains consistent;

[0072] The background replacement method is used to enrich the training data set. When replacing the background, the annotated binary mask image is slightly eroded at the edges to generate a synthetic image. A new synthetic image I'=M*I+(1-M)*B is generated. g , where B g is the background image;

[0073] The optimization of the network model records the weight information during the model training process through the optimizer, calculates the weight information on average every K times, and updates the network model parameters, where K is the number of recorded iterations.

[0074] The segmentation of human feature information includes enlarging the low-resolution feature information to the resolution of the feature information F through bilinear interpolation, and inputting the enlarged feature map into the low-resolution branch, where the low-resolution branch contains a channel attention module and a convolution layer, and outputs the predicted binary mask map MASK;

[0075] The predicted binary mask map MASK and the input image are input into the high-resolution branch, and a binary mask map with high-precision edge information is output; wherein the high-resolution branch contains 6 convolutional layers.

[0076] The training of network models also includes preventing the network models from overfitting;

[0077] Preventing the network model from overfitting includes calculating the loss value for back-propagation to update the parameters when the loss value of the loss function is greater than the threshold h, and calculating the loss value and using the negative value of the loss for back-propagation to update the parameters when the loss value is less than the threshold h.

[0078] Example 2

[0079] Based on Example 1, the calculation of the loss function of this embodiment also includes key point feature loss, constraining the human feature map F, reducing the interference of background information, and predicting human key point information, using MSE loss to calculate the difference between the predicted human key points and the real human key points.

[0080] Example 3

[0081] Based on the above embodiments, this embodiment collects corresponding human body data sets for different scenes, such as gymnastics, diving and other sports, and collects enough materials with different human body postures.

[0082] Label all the materials. When labeling, the edges of the foreground human body need to be feathered. This can be done using image editing software such as PS or GIMP. In addition, you need to prepare enough background images, such as natural scenery images. The selection of background images needs to be adjusted according to the specific application scenario and be as close to the background of the real scene as possible.

[0083] Network structure Figure 2 As shown in Figure 1, the resolution of the BGR input image I of the network model is fixed to 256×256, and a symmetric structure of encoder-decoder is adopted.

[0084] The encoder is composed of a convolutional network, which extracts different feature information from the input image. In order to reduce the amount of computation of the network model, the size of the feature map is reduced layer by layer, and an 8×8 feature map is output. Since the 8×8 feature map loses a lot of texture details, the feature maps with different scales generated in the middle will also be used. By fusing features of different scales, the fused multi-scale pyramid feature map is output, and finally the feature map output by the encoder network; the feature map contains 5 scales, namely F 1 (128×128), F 2 (64×64), F 3 (32×32), F 4 (16×16), F 5(8×8), where the high-resolution feature map contains rich edge information, which helps to reconstruct the mask edge with high precision, while the low-resolution feature map contains rich semantic information, which helps the model distinguish between the foreground and the background.

[0085] The decoder network consists of a convolutional network, with the input being the feature maps of four different scales output by the encoder and the input image I. It includes a main branch for segmentation prediction and an auxiliary branch for key point reasoning.

[0086] The key point prediction auxiliary branch consists of a convolutional layer, with the input feature map F 2 , output 17 channels of key point feature map F P , the human body key point distribution used is shown in Figure 3 shown.

[0087] The main branch of segmentation prediction consists of two sub-branches: a low-resolution branch and a high-resolution branch. The input of the low-resolution branch is F 2 、F 3 、F 4 、F 5 .

[0088] First, F 2 、F 3 、F 4 、F 5 Scale up to F by bilinear interpolation 2 The resolution of the feature map;

[0089] The features are concatenated along the channel dimension to form a fusion feature Fs, which is then output as a binary mask M through a convolutional network including channel attention. LR ;

[0090] The input of the high-resolution branch is F S , F 1 , I and M LR , after the convolutional network outputs a high-precision binary mask map M.

[0091] See the training flow chart Figure 4 As shown in the figure, this network model uses the following methods and strategies in the end-to-end training process to improve the robustness of the model in real scenarios, including data enhancement, optimizer and loss function.

[0092] In terms of data enhancement, we first use pixel-based data enhancement and spatial-based data enhancement methods to generate two new images. a and I b ;

[0093] Then the two images are weighted averaged to generate I Aug1 , I Aug1 =w*Ia +(1-w)*I b , where w is between 0 and 1, and then the input original image and Aug1 are weighted averaged to generate I Aug2 , I Aug2 =m*I+(1-m)*I Aug2 , where m is between 0 and 1;

[0094] In the training phase, input the original image I, I Aug1 and I Aug2 Three pictures, and then predict these three pictures and output M, M 1 and M 2 , and then use KL divergence to constrain the distribution of the three outputs to be as consistent as possible;

[0095] In order to enrich the training data, the background replacement method is used. When replacing the background, in order to ensure the naturalness of the foreground embedding, the annotated binary mask image needs to be slightly eroded at the edges. The degree of erosion is adjusted according to the annotation accuracy and task requirements, and then a new synthetic image I'=M*I+(1-M)*B is generated. g , where B g For the background image.

[0096] In the loss function, the key point feature loss is used to calculate the human feature map F 2 Constraints are imposed to reduce the interference of background information and predict the key points of the human body. The difference between the predicted key points of the human body and the real key points of the human body is calculated using the MSE loss. The edge pixel loss is used to improve the edge accuracy of the Mask generated by the model.

[0097] First, according to the marked binary mask image, through the corrosion operation O e and the expansion operation O d To obtain the edge area B of the human body, B = ~ (O e (I)|~O d (I)) Where ~ is a negation operation, | is an OR operation; in the training stage, the edge area of ​​the image is obtained through the mask image of the edge area, and then supervised learning is performed on the area;

[0098] The spatial transformation content consistency loss is used to improve the model's robustness to spatial position changes. First, the input image I is pixel transformed, and then the spatial transformation T is performed, such as translation, rotation and other transformation operations. The network model outputs the predicted result M, and then the image I without spatial transformation is input into the network model. At this time, the network model is set to eval mode, and the gradient information is not recorded. The result M' is predicted, and then the pixel loss constraint M'=T'(M) is used. Similarly, the same constraint method F is used for the feature map of the key point. P’ =T'(FP ), where T' is the inverse transformation of T;

[0099] To prevent the model from overfitting, a threshold h is added to the loss function, which is adjusted according to the loss value when the model converges.

[0100] When the loss value converges to 0.17, the threshold h can be set to 0.2. When the model is in the training stage, the loss value is greater than the threshold h at the beginning. At this time, the network model is the same as the normal case. The loss value is calculated and back-propagated to update the parameters. After that, the model converges to a certain extent.

[0101] The loss value starts to be less than the threshold h, the loss value is calculated, and the negative value of the loss is used for back propagation to update the parameters to prevent the model from overfitting.

[0102] In the optimizer, a multi-model weight averaging method is used to improve the stability and generalization of model training. First, the optimizer records the weight information during the model training process, and then averages the weight information every k times to update the model parameters, where k is the number of recording iterations, and in this embodiment, k=10.

[0103] Example 4

[0104] Based on the above embodiments, the present invention further provides a moving human body segmentation system based on deep learning, which includes a multi-scale human body feature information acquisition module, a human body feature information constraint module, a human body feature information segmentation module, a network model training module and a network model update module;

[0105] A module for acquiring multi-scale human feature information, wherein the module is used to input a human image into an encoder network model to acquire multi-scale human feature information, wherein the multi-scale human feature information includes high-resolution human feature information and low-resolution human feature information, wherein the high-resolution human feature information includes high-precision edge information, and the low-resolution human feature information includes distinct semantic information;

[0106] A human feature information constraint module, which is used to constrain the human feature information of the multi-scale human feature information acquisition module and output predicted human key point information;

[0107] A human feature information segmentation module, which is used to input the human feature information of the multi-scale human feature information acquisition module into the decoder to obtain the segmentation result of the human feature information, and the output result includes a binary mask image;

[0108] The training module of the network model calculates the loss function and trains the network model based on the human key point information predicted by the constraint module of the human feature information and the binary mask image predicted by the segmentation module of the human feature information;

[0109] The updating module of the network model updates the parameters of the network model according to the loss function calculated by the training module of the network model.

[0110] The training module of the network model includes a loss function calculation module, a data enhancement module and a network model optimization module. The loss function calculation module is used to evaluate the degree of difference between the network model prediction results and the actual annotation results. The data enhancement module is used to augment the input data of the network model. The network model optimization module is used to improve the robustness and generalization of the network model.

[0111] Example 5

[0112] In the above-mentioned technical embodiment, the present invention further provides an electronic device, characterized in that it includes: at least one processor and a memory;

[0113] The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs a moving human body segmentation method such as deep learning.

[0114] Example 6

[0115] Based on the above embodiments, this embodiment further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, a deep learning method for segmenting a moving human body is implemented.

Claims

1. A moving human body segmentation method based on deep learning, characterized in that: Methods include: S1, obtaining multi-scale human feature information, inputting a human image into an encoder network model to obtain multi-scale human feature information, the multi-scale human feature information including high-resolution human feature information and low-resolution human feature information, wherein the high-resolution human feature information includes high-precision edge information, and the low-resolution human feature information includes clear semantic information; S2, human body feature information constraint, constrains the human body feature information of S1, and outputs the predicted human body key point information; S3, segmentation of human feature information, inputting the human feature information of S1 into the decoder to obtain the segmentation result of the human feature information, and the output result includes a binary mask image; S4, training of the network model, based on the human key point information predicted by S2 and the binary mask image predicted by S3, calculates the loss function and trains the network model; S5, updating the network model, updating the network model parameters according to the loss function calculated in S4; S4, training the network model, the training of the network model includes the calculation of the loss function, and the calculation of the loss function includes: spatial transformation content consistency loss and edge pixel loss; Edge pixel loss, the labeled binary mask image is eroded by an erosion operation O e and the expansion operation O d To obtain the edge area B of the human body, B=~(O e (I)|~O d (I)) Among them, ~ is the negation operation, | is the OR operation; In the training stage, the edge region of the image is obtained through the mask map of the edge region, and then supervised learning is performed on the edge region of the image; Spatial transformation content consistency loss, Perform pixel transformation on the input image i, and then perform spatial transformation T, and the network model outputs the predicted result M; Input the image I without spatial transformation into the network model and predict the result M ’ , loss constraint M ’ =T ’ (M), where T' is the inverse transformation of T; The segmentation of human feature information includes enlarging the low-resolution feature information to the resolution of the feature information F through bilinear interpolation, and inputting the enlarged feature map into the low-resolution branch, where the low-resolution branch contains a channel attention module and a convolution layer, and outputs the predicted binary mask map Mask; The predicted binary mask map Mask and the input image are input into the high-resolution branch, and a binary mask map with high-precision edge information is output; wherein the high-resolution branch contains 6 convolutional layers.

2. The method for segmenting a moving human body based on deep learning according to claim 1, characterized in that ,S2,The calculation of the loss function also includes the key point feature loss, which constrains the human feature map f, reduces the interference of background information, and predicts the human key point information. The MSE loss is used to calculate the difference between the predicted human key points and the real human key points.

3. The method for segmenting a moving human body based on deep learning according to claim 1, characterized in that: The training of the network model also includes data enhancement and optimization of the network model. Data enhancement includes: Generation of new data enhancement images, data enhancement based on pixel conversion and data enhancement based on spatial conversion methods to generate two new images I a and I b ; The first weighted average, for I a and I b The two images are weighted averaged to generate I Aug1 , I Aug1 =w*I a +(1-w)*I b , where w is between 0 and 1; The second weighted average is to generate I by weighted average of the input original image and Aug1. Aug2 , I Aug2 =m*I+(1-m)*I Aug2 , where m is between 0 and 1; Model prediction, input original image I, I Aug1 and I Aug2 Three pictures, predict these three pictures and output M, M1 and M2; Model constraints are constrained through KL divergence, so that the distribution of the binary mask images output after the three images are predicted remains consistent; The background replacement method is used to enrich the training data set. When replacing the background, the annotated binary mask image is slightly eroded at the edges to generate a synthetic image. A new synthetic image I'=M*i+(1-M)*B is generated. g , where B g is the background image; The optimization of the network model records the weight information during the model training process through the optimizer, averages the weight information every k times, and updates the network model parameters, where k is the number of recorded iterations.

4. The method for segmenting a moving human body based on deep learning according to claim 1, characterized in that: The training of network models also includes preventing the network models from overfitting; Preventing the network model from overfitting includes calculating the loss value for back-propagation to update the parameters when the loss value of the loss function is greater than the threshold h, and calculating the loss value and using the negative value of the loss for back-propagation to update the parameters when the loss value is less than the threshold h.

5. A moving human body segmentation system based on deep learning, characterized in that: It includes a module for acquiring multi-scale human feature information, a module for constraining human feature information, a module for segmenting human feature information, a network model training module and a network model updating module; A module for acquiring multi-scale human feature information, wherein the module is used to input a human image into an encoder network model to acquire multi-scale human feature information, wherein the multi-scale human feature information includes high-resolution human feature information and low-resolution human feature information, wherein the high-resolution human feature information includes high-precision edge information, and the low-resolution human feature information includes distinct semantic information; A human feature information constraint module, which is used to constrain the human feature information of the multi-scale human feature information acquisition module and output predicted human key point information; A human feature information segmentation module, which is used to input the human feature information of the multi-scale human feature information acquisition module into the decoder to obtain the segmentation result of the human feature information, and the output result includes a binary mask image; The training module of the network model calculates the loss function and trains the network model based on the human key point information predicted by the constraint module of the human feature information and the binary mask image predicted by the segmentation module of the human feature information; An updating module of the network model, which updates the parameters of the network model according to the loss function calculated by the training module of the network model; S5, updating the network model, updating the network model parameters according to the loss function calculated in S4; S4, training the network model, the training of the network model includes the calculation of the loss function, and the calculation of the loss function includes: spatial transformation content consistency loss and edge pixel loss; Edge pixel loss, the labeled binary mask image is eroded by an erosion operation O e and the expansion operation O d To obtain the edge area B of the human body, B=~(O e (I)|~O d (I)) Among them, ~ is the negation operation, | is the OR operation; In the training stage, the edge region of the image is obtained through the mask map of the edge region, and then supervised learning is performed on the edge region of the image; Spatial transformation content consistency loss, Perform pixel transformation on the input image i, and then perform spatial transformation T, and the network model outputs the predicted result M; Input the image I without spatial transformation into the network model and predict the result M ’ , loss constraint M ’ =T ’ (M), where T' is the inverse transformation of T; The segmentation of human feature information includes enlarging the low-resolution feature information to the resolution of the feature information F through bilinear interpolation, and inputting the enlarged feature map into the low-resolution branch, where the low-resolution branch contains a channel attention module and a convolution layer, and outputs the predicted binary mask map Mask; The predicted binary mask map Mask and the input image are input into the high-resolution branch, and a binary mask map with high-precision edge information is output; wherein the high-resolution branch contains 6 convolutional layers.

6. The deep learning-based moving human body segmentation system according to claim 5, characterized in that: The training module of the network model includes a loss function calculation module, a data enhancement module and a network model optimization module. The loss function calculation module is used to evaluate the degree of difference between the prediction results of the network model and the actual annotation results. The data enhancement module is used to augment the input data of the network model. The network model optimization module is used to improve the robustness and generalization of the network model.

7. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the deep learning moving human body segmentation method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the deep learning moving human body segmentation method as described in any one of claims 1 to 4 is implemented.

Images