A method and system for estimating human body posture

Abstract

The present invention provides a method and system for estimating human posture, including inputting an image into a codec network including a four-layer coding layer and a four-layer decoding layer structure for training, and outputting a semantic segmentation result; using an energy function to convert a pixel semantic probability map obtained in the first two coding layers into an edge activation pixel map, in response to an activation value of a pixel being greater than an activation value threshold, the pixel is an edge pixel; based on the semantic label in the semantic segmentation result, pixels belonging to the same instance are aggregated to obtain an instance segmentation result, and the instance segmentation result includes a mask indicating the instance to which each pixel belongs; using a fully convolutional network to generate a human skeleton confidence map, outputting a label of a skeleton component to which each pixel belongs in each instance, regressing the joint point position through a fully connected network, and connecting the joint points in each instance to construct a human skeleton structure to obtain human posture information. The method and system improve the crossover phenomenon existing in posture estimation between people.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a method and system for estimating human body posture. Background Art

[0002] Human pose estimation technology, also known as human pose recognition technology, aims to map human body pixels in images or video streams to the three-dimensional surface of the limbs. It can continuously, automatically and quickly extract the position information of each joint of the human body in the image or video stream, which involves many computer vision tasks, such as target detection, pose estimation, segmentation, etc. A large number of modeling can be performed based on the joint point position information obtained by this technology. Its application scenarios include not only human joint point positioning, such as graphics, augmented reality (AR), human-computer interaction, but also 3D target recognition and many other aspects.

[0003] Human posture estimation technology has very broad application prospects in the fields of behavior recognition, human-computer interaction, games, animation, etc., such as somatosensory games in the game field; it can be applied to home monitoring in the security field, such as identifying falls of elderly people living alone at home, and identifying special human postures through monitoring so as to respond in time; it can be applied to smart classrooms in the education field, such as identifying students' attendance in teaching activities, such as the perception of behaviors such as raising hands and sleeping in class.

[0004] However, traditional human posture estimation technology is highly dependent on hardware. For example, human posture estimation based on Kinect has been successfully applied to the field of somatosensory games and has achieved success. However, this technology has very high hardware requirements and has limitations in perception distance and perception accuracy. Inevitably, there are problems such as complex hardware configuration and operation, insensitivity to long-distance portraits, and great influence from environmental factors.

[0005] The rapid development of deep learning in the field of computer vision has enabled human posture estimation technology to gradually get rid of its dependence on hardware and develop into human posture estimation based on images using deep learning. The deep learning-based method is free from the constraints of hardware and only relies on the input image information. This method has achieved breakthroughs in terms of high accuracy, high speed, sensitivity to human portraits, and little impact from environmental factors.

[0006] At present, the more popular human posture estimation methods are mainly based on the following two ideas.

[0007] One is the top-down idea. This idea is to regress the bounding box area of ​​all human bodies in the image, and then estimate the human body posture of each person in each box to obtain the position information of each person's joint points, that is, first obtain the position information of all people, and then obtain the position information of the joint points from the position information of each person; the other is the bottom-up idea. This idea is to first regress the position of the joint points of each part of all people in the input image, and then gradually connect all the related nodes in the image to obtain the human skeleton structure of each person. That is, first obtain the position information of all related nodes, and then obtain the position information of the person from the position information of the joint points.

[0008] The human pose estimation method based on the top-down idea predicts a rich human bounding box from the image, then obtains the final bounding box through operations such as non-maximum suppression, and then estimates the human pose of each person in each box to obtain the position information of each person's joint points. This not only increases the amount of calculation, but also the human pose estimation result depends on the bounding box regression accuracy. The existing human pose estimation method based on the bottom-up idea, such as OpenPose, first detects each joint point of the human body, then connects the joint points to construct the human skeleton structure. However, this method is prone to joint point misconnection in scenarios where multiple people interact.

[0009] In other words, although the top-down approach can effectively improve the situation of joint point misconnection, it is not applicable in complex scenarios because its running speed is proportional to the number of people. The bottom-up approach can effectively improve the above problems, but it is prone to joint point misconnection. Summary of the invention

[0010] In order to solve the technical problems in the prior art that the running speed of the top-down human posture estimation method is proportional to the number of people, which makes it unsuitable in complex scenes, and the bottom-up human posture estimation method is prone to misconnection of joints, the present invention proposes a method and system for estimating human posture, which solves the above technical problems.

[0011] According to one aspect of the present invention, a method for estimating human body posture is proposed, comprising:

[0012] S1: Input the image into the encoder-decoder network with four encoding layers and four decoding layers for training, and output the semantic segmentation result;

[0013] S2: The pixel semantic probability map obtained in the first two coding layers is converted into an edge activation pixel map using an energy function. In response to the activation value of a pixel being greater than an activation value threshold, the pixel is an edge pixel.

[0014] S3: Based on the semantic labels in the semantic segmentation result, pixels belonging to the same instance are clustered to obtain an instance segmentation result, where the instance segmentation result includes a mask indicating the instance to which each pixel belongs; and

[0015] S4: Generate a human skeleton confidence map using a fully convolutional network, output the label of the skeleton component to which each pixel in each instance belongs, regress the joint point position through a fully connected network, connect the joint points in each instance to construct the human skeleton structure and obtain human posture information.

[0016] In some specific embodiments, the encoding layer includes a 3*3 convolutional network, and the encoder obtains features of different scales through a maximum pooling operation.

[0017] In some specific embodiments, the decoding layer includes a 2*2 convolutional network, the decoder adjusts the image size by upsampling, returns to the initial image size through four decoding layers, and a 1*1 convolution is added after the last decoding layer in the decoder to obtain the category of each pixel in the image.

[0018] In some specific embodiments, step S2 specifically includes: i =col i (|M*Soft max(p)-Soft max(p)|) converts the pixel semantic probability map obtained in the first two encoding layers into an edge activation pixel map {m1,m2,...,m i}(m i ∈R N ), where p represents the probability that the pixel value belongs to the i-th semantic category, Softmax is the cross entropy function, M represents the average filter, and col i represents the i-th column, m i Represents the pixel mask of the activation point of the i-th category.

[0019] In some specific embodiments, pixels close to the prediction boundary have larger activation values.

[0020] In some specific embodiments, step S3 specifically performs k-means clustering on pixels with a semantic label of person, and clusters pixels belonging to the same instance together to obtain an instance segmentation result.

[0021] In some specific embodiments, step S3 further includes using a weighted edge pixel loss function:

[0022] Optimize instance segmentation results, where P i represents the probability of edge pixels among pixels of the i-th semantic category.

[0023] In some specific embodiments, step S4 also includes optimizing the human skeleton confidence map using a conditional random field.

[0024] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, any of the above methods is implemented.

[0025] According to a third aspect of the present invention, a system for estimating human body posture is proposed, the system comprising:

[0026] Semantic segmentation unit: configured to input an image into a codec network including a four-layer encoding layer and a four-layer decoding layer structure for training, and output a semantic segmentation result;

[0027] An edge pixel acquisition unit: configured to convert the pixel semantic probability map obtained in the first two coding layers into an edge activation pixel map using an energy function, and in response to the activation value of the pixel being greater than the activation value threshold, the pixel is an edge pixel;

[0028] An instance segmentation unit: configured to aggregate pixels belonging to the same instance based on the semantic labels in the semantic segmentation result to obtain an instance segmentation result, wherein the instance segmentation result includes a mask indicating the instance to which each pixel belongs; and

[0029] Posture estimation unit: Use a fully convolutional network to generate a human skeleton confidence map, output the label of the skeleton component to which each pixel in each instance belongs, regress the joint point position through a fully connected network, connect the joint points in each instance to construct the human skeleton structure and obtain human posture information.

[0030] The method and system for estimating human posture of the present invention perform instance segmentation on a two-dimensional image, segment different people from the image, and mark different people with different labels, thereby essentially preventing the phenomenon of cross-pose estimation between people, and the result of instance segmentation is closer to the target of human posture estimation. Then, the component segmentation of human joints is performed, and the component segmentation can achieve the segmentation of various skeletal components of the human body. As an auxiliary optimization network for human joint point detection, it can prevent joint point prediction errors to a large extent. Finally, the joint points are connected to construct the skeleton structure of the human body. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0032] Figure 1 is a flow chart of a method for estimating human body posture according to an embodiment of the present application;

[0033] Figure 2 is a flow chart of a method for estimating human body posture according to a specific embodiment of the present application;

[0034] Figure 3 is a framework diagram of a system for estimating human posture according to an embodiment of the present application;

[0035] Figure 4 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0037] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] According to a method for estimating human body posture according to an embodiment of the present application, Figure 1 FIG. 1 shows a flow chart of a method for estimating human posture according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0039] S101: Input the image into a codec network including a four-layer coding layer and a four-layer decoding layer structure for training, and output a semantic segmentation result.

[0040] In a specific embodiment, the encoding layer includes a 3*3 convolutional network, and the encoder obtains features of different scales through a maximum pooling operation. The decoding layer includes a 2*2 convolutional network, and the decoder adjusts the image size through upsampling, returns to the initial image size after four layers of decoding layers, and a 1*1 convolution is added after the last decoding layer in the decoder to obtain the category of each pixel in the image.

[0041] S102: Using the energy function, the pixel semantic probability map obtained in the first two coding layers is converted into an edge activation pixel map. In response to the activation value of a pixel being greater than the activation value threshold, the pixel is an edge pixel. i =col i (|M*Soft max(p)-Soft max(p)|Convert the pixel semantic probability map obtained in the first two encoding layers into an edge activation pixel map {m1,m2,...,m i}(m i ∈R N ), where p represents the probability that the pixel value belongs to the i-th semantic category, Softmax is the cross entropy function, M represents the average filter, and col i represents the i-th column, m i Represents the pixel mask of the activation point of the i-th category. Pixels close to the prediction boundary have larger activation values.

[0042] S103: Based on the semantic labels in the semantic segmentation results, pixels belonging to the same instance are clustered to obtain instance segmentation results. The instance segmentation results include a mask indicating the instance to which each pixel belongs. Specifically, k-means clustering is performed on pixels with the semantic label of person, and pixels belonging to the same instance are clustered together to obtain the instance segmentation result. The output of instance segmentation is a mask for each pixel, which indicates which instance the pixel belongs to (here only person instance segmentation is performed, so the instances are divided into person 1, person 2... other pixels are regarded as background pixels, and the binary mask is 0).

[0043] In a preferred embodiment, a weighted edge pixel loss function is used Optimize instance segmentation results, where P i The probability of edge pixels in pixels representing the i-th semantic category. The optimized instance segmentation results can facilitate more accurate posture analysis.

[0044] S104: Generate a human skeleton confidence map using a fully convolutional network, output the skeleton component label of each pixel in each instance, regress the joint point position through a fully connected network, connect the joint points in each instance to construct the human skeleton structure and obtain human posture information.

[0045] In a specific embodiment, a fully convolutional network is used to generate a human skeleton confidence map, and then the confidence map is optimized through a conditional random field, and finally the label of the skeleton component to which each pixel belongs is output. In each joint component, a key point is regressed through a fully connected network, and the key point is the position of the joint point. At the same time, each joint point also contains its skeleton component label, so that the joint points are connected in each instance according to predefined prior knowledge to construct the human skeleton structure, avoiding cross-connection of different human joint points.

[0046] The key points of the above-mentioned human posture estimation method of the present invention are to convert the semantic probability map into an edge activation pixel map through an energy function; optimize the instance segmentation result through a weighted edge pixel loss function; improve the cross-personal posture estimation phenomenon in the bottom-up method through instance segmentation; and use component segmentation as an auxiliary network to optimize human joint point detection. This method can effectively reduce the occurrence of misconnection of human joint points during the estimation process, and make human posture estimation more accurate. The present invention optimizes the effect of joint point position regression without changing the size of the original model by combining the top-down and bottom-up ideas.

[0047] Continue to refer Figure 2 , Figure 2 A flowchart for estimating human posture according to a specific embodiment of the present application is shown, as shown in FIG. Figure 2 As shown, including:

[0048] Step 201: Input the image into the encoder-decoder network for training. A four-layer encoding layer and a four-layer decoding layer structure are used to output the semantic segmentation result. The encoder adopts a four-layer encoding layer structure. Each encoding layer contains a 3x3 convolutional network, and through the maximum pooling operation, features of different scales are obtained. The decoder adopts a four-layer decoding layer structure. Each decoding layer contains a 2x2 convolutional network, and the image size is continuously adjusted by upsampling operations, and finally returned to the initial image size through four decoding layers. The last step of the last decoding layer adds a 1x1 convolution, and finally obtains the category of each pixel in the image (people are a category).

[0049] Step 202: The pixel semantic probability maps obtained in the first two coding layers are converted into edge activation pixel maps through the energy function. The images of the first two coding layers have high resolution, so the extracted features contain rich character edge pixel information. i =col i (|M*Soft max(p)-Soft max(p)|Convert the pixel semantic probability map obtained in the first two encoding layers into an edge activation pixel map {m1,m2,...,m i}(m i ∈R N). p represents the probability that the pixel value belongs to the i-th semantic category. Softmax is the cross entropy function. M represents the average filter. col i Represents the i-th column. m i Represents the pixel mask of the activation point of the i-th category. Points close to the prediction boundary will have larger activation values.

[0050] Step 203: Set an activation value threshold, and pixels exceeding the threshold are considered edge pixels.

[0051] Step 204: Perform k-means clustering on the pixels with the semantic label of person, and cluster the pixels belonging to the same instance together to obtain the instance segmentation result. Perform k-means clustering on the pixels with the semantic label of person, and cluster the pixels belonging to the same instance together to obtain the instance segmentation result. The output of instance segmentation is a mask for each pixel, which represents which instance the pixel belongs to (here only person instance segmentation is performed, so the instances are divided into person 1, person 2... other pixels are regarded as background pixels, and the binary mask is 0).

[0052] Step 205: Use a weighted edge pixel loss function to optimize the instance segmentation result. Since each instance has edge pixels, this patent uses a weighted edge pixel loss function Optimize instance segmentation results. i represents the probability of edge pixels among pixels of the i-th semantic category.

[0053] Step 206: Perform component segmentation on each instance (person). Perform component segmentation on each instance (person), and segment the various skeletal components of the human body. This step first uses a fully convolutional network to generate a human skeleton confidence map, then optimizes the confidence map through a conditional random field, and finally outputs the skeletal component label to which each pixel belongs.

[0054] Step 207: regress the joint points in different joint components. Regress the key points, i.e. the joint point positions, in each joint component through a fully connected network. At the same time, each joint point also contains its skeleton component label.

[0055] Step 208: Connect the joints in the same instance to construct the human skeleton. Since the joints contain their skeletal component labels, the joints are connected in each instance according to predefined prior knowledge (for example, the top of the head and the neck need to be connected, and the wrist and the elbow need to be connected) to construct the human skeleton structure, avoiding cross-connection of different human joints.

[0056] This method of estimating human posture combines the top-down and bottom-up ideas and innovates and promotes them. First, a top-down approach is used to perform instance segmentation on the two-dimensional image to segment different people from the image. Different people are marked with different labels, thereby essentially preventing the phenomenon of cross-pose estimation between people. This is very different from the currently commonly used target detection method. The result of instance segmentation is closer to the target of human posture estimation. Then the bottom-up idea is adopted, but the component segmentation of the human joint is first performed instead of human joint point detection. Component segmentation can segment the various skeletal parts of the human body. As an auxiliary optimization network for human joint point detection, it can prevent joint point prediction errors to a large extent. Finally, the joint points are connected to construct the human skeleton structure.

[0057] Continue to refer Figure 3 , Figure 3 The framework diagram of a human body posture estimation system according to an embodiment of the present application is shown. The system specifically includes a semantic segmentation unit 301 , an edge pixel acquisition unit 302 , an instance segmentation unit 303 and a posture estimation unit 304 .

[0058] In a specific embodiment, the semantic segmentation unit 301 is configured to input the image into a codec network including a four-layer coding layer and a four-layer decoding layer structure for training, and output a semantic segmentation result; the edge pixel acquisition unit 302 is configured to use an energy function to convert the pixel semantic probability map obtained in the first two coding layers into an edge activation pixel map, and in response to the activation value of the pixel being greater than the activation value threshold, the pixel is an edge pixel; the instance segmentation unit 303 is configured to cluster pixels belonging to the same instance based on the semantic label in the semantic segmentation result to obtain an instance segmentation result, and the instance segmentation result includes a mask representing the instance to which each pixel belongs; the posture estimation unit 304 is configured to generate a human skeleton confidence map using a fully convolutional network, output the label of the skeleton component to which each pixel in each instance belongs, regress the joint point position through a fully connected network, connect the joint points in each instance to construct the human skeleton structure to obtain human posture information.

[0059] Reference below Figure 4 , which shows a schematic diagram of the structure of a computer system 400 suitable for implementing an electronic device of an embodiment of the present application. Figure 4 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0060] like Figure 4As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the system 400 are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0061] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0062] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0063] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0064] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0065] The modules involved in the embodiments of the present application may be implemented by software or hardware.

[0066] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist alone and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device: inputs the image into a codec network including a four-layer coding layer and a four-layer decoding layer structure for training, and outputs a semantic segmentation result; uses an energy function to convert the pixel semantic probability map obtained in the first two coding layers into an edge activation pixel map, and in response to the activation value of the pixel being greater than the activation value threshold, the pixel is an edge pixel; based on the semantic label in the semantic segmentation result, the pixels belonging to the same instance are aggregated to obtain an instance segmentation result, and the instance segmentation result includes a mask representing the instance to which each pixel belongs; uses a fully convolutional network to generate a human skeleton confidence map, outputs the label of the skeleton component to which each pixel belongs in each instance, regresses the joint point position through a fully connected network, and connects the joint points in each instance to construct a human skeleton structure to obtain human posture information.

[0067] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for estimating human body posture, characterized in that: include: S1: Input the image into the encoder-decoder network with four encoding layers and four decoding layers for training, and output the semantic segmentation result; S2: using an energy function to convert the pixel semantic probability map obtained in the first two coding layers into an edge activation pixel map, in response to the activation value of a pixel being greater than an activation value threshold, the pixel is an edge pixel; S3: Based on the semantic label in the semantic segmentation result, pixels belonging to the same instance are clustered to obtain an instance segmentation result, wherein the instance segmentation result includes a mask indicating the instance to which each pixel belongs. Specifically, k-means clustering is performed on pixels whose semantic label is person, and pixels belonging to the same instance are clustered together to obtain an instance segmentation result; as well as S4: Generate a human skeleton confidence map using a fully convolutional network, output the label of the skeleton component to which each pixel in each instance belongs, regress the joint point position through a fully connected network, connect the joint points in each instance to construct the human skeleton structure and obtain human posture information.

2. The method for estimating human body posture according to claim 1, characterized in that: The encoding layer includes a 3*3 convolutional network, and the encoder obtains features of different scales through maximum pooling operation.

3. The method for estimating human body posture according to claim 2, characterized in that: The decoding layer includes a 2*2 convolutional network. The decoder adjusts the image size by upsampling, and returns to the initial image size after four layers of decoding layers. In addition, a 1*1 convolution is added after the last decoding layer in the decoder to obtain the category of each pixel in the image.

4. The method for estimating human body posture according to claim 1, characterized in that: The step S2 specifically includes: i =col i (|M*Softmax(p)-Softmax(p)|) converts the pixel semantic probability map obtained in the first two encoding layers into an edge activation pixel map {m1,m2,...,m i }(m i ∈R N ), where p represents the probability that the pixel value belongs to the i-th semantic category, Softmax is the cross entropy function, M represents the average filter, and col i represents the i-th column, m i Represents the pixel mask of the activation point of the i-th category.

5. The method for estimating human body posture according to claim 4, characterized in that: Pixels close to the prediction boundary have larger activation values.

6. The method for estimating human posture according to claim 1, characterized in that: The step S3 further includes adopting a weighted edge pixel loss function: Optimize instance segmentation results, where P i represents the probability of edge pixels in pixels of the i-th semantic category, m i represents the pixel mask of the activation point of the i-th category, represents the predicted pixel mask of the i-th semantic category.

7. The method for estimating human body posture according to claim 1, characterized in that: The step S4 also includes optimizing the human skeleton confidence map using a conditional random field.

8. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 7 is implemented.

9. A system for estimating human body posture, characterized in that: The system comprises: Semantic segmentation unit: configured to input an image into a codec network including a four-layer encoding layer and a four-layer decoding layer structure for training, and output a semantic segmentation result; An edge pixel acquisition unit: configured to convert the pixel semantic probability maps obtained in the first two coding layers into edge activation pixel maps using an energy function, and in response to an activation value of a pixel being greater than an activation value threshold, the pixel is an edge pixel; An instance segmentation unit: configured to cluster pixels belonging to the same instance based on the semantic label in the semantic segmentation result to obtain an instance segmentation result, wherein the instance segmentation result includes a mask indicating the instance to which each pixel belongs, specifically, performing k-means clustering on pixels whose semantic label is person, and clustering pixels belonging to the same instance together to obtain an instance segmentation result; and Posture estimation unit: It is configured to generate a human skeleton confidence map using a fully convolutional network, output the label of the skeleton component to which each pixel in each instance belongs, regress the joint point position through a fully connected network, connect the joint points in each instance to construct the human skeleton structure and obtain human posture information.

Images