A human pose estimation method and related apparatus

Abstract

The application provides a human posture estimation method and related device, the human posture estimation method comprises the following steps: obtaining a background region image set based on a first original image with a continuous preset frame number; performing background modeling by using the background region image set to obtain a complete background image; performing difference operation on a second original image with a frame number greater than the preset frame number and the complete background image to obtain a foreground target image corresponding to the second original image; and performing feature extraction and prediction on the foreground target image to obtain a human posture estimation result corresponding to the second original image. Through the implementation of the application, background modeling is performed on multiple original images to obtain a relatively complete background image, then background removal is performed on the to-be-detected image based on the complete background image, and posture estimation is performed on the foreground target, so that most of the false detections caused by background interference can be excluded, and the accuracy of the human skeleton point detection result is effectively improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a human pose estimation method and related apparatus. Background Technology

[0002] In recent years, with the continuous advancement of deep learning technology, human pose estimation (also known as human skeleton point detection) has made significant progress. Human skeleton points include joints, facial features, etc., and these key points can be used to describe human skeletal information. Currently, human pose estimation technology is being applied more and more widely in real-world scenarios, such as entertainment and fitness, rehabilitation training, and action recognition.

[0003] However, in real-world applications, the images to be detected may contain complex backgrounds, which can easily lead to false detections of human skeleton points during human pose estimation due to background interference. To overcome this problem, related technologies typically use the background region of the previous frame as a reference to obtain the foreground target of the current frame, and then perform human skeleton point detection on the obtained foreground target. However, in motion scenarios, the background region of the previous frame and the actual background region of the current frame only have a certain similarity. In complex motion scenarios, this cannot provide an accurate reference for human skeleton point detection in the current frame, and there is still a high possibility of false detection in practical applications, resulting in relatively low overall accuracy of human skeleton point detection results. Summary of the Invention

[0004] This application provides a human pose estimation method and related apparatus, which can at least solve the problems of high false detection probability and relatively low overall accuracy of human pose estimation schemes provided in related technologies.

[0005] The first aspect of this application provides a human pose estimation method, comprising: acquiring a background region image set based on a first original image with a consecutive preset number of frames; wherein the background region image set includes multiple background region images corresponding to multiple frames of the first original image; performing background modeling using the background region image set to obtain a complete background image; performing a difference operation on a second original image and the complete background image to obtain a foreground target image corresponding to the second original image; wherein the frame number of the second original image is greater than the preset number of frames; and performing feature extraction and prediction on the foreground target image to obtain a human pose estimation result corresponding to the second original image.

[0006] A second aspect of this application provides a human pose estimation device, comprising: an acquisition module, configured to acquire a background region image set based on a first original image with a consecutive preset number of frames; wherein the background region image set includes multiple background region images corresponding to multiple frames of the first original image; a modeling module, configured to perform background modeling using the background region image set to obtain a complete background image; a calculation module, configured to perform a difference operation on a second original image and the complete background image to obtain a foreground target image corresponding to the second original image; wherein the frame number of the second original image is greater than the preset number of frames; and an estimation module, configured to perform feature extraction and prediction on the foreground target image to obtain a human pose estimation result corresponding to the second original image.

[0007] A third aspect of this application provides an electronic device, including: an image acquisition device and a processor, wherein: the image acquisition device is used to acquire an original image and transmit it to the processor; the processor is used to process the original image using the human pose estimation method provided in the first aspect of this application to obtain a human pose estimation result.

[0008] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps in the human pose estimation method provided in the first aspect of this application.

[0009] As can be seen from the above, according to the human pose estimation method and related apparatus provided in this application, a background region image set is obtained based on a first original image with a preset number of consecutive frames; background modeling is performed using the background region image set to obtain a complete background image; a difference operation is performed on a second original image with a frame number greater than a preset number of frames and the complete background image to obtain a foreground target image corresponding to the second original image; feature extraction and prediction are performed on the foreground target image to obtain the human pose estimation result corresponding to the second original image. Through the implementation of this application, background modeling of multiple original images can obtain a relatively complete background image. Then, based on the complete background image, background removal processing is performed on the image to be detected, and then pose estimation is performed on the foreground target. This can eliminate most false detections caused by background interference and effectively improve the accuracy of human skeleton point detection results. Attached Figure Description

[0010] Figure 1 A schematic diagram illustrating an application scenario provided in one embodiment of this application;

[0011] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0012] Figure 3A schematic diagram of the basic flowchart of a human pose estimation method provided in an embodiment of this application;

[0013] Figure 4 A schematic diagram of the detection process of a human pose estimation model provided in an embodiment of this application;

[0014] Figure 5 This is a schematic diagram of the structure of a feature extraction network provided in an embodiment of this application;

[0015] Figure 6 This is a schematic diagram of the structure of a pose prediction network provided in an embodiment of this application;

[0016] Figure 7 A detailed flowchart illustrating a human pose estimation method provided in an embodiment of this application;

[0017] Figure 8 A schematic diagram of the program modules of a human posture estimation device provided in an embodiment of this application. Detailed Implementation

[0018] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the description of the embodiments of this application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0021] In the embodiments of this application, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0022] The following will describe in detail, with reference to the accompanying drawings, a human posture estimation method and related apparatus according to an embodiment of this application.

[0023] To improve the accuracy of human skeleton point detection results, one embodiment of this application provides a human pose estimation method, applicable to, for example... Figure 1 The scenario shown can include an image acquisition device 10 and an electronic device 20. In one implementation, the image acquisition device 10 can be a camera, and its sensor type can be determined according to the actual application scenario. In a typical implementation, the camera can be any one or more combinations of a color camera, a depth camera, a grayscale camera, etc. The electronic device 20 is a variety of terminal devices with data processing functions, including but not limited to televisions, smartphones, tablets, laptops, and desktop computers.

[0024] exist Figure 1 In the application scenario shown, multiple frames of images can be continuously acquired by the image acquisition device 10, and then the continuously acquired images can be sent to the electronic device 20. The electronic device 20 acquires a background region image set from the first N frames of received images, i.e., the first original image; then, it uses the background region image set to perform background modeling to obtain a complete background image; next, when a second original image with a frame number greater than N is received, it performs a difference operation with the complete background image to obtain a foreground target image corresponding to the second original image; finally, it performs feature extraction and prediction on the foreground target image to obtain a human pose estimation result corresponding to the second original image.

[0025] like Figure 2The diagram shows a schematic of an electronic device according to an embodiment of this application. The electronic device mainly includes a memory 201 and a processor 202. The number of processors 202 can be one or more. The memory 201 stores a computer program 203 that can run on the processor 202. The memory 201 and the processor 202 are communicatively connected. When the processor 202 executes the computer program 203, it implements the following human pose estimation method: Based on a first original image with a preset number of consecutive frames, a background region image set is obtained; wherein the background region image set includes multiple background region images corresponding to multiple frames of the first original image; background modeling is performed using the background region image set to obtain a complete background image; a difference operation is performed on the second original image and the complete background image to obtain a foreground target image corresponding to the second original image; wherein the frame number of the second original image is greater than a preset number of frames; feature extraction and prediction are performed on the foreground target image to obtain a human pose estimation result corresponding to the second original image.

[0026] In one embodiment, the processor 202 may be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), neural network chips, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0027] In one embodiment, the memory 201 can be an internal storage unit, such as a hard disk or RAM; the memory can also be an external storage device, such as an external hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Furthermore, the memory can include both internal storage units and external storage devices, and can also be used to temporarily store data that has been output or will be output. It should be noted that when the processor is a neural network chip, the electronic device may not include memory; whether the electronic device needs to use memory to store the corresponding computer program depends on the type of processor.

[0028] In one embodiment, the electronic device further includes an image acquisition device for acquiring raw images and transmitting them to the processor 202, so that the processor 202 can process the images according to the human pose estimation method provided in this application. The image acquisition device may be integrated into the electronic device or connected to the electronic device via wired or wireless communication; the image acquisition device may include any one or more combinations of color cameras, depth cameras, grayscale cameras, etc., and the depth camera may be a structured light camera, time-of-flight camera, binocular camera, or multi-view camera, etc., without limitation herein.

[0029] One embodiment of this application also provides a computer-readable storage medium, which may be disposed in the aforementioned electronic device. The computer-readable storage medium may be as described above. Figure 2 The memory in the illustrated embodiment.

[0030] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the aforementioned human pose estimation method. Furthermore, the computer-readable storage medium can also be a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk, or any other medium capable of storing program code.

[0031] like Figure 3 This is a basic flowchart of the human pose estimation method provided in this embodiment. The human pose estimation method can be derived from... Figure 1 or Figure 2 The electronic device in the process executes the following steps:

[0032] Step 301: Obtain a set of background region images based on the first original image with a consecutive preset number of frames.

[0033] Specifically, in practical applications, the camera continuously captures images (e.g., RGB images) of the actual scene. When the electronic device receives the first N frames of images captured by the camera, it can obtain the corresponding background region images. The background region images are the images obtained after removing the foreground objects from the original images. The N background region images form a background region image set.

[0034] In one optional embodiment of this example, the step of obtaining a background region image set based on a first original image with a consecutive preset number of frames includes: normalizing the first original image with a consecutive preset number of frames; and obtaining a background region image set based on the normalized first original image.

[0035] Specifically, in this embodiment, the original image can first be normalized, that is, the original image to be processed can be converted into a unique standard form. This standard form has invariant properties to affine transformations such as translation, rotation, and scaling, so that the data of different images are all within the same range. Then, the corresponding background area images are obtained separately to facilitate subsequent processing using the background area images.

[0036] Step 302: Use the background region image set to perform background modeling to obtain a complete background image.

[0037] Specifically, in practical applications, since the background area output from the previous frame is basically similar to the background area of ​​the current frame, the foreground target can be obtained using the background area of ​​the previous frame when a complete background image cannot be obtained. However, the background area output from the previous frame is usually only a rough target area relative to the current frame. Especially in complex scenes or complex actions, false detections or missed detections may occur. If the foreground target obtained by directly using the difference operation between the background area and the image of the current frame for subsequent detection is performed, the resulting skeleton points will also produce false detections or missed detections, thus reducing the accuracy of the skeleton points or increasing their false detection rate. Based on this, in this embodiment, background modeling is performed based on multiple frames of original images to obtain a complete background, thereby obtaining a more accurate foreground target.

[0038] In one optional embodiment of this example, the step of using a background region image set to perform background modeling to obtain a complete background image includes: performing background modeling on the background region image set and the corresponding first original image based on a preset background modeling algorithm to obtain a complete background image.

[0039] The background modeling algorithm in this embodiment includes any one of the following: Gaussian mixture background modeling method and frame difference method.

[0040] Step 303: Perform a difference operation on the second original image and the complete background image to obtain the foreground target image corresponding to the second original image.

[0041] Specifically, in this embodiment, the frame number of the second original image is greater than the preset frame number. That is, for images continuously input by the camera, when the original image after the Nth frame is received, the complete background image obtained from the previous N original images can be used to perform a difference operation on the current frame original image to obtain the foreground target image of the current frame original image.

[0042] Step 304: Extract and predict features from the foreground target image to obtain the human pose estimation result corresponding to the second original image.

[0043] Specifically, in this embodiment, the foreground target image of the original image of the current frame can be input into the human pose estimation model for feature extraction and prediction processing to obtain the human skeleton point inference result.

[0044] In one optional embodiment of this example, the steps of extracting and predicting features from the foreground target image to obtain a human pose estimation result corresponding to the second original image include: inputting the foreground target image into the feature extraction network of the trained human pose estimation model and outputting a target feature image; inputting the target feature image into the pose prediction network of the human pose estimation model and outputting a human pose estimation result corresponding to the second original image.

[0045] like Figure 4 The diagram shows a detection process of a human pose estimation model provided in this embodiment. The network represented by Backbone in the diagram represents the aforementioned feature extraction network, and the network composed of Stage 1 to Stage k represents the aforementioned pose prediction network.

[0046] like Figure 5 The diagram shown is a schematic of a feature extraction network provided in this embodiment. In an optional implementation of this embodiment, the feature extraction network includes a max pooling module (i.e., ... Figure 5 Maxpool), multiple cascaded convolutional modules (i.e., maxpool), and multiple convolutional modules. Figure 5 The middle convolution module and the average pooling module (i.e., conv) Figure 5 (Average pooling). Accordingly, the steps described above for inputting the foreground target image into the feature extraction network of the trained human pose estimation model and outputting the target feature image include: inputting the foreground target image into the feature extraction network of the trained human pose estimation model, performing max pooling on the foreground target image using a max pooling module to obtain a first feature image; fusing the inputs and outputs of each convolutional module and using them as the input of the next convolutional module, performing convolution on the first feature image using multiple cascaded convolutional modules to obtain a second feature image; and performing average pooling on the second feature image using an average pooling module to obtain the target feature image.

[0047] Specifically, in this embodiment, the output of the max pooling module is connected to the input of the first convolutional module, and the output of the last convolutional module is connected to the input of the average pooling module. This embodiment has eight convolutional modules, each comprising two convolutional layers. The scale of the convolutional layers in all convolutional modules can be 3x3. The first and second convolutional modules have 64 channels, the third and fourth convolutional modules have 128 channels, the fifth and sixth convolutional modules have 256 channels, and the seventh and eighth convolutional modules have 512 channels. Furthermore, the input of the max pooling module in this embodiment can also be connected to the output of a separate convolutional layer. That is, the feature map after the input image has undergone convolution processing by this separate convolutional layer is input to the max pooling module. This separate convolutional layer has a scale of 7x7 and 64 channels. It should also be noted that the output of the average pooling module is connected to a fully connected layer (FC). The feature map after average pooling is processed by the fully connected layer to obtain the final target feature image.

[0048] like Figure 6 The diagram shown is a structural schematic of a pose prediction network provided in this embodiment. In an optional implementation of this embodiment, the pose prediction network includes multiple cascaded stage modules (such as...). Figure 6 The model is structured as stage1, stage2, etc., where each stage module includes a keypoint confidence prediction network and a keypoint affinity vector field prediction network. Correspondingly, the step of inputting the target feature image into the pose prediction network of the human pose estimation model and outputting a human pose estimation result corresponding to the second original image includes: inputting the target feature image into the pose prediction network of the human pose estimation model; sequentially using the keypoint confidence prediction network of each stage module to obtain a keypoint confidence image, and using the keypoint affinity vector field prediction network to obtain a keypoint affinity vector field; fusing the keypoint confidence image and the keypoint affinity vector field of the last stage module to obtain the output as the human pose estimation result corresponding to the second original image; wherein, in two adjacent stage modules, the output of the previous stage module is the input of the next stage module.

[0049] Specifically, in this embodiment, after extracting features through the backbone network, the extracted features are input into the first stage module of a series of sequential stage modules. All stage modules have the same structure and function. In this embodiment, each stage module includes two branches, both composed of five-layer convolutional networks. One branch generates a part confidence map (PCM), i.e., the confidence map of each skeleton point. The other branch generates a part affinity field (PAF), a vector field composed of 2D vectors of each limb of the body part. Each limb consists of two skeleton points, preserving the positional and directional information between limbs. It should be understood that this embodiment sequentially performs predictions through multiple stage modules, continuously deepening the prediction depth. Subsequent stage modules can utilize the prediction results of earlier stage modules to further optimize their own predictions. For keypoints in areas with complex visual features, progressively optimizing predictions through multiple stage modules can achieve both accuracy and comprehensiveness in the prediction results.

[0050] In one optional embodiment of this example, the step of obtaining a background region image set based on a first original image with a consecutive preset number of frames includes: inputting the first original image with a consecutive preset number of frames into the feature extraction network and the pose prediction network in the human pose estimation model for feature extraction and prediction, and outputting feature images and human pose estimation results corresponding to the first original image; inputting the feature images of each first original image into the background analysis network in the human pose estimation model and fusing the features obtained from each stage module in the pose prediction network to obtain each foreground region; using each foreground region and each first original image to obtain background region images of the first N original images; and summarizing all background region images to obtain a background region image set.

[0051] For details, please refer to the following document again. Figure 4The human pose estimation model in this embodiment also includes a background parsing network. In this embodiment, for the first N frames of original images, the human pose estimation is also implemented using the feature extraction network and pose prediction network of the aforementioned human pose estimation model. The difference is that, since the first N frames of original images do not have complete background images, the first N frames of original images are input into the background parsing network of the human pose estimation model to obtain foreground targets, and the complete background image is obtained by comparing the foreground region with the first N frames of original images. It should be noted that this embodiment performs multi-task detection on the first N frames of original images. In addition to performing the human pose estimation task, the background analysis task is also performed simultaneously. That is, the feature images obtained from the first N frames of original images through the feature extraction network are input into the background parsing network. In the background parsing network, the features obtained from each stage module of the pose prediction network are fused to obtain the foreground region, and the foreground region is used to perform a difference operation with the first N frames of original images to obtain the corresponding background analysis result of the first N frames of original images, that is, the background region image. The N frames of background region images constitute the background region image set. In addition, by fusing the features obtained from each stage module of the pose prediction network into the background analysis network, it can not only assist the pose prediction network in making pose predictions and ensure the detection accuracy of human pose in the first N frames, but also obtain a more accurate foreground region to obtain a better background region.

[0052] It should be noted that in this embodiment, background region images can also be obtained through the background analysis network for images after N frames, so as to continuously iterate and update the background region image set. Correspondingly, the complete background image can be further updated through the background modeling algorithm. Thus, even if multiple people enter the scene, the complete human body of multiple people can be obtained, thereby obtaining the skeleton of multiple people.

[0053] Figure 7 The method described in this application is a refined human pose estimation method provided in an embodiment of the present application. This human pose estimation method includes:

[0054] Step 701: Input the first N frames of original images into the human pose estimation model, and use the feature extraction network and pose prediction network in the human pose estimation model to perform feature extraction and prediction, and output the feature images and human pose estimation results corresponding to the first N frames of original images.

[0055] Step 702: Input the feature images corresponding to the first N original images into the background analysis network in the human pose estimation model, and fuse the features obtained from each module of the pose prediction network to obtain each foreground region. Use the first N original images and their corresponding foreground regions to output N background region images to form a background region image set.

[0056] Specifically, this embodiment performs multi-task detection on the first N frames of original images captured by the camera. Besides performing human pose estimation, it also performs background analysis simultaneously. That is, the features obtained from the feature extraction network and pose prediction network of the first N frames are input into the background analysis network to obtain the foreground region. The foreground region is then compared with the original image to obtain the corresponding background analysis result, i.e., the background region image. It should be noted that in this application, the human pose estimation task and the background analysis task share a feature extraction network, which allows the background analysis network to constrain the pose prediction network, thereby improving the accuracy of human pose estimation.

[0057] Step 703: Based on the preset background modeling algorithm, perform background modeling on the background region image set and the corresponding first N frames of original images to obtain a complete background image.

[0058] Specifically, the background modeling algorithm in this embodiment includes any one of the following: Gaussian mixture background modeling method and frame difference method.

[0059] Step 704: Perform a difference operation between the original image with a frame number after N frames and the complete background image to obtain the corresponding foreground target image.

[0060] Specifically, in this embodiment, background modeling is performed based on multiple frames of original images to obtain a complete background. Based on this, background removal processing is performed on the original images to obtain a more accurate foreground target.

[0061] Step 705: Input the foreground target image into the human pose estimation model, use the feature extraction network and pose prediction network to perform feature extraction and prediction, and output the human pose estimation result corresponding to the original image after N frames.

[0062] Specifically, in this embodiment, for the original image after N frames, its foreground target image is input into the human pose estimation model for feature extraction and prediction processing. By directly estimating the human pose in the foreground region, the accuracy of the human skeleton point inference results can be improved, and false detections caused by background interference can be effectively eliminated.

[0063] It should be understood that the sequence number of each step in this embodiment does not imply the order in which the steps are executed. The execution order of each step should be determined by its function and internal logic, and should not constitute a unique limitation on the implementation process of this application embodiment.

[0064] Based on the technical solutions of the above embodiments of this application, background modeling of multiple frames of original images can obtain a relatively complete background image. Then, based on the complete background image, background removal processing is performed on the image to be detected, and pose estimation is performed on the foreground target. This can eliminate most false detections caused by background interference and effectively improve the accuracy of human skeleton point detection results.

[0065] Figure 8 A human pose estimation device is provided as an embodiment of this application. This human pose estimation device can be used to implement the human pose estimation method in the foregoing embodiments. The human pose estimation device mainly includes:

[0066] The acquisition module 801 is used to acquire a background region image set based on a first original image with a consecutive preset number of frames; wherein, the background region image set includes multiple background region images corresponding to multiple frames of the first original image;

[0067] Modeling module 802 is used to perform background modeling using a set of background region images to obtain a complete background image;

[0068] The calculation module 803 is used to perform a difference operation on the second original image and the complete background image to obtain a foreground target image corresponding to the second original image; wherein, the frame number of the second original image is greater than a preset number of frames.

[0069] The estimation module 804 is used to extract and predict features from the foreground target image to obtain the human pose estimation result corresponding to the second original image.

[0070] In some implementations of this embodiment, the estimation module is specifically used to: input the foreground target image into the feature extraction network in the trained human pose estimation model, and output the target feature image; input the target feature image into the pose prediction network in the human pose estimation model, and output the human pose estimation result corresponding to the second original image.

[0071] In some embodiments of this example, the feature extraction network includes a max pooling module, multiple cascaded convolutional modules, and an average pooling module; the pose prediction network includes multiple cascaded stage modules, and the stage modules include a keypoint confidence prediction network and a keypoint affinity vector field prediction network.

[0072] Accordingly, the estimation module is specifically used for: inputting the foreground target image into the feature extraction network of the trained human pose estimation model, performing max pooling on the foreground target image using the max pooling module to obtain the first feature image; fusing the inputs and outputs of each convolution module as the input of the next convolution module, performing convolution on the first feature image using multiple cascaded convolution modules to obtain the second feature image; performing average pooling on the second feature image using the average pooling module to obtain the target feature image; inputting the target feature image into the pose prediction network of the human pose estimation model, sequentially using the keypoint confidence prediction network of each stage module to obtain the keypoint confidence image, and using the keypoint affinity vector field prediction network to obtain the keypoint affinity vector field; and fusing the keypoint confidence image and the keypoint affinity vector field of the last stage module to obtain the output as the human pose estimation result corresponding to the second original image; wherein, in two adjacent stage modules, the output of the previous stage module is the input of the next stage module.

[0073] In some embodiments of this example, the acquisition module is specifically used to: normalize a first original image for a consecutive preset number of frames; and acquire a background region image set based on the normalized first original image.

[0074] In other embodiments of this example, the acquisition module is specifically used to: input a first original image of a consecutive preset number of frames into the feature extraction network and the pose prediction network in the human pose estimation model for feature extraction and prediction, and output feature images corresponding to the first original images and human pose estimation results; input the feature images of each first original image into the background analysis network in the human pose estimation model and fuse the features obtained by each stage module in the pose prediction network to obtain each foreground region; use each foreground region and each first original image to obtain background region images of the first N frames; and summarize all background region images to obtain a background region image set.

[0075] In some embodiments of this example, the modeling module is specifically used to: perform background modeling on the background region image set and the corresponding first original image based on a preset background modeling algorithm to obtain a complete background image; wherein, the background modeling algorithm includes any one of the following: Gaussian mixture background modeling method, frame difference method.

[0076] It should be noted that the human pose estimation methods in the foregoing embodiments can all be implemented based on the human pose estimation device provided in this embodiment. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the human pose estimation device described in this embodiment can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0077] According to the human pose estimation device provided in this embodiment, a background region image set is obtained based on a first original image with a preset number of consecutive frames; background modeling is performed using the background region image set to obtain a complete background image; a difference operation is performed on a second original image with a frame number greater than a preset number of frames and the complete background image to obtain a foreground target image corresponding to the second original image; feature extraction and prediction are performed on the foreground target image to obtain the human pose estimation result corresponding to the second original image. Through the implementation of this application's solution, background modeling of multiple original images can obtain a relatively complete background image. Then, based on the complete background image, background removal processing is performed on the image to be detected, and then pose estimation is performed on the foreground target. This can eliminate most false detections caused by background interference, effectively improving the accuracy of human skeleton point detection results.

[0078] It should be noted that the apparatuses and methods disclosed in the several embodiments provided in this application can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0079] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0080] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0081] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0082] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0083] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0084] The above is a description of the human posture estimation method and related apparatus provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for estimating human pose, characterized in that, include: A background region image set is obtained based on a first original image with a consecutive preset number of frames; wherein, the background region image set includes multiple background region images corresponding to multiple frames of the first original image; Background modeling is performed using the aforementioned background region image set to obtain a complete background image; A difference operation is performed on the second original image and the complete background image to obtain a foreground target image corresponding to the second original image; wherein the frame number of the second original image is greater than the preset frame number; Feature extraction and prediction are performed on the foreground target image to obtain the human pose estimation result corresponding to the second original image; The step of obtaining a background region image set based on a first original image with a consecutive preset number of frames includes: The first original image with a preset number of consecutive frames is input into the feature extraction network and the pose prediction network in the human pose estimation model to extract and predict features, and outputs the feature image and human pose estimation result corresponding to the first original image. The feature images of each of the first original images are input into the background analysis network in the human pose estimation model and the features obtained from the pose prediction network are fused to obtain each foreground region. The background region images of the first N frames of original images are obtained by using each foreground region and each of the first original images. All the background region images are summarized to obtain a background region image set.

2. The human posture estimation method according to claim 1, characterized in that, The step of extracting and predicting features from the foreground target image to obtain a human pose estimation result corresponding to the second original image includes: The foreground target image is input into the feature extraction network of the trained human pose estimation model, and the target feature image is output. The target feature image is input into the pose prediction network in the human pose estimation model, and the human pose estimation result corresponding to the second original image is output.

3. The human posture estimation method according to claim 2, characterized in that, The feature extraction network includes a max pooling module, multiple cascaded convolutional modules, and an average pooling module. The step of inputting the foreground target image into the feature extraction network of the trained human pose estimation model and outputting the target feature image includes: The foreground target image is input into the feature extraction network of the trained human pose estimation model, and the foreground target image is subjected to max pooling processing by the max pooling module to obtain the first feature image. The inputs and outputs of each convolutional module are fused to form the input of the next convolutional module. The first feature image is then convolved using multiple cascaded convolutional modules to obtain the second feature image. The second feature image is processed by average pooling using the average pooling module to obtain the target feature image.

4. The human posture estimation method according to claim 2, characterized in that, The attitude prediction network includes multiple cascaded stage modules, each stage module including a keypoint confidence prediction network and a keypoint affinity vector field prediction network. The step of inputting the target feature image into the pose prediction network of the human pose estimation model and outputting the human pose estimation result corresponding to the second original image includes: The target feature image is input into the pose prediction network in the human pose estimation model. The key point confidence prediction network of each stage module is used to obtain the key point confidence image, and the key point affinity vector field is used to obtain the key point affinity vector field. The output obtained by fusing the keypoint confidence image and the keypoint affinity vector field of the last stage module is used as the human pose estimation result corresponding to the second original image; wherein, in two adjacent stage modules, the output of the previous stage module is the input of the next stage module.

5. The human posture estimation method according to claim 1, characterized in that, The step of obtaining a background region image set based on a first original image with a consecutive preset number of frames includes: Normalize the first original image for a consecutive preset number of frames; A background region image set is obtained based on the first original image after normalization processing.

6. The human pose estimation method according to any one of claims 1 to 5, characterized in that, The step of performing background modeling using the background region image set to obtain a complete background image includes: Based on a preset background modeling algorithm, background modeling is performed on the background region image set and the corresponding first original image to obtain a complete background image; wherein, the background modeling algorithm includes any one of the following: Gaussian mixture background modeling method and frame difference method.

7. A human posture estimation device, characterized in that, include: The acquisition module is used to input a first original image of a consecutive preset number of frames into the feature extraction network and the pose prediction network in the human pose estimation model for feature extraction and prediction, and output a feature image and human pose estimation result corresponding to the first original image; input the feature images of each first original image into the background analysis network in the human pose estimation model and fuse the features obtained from the pose prediction network to obtain each foreground region; use each foreground region and each first original image to obtain the background region images of the first N frames of original images; summarize all the background region images to obtain a background region image set; wherein, the background region image set includes multiple background region images corresponding to multiple frames of the first original image; The modeling module is used to perform background modeling using the background region image set to obtain a complete background image; The calculation module is used to perform a difference operation on the second original image and the complete background image to obtain a foreground target image corresponding to the second original image; wherein the frame number of the second original image is greater than the preset frame number; The estimation module is used to extract and predict features from the foreground target image to obtain a human pose estimation result corresponding to the second original image.

8. An electronic device, characterized in that, Includes an image acquisition device and a processor, wherein: The image acquisition device is used to acquire raw images and transmit them to the processor; The processor is configured to process the original image using the human pose estimation method according to any one of claims 1 to 6 to obtain human pose estimation results.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the human pose estimation method according to any one of claims 1 to 6.

Images