Human body posture estimation method, device, terminal and storage medium

By combining RGBD image data and wireless communication signal data for human posture estimation, the problem of inaccurate posture estimation in the case of occlusion is solved, and higher estimation accuracy is achieved.

CN115082962BActive Publication Date: 2025-08-12SHENZHEN ORBBEC CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210742382.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-28
Publication Date
2025-08-12
Estimated Expiration
2042-06-28

AI Technical Summary

Technical Problem

In the prior art, the target has the problem of low accuracy in estimation of human posture during occlusion.

Method used

RGBD image data and wireless communication signal data are used to estimate the first and second postures of the target user, respectively, and fuse them to improve the estimation accuracy.

Benefits of technology

By fusing the recognition results of RGBD image data and wireless communication signal data, the accuracy of human posture estimation in the case of occlusion is effectively improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115082962B_ABST
    Figure CN115082962B_ABST
Patent Text Reader

Abstract

The present invention discloses a human pose estimation method, device, terminal, and storage medium. The method comprises: obtaining RGBD image data and wireless communication signal data of the same target scene; wherein the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data; estimating a first pose and a second pose of a target user in the target scene based on the RGBD image data and the wireless communication signal data; and fusing the first pose and the second pose to obtain a target human pose of the target user. The present invention can improve the accuracy of human pose estimation in the presence of occlusion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of human posture estimation, and in particular to a human posture estimation method, device, terminal and storage medium. Background Art

[0002] Human pose estimation has a wide range of applications, including human-computer interaction and motion analysis. Traditional human pose estimation generally uses a camera to acquire images and identify key points of the human body, such as joints and facial features, from the images to describe the human pose. However, camera-based methods cannot accurately identify the target when it is obscured.

[0003] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0004] In view of the above-mentioned defects of the prior art, the present invention provides a human body posture estimation method, device, terminal and storage medium, aiming to solve the problem of low accuracy of human body posture estimation when the target is occluded in the prior art.

[0005] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0006] In a first aspect of the present invention, a method for estimating a human body posture is provided. The method comprises: acquiring RGBD image data and wireless communication signal data of the same target scene; wherein the wireless communication signal data comprises Bluetooth signal data and / or wireless network channel state information data; estimating a first posture and a second posture of a target user in the target scene based on the RGBD image data and the wireless communication signal data; and fusing the first posture and the second posture to obtain a target human body posture of the target user.

[0007] In a second aspect of the present invention, a human body posture estimation device is provided, comprising: a data acquisition module for acquiring RGBD image data and wireless communication signal data of the same target scene, wherein the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data; a posture acquisition module for estimating a first posture and a second posture of a target user in the target scene based on the RGBD image data and the wireless communication signal data, respectively; and a fusion module for fusing the first posture and the second posture to obtain a target human body posture of the target user.

[0008] The third aspect of the present invention provides a terminal, including a processor and a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium being suitable for storing multiple instructions, and the processor being suitable for calling the instructions in the computer-readable storage medium to execute the steps of implementing the above-mentioned human body posture estimation method.

[0009] A fourth aspect of the present invention provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the above-mentioned human body posture estimation method.

[0010] Compared with the existing technology, the present invention not only performs posture recognition based on image data, but also uses wireless communication signal data for posture recognition as a supplement to the image posture recognition results. The final human body posture is obtained by fusing the recognition results of the two. Based on the fact that occlusion between targets will affect the changes in wireless communication signal data, the problem of target occlusion that cannot be solved by image data is solved, and the accuracy of human body posture estimation is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flowchart of an embodiment of a method for estimating a human body posture provided by the present invention;

[0012] Figure 2 A schematic diagram of a human body posture in an embodiment of the human body posture estimation method provided by the present invention;

[0013] Figure 3 A schematic diagram of an application scenario of an embodiment of the human body posture estimation method provided by the present invention;

[0014] Figure 4 Schematic diagram of Bluetooth signal data acquisition in an embodiment of the human posture estimation method provided by the present invention Figure 1 ;

[0015] Figure 5 Schematic diagram of Bluetooth signal data acquisition in an embodiment of the human posture estimation method provided by the present invention Figure 2 ;

[0016] Figure 6 A schematic diagram of using a neural network to extract human posture in an embodiment of the human posture estimation method provided by the present invention;

[0017] Figure 7 A schematic diagram of the structure of an embodiment of the human body posture estimation device provided by the present invention;

[0018] Figure 8 This is a schematic diagram of the principles of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] The human body posture estimation method provided by the present invention can be applied to a terminal with computing capabilities. The terminal can execute the human body posture estimation method provided by the present invention to perform human body posture estimation. The terminal can be, but is not limited to, various computers, mobile terminals, smart home appliances, wearable devices, routers, etc.

[0021] Figure 1 The following is a flow chart of a method for estimating a human body posture according to the present application. In one embodiment, the method includes the following steps:

[0022] S100: Acquire RGBD image data and wireless communication signal data of the same target scene.

[0023] Specifically, RGBD image data is the RGB image and depth image of the target scene captured by an acquisition device. In this embodiment, the acquisition device includes a depth camera and a color camera, which are used to respectively capture the depth image and color image of the target scene. It should be noted that the depth camera and the color camera can be integrated or independently configured; the depth camera can be a structured light camera, a time-of-flight camera, a binocular camera, or a combination of multiple cameras, without limitation.

[0024] However, extracting the human pose in an image solely through image data can lead to inaccurate recognition due to the obstruction of other human bodies or objects. One possible implementation could address this issue by using multiple cameras to capture images from multiple angles. However, this approach would obviously incur higher costs and compromise the privacy of users within the target scene.

[0025] Therefore, in order to improve the accuracy of human body posture estimation when there is occlusion, in another embodiment, as Figure 3 As shown, wireless communication devices are also used to obtain wireless communication signal data for the same target scene; the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data. During signal transmission, the signal data from the wireless communication device is affected by the presence of objects. Different object occlusions can cause different changes in the wireless communication signal data. In other words, even in the presence of object occlusion, the signal data from the wireless communication device can accurately reflect the current body posture of the target user.

[0026] In one embodiment, when the wireless communication signal data is Bluetooth signal data, obtaining the wireless communication data in the target scene includes: obtaining a first signal received by a first Bluetooth device, and calculating the Bluetooth signal data based on the first signal; wherein the first signal is a Bluetooth signal sent by a second Bluetooth device. In this embodiment, since the one-to-one signal transmission between the transmitting antenna and the receiving antenna cannot reflect the existence of an object between the transmitting antenna and the receiving antenna, it is preferred to use AoA (Angle of Arrival) or AoD (Angle of Departure) to obtain the first signal, such as Figure 4-5 As shown, it more specifically includes: obtaining first signals sent by a transmitting antenna of the second Bluetooth device and received by multiple receiving antennas on the first Bluetooth device; or obtaining multiple first signals sent by multiple transmitting antennas on the second Bluetooth device and received by a receiving antenna on the first Bluetooth device.

[0027] Furthermore, in one embodiment, when multiple receiving antennas on the first Bluetooth device respectively receive the first signal sent by a transmitting antenna of the second Bluetooth device, the first signals received by the multiple receiving antenna arrays will have a phase difference due to different distances, and thus the phase difference can be calculated based on each first signal to obtain Bluetooth signal data; or, in another embodiment, when the first Bluetooth device receives the first signal sent by a transmitting antenna of the second Bluetooth device respectively received by multiple receiving antennas on the first Bluetooth device with one receiving antenna, the phase difference can be directly calculated based on the received first signal to obtain Bluetooth signal data; wherein, the Bluetooth signal data consists of a 1*n dimensional vector, and n is the number of received first signals.

[0028] It should be noted that, in order to improve the accuracy of the Bluetooth signal data, the Bluetooth signal data may be filtered to obtain filtered Bluetooth signal data.

[0029] In one embodiment, when the wireless communication signal data is wireless network channel state information data, obtaining the wireless communication data in the target scene includes: obtaining the wireless network channel state information data of the first wireless network device; wherein the first wireless network device can be a wireless network router, and the channel state information data, also known as CSI (Channel State Information), is the state information of multiple channels of the wireless network device, and the channel state information is related to objects encountered during the signal transmission process. Therefore, the wireless network channel state information data of the first wireless network device can reflect the human body posture within the signal transmission range of the first wireless network device.

[0030] S200 : Estimate a first posture and a second posture of a target user in a target scene according to the RGBD image data and the wireless communication signal data.

[0031] Specifically, if Figure 2 As shown, a human body pose can be composed of a series of key points (such as joints, facial features, etc.) and the lines connecting these key points, which can also be called a 3D skeleton or 3D bones. Acquiring the first pose based on RGBD image data can use existing algorithms for recognizing human poses from images or deep learning-based neural networks, such as OpenPose or DeepCut.

[0032] In one embodiment, estimating a first posture of a target user in a target scene based on RGBD image data more specifically includes: preferably inputting the RGBD image data into a first neural network to obtain the first posture of the target user; wherein the first neural network includes a base network and a two-dimensional branch network and a three-dimensional branch network respectively connected to the base network, the base neural network respectively extracting features from the RGB image and the depth image to obtain corresponding feature images, then inputting the feature images corresponding to the RGB image into the two-dimensional branch network to obtain two-dimensional posture data, and then inputting the two-dimensional posture data and the depth image data into the three-dimensional branch network to obtain the first posture of the target user. It should be noted that the two-dimensional branch network and the two-dimensional branch network of the first neural network share a shared network, so that the two-dimensional branch network acts as a constraint on the three-dimensional branch network, which can improve the accuracy of the target user's human body posture estimation.

[0033] In another embodiment, in order to improve the accuracy of obtaining the second posture based on the wireless communication signal data, estimating the second posture in the target scene based on the wireless communication signal data more specifically includes: using a neural network to extract the second posture. Specifically, obtaining the second posture based on the wireless communication signal data includes: inputting the wireless communication signal data into a second neural network to obtain the second posture of the target user.

[0034] In one embodiment, based on Figure 6 In the second neural network shown, the wireless communication signal data includes two formats: vertical heat map and horizontal heat map. The wireless communication signal is projected on a plane perpendicular to the ground as a vertical heat map, and projected on a plane parallel to the ground as a horizontal heat map, which is equivalent to a depth map. The higher the corresponding value, the stronger the signal reflection intensity. Furthermore, the vertical heat map and the horizontal heat map are convolutionally encoded and spliced to obtain a channel-merged heat map. More specifically, the vertical heat map and the horizontal heat map are convolutionally performed in the spatial dimension. After each layer of convolution, an activation function is executed and batch normalization is performed to remove the spatial dimension so as to obtain relevant information about the human body from the wireless communication signal. After obtaining the channel-merged heat map, it is necessary to perform deconvolution decoding on the channel-merged heat map to obtain the key points of the human body, thereby realizing human body posture estimation.

[0035] In one possible implementation, the second neural network can be trained using the human pose estimation results of the sample image data as supervision, such as Figure 6 As shown, sample image data including a target foreground is input into a posture recognition network with a complex structure and high depth (such as Resnet 152) to obtain a standard human posture estimation based on the sample image data, and several wireless communication signal data including the same target foreground are input into a second neural network as a training set to obtain a predicted human posture estimation output by the second neural network. The standard human posture estimation is used as the supervisory data of the output of the second neural network to train the second neural network, so that the predicted human posture estimation output by the second neural network meets the standard.

[0036] It should be noted that, since the wireless communication signal data may include Bluetooth signal data and / or wireless network channel status information data, the corresponding wireless communication signal data can be adaptively selected based on the cost, accuracy requirements and the situation of existing wireless communication devices in the target scenario, that is, the second posture can be obtained only based on the Bluetooth signal data, or the second posture can be obtained only based on the wireless network channel status information data, or the corresponding second posture can be obtained based on the Bluetooth signal data and the wireless network channel status information data at the same time; wherein, using the Bluetooth signal data and the wireless network channel status information data at the same time to obtain the corresponding second posture can improve the accuracy of the final human body posture estimation result, but it will face the problems of computational complexity and cost, so the specific selection can be based on actual conditions and is not limited here.

[0037] like Figure 1 As shown, after obtaining the first posture and the second posture, the human body posture estimation method provided by this embodiment further includes the steps of:

[0038] S300: Fusing the first posture and the second posture to obtain a target human body posture of a target user.

[0039] In one embodiment, the fusion of the first and second postures can be achieved through a neural network. Specifically, the first and second postures having the same timestamp are input into a preset third neural network, and the posture data output by the third neural network is obtained as the target human posture corresponding to the target user at the current timestamp. It should be noted that the preset third neural network can adopt an existing neural network structure, such as FCN, GCN, etc., and is not limited here.

[0040] In actual operation, the timestamps corresponding to each first posture and each second posture are determined based on the acquisition time of the image acquired by the acquisition device and the time when the wireless communication device acquires the acquired data, that is, the timestamp corresponding to each first posture is the time corresponding to the acquisition device acquiring the corresponding image, and the timestamp corresponding to each second posture is determined based on the time corresponding to the acquisition of the wireless communication signal data.

[0041] Obviously, human body postures with different timestamps are different. To prevent issues arising from inconsistent timestamps between different devices, in this embodiment, before fusing the first and second postures, the timestamps of the first and second postures must be aligned. After the timestamps of the first and second postures are aligned, the real time corresponding to the second posture at time t on the timestamp of the second posture is consistent with the real time corresponding to the first posture at time t on the timestamp of the first posture. Therefore, when fusing the first and second postures, the second posture corresponding to the same time is fused with the first posture to obtain the target posture corresponding to the current time, that is, the target human body posture corresponding to the target user at the current time.

[0042] In one possible implementation, to further improve the accuracy of gesture recognition results, when extracting the first and second gestures, the confidence levels of the first and second gestures can also be extracted. This confidence level reflects the accuracy and reliability of the acquired gestures; that is, the higher the confidence level, the more likely the gesture is accurate. Fusion of the first and second gestures based on their confidence levels can reduce the impact of potentially inaccurate data on the final gesture recognition results.

[0043] In one embodiment, the second posture and the first posture at the same moment are fused, specifically including: obtaining the confidence levels corresponding to the second posture and the first posture, respectively; and fusing the second posture and the first posture at the same moment according to the corresponding confidence levels. More specifically, when the second posture and the first posture are obtained respectively using the second neural network and the first neural network, the coordinates of each human skeleton point in the second posture and the first posture and the corresponding confidence level of each coordinate can be calculated using a softmax function in the last layer of the second neural network and the first neural network; after obtaining the coordinates of each human skeleton point and the corresponding confidence level, the first posture and the second posture can be fused using a preset third neural network or a concat function based on the coordinates of each human skeleton point in the first posture and the corresponding confidence level and the coordinates of each human skeleton point in the second posture and the corresponding confidence level to obtain the target human posture.

[0044] Furthermore, after obtaining the target posture, human behavior recognition can be performed based on the target posture, such as Figure 3 As shown, the target posture can be sent to a remote server for human behavior recognition.

[0045] In summary, this embodiment provides a method for estimating human posture. In addition to acquiring image data and performing posture recognition based on the image data, wireless communication signal data is also used for posture recognition as a supplement to the image posture recognition results. The recognition results of the two are fused to obtain the final human posture. Occlusion between targets will cause changes in the wireless communication signal data, that is, the wireless communication signal data can reflect the occlusion situation of the target. Fusion of the two can effectively improve the accuracy of human posture estimation.

[0046] It should be understood that, although the steps in the flowcharts provided in the accompanying drawings of the present invention are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowcharts may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be performed in rotation or alternation with other steps or at least a portion of sub-steps or stages of other steps.

[0047] Based on the above embodiments, Figure 7 This is a schematic diagram of the structure of a human body posture estimation device provided by the present invention. For details not described in detail in the device, please refer to the description of the human body posture estimation method embodiment above, and will not be repeated below. In one embodiment, the device includes:

[0048] A data acquisition module, configured to acquire RGBD image data and wireless communication signal data of the same target scene, wherein the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data;

[0049] A posture acquisition module, configured to estimate a first posture and a second posture of a target user in a target scene based on the RGBD image data and the wireless communication signal data;

[0050] A fusion module is used to fuse the first posture and the second posture to obtain a target human body posture of the target user.

[0051] Based on the above embodiments, the present invention also provides a terminal, such as Figure 8 As shown, the terminal includes a processor 10 and a memory 20. Figure 8 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0052] In one embodiment, the memory 20 is an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Furthermore, the memory 20 may also include both the internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or is to be output. In one embodiment, a human body posture estimation program 30 is stored on the memory 20, and the human body posture estimation program 30 can be executed by the processor 10, thereby realizing the human body posture estimation method in the present application.

[0053] In one embodiment, the processor 10 is a central processing unit (CPU), a microprocessor or other chip, which is used to run program codes or process data stored in the memory 20, such as executing a human posture estimation method.

[0054] In one embodiment, when the processor 10 executes the human body posture estimation program 30 in the memory 20, the following steps are implemented: obtaining RGBD image data and wireless communication signal data of the same target scene; wherein the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data; based on the RGBD image data and the wireless communication signal data, respectively estimating the first posture and the second posture of the target user in the target scene; and fusing the first posture and the second posture to obtain the target human body posture of the target user.

[0055] In one embodiment, the terminal further includes a capture device and a wireless device for respectively capturing RGBD image data and wireless communication signal data of the same target scene and transmitting these data to the processor for human pose estimation. It should be noted that the capture device and the wireless device can be arranged in an interleaved or opposed arrangement, as long as the data captured is the same target scene, and this is not a limitation herein.

[0056] The present invention also provides a computer-readable storage medium, in which one or more programs are stored. The one or more programs can be executed by one or more processors to implement the steps of the above human body posture estimation method.

[0057] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for estimating a human body posture, characterized in that: The method comprises: Acquire RGBD image data and wireless communication signal data of the same target scene; wherein the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data; estimating a first posture and a second posture of a target user in the target scene based on the RGBD image data and the wireless communication signal data, respectively; wherein the first posture of the target user is estimated based on the RGBD image data, and the second posture of the target user is estimated based on the wireless communication signal data; aligning the timestamps of the second posture and the first posture; fusing the second posture and the first posture with the same timestamp to obtain a target human body posture corresponding to the current moment; wherein the fusion of the second posture and the first posture is performed based on a preset third neural network; The first posture of the target user is estimated according to the following steps: RGBD image data is input into a first neural network to obtain the first posture of the target user; wherein, the first neural network includes a basic network for feature extraction and a two-dimensional branch network and a three-dimensional branch network respectively connected to the basic network, the two-dimensional branch network is used to obtain two-dimensional posture data using the RGB image, and the three-dimensional branch network is used to estimate the first posture of the target user using the two-dimensional posture data and the depth image; the two-dimensional branch network and the three-dimensional branch network of the first neural network share a shared network.

2. The human body posture estimation method according to claim 1, wherein When the wireless communication signal data is Bluetooth signal data, obtaining the wireless communication signal data in the target scene includes: Acquire a first signal sent by a transmitting antenna of the second Bluetooth device and received by multiple receiving antennas on the first Bluetooth device respectively; The phase difference is calculated according to each first signal received by the multiple receiving antenna arrays to obtain the Bluetooth signal data.

3. The method for estimating human body posture according to claim 1, wherein: When the wireless communication signal data is Bluetooth signal data, obtaining the wireless communication signal data in the target scene includes: Acquire a first signal received by a receiving antenna on the first Bluetooth device and sent respectively by multiple transmitting antennas on the second Bluetooth device; The phase difference is calculated according to the multiple first signals received by the one receiving antenna to obtain the Bluetooth signal data.

4. The method for estimating human body posture according to claim 1, wherein: Estimating a second posture of a target user in the target scene according to the wireless communication signal data includes: The wireless communication signal data is input into a second neural network to obtain a second posture of the target user; wherein the wireless communication signal data includes two formats of vertical heat map and horizontal heat map, the second neural network performs convolution encoding on each heat map and merges channels, and decodes the heat map after merging channels to estimate the second posture of the target user.

5. The method for estimating human body posture according to claim 1, wherein The fusing the second posture and the first posture at the same moment includes: respectively obtaining the coordinates of each human skeleton point in the first posture and the second posture and the corresponding confidence level of each coordinate; The second posture and the first posture are fused based on the corresponding confidence levels and the same timestamp.

6. A human body posture estimation device, characterized in that: include: a data acquisition module, configured to acquire RGBD image data and wireless communication signal data of the same target scene, wherein the wireless communication signal data includes Bluetooth signal data and / or wireless network channel state information data; a posture acquisition module, configured to estimate a first posture and a second posture of a target user in the target scene based on the RGBD image data and the wireless communication signal data, respectively; wherein the first posture of the target user is estimated based on the RGBD image data, and the second posture of the target user is estimated based on the wireless communication signal data; a fusion module, configured to align the timestamps of the second posture and the first posture; and fuse the second posture and the first posture having the same timestamp to obtain a target human body posture corresponding to the current moment; wherein the fusion of the second posture and the first posture is performed based on a preset third neural network; The posture acquisition module is specifically used to: input RGBD image data into a first neural network to obtain a first posture of a target user; wherein, the first neural network includes a basic network for feature extraction and a two-dimensional branch network and a three-dimensional branch network respectively connected to the basic network, the two-dimensional branch network is used to obtain two-dimensional posture data using the RGB image, and the three-dimensional branch network is used to estimate the first posture of the target user using the two-dimensional posture data and the depth image; the two-dimensional branch network and the three-dimensional branch network of the first neural network share a shared network.

7. A terminal, characterized in that: The terminal includes: a processor, a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium being suitable for storing a plurality of instructions, and the processor being suitable for calling the instructions in the computer-readable storage medium to execute the steps of the human body posture estimation method according to any one of claims 1 to 5. The terminal according to claim 7 , wherein: It also includes an acquisition device and a wireless device for respectively acquiring RGBD image data and wireless communication signal data of the same target scene and transmitting them to the processor to implement the steps of the human body posture estimation method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the human body posture estimation method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Human body posture estimation method and device, electronic equipment and readable storage medium

    CN112668413A