Three-dimensional posture information restoration method and device

Abstract

The present invention discloses a method and device for restoring three-dimensional posture information. In the above method, a first frontal image of a current human body facing a camera device is obtained; multiple joints in the first frontal image are identified, and the first length information of the rigid bones corresponding to each of the multiple joints in the first frontal image is determined; a second frontal image of the current human body facing the camera device and photographed after the human body moves and changes its posture is obtained; multiple joints in the second frontal image are identified, and the second length information corresponding to the rigid bones corresponding to each of the multiple joints after the movement is determined; and the posture information of the current human body in the third dimension orthogonal to the imaging plane is restored according to the projection coordinate position information and the second length information of the rigid bones corresponding to each of the multiple joints on the image imaging plane. The acquisition cost of model samples and the computing power requirements for training models are reduced, and the amount of calculation is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a method and device for restoring three-dimensional posture information. Background Art

[0002] In recent years, with the rapid development of artificial intelligence, especially in the field of image recognition, many excellent algorithms have been born, and the level of image recognition has been continuously improved. However, as a very popular sub-field in the current field of image recognition, the accuracy of human posture recognition has been continuously improved through the use of various algorithms.

[0003] In recent years, the level of refinement of human body recognition has been improved from two-dimensional to three-dimensional. To perform accurate three-dimensional human posture recognition, related technologies generally require the use of three-dimensional information of the human body for training. Such information includes voxels (points in three-dimensional space, corresponding to pixels in two-dimensional space), point clouds or multiple two-dimensional images, etc. The amount of data of these data types is an order of magnitude higher than the pixels of two-dimensional images. Therefore, the computing power required for training and detection is much greater than that of two-dimensional posture recognition. First, the collection of "point cloud" data is relatively difficult. To collect accurate "point cloud" data, a relatively advanced depth camera is required, and the cost of such equipment is generally relatively high; secondly, due to the huge amount of "point cloud" data, the design and training of the model face a huge challenge.

[0004] Therefore, in order to reduce the sample collection cost and the amount of calculation, most of the later 3D posture recognition models use multiple 2D images taken from different angles as samples, which greatly reduces the amount of sample information. A large number of 2D samples are used to infer the position of the 3D space. Compared with the model that uses "point cloud" as the sample, this model has greatly reduced the sample collection cost and the amount of calculation.

[0005] However, this solution still involves a relatively large amount of spatial geometry calculations, and the computing power required for model training is still relatively high. Summary of the invention

[0006] The main purpose of the present invention is to disclose a method and device for restoring three-dimensional posture information, so as to at least solve the problem that the three-dimensional posture recognition model used in the related art has a large amount of spatial geometry calculation and the computing power required for model training is relatively high.

[0007] According to one aspect of the present invention, a method for restoring three-dimensional posture information is provided.

[0008] The three-dimensional posture information restoration method according to the present invention comprises: obtaining a first front image taken by a current human body facing a camera device, wherein the rigid bones corresponding to the joints of the current human body are parallel to the image imaging plane; identifying multiple joints in the first front image, and determining first length information of the rigid bones corresponding to each of the multiple joints in the first front image; obtaining a second front image taken by the current human body after it moves and changes its posture facing the camera device, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera, and the distance between the current human body and the image imaging plane changes; identifying the multiple joints in the second front image, and determining the second length information corresponding to the rigid bones corresponding to each of the multiple joints after the movement according to the change rate of the length information of the hip joint line in the second front image relative to the length information of the hip joint line in the first front image, and the first length information; determining the projection coordinate position information of the rigid bones corresponding to each of the multiple joints on the image imaging plane; and restoring the posture information of the current human body in a third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information.

[0009] According to another aspect of the present invention, a three-dimensional posture information restoration device is provided.

[0010] The three-dimensional posture information restoration device according to the present invention comprises: a first acquisition module, used to acquire a first front image taken by the current human body facing the camera device, wherein the rigid bones corresponding to the joints of the current human body are parallel to the image imaging plane; a first determination module, used to identify multiple joints in the first front image, and determine the first length information of the rigid bones corresponding to each of the multiple joints in the first front image; a second acquisition module, used to acquire a second front image taken after the current human body moves and changes its posture facing the camera device, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera device, and the current human body is parallel to the imaging plane of the image. The distance changes; a second determination module is used to identify the above-mentioned multiple joints in the above-mentioned second frontal image, and determine the second length information corresponding to the rigid bone corresponding to each joint of the multiple joints after moving according to the change rate of the length information of the hip joint line in the second frontal image relative to the length information of the hip joint line in the first frontal image, and the first length information; a third determination module is used to determine the projection coordinate position information of the rigid bone corresponding to each joint of the multiple joints on the imaging plane of the image; a restoration module is used to restore the posture information of the above-mentioned current human body in a third dimension orthogonal to the above-mentioned imaging plane according to the above-mentioned projection coordinate position information and the above-mentioned second length information.

[0011] The three-dimensional posture information restoration scheme adopted by the present invention builds a human posture recognition model based on a single two-dimensional image of the rigid bones corresponding to each joint of the human body parallel to the image imaging plane, and restores the posture information of the current human body in the third dimension orthogonal to the above imaging plane according to the projection coordinate position information of the rigid bones on the above imaging plane and the length information of the rigid bones. Therefore, the acquisition cost of the model samples and the computing power requirements for the training model are reduced, and the amount of calculation is greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of a method for restoring three-dimensional posture information according to an embodiment of the present invention;

[0013] Figure 2 is a schematic diagram of imaging when a human body rigid skeleton and an imaging plane of a camera are parallel according to a preferred embodiment of the present invention;

[0014] Figure 3 is a schematic diagram of imaging when the rigid skeleton of a human body and the imaging plane of a camera are not parallel according to a preferred embodiment of the present invention;

[0015] Figure 4 is a three-dimensional coordinate diagram of the rigid skeleton rotated along the Z axis according to an example of the present invention;

[0016] Figure 5 is a three-dimensional coordinate diagram of the rigid skeleton rotated along the Y axis according to an example of the present invention;

[0017] Figure 6 is a structural block diagram of a three-dimensional posture information restoration device according to an embodiment of the present invention;

[0018] Figure 7 It is a structural block diagram of a three-dimensional posture information restoration device according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0019] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0020] According to an embodiment of the present invention, a method for restoring three-dimensional posture information is provided.

[0021] Figure 1 is a flow chart of a method for restoring three-dimensional posture information according to an embodiment of the present invention. Figure 1 As shown, the three-dimensional posture information restoration method includes:

[0022] Step S101: obtaining a first frontal image of a current human body facing a camera device, wherein the rigid bones corresponding to the joints of the current human body are parallel to the imaging plane of the image;

[0023] Step S103: identifying multiple joints in the first frontal image, and determining first length information of a rigid bone corresponding to each of the multiple joints in the first frontal image;

[0024] Step S105: obtaining a second frontal image taken after the current human body moves and changes its posture facing the camera device, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera, and the distance between the current human body and the imaging plane of the image changes;

[0025] Step S107: identifying the plurality of joints in the second front image, and determining second length information corresponding to the rigid bones corresponding to each of the plurality of joints after movement according to the change rate of the length information of the hip joint line in the second front image relative to the length information of the hip joint line in the first front image and the first length information;

[0026] Step S109: determining the projection coordinate position information of the rigid bone corresponding to each of the plurality of joints on the image plane;

[0027] Step S111: restoring the posture information of the current human body in a third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information.

[0028] use Figure 1 The three-dimensional posture information restoration scheme shown in the figure constructs a human posture recognition model based on a single two-dimensional image in which the rigid bones corresponding to each joint of the human body are parallel to the imaging plane of the image. The human body is required to remain facing the camera (e.g., a camera), that is, the line connecting the two hip joints of the human body is parallel to the imaging plane of the camera (not sideways) and moves and changes its posture. According to the change rate of the length information of the hip joint line in the second front image relative to the length information of the hip joint line in the first front image, and the first length information, the second length information corresponding to the rigid bones corresponding to each joint of the multiple joints after movement is determined; according to the projection coordinate position information of the rigid bones on the above-mentioned imaging plane and the second length information of the rigid bones, the posture information of the above-mentioned current human body in the third dimension orthogonal to the above-mentioned imaging plane is restored. Therefore, the acquisition cost of the model sample and the computing power requirements for the training model are reduced, and the amount of calculation is greatly reduced.

[0029] In the preferred implementation process, a human posture recognition model is constructed based on a single two-dimensional image in which the rigid bones corresponding to each joint of the human body are parallel to the imaging plane of the image, which specifically includes: requiring the human body to be in an upright posture, that is, with legs standing straight and hands hanging naturally on both sides of the human body, and performing posture recognition on the human body. The rigid bones of the human body are approximately regarded as directed line segments between its two endpoints (two joints). Since the rigid bones of the human body are parallel to the imaging plane of the camera device (for example, a camera), the "image" (that is, the first frontal image) of the bones through the camera lens should be within the imaging plane of the camera. Therefore, the rigid bones of the human body in the captured image represent the "image" of the bones through the camera lens, and the actual length of the rigid bones of the human body corresponds to the length of the "image" of the bones (that is, the length information of the rigid bones corresponding to the above-mentioned joints in the first frontal image). For details, please refer to Figure 2 .

[0030] Then, the human body is moved (the distance between the current human body and the imaging plane changes) and the posture is changed while the human body torso remains facing the camera, that is, the line connecting the two hip joints of the human body remains parallel to the imaging plane of the camera (not sideways), to obtain the result of two-dimensional posture estimation.

[0031] Preferably, the second length information corresponding to the rigid bone corresponding to each joint in the multiple joints after the movement can be determined in the following manner:

[0032] The second length information = (the length information of the hip joint line in the second frontal image * the first length information) / the length information of the hip joint line in the first frontal image

[0033] In the preferred implementation process, when the human body is displaced, that is, the human body changes its position under the condition that the two hip joint lines are kept parallel to the imaging plane of the camera device, since the hip joint line is always parallel to the imaging plane, the "image" formed by the line between the two joints behind the camera lens is still in the camera imaging plane, so the length of the hip joint line in the image captured by the camera corresponds to the length of the "image", and before and after the human body changes its position, the ratio of these two lengths (i.e.: the length information of the hip joint line in the second front image / the length information of the hip joint line in the first front image) is equal to the ratio of the length of other bones in the "image" formed by the human body (all "images" formed by the human body are "similar", and the lengths of all corresponding rigid bones are proportional). Therefore, the length of each rigid bone in the "image" formed by the human body after the displacement (i.e., the second length information) can be inferred by the rate of change of the length of the hip joint line after the displacement. And with the length of each bone in the "image" formed by the human body, the three-dimensional orientation of each bone can be calculated in the following way.

[0034] Specifically, when the position of the human body changes (i.e., the distance between the current human body and the imaging plane of the image changes), the human body changes its posture. Since the human skeleton changes its orientation, its "image" formed by the camera lens may no longer be located in the imaging plane of the camera. However, its corresponding image in the imaging plane (i.e., the second frontal image) can be approximately regarded as the projection of the "image" in the imaging plane. For details, please refer to Figure 3 .

[0035] The length of each rigid bone in the "image" of the human body (i.e., the second length information corresponding to the movement of the rigid bone) can be regarded as the vector sum of its projection in the camera imaging plane and the projection in the direction perpendicular to the imaging plane. The lengths of the three satisfy the Pythagorean theorem. By obtaining the length of the "image" and the length of its projection on the imaging plane, the length of its projection in the direction perpendicular to the projection plane can be inferred. According to the range of motion of the bones and some preset conditions, the direction of the projection can be determined, thus obtaining the orientation of the image of the bones in three-dimensional space. Then, according to the "point symmetry" of the bones and the images of the bones relative to the center of the camera lens, the three-dimensional orientation of the bones can be inferred.

[0036] The following combination Figure 4 and Figure 5 .

[0037] In the preferred implementation process, the first frontal image of the human body facing the image plane is first captured by a shooting device, wherein the rigid bones corresponding to the current joints of the human body are parallel to the image plane; specifically, after the first frontal image is captured, the current human posture recognition technology can be used to identify the various joints of the human body in the first frontal image. For example, the "bottom-up" recognition technology can be used to first find the various joints of the human body, and then determine the various rigid bones (such as neck, forearm, upper arm, thigh, calf, etc.) corresponding to these joints. Since the rigid bones corresponding to the various joints of the human body are parallel to the image plane, in the first frontal image, the length of each rigid bone corresponds to the real bone length of the human body.

[0038] After obtaining the lengths of the rigid bones corresponding to the joints of the human body (such as the corresponding rigid bones in the neck, forearm, upper arm, thigh, calf, etc.) by identification, the current human body is moved and changed in posture while facing the camera, and a second front image taken after the current human body moves and changes in posture facing the camera is obtained. The above-mentioned multiple joints in the second front image are identified by a two-dimensional human posture recognition technology, and the second length information corresponding to the rigid bones corresponding to each joint of the multiple joints after movement is determined according to the change rate of the length information of the hip joint line in the second front image relative to the length information of the hip joint line in the first front image, and the first length information, and the projection coordinate position information of the rigid bones corresponding to each joint of the multiple joints on the above-mentioned image imaging plane (that is, the positions of the multiple joints on the two-dimensional image imaging plane) is determined, and then the posture information of the above-mentioned current human body in the third dimension orthogonal to the above-mentioned imaging plane is restored by the second length information and the projection coordinate position information of the rigid bones corresponding to each joint in the posture, so as to obtain a three-dimensional human posture estimation result.

[0039] The following takes a rigid bone of the human body as an example to describe the relationship between the projection coordinates of the rigid bone, the second length information of the rigid bone, and the trigonometric functions of the rotation angle information of the rigid bone in the second front image relative to the first front image. First, set one end point of the "image" of the rigid bone to always be located in the imaging plane of the camera, take this end point as the coordinate origin, take the direction pointing horizontally to the side of the right hand of the human body when the person faces the imaging plane as the X-axis, take the vertical upward Y-axis, and take the direction perpendicular to the X and Y axes and pointing to the observer as the Z-axis. When one end of the rigid bone is set as the coordinate origin, assuming that the "image" of the bone starts from the Y-axis direction, first rotate along the Z-axis by an angle of θ1 (rotate in the XY plane, such as Figure 4 Then rotate along the Y axis by θ2 to reach the final position (as shown in Figure 5 Note: When rotating, use the left-hand rule, that is, the thumb of the left hand points to the direction of the axis along which it is rotating, and the other four fingers point to the direction of rotation. Then θ1 and θ2 should satisfy the following relationship:

[0040] x=l·cos(θ1) Formula (1)

[0041] y=l·sin(θ1)·cos(θ2) Formula (2)

[0042] z=l·sin(θ1)·sin(θ2) Formula (3)

[0043] Among them, x, y, z are the coordinates of the final position of the other end point of the rigid bone on the X-axis, Y-axis and Z-axis respectively, and l is the length of the rigid bone in the "image" of the human body after the displacement (that is, the second length information corresponding to the movement of the rigid bone). The possible values ​​of θ1 and θ2 can be inferred from the values ​​of l and x, y, z. Then, based on the range of motion of the human joints and some limiting conditions, some impossible values ​​can be eliminated, and finally a unique set of values ​​can be obtained.

[0044] Preferably, in step S109, restoring the posture information of the current human body in the third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information may further include: for the rigid bone corresponding to each of the multiple joints, determining the rotation angle information of the rigid bone in the second front image relative to the first front image according to the projection coordinate position information and the second length information of the rigid bone.

[0045] In a preferred implementation process, determining the rotation angle information of the rigid bone in the second frontal image relative to the first frontal image based on the projection coordinate position information and the second length information of the rigid bone can further include: setting one endpoint of the rigid bone to be located at the coordinate origin; obtaining a calculation result through the trigonometric function relationship and the inverse trigonometric function formula between the projection coordinates and the second length information based on the projection coordinates of the other endpoint of the rigid bone on the imaging plane and the second length information of the rigid bone; determining the rotation angle in the second frontal image relative to the first frontal image from the above calculation result based on a preset angle range or a judgment condition of whether it conforms to the law of human bone activity.

[0046] For example, from the values ​​of l, x, and y in the above formula, through formula (1) and formula (2), the possible values ​​of θ1 and θ2 can be determined, and then some impossible values ​​can be eliminated according to the range of motion of the human joints (that is, the above judgment conditions for whether it conforms to the laws of human skeletal motion) and some limiting conditions (preset angle range), and finally a unique rotation angle value can be obtained.

[0047] Preferably, after determining the rotation angle information of the rigid bone in the second frontal image relative to the first frontal image, it can also include: determining the component information of the rigid bone in the third dimension orthogonal to the above-mentioned imaging plane according to the rotation angle information corresponding to the rigid bone and the second length information of the rigid bone.

[0048] In a preferred implementation process, determining the component information of the rigid bone in the third dimension orthogonal to the above-mentioned imaging plane according to the rotation angle information corresponding to the rigid bone and the second length information of the rigid bone can further include: determining the component information of the rigid bone in the third dimension orthogonal to the above-mentioned imaging plane according to the second length information of the rigid bone and the trigonometric function value of the rotation angle corresponding to the rigid bone, and according to a preset value range or a judgment condition of whether it conforms to the law of human bone activity.

[0049] For example, after θ1 and θ2 are determined, the component information of the rigid bone in the third dimension orthogonal to the imaging plane can be obtained through formula (3), for example, the coordinate of the other end point of the rigid bone on the z-axis.

[0050] Preferably, in the above step S109, restoring the posture information of the above current human body in the third dimension orthogonal to the above imaging plane according to the above projection coordinate position information and the above second length information may further include: determining the component information of the rigid bone in the third dimension orthogonal to the above imaging plane according to the projection coordinate position information and the second length information of the rigid bone, and in accordance with a preset value range or a judgment condition of whether it conforms to the law of human bone activity.

[0051] For example, according to the formula l 2 =x 2 +y 2 +z 2 The values ​​of l, x, and y in can determine the absolute value (modulus) of z. Then, based on the range of motion of the human joints and some limiting conditions, some impossible values ​​can be eliminated, and finally a unique z value can be obtained.

[0052] The preferred implementation scheme described above is further described below in conjunction with an example, wherein a first frontal image is acquired, a plurality of joints in the first frontal image are identified, a first length information of a rigid bone corresponding to a certain joint in the first frontal image is determined, and a second frontal image taken after the current human body moves and changes its posture facing the camera device is acquired, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera device, and the distance between the current human body and the imaging plane of the image changes, the plurality of joints in the second frontal image are identified, and the second length information corresponding to the rigid bone corresponding to each of the plurality of joints after the movement is determined according to the rate of change of the length information of the hip joint line in the second frontal image relative to the length information of the hip joint line in the first frontal image, and the first length information, that is, the following formula is used to determine:

[0053] The second length information = (the length information of the hip joint line in the second frontal image * the first length information) / the length information of the hip joint line in the first frontal image

[0054] For example, using the above scheme, the calculated second length information l is 50 pixels, one endpoint of the bone is set at the origin of the coordinates, and the coordinates of the projection vector of the other endpoint of the bone on the XY plane (i.e., the image imaging plane) are (30, 30). The value range of θ1 is first limited to [0°, 180°], and the value range of θ2 is [0°, 360°). Because the value ranges of the above two angles can ensure that the final position of the other endpoint can cover all possible positions of all quadrants, the above preset angle range is adopted.

[0055] According to the above formula (1), cos(θ1) can be calculated to be 0.6, and the possible value of θ1 can be calculated by inverse trigonometric function to be 53.13° or 306.87°. According to the preset range of θ1 values, 306.87° can be excluded, so θ1 is 53.13°. Substituting into formula (2), cos(θ2) is 0.75, and θ2 is deduced to be 41.41° or 318.59° by inverse trigonometric function. If z is determined to be positive (or z is pre-defined as positive) based on the judgment condition of whether it meets the law of human bone activity (i.e. the normal range of bone activity), then the value of z being negative is excluded, and then sin(θ2) can be deduced to be positive by formula (3), so θ2 can only be 41.41°. Finally, in addition to θ1 and θ2, the third dimension can be obtained by formula (3), that is, the coordinate of the final position of the other end point of the rigid bone on the Z axis: 26.4. The three-dimensional orientation of the "image" of the bone is obtained, and based on the point symmetry relationship, the orientation of the bone in three-dimensional space can be deduced.

[0056] Of course, in addition to the above determination method, you can also use formula l 2 =x 2 +y 2 +z 2 The values ​​of l, x, and y in are used to determine the modulus of z, and then the possible value of z. Then, according to the preset value range or the judgment condition of whether it conforms to the law of human bone activity, it is determined that z must be positive, and the coordinate of the final position of the other end point of the rigid bone on the Z axis is obtained.

[0057] According to an embodiment of the present invention, a three-dimensional posture information restoration device is provided.

[0058] Figure 6 is a structural block diagram of a three-dimensional posture information restoration device according to an embodiment of the present invention. Figure 6As shown, the three-dimensional posture information restoration device includes: a first acquisition module 60, which is used to acquire a first front image taken by the current human body facing the camera device, wherein the rigid bones corresponding to the joints of the current human body are parallel to the image imaging plane; a first determination module 62, which is used to identify multiple joints in the first front image, and determine the length information of the rigid bones corresponding to each of the multiple joints in the first front image; a second acquisition module 64, which is used to acquire a second front image taken after the current human body moves and changes its posture facing the camera device, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera device, and the distance between the current human body and the image imaging plane is changes; a second determining module 66 is used to identify the above-mentioned multiple joints in the above-mentioned second frontal image, and determine the second length information corresponding to the rigid bones corresponding to each of the multiple joints after movement according to the change rate of the length information of the hip joint line in the second frontal image relative to the length information of the hip joint line in the first frontal image, and the first length information; a third determining module 68 is used to determine the projection coordinate position information of the rigid bones corresponding to each of the above-mentioned multiple joints on the imaging plane of the above-mentioned image; a restoration module 70 is used to restore the posture information of the above-mentioned current human body in a third dimension orthogonal to the above-mentioned imaging plane according to the above-mentioned projection coordinate position information and the above-mentioned second length information.

[0059] use Figure 6 The three-dimensional posture information restoration device shown builds a human posture recognition model based on a single two-dimensional image in which the rigid bones corresponding to each joint of the human body are parallel to the imaging plane of the image, requiring the human body to remain facing the camera (e.g., camera), that is, the line connecting the two hip joints of the human body is parallel to the imaging plane of the camera (not sideways) and moves and changes posture, and determines the second length information corresponding to the rigid bones corresponding to each joint of the multiple joints after movement according to the change rate of the length information of the hip joint line in the second front image relative to the length information of the hip joint line in the first front image, and the first length information; restores the posture information of the current human body in the third dimension orthogonal to the above-mentioned imaging plane according to the projection coordinate position information of the rigid bones on the above-mentioned imaging plane and the second length information of the rigid bones. Therefore, the acquisition cost of the model samples and the computing power requirements for the training model are reduced, and the amount of calculation is greatly reduced.

[0060] Preferably, if Figure 7As shown, the restoration module 70 may further include: a first determination unit 700, for determining, for a rigid bone corresponding to each of the above-mentioned multiple joints, rotation angle information of the rigid bone in the second front image relative to the first front image based on the projection coordinate position information and the second length information of the rigid bone.

[0061] Preferably, if Figure 7 As shown, the restoration module 70 may further include: a second determination unit 702, configured to determine component information of the rigid bone in a third dimension orthogonal to the imaging plane according to the rotation angle information corresponding to the rigid bone and the second length information of the rigid bone.

[0062] Preferably, if Figure 7 As shown, the restoration module 70 may also include: a third determination unit 704, which is used to determine the component information of the rigid bone in the third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information of the rigid bone and in accordance with a preset value range or a judgment condition of whether it conforms to the law of human bone activity.

[0063] The preferred implementation of the combination of each module and each unit of the above device can be found in Figures 1 to 3 The description is not repeated here.

[0064] With the above-mentioned embodiment provided by the present invention, it is required that the human body must face the camera device, that is, the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera device, and the distance between the current human body and the imaging plane of the image changes. According to the change rate of the length information of the hip joint line in the second front image relative to the length information of the hip joint line in the first front image, and the first length information, the second length information corresponding to the rigid bone corresponding to each joint of the multiple joints after the movement is determined; according to the projection coordinate position information of the rigid bone on the above-mentioned imaging plane and the second length information of the rigid bone, the posture information of the current human body in the third dimension orthogonal to the above-mentioned imaging plane is restored. The two-dimensional projection of the three-dimensional vector and its length are used to realize the scheme of obtaining three-dimensional human body posture estimation using a single two-dimensional image. The scheme is simple and easy to implement, reduces the acquisition cost of model samples and the computing power requirements of the training model, greatly reduces the amount of calculation, and effectively improves the system operation efficiency.

[0065] The above disclosures are only several specific embodiments of the present invention; however, the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A three-dimensional posture information restoration method, characterized in that: include: Acquire a first frontal image of the current human body taken by the camera device, wherein the rigid bones corresponding to the joints of the current human body are parallel to the image plane; Identify multiple joints in the first frontal image, and determine first length information of a rigid bone corresponding to each of the multiple joints in the first frontal image; Acquire a second frontal image captured after the current human body moves and changes its posture toward the camera device, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera device, and the distance between the current human body and the imaging plane of the image changes; Identify the multiple joints in the second frontal image, and determine second length information corresponding to the rigid bone corresponding to each joint of the multiple joints after movement according to the change rate of the length information of the hip joint line in the second frontal image relative to the length information of the hip joint line in the first frontal image and the first length information; Determine projection coordinate position information of a rigid bone corresponding to each of the plurality of joints on the image plane; The posture information of the current human body in a third dimension orthogonal to the imaging plane is restored according to the projection coordinate position information and the second length information.

2. The method according to claim 1, characterized in that The second length information corresponding to the rigid bone corresponding to each joint in the multiple joints after the movement is determined in the following manner: The second length information=(the length information of the hip joint line in the second front image*the first length information) / the length information of the hip joint line in the first front image.

3. The method according to claim 1, characterized in that: Restoring the posture information of the current human body in a third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information includes: For the rigid bone corresponding to each of the multiple joints, the rotation angle information of the rigid bone in the second front image relative to the first front image is determined according to the projection coordinate position information and the second length information of the rigid bone.

4. The method according to claim 3, characterized in that: Determining the rotation angle information of the rigid bone in the second frontal image relative to the first frontal image according to the projection coordinate position information and the second length information of the rigid bone includes: Set one endpoint of the rigid bone to be at the coordinate origin; According to the projection coordinates of the other end point of the rigid bone on the imaging plane and the second length information of the rigid bone, a calculation result is obtained through a trigonometric function relationship between the projection coordinates and the length value and an inverse trigonometric function formula; According to a preset angle range or a determination condition of whether it complies with the law of human skeletal movement, the rotation angle in the second frontal image relative to the first frontal image is determined from the calculation result.

5. The method according to claim 3, characterized in that: After determining the rotation angle information of the rigid skeleton in the second frontal image relative to the first frontal image, the method further includes: According to the rotation angle information corresponding to the rigid bone and the second length information of the rigid bone, component information of the rigid bone in a third dimension orthogonal to the imaging plane is determined.

6. The method according to claim 5, characterized in that Determining component information of the rigid bone in a third dimension orthogonal to the imaging plane according to the rotation angle information corresponding to the rigid bone and the second length information of the rigid bone includes: According to the second length information of the rigid bone and the trigonometric function value of the rotation angle corresponding to the rigid bone, and in accordance with the preset value range or the judgment condition of whether it conforms to the law of human bone activity, the component information of the rigid bone in the third dimension orthogonal to the imaging plane is determined.

7. The method according to claim 1, characterized in that Restoring the posture information of the current human body in a third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information includes: According to the projection coordinate position information and the second length information of the rigid bone, and in accordance with a preset value range or a determination condition of whether it conforms to the law of human bone activity, component information of the rigid bone in a third dimension orthogonal to the imaging plane is determined.

8. A three-dimensional posture information restoration device, characterized in that: include: A first acquisition module is used to acquire a first frontal image of the current human body facing the camera device, wherein the rigid bones corresponding to the joints of the current human body are parallel to the image plane; A first determination module, configured to identify a plurality of joints in the first frontal image, and determine first length information of a rigid bone corresponding to each of the plurality of joints in the first frontal image; A second acquisition module is used to acquire a second frontal image taken after the current human body moves and changes its posture facing the camera device, wherein the current human body moves and changes its posture while keeping the hip joint line parallel to the imaging plane of the camera device, and the distance between the current human body and the imaging plane of the image changes; a second determination module, for identifying the plurality of joints in the second frontal image, and determining second length information corresponding to the rigid bone corresponding to each of the plurality of joints after movement according to a rate of change of the length information of the hip joint line in the second frontal image relative to the length information of the hip joint line in the first frontal image, and the first length information; A third determination module is used to determine the projection coordinate position information of the rigid bone corresponding to each joint of the plurality of joints on the image imaging plane; A restoration module is used to restore the posture information of the current human body in a third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information.

9. The device according to claim 8, characterized in that The restoration module comprises: A first determining unit is used to determine, for a rigid bone corresponding to each of the plurality of joints, rotation angle information of the rigid bone in the second frontal image relative to the first frontal image according to the projection coordinate position information and the second length information of the rigid bone; The second determining unit is used to determine the component information of the rigid bone in a third dimension orthogonal to the imaging plane according to the rotation angle information corresponding to the rigid bone and the second length information of the rigid bone.

10. The device according to claim 8, characterized in that The restoration module comprises: The third determination unit is used to determine the component information of the rigid bone in the third dimension orthogonal to the imaging plane according to the projection coordinate position information and the second length information of the rigid bone and in accordance with a preset value range or a judgment condition of whether it conforms to the law of human bone activity.

Images