A three-dimensional space positioning method, system, device and storage medium

Abstract

The present application discloses a three-dimensional spatial positioning method, system, device and storage medium, wherein the method comprises the following steps: obtaining first posture data of inertial motion capture of human skeleton; determining the motion state of a human virtual model according to the first posture data; determining that the human virtual model is in the first motion state, obtaining first position information of a static skeleton node of the human virtual model; determining second position information of a target skeleton node of the human virtual model according to the first position information and the first posture data; determining that the human virtual model is in the second motion state, obtaining speed information of a moving skeleton node of the human virtual model; the speed information comprises gravity-free acceleration and motion speed. The present method can reduce the error of three-dimensional spatial positioning of human skeleton and improve the accuracy of three-dimensional spatial positioning. The present application can be widely used in the field of three-dimensional spatial positioning technology.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional space positioning technology, and in particular to a three-dimensional space positioning method, system, device and storage medium. Background Art

[0002] The pedestrian navigation system can monitor and locate the position of mission personnel in real time, and enable the command center to understand the movement status of mission personnel, thereby greatly improving the rapid response capability of emergency search and rescue personnel and the command capability of the command center. Therefore, the development of real-time and accurate pedestrian navigation and positioning technology has broad application prospects.

[0003] In recent years, the technology of micro-electromechanical (MEMS) sensors has gradually matured, and pedestrian navigation technology based on MEMS technology has developed rapidly. This technology usually fixes the MEMS sensor on the foot, obtains position information and orientation information through gyroscope and accelerometer integration, and then reduces the drift error of the integration through zero-speed correction of human gait characteristics. However, consumer-grade MEMS sensors are generally of low accuracy, and this method cannot effectively estimate and correct the drift error of the integration. The position error caused by the acceleration error during the integration process cannot be accurately estimated, and the three-dimensional spatial position of the human body cannot be accurately located back and forth; therefore, a new three-dimensional spatial positioning method is urgently needed. Summary of the invention

[0004] The purpose of this application is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0005] To this end, an object of an embodiment of the present application is to provide a three-dimensional spatial positioning method, system, device and storage medium, which can reduce the error of three-dimensional spatial positioning of human bones and improve the accuracy of three-dimensional spatial positioning.

[0006] In order to achieve the above-mentioned technical objectives, the technical scheme adopted in the embodiment of the present application includes: obtaining the first posture data of inertial motion capture of the human skeleton; determining the motion state of the human virtual model based on the first posture data; determining that the human virtual model is in the first motion state, and obtaining the first position information of the stationary bone node of the human virtual model; determining the second position information of the target bone node of the human virtual model based on the first position information and the first posture data; determining that the human virtual model is in the second motion state, and obtaining the speed information of the moving bone node of the human virtual model; the speed information includes gravity-free acceleration and motion speed; determining the third position information of the target bone node of the human virtual model based on the speed information; determining the target position information of the target bone node based on the second position information and the third position information; determining the position information of all nodes of the human skeleton based on the target position information.

[0007] In addition, the three-dimensional space positioning method according to the above embodiment of the present invention may also have the following additional technical features:

[0008] Further, in the embodiment of the present application, the step of determining the second position information of the target bone node of the human virtual model according to the first position information and the first posture data specifically includes: determining the first posture matrix according to the first posture data; determining the second position information of the target bone node of the human virtual model according to the first position information, the first posture matrix and the displacement iteration formula; wherein the displacement iteration formula includes:

[0009] Pos[j]=Pos[i]-ROTM[i]×(IntP[i]-IntP[j]);

[0010] In the above formula, IntP[i] and IntP[j] represent the original coordinate information of the virtual model skeleton node, Pos[i] is the first position information of the stationary skeleton node, Pos[j] is the second position information of the target skeleton node; ROTM[i] is the first posture matrix.

[0011] Further, in the embodiment of the present application, the step of determining the first posture matrix according to the first posture data specifically includes: performing posture calibration on the first posture data to obtain second posture data; determining the first posture matrix according to the second posture data and a matrix calculation formula; wherein the matrix calculation formula is

[0012]

[0013] In the above matrix calculation formula, Q[j] represents the calibrated second posture data, ROTM[j] represents the corresponding posture matrix, and j represents the serial number of the key bone of the virtual model.

[0014] Further, in the embodiment of the present application, the step of determining the third position information of the target bone node of the human virtual model according to the speed information specifically includes: obtaining the speed information of the moving bone node in two consecutive frames of data; the speed information includes gravity-free acceleration and movement speed; according to the speed information and the node position formula, obtaining the third position information of the target bone node; wherein the node position formula is

[0015]

[0016] In the above formula, acc[i] is the gravity-free acceleration of the current frame, _acc[i] is the gravity-free acceleration of the previous frame, Vel[i] is the speed of the current frame, _Vel[i] is the speed of the previous frame, the initial moment _Vel[i] is 0, Δt is the time of one frame, _Pos[i] is the position information of the target node of the previous frame, and Pos[i] is the position information of the target node of the current frame.

[0017] Furthermore, in the embodiment of the present application, the steps are also included: determining that the human virtual model is in a third motion state, and adjusting the position information of all nodes of the human skeleton.

[0018] Further, in the embodiment of the present application, the step of performing posture calibration on the first posture data to obtain the second posture data specifically includes: obtaining the first posture data; obtaining the second posture data according to the first posture data and the calibration formula; wherein the calibration formula is

[0019] Q[j]=CaliM[j]×Q int [j];

[0020] In the above formula, Q int [j] represents the initial posture data output by the inertial motion capture module, and CaliM[j] represents the calibration matrix of the inertial motion capture module.

[0021] Furthermore, in an embodiment of the present application, the step of adjusting the position information of all nodes of the human skeleton specifically includes: determining the height difference of the changes in the nodes of the human skeleton; and adjusting the position information of all nodes of the human skeleton according to the height difference.

[0022] On the other hand, the embodiment of the present application also provides a three-dimensional space positioning system, including:

[0023] A first acquisition unit, for first posture data of inertial motion capture of human skeleton;

[0024] A first processing unit, configured to determine a motion state of the human virtual model according to the first posture data;

[0025] A second acquisition unit, used to acquire first position information of a stationary skeletal node of the human virtual model;

[0026] A second processing unit, used for determining second position information of a target bone node of the human virtual model according to the first position information and the first posture data;

[0027] A third acquisition unit is used to acquire speed information of motion bone nodes of the human virtual model;

[0028] A third processing unit, used to determine third position information of a target skeleton node of the human virtual model according to the speed information;

[0029] a fourth processing unit, configured to determine target position information of a target skeletal node according to the second position information and the third position information;

[0030] The fifth processing unit is used to determine the position information of all nodes of the human skeleton according to the target position information.

[0031] On the other hand, the present application also provides a three-dimensional space positioning device, comprising:

[0032] at least one processor;

[0033] at least one memory for storing at least one program;

[0034] When the at least one program is executed by the at least one processor, the at least one processor implements a three-dimensional space positioning method as described in any one of the invention contents.

[0035] In addition, the present application also provides a storage medium, which stores processor-executable instructions, and the processor-executable instructions are used to execute a three-dimensional space positioning method as described in any of the above items when executed by the processor.

[0036] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0037] The present application can determine the different motion states of a human virtual model based on the first posture data of inertial motion capture of the human skeleton, determine the position information of the final target node based on the position information of the target node obtained under different motion states, and then determine the position information of all nodes of the human skeleton through the position information of the target node, thereby reducing the error of three-dimensional spatial positioning of the human skeleton and improving the accuracy of three-dimensional spatial positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the steps of a three-dimensional space positioning method in a specific embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of the structure of an inertial motion capture system in a specific embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the steps of determining the second position information of the target bone node of the human virtual model according to the first position information and the first posture data in a specific embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the steps of determining a first posture matrix according to the first posture data in a specific embodiment of the present invention;

[0042] Figure 5 A schematic diagram of the steps of performing posture calibration on the first posture data to obtain second posture data in a specific embodiment of the present invention;

[0043] Figure 6 A schematic diagram of a step of determining the third position information of a target skeleton node of a human virtual model according to the speed information in a specific embodiment of the present invention;

[0044] Figure 7 It is a structural schematic diagram of a three-dimensional space positioning system in a specific embodiment of the present invention;

[0045] Figure 8 It is a structural schematic diagram of a three-dimensional space positioning device in a specific embodiment of the present invention;

[0046] Fig. 9 It is a waveform diagram of zero speed information in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0047] The embodiments of the present invention are described in detail below with reference to the accompanying drawings to illustrate the principles and processes of the three-dimensional space positioning method, system, device and storage medium in the embodiments of the present invention.

[0048] Reference Figure 1 The present invention provides a three-dimensional space positioning method, comprising the following steps:

[0049] S1, obtaining the first posture data of human skeleton inertial motion capture;

[0050] In some embodiments of the present application, an inertial motion capture module may be worn at 17 key parts of the human body, numbered 1 to 17. Figure 2As shown in the figure: each inertial motion capture module is composed of a 9-axis sensor (gyroscope, accelerometer, magnetometer) and a microprocessor. The microprocessor of the inertial motion capture module is connected to the 9-axis sensor respectively, and is used for calibrating the 9-axis sensor and performing 9-axis data fusion posture solution to generate the first posture data; and the first posture data can be transmitted to the controller module; the controller module includes a battery, a data communication module and a microprocessor. The battery supplies power to the entire inertial motion capture hardware, and the microprocessor synthesizes the posture data transmitted by each inertial motion capture module into a frame, which is sent to the data terminal by the data communication module. After calibration according to the sent posture data, the data terminal can display the wearer's various movements in real time through a virtual model, and can also save the movement data; the parts worn are: waist, right thigh, right calf, right foot, left thigh, left calf, left foot, back, head, right shoulder, right upper arm, right lower arm, right hand, left shoulder, left upper arm, left lower arm and left hand. For pedestrian navigation, it is usually the feet that move, so you only need to wear inertial motion capture modules 1-7 on the waist, right thigh, right calf, right foot, left thigh, left calf and left foot respectively.

[0051] S2, determining the motion state of the human body virtual model according to the first posture data;

[0052] In some embodiments of the present application, data from the lower limb inertial motion capture module during walking can be collected, and data from other inertial motion capture modules can also be collected, and the gait characteristics of the pedestrians can be statistically analyzed. Through the output of the neural network model, different motion states of the human virtual model can be obtained, where the motion state may include but is not limited to a static state, a state in which the left and right feet leave the ground at the same time, a state in which the left and right feet move forward alternately, and a state in which the left and right feet leave the ground at the same time with a height difference greater than a certain level.

[0053] S3, determining that the human virtual model is in a first motion state, and obtaining first position information of a stationary skeletal node of the human virtual model;

[0054] In some embodiments of the present application, the first motion state refers to a state in which the human body is in a state in which the left and right feet alternately move forward. In combination with a specific application scenario, when the human body moves forward, at a certain moment, its stationary bone node may be the left foot or the right foot, and the first position information may correspond to the position information of the stationary node in three-dimensional coordinates. Therefore, when it is determined that the human body virtual model is in forward motion, the position information of the bone node may be the left foot or the right foot. The specific acquisition method includes but is not limited to wireless transmission and wired transmission of data between the device and the host computer, etc.

[0055] S4, determining second position information of a target skeletal node of the human body virtual model according to the first position information and the first posture data;

[0056] In some embodiments of the present application, a corresponding posture matrix can be obtained based on the first posture data of each bone segment of the human skeleton, and the second position information can be obtained based on the first position information and the posture matrix; in the embodiment of the present application, the second position information is the position information of the target bone node obtained through the static nodes of the human skeleton and the posture matrix, and any node can be selected as the target bone node among the human skeleton nodes, and correspondingly, the position information of any static node can also be selected as the first position information. In order to facilitate subsequent calculations, the target node can select the node with the most connections to other nodes, that is, the waist node as the preferred target node, and the static node can select the static node in the lower limbs of the human skeleton.

[0057] S5, determining that the human body virtual model is in a second motion state, and obtaining velocity information of motion bone nodes of the human body virtual model; the velocity information includes gravity-free acceleration and motion velocity;

[0058] In some embodiments of the present application, the second motion state may refer to a motion state in which the left and right feet of the human body leave the ground at the same time, such as a jumping state. After determining that the human virtual model is in a motion state in which the left and right feet of the human body leave the ground at the same time, the speed information of the motion bone nodes of the human virtual model can be obtained. In the motion state in which the left and right feet of the human body leave the ground at the same time, the entire human skeleton can be a motion bone node, such as the speed information of the left and right foot nodes can be obtained, and the speed information of the waist node can also be obtained, wherein the speed information may include gravity-depleted acceleration and motion speed, and the gravity-depleted acceleration and motion speed can be obtained by collecting data recorded by the inertial motion capture module.

[0059] S6, determining third position information of a target skeleton node of the human virtual model according to the speed information;

[0060] In some embodiments of the present application, the third position information of the target bone node of the human body virtual model can be determined by the speed information of two consecutive frames of the motion node; the third position information can be the position information of the target node calculated by collecting the speed information of two consecutive frames of the target node by the inertial motion capture module; the third position information can be the position information of any target node, and the target node can be any node of the target bone node of the human body virtual model. In order to subsequently solve the position of other nodes of the human skeleton, the waist node can be selected as the target node. The waist node has the most connections with other nodes of the human skeleton, and the position information of the nodes of the upper limbs of the human body can be solved by the waist node.

[0061] S7, determining target position information of a target skeletal node according to the second position information and the third position information;

[0062] In some embodiments of the present application, after obtaining the second position information of the target bone node through the human skeleton static nodes and the posture matrix and the target node position information calculated by collecting the speed information of two consecutive frames of the target node through the inertial motion capture module, more accurate target position information can be obtained through weighted fusion and filtering processing.

[0063] S8. Determine the position information of all nodes of the human skeleton according to the target position information.

[0064] In some embodiments of the present application, the target position information of the target node is obtained, and the position information of other nodes of the human skeleton can be determined through a preset algorithm, thereby obtaining the position information of all skeletal nodes of the human body.

[0065] Further, refer to Figure 3 The step of determining the second position information of the target skeleton node of the human virtual model according to the first position information and the first posture data may specifically include:

[0066] S101, determining a first posture matrix according to the first posture data;

[0067] S102, determining second position information of a target bone node of the human body virtual model according to the first position information, the first posture matrix and the displacement iteration formula;

[0068] The displacement iteration formula includes:

[0069] Pos[j]=Pos[i]-ROTM[i]×(IntP[i]-IntP[j]);

[0070] In the above formula, IntP[i] and IntP[j] represent the original coordinate information of the virtual model skeleton node, Pos[i] is the first position information of the static skeleton node, Pos[j] is the second position information of the target skeleton node; ROTM[i] is the first posture matrix;

[0071] Specifically, in some embodiments of the present application, a first posture matrix can be determined according to the first posture data, and the first matrix and the first position information can be input into a displacement iteration formula to determine the second position information of the target node, wherein the displacement iteration formula is:

[0072] Pos[j]=Pos[i]-ROTM[i]×(IntP[i]-IntP[j])

[0073] Among them, IntP[i] and IntP[j] represent the original coordinate information of the virtual model skeleton node. In the present application, the original coordinate information of the virtual model skeleton node is the coordinate information of the virtual model determined at the beginning, Pos[i] is the first position information of the skeleton node corresponding to the previous frame in the continuous frame, and Pos[j] is the position information of the nodes other than Pos[i] in the current frame determined according to Pos[i]; ROTM[i] is the first posture matrix, specifically, taking the previous frame as the left foot moving and the right foot still; the motion state of the current frame is the left foot still and the right foot moving as an example; at this time, the position information of the left foot as the starting node in the current frame is the position information obtained after calculating with the right foot as the starting node in the previous frame, and the right foot position information inferred in the current frame can be used in subsequent frames to infer the position information of the left foot in subsequent frames with this as the node.

[0074] Further, refer to Figure 4 The step of determining the first posture matrix according to the first posture data may specifically include:

[0075] S201, performing posture calibration on the first posture data to obtain second posture data;

[0076] S202, determining a first posture matrix according to the second posture data and a matrix calculation formula; wherein the matrix calculation formula is:

[0077]

[0078] In the above matrix calculation formula, Q[j] represents the calibrated second posture data, ROTM[j] represents the corresponding posture matrix, and j represents the serial number of the key bone of the virtual model;

[0079] Specifically, in the application embodiment, after obtaining the first posture data, the first posture data needs to be calibrated to obtain the calibrated second posture data. The second posture data is input into the calculation matrix to obtain the first posture matrix, and the first posture matrix can be used to determine the position information of the target node.

[0080] Further, refer to Figure 5 The step of performing posture calibration on the first posture data to obtain the second posture data specifically includes:

[0081] S301, obtaining first posture data;

[0082] S302, obtaining second posture data according to the first posture data and a calibration formula; wherein the calibration formula is:

[0083] Q[j]=CaliM[j]×Q int [j];

[0084] In the above formula, Q int [j] represents the initial posture data output by the inertial motion capture module, and CaliM[j] represents the calibration matrix of the inertial motion capture module;

[0085] Specifically, during the calibration process, the first posture data obtained needs to be input into the calibration formula, and the calibration formula is:

[0086] Q[j]=CaliM[j]×Q int [j];

[0087] Among them, Q int [j] represents the initial posture data output by the inertial motion capture module, and CaliM[j] represents the calibration matrix of the inertial motion capture module. Both data can be obtained by setting the parameters of the inertial motion capture module in advance.

[0088] Further, refer to Figure 6 The step of determining the third position information of the target skeleton node of the human virtual model according to the speed information may specifically include:

[0089] S401, obtaining speed information of moving bone nodes in two consecutive frames of data; the speed information includes gravity-free acceleration and movement speed;

[0090] S402, obtaining third position information of the target bone node according to the speed information and the node position formula;

[0091] Among them, the node position formula is:

[0092]

[0093] In the above formula, acc[i] is the gravity-free acceleration of the current frame, _acc[i] is the gravity-free acceleration of the previous frame, Vel[i] is the speed of the current frame, _Vel[i] is the speed of the previous frame, the initial moment _Vel[i] is 0, Δt is the time of one frame, _Pos[i] is the node position of the previous frame, and Pos[i] is the node position of the current frame.

[0094] Specifically, in the embodiment of the present application, the velocity information of the moving bone node in two consecutive frames of data including the current frame can be obtained through the inertial motion capture module, wherein the velocity information may include the gravity-free acceleration and the motion speed, and the third position information of the target bone node can be obtained according to the following formula;

[0095]

[0096] In the above formula, acc[i] is the gravity-free acceleration of the current frame in the continuous frame data, _acc[i] is the gravity-free acceleration of the previous frame in the continuous frame data, Vel[i] is the speed of the current frame, _Vel[i] is the speed of the previous frame, the initial moment _Vel[i] is 0, Δt is the time of one frame, __Pos[i] is the node position of the previous frame, and Pos[i] is the node position of the current frame.

[0097] Furthermore, the method further comprises the steps of: determining that the human virtual model is in a third motion state, and adjusting position information of all nodes of the human skeleton;

[0098] Specifically, since the human virtual model can be in a third motion state, the third motion state is a motion state in which the left and right feet leave the ground at the same time with a height difference greater than a certain level, such as the motion state of the human virtual model going up and down stairs. When the human virtual model is in the third motion state, the position information of all nodes of the human skeleton can be adjusted.

[0099] Furthermore, the step of adjusting the position information of all nodes of the human skeleton specifically includes:

[0100] S501, determining the height difference of the human skeleton nodes;

[0101] S502, adjusting the position information of all nodes of the human skeleton according to the height difference;

[0102] Specifically, the height difference of the changes in the human skeleton nodes can be determined through the inertial motion capture module, and the position information of all the human skeleton nodes can be adjusted according to the height difference. Specifically, if the height difference between the previous and next frames is a positive number, the height difference value will be increased in the position information of all the human skeleton nodes. If the height difference between the previous and next frames is a negative number, the height difference value will be increased in the position information of all the human skeleton nodes.

[0103] Taking the target node as a waist node as an example, the three-dimensional spatial positioning method of the present application is described below:

[0104] First, before three-dimensional spatial positioning, an inertial motion capture module is worn on 17 key parts of the human body, numbered 1 to 17. Each inertial motion capture module consists of a 9-axis sensor (gyroscope, accelerometer, magnetometer) and a microprocessor. The microprocessor of the inertial motion capture module is connected to the 9-axis sensor respectively, used for 9-axis sensor calibration and 9-axis data fusion posture solution, generates posture information, and transmits the posture data to the controller module; the controller module contains a battery, a data communication module and a microprocessor. The battery supplies power to the entire inertial motion capture hardware, and the microprocessor synthesizes the posture data transmitted by each inertial motion capture module into a frame, which is sent to the data terminal by the data communication module. After calibration, the data terminal can display the wearer's various movements in real time through a virtual model according to the sent posture data, and can also save the movement data; the parts worn are: waist, right thigh, right calf, right foot, left thigh, left calf, left foot, back, head, right shoulder, right upper arm, right lower arm, right hand, left shoulder, left upper arm, left lower arm and left hand. Specifically, for pedestrian navigation, both feet are usually in motion, and inertial motion capture modules 1-7 need to be worn on the waist, right thigh, right calf, right foot, left thigh, left calf and left foot respectively.

[0105] The specific steps to achieve pedestrian navigation are as follows:

[0106] Step 1: Posture output of inertial motion capture module;

[0107] Statistical analysis is performed on the 9-axis data of each inertial motion capture module in each orientation, and a least squares ellipse fitting model of zero bias and orthogonal factors is established. The inertial motion capture module is calibrated to obtain accurate 9-axis real-time theoretical values ​​for Kalman filter attitude estimation, and finally the attitude of the inertial motion capture module is output.

[0108] Step 2: virtual model posture calibration;

[0109] Due to the differences between wearers and the inconsistency of each wearing position, the posture data output by the inertial motion capture module needs to be calibrated. Only through calibration can the posture data of the inertial motion capture module drive the virtual model. The calibration formula is:

[0110] Q[j]=CaliM[j]×Q int [j]

[0111] Among them, Q int [j] represents the initial posture data output by each inertial motion capture module, CaliM[j] represents the calibration matrix of each inertial motion capture module, Q[j](Q 1 , Q 2 , Q 3 , Q 4) represents the posture data of a certain key bone of the virtual model, j represents the serial number of a certain key bone from 1 to 17, the bones of the virtual model are not limited to this, and can be expanded to a standard 23-segment bone. The quaternion of the middle bone can be obtained by interpolating the quaternions of two adjacent key bones.

[0112] Step 3: Acquisition of motion status;

[0113] By collecting data from the lower limb inertial motion capture module during walking, statistically analyzing the pedestrian's gait characteristics, and outputting them through the neural network model, we can obtain real-time and accurate zero-speed information, through which we can determine the motion state of the human virtual model. Fig. 9 As shown: ZUPT_RF indicates the right foot zero speed state, ZUPT_LF indicates the left foot zero speed state, when ZUPT_RF is 0, it indicates that the right foot is in motion, when ZUPT_RF is 1, it indicates that the right foot is in a stationary state, when ZUPT_LF is 0, it indicates that the left foot is in motion, when ZUPT_LF is 1, it indicates that the left foot is in a stationary state, at this time, the human virtual model is in the first motion state, as shown in time segments 2, 4, 6, 8 and 10 in the figure; when ZUPT_RF and ZUPT_LF are 1 at the same time, it indicates that the pedestrian's feet touch the ground, at this time, the human virtual model is in a short stationary state, as shown in Fig. 9 In the time segment 1, time segment 3, time segment 5, time segment 7, time segment 9 and time segment 12; when ZUPT_RF is 0 and ZUPT_LF is 0 at the same time, it means that the pedestrian's feet are off the ground, both left and right feet are moving, and they are in a jumping state, such as Fig. 9 Middle time segment 11; Generally, when pedestrians walk, their feet touch the ground at the same time for a short moment. Fig. 9 In time segments 3, 5, 7 and 9, the posture can be corrected by utilizing the process of both feet touching the ground.

[0114] Step 4: Three-dimensional spatial positioning solution;

[0115] According to each key bone posture data obtained in step 2, the corresponding posture matrix is ​​obtained. The formula is as follows:

[0116]

[0117] Among them, Q[j] represents the quaternion of each key bone posture of the virtual model, ROTM[j] represents the corresponding posture matrix, and j represents the serial number [1-17] of the key bone of the virtual model.

[0118] By using the zero-speed interval obtained in step 3, according to the kinematics of the rigid body, the position information of each joint point of the virtual model can be obtained. The initial position can be obtained by the following formula:

[0119] Pos[j]=Pos[i]-ROTM[i]×(IntP[i]-IntP[j])

[0120] The above formula is called the displacement iteration formula, where IntP[i] and IntP[j] represent the known virtual model skeleton node coordinate information (initial node). The model skeleton information can be manually edited to be equivalent to the real wearer. Pos[i] and Pos[j] represent the position information of each joint node of the real-time driven virtual model. i represents the serial number of the previous joint (parent node), and j represents the serial number of the next joint (child node).

[0121] When ZUPT_RF is 1 and ZUPT_LF is 1, both feet are stationary. Select the left foot or the right foot as the initial solution root node. If the right foot is selected as the initial solution root node, assign Pos[4] of the previous frame to Pos[4] of the current frame, i value is [4, 3, 2, 1, 5, 6], j value is [3, 2, 1, 5, 6, 7], and complete the solution from the right foot node to the right calf node, then from the right calf node to the right thigh node, and then from the right thigh node to the waist node. , then from the waist node to the left thigh node, then from the left thigh node to the left calf node, and then from the left calf node to the left foot node, thus completing the node coordinate solution of the entire motion chain; you can also choose the left foot as the initial solution root node, assign the Pos[7] of the previous frame to the Pos[7] of the current frame, the i value is [7, 6, 5, 1, 2, 3], and the j value is [6, 5, 1, 2, 3, 4] to complete the node coordinate solution of the entire motion chain from the left foot to the right foot, or perform fusion solution in both directions at the same time.

[0122] When ZUPT_RF is 1 and ZUPT_LF is 0, the right foot is stationary and the left foot is in motion. The right foot is used as the initial solution root node, and the Pos[4] of the previous frame of the continuous frame is assigned to the Pos[4] of the current frame. Then the i value is [4, 3, 2, 1, 5, 6], and the j value is [3, 2, 1, 5, 6, 7]. The solution is completed from the right foot node to the right calf node, then from the right calf node to the right thigh node, then from the right thigh node to the waist node, then from the waist node to the left thigh node, then from the left thigh node to the left calf node, and then from the left calf node to the left foot node, thus completing the node coordinate solution of the entire chain. Calculate; or, when ZUPT_RF is 0 and ZUPT_LF is 1, the right foot is in motion and the left foot is stationary. The left foot is used as the initial calculation root node, and the Pos[7] of the previous frame is assigned to the Pos[7] of the current frame. Then the i value is [7, 6, 5, 1, 2, 3], and the j value is [6, 5, 1, 2, 3, 4]. The calculation is performed from the left foot node to the left calf node, and then from the left calf node to the left thigh node, and then from the left thigh node to the waist node, and then from the waist node to the right thigh node, and then from the right thigh node to the right calf node, and then from the right calf node to the right foot node, thereby completing the node coordinate calculation of the entire chain.

[0123] When ZUPT_RF is 0 and ZUPT_LF is 0, both feet are in motion, and the three-dimensional spatial positions of the left and right feet are calculated respectively. The formula is as follows:

[0124]

[0125] Among them, acc[i] is taken from the output of the inertial motion capture module, which represents the gravity-free acceleration of the current frame. The value of i is 1 for the waist, 4 for the right foot, and 7 for the left foot. _acc[i] represents the gravity-free acceleration of the previous frame. Vel[i] represents the speed of the current frame, _Vel[i] represents the speed of the previous frame, and the initial moment _Vel[i] is 0. Δt represents the time of one frame. _Pos[i] represents the node position of the previous frame, and Pos[i] represents the node position of the current frame. Due to the existence of the integral error of the inertial motion capture module, the information of the seven nodes of the left and right legs is further integrated. The position of the left foot is iterated to the waist node according to the displacement iteration formula to obtain the waist position information Pos1[1]. Then, the position of the right foot is iterated to the waist node according to the displacement iteration formula to obtain the waist position information Pos2[1]. Then, the three information of Pos[1], Pos1[1] and Pos2[1] are weighted, fused and filtered to obtain more accurate position information. Finally, the accurate position information of the waist is used to obtain the position information of all nodes of the human skeleton.

[0126] Step 5: Three-dimensional space positioning correction;

[0127] While performing the three-dimensional spatial positioning solution in step 4 above, the characteristics of the height difference ΔH between the lowest points of the left and right feet when the pedestrian changes feet during walking are judged to distinguish whether the pedestrian is walking on a flat surface or going up and down stairs; if ΔH is less than the threshold β when changing feet, it means walking on a flat surface, and the ground height H ground No update is done; on the contrary, when going up or down stairs, determine whether it is going up or down stairs. If it is going up stairs, update the ground height H ground =H ground +H stair , H stair is the step height; if it is downstairs, update the ground height H ground =H ground -H stair In addition, when going up and down stairs, the height H of the steps is also stair The system performs paired statistics based on the number of stairs and the number of steps, so that when going up or down a certain stair, the vertical height information of the three-dimensional space when changing feet can be accurately controlled according to the recorded information of the stair.

[0128] In addition, refer to Figure 7 ,and Figure 1 Corresponding to the method, an embodiment of the present application also provides a three-dimensional space positioning system, including: a first acquisition unit 1001, which is used for first posture data of inertial motion capture of human skeleton; a first processing unit 1002, which is used to determine the motion state of a human virtual model according to the first posture data; a second acquisition unit 1003, which is used to acquire first position information of a stationary skeleton node of the human virtual model; a second processing unit 1004, which is used to determine second position information of a target skeleton node of the human virtual model according to the first position information and the first posture data; a third acquisition unit 1005, which is used to acquire speed information of a moving skeleton node of the human virtual model; a third processing unit 1006, which is used to determine third position information of a target skeleton node of the human virtual model according to the speed information; a fourth processing unit 1007, which is used to determine target position information of a target skeleton node according to the second position information and the third position information; a fifth processing unit 1008, which is used to determine position information of all nodes of the human skeleton according to the target position information.

[0129] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0130] and Figure 8 Corresponding to the method, the embodiment of the present application further provides a three-dimensional space positioning device, the specific structure of which can be referred to Figure X, including:

[0131] at least one processor 1011;

[0132] At least one memory 1012, used to store at least one program;

[0133] When the at least one program is executed by the at least one processor, the at least one processor implements the three-dimensional space positioning method.

[0134] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0135] and Figure 1 Corresponding to the method, an embodiment of the present application further provides a storage medium, in which processor-executable instructions are stored, and the processor-executable instructions are used to execute the three-dimensional space positioning method when executed by the processor.

[0136] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are expected, wherein the order of various operations is changed and the sub-operation described as a part of a larger operation is performed independently.

[0137] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several programs to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable programs for implementing the logical functions, and may be embodied in any computer-readable medium for use by a program execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch and execute a program from a program execution system, device or apparatus), or in conjunction with such program execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by a program execution system, device or apparatus, or in conjunction with such program execution systems, devices or apparatuses.

[0140] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0141] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0143] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0144] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments. Technical personnel familiar with the field may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A three-dimensional spatial positioning method, It is characterized in that The following steps are involved: Acquire the first posture data of human skeleton inertial motion capture; Determine the motion state of the human body virtual model according to the first posture data; Determine that the human virtual model is in a first motion state, and obtain first position information of a stationary skeletal node of the human virtual model; Determine second position information of a target skeletal node of the human virtual model according to the first position information and the first posture data; This step includes: Determine a first posture matrix according to the first posture data; Determine second position information of a target bone node of the human virtual model according to the first position information, the first posture matrix and the displacement iteration formula; The displacement iteration formula includes: ；（1） In formula (1), and Represents the original coordinate information of the virtual model skeleton node. is the first position information of the static bone node, The second position information of the target bone node; is the first posture matrix; According to the first posture data, the step of determining a first posture matrix comprises: Acquire first posture data; According to the first posture data and the calibration formula, the second posture data is obtained; the calibration formula is: ；（2） In formula (2), Represents the initial posture data output by the inertial motion capture module. Represents the calibration matrix of the inertial motion capture module; According to the second posture data and the matrix calculation formula, the first posture matrix is ​​determined; the matrix calculation formula is: （3） In formula (3), [j] represents the second posture data after calibration, represents the corresponding posture matrix, j represents the serial number of the key bone of the virtual model, and i represents the serial number of the parent node corresponding to the key bone j in the virtual model; Determine that the human virtual model is in a second motion state, and obtain velocity information of motion bone nodes of the human virtual model; the velocity information includes gravity-deducted acceleration and motion velocity; Determine the third position information of the target bone node of the human virtual model according to the speed information; this step includes: obtaining the speed information of the moving bone node in two consecutive frames of data; the speed information includes the acceleration due to gravity and the moving speed; Obtaining third position information of the target bone node according to the speed information and the node position formula; Among them, the node position formula is: （4） In formula (4), is the gravity-free acceleration of the current frame, Indicates the gravity-free acceleration of the previous frame, Indicates the speed of the current frame. Indicates the speed of the previous frame, the initial time is 0, Indicates the time of one frame. Indicates the location information of the target node in the previous frame. Indicates the location information of the target node in the current frame; Determine target position information of the target bone node according to the second position information and the third position information; According to the target position information, the position information of all nodes of the human skeleton is determined.

2. A three-dimensional space positioning method according to claim 1, It is characterized in that The method also includes the steps of: determining that the human virtual model is in a third motion state, and adjusting position information of all nodes of the human skeleton.

3. A three-dimensional space positioning method according to claim 2, It is characterized in that The step of adjusting the position information of all nodes of the human skeleton specifically includes: Determine the height difference of the human skeleton node changes; According to the height difference, the position information of all nodes of the human skeleton is adjusted.

4. A three-dimensional spatial positioning system, It is characterized in that For implementing the three-dimensional space positioning method according to any one of claims 1 to 3, the system comprises: A first acquisition unit, for first posture data of inertial motion capture of human skeleton; A first processing unit, configured to determine a motion state of the human virtual model according to the first posture data; A second acquisition unit, used to acquire first position information of a stationary skeletal node of the human virtual model; The second processing unit is used to determine the second position information of the target bone node of the human virtual model according to the first position information and the first posture data; the determining the second position information of the target bone node of the human virtual model according to the first position information and the first posture data comprises: Determine a first posture matrix according to the first posture data; Determine second position information of a target bone node of the human virtual model according to the first position information, the first posture matrix and the displacement iteration formula; The displacement iteration formula includes: ；（1） In formula (1), and Represents the original coordinate information of the virtual model skeleton node. is the first position information of the static bone node, The second position information of the target bone node; is the first posture matrix; According to the first posture data, the step of determining a first posture matrix comprises: Acquire first posture data; According to the first posture data and the calibration formula, the second posture data is obtained; the calibration formula is: ；（2） In formula (2), Represents the initial posture data output by the inertial motion capture module. Represents the calibration matrix of the inertial motion capture module; According to the second posture data and the matrix calculation formula, the first posture matrix is ​​determined; the matrix calculation formula is: （3） In formula (3), [j] represents the second posture data after calibration, represents the corresponding posture matrix, j represents the serial number of the key bone of the virtual model, and i represents the serial number of the parent node corresponding to the key bone j in the virtual model; A third acquisition unit is used to acquire speed information of motion bone nodes of the human virtual model; A third processing unit is used to determine the third position information of the target bone node of the human virtual model according to the speed information; the determining the third position information of the target bone node of the human virtual model according to the speed information includes: obtaining the speed information of the moving bone node in two consecutive frames of data; the speed information includes the gravity-reduced acceleration and the moving speed; Obtaining third position information of the target bone node according to the speed information and the node position formula; Among them, the node position formula is: （4） In formula (4), is the gravity-free acceleration of the current frame, Indicates the gravity-free acceleration of the previous frame, Indicates the speed of the current frame. Indicates the speed of the previous frame, the initial time is 0, Indicates the time of one frame. Indicates the location information of the target node in the previous frame. Indicates the location information of the target node in the current frame; a fourth processing unit, configured to determine target position information of a target skeletal node according to the second position information and the third position information; The fifth processing unit is used to determine the position information of all nodes of the human skeleton according to the target position information.

5. A three-dimensional spatial positioning device, Features include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a three-dimensional space positioning method as described in any one of claims 1-3.

6. A storage medium having stored therein instructions executable by a processor, It is characterized in that The processor-executable instructions are used to execute a three-dimensional space positioning method as described in any one of claims 1-3 when executed by the processor.

Images