Human body posture estimation method and device based on progressive particle filter and medium

Abstract

The invention belongs to the field of human body posture estimation, and discloses a human body posture estimation method and device based on progressive particle filtering and a medium, and the method comprises the steps: 1, initializing parameters of a human body posture estimation system, and determining a state vector of the human body posture estimation system; 2, taking a state transfer function in the state space model as suggested distribution, and sampling prediction particles; step 3, updating progressive measurement, and calculating the weight in the progressive process; step 4, performing resampling to obtain a new particle set; 5, judging whether the measured fusion improves the estimation performance of the filter or not; step 6, obtaining a state vector and a covariance thereof according to the particle set; and 7, repeatedly executing the steps 2-6 to complete state estimation of each joint point of the human body so as to obtain human body posture estimation based on the progressive particle filter. According to the method, the particle weight at the progressive moment is obtained through progressive measurement updating, the particle degradation problem is solved, and therefore the filtering precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of human body posture estimation, and in particular to a human body posture estimation method, device and medium based on progressive particle filtering. Background Art

[0002] Human posture estimation is to estimate the posture of the human body by connecting the joints of the human body, so as to further judge the state and behavior of the human body. With the development of technologies such as artificial intelligence and 3D vision, human posture estimation based on visual sensors has been applied to fields such as robotics and autonomous driving. For example, it can analyze the movements of pedestrians, avoid collisions between vehicles and pedestrians, and ensure the safety of vehicles and pedestrians.

[0003] In practical applications, the complex structure of the human body and the changing environmental factors make the human posture estimation system nonlinear and non-Gaussian. For such a system, the Kalman filter and its variants are difficult to obtain good estimation results. Particle filtering, combining Monte Carlo thinking and Bayesian filtering, is a widely used nonlinear non-Gaussian filter, but there is a problem of particle degradation. Affected by the surrounding lighting and obstructions, visual sensors are prone to measurement anomalies. The emergence of outliers will aggravate particle degradation, resulting in the performance of particle filters. Even divergence. Existing methods alleviate the particle degradation problem by improving the proposed distribution, such as extended Kalman particle filtering and auxiliary particle filtering. However, existing methods do not consider outliers in the proposed distribution, resulting in the inability to obtain good estimation accuracy and robustness. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device and medium for human posture estimation based on progressive particle filtering to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A human body posture estimation method based on progressive particle filtering, comprising:

[0007] Step 1, construct a human posture estimation system, initialize the parameters of the human posture estimation system, initialize the particle set, and determine the state vector of the human posture estimation system;

[0008] Step 2, taking the state transition function in the state space model as the proposed distribution and sampling the predicted particles;

[0009] Step 3: Progressive measurement update, calculate the weights in the progressive process;

[0010] Step 4, resampling to obtain a new particle set;

[0011] Step 5, determine whether the integration of the measurement improves the estimation performance of the filter, if yes, go to step 6, otherwise go back to step 3;

[0012] Step 6, obtain the state vector and its covariance according to the particle set;

[0013] Step 7, repeat steps 2 to 6 to complete the state estimation of each joint of the human body, and obtain the human body posture estimation based on progressive particle filtering.

[0014] Furthermore, the step 1 comprises:

[0015] Establish a nonlinear state space model for the human posture estimation system, which includes the kinematic model of each joint of the human body and the measurement model based on visual sensors; determine the process noise covariance , measurement noise covariance , the initial state of the human joints and covariance And the initial particle set , where n is the serial number of the human joint point, is the number of particles.

[0016] Furthermore, the kinematic model of each joint of the human body established in step 1 is:

[0017] (1)

[0018] in, is a discrete time series, is the state vector of the human body posture estimation system at human joint point n at time k, , , and Respectively represent the human joint point n in the visual sensor coordinate system axis, Axis and The position on the axis, T is the transposed matrix, For human joint point n Time has come The state function at time t, process noise It follows a Gaussian distribution with a mean of zero and a covariance of ;

[0019] The measurement model based on visual sensors is described as:

[0020] (2)

[0021] in, is the three-dimensional identity matrix, is the measurement information of human joint point n at time k, is the measurement information of human joint point n at time k, , and They are the human joint point n on the visual sensor axis, Axis and Measurement values ​​on axis, measurement noise It follows a Gaussian distribution with a mean of zero and a covariance of , the initial state and covariance of the human joint point n are and , and is independent of each noise.

[0022] Furthermore, the step 2 comprises:

[0023] In the progressive particle filter, the state transfer function As a proposal distribution in particle filtering Importance sampling is performed; according to the kinematic model of each joint of the human body (1) and All particles related to the human joint point n at the moment , the importance sampling process is expressed as:

[0024] (3)

[0025] get Particles related to human joint point n at moment , according to the obtained particles and The weights of all particles related to the human joint point n at the moment , calculate the state prediction value of human joint point n and its corresponding covariance :

[0026] (4)

[0027] (5).

[0028] Furthermore, the step 3 comprises:

[0029] Decompose the original measurement update process into A gradual measurement update process is used to construct the covariance of the measurement noise of the human joint point n Asymptotic progression from infinity to the true covariance:

[0030] (6)

[0031] in, is the asymptotic covariance of the measurement noise of human joint point n in the jth step of the asymptotic measurement update process, is a small constant, is a progressive parameter that controls the integration of measurement information. , an extreme case is , indicating that there is no measurement information at present; the other extreme case is , indicating that all current measurement information has been integrated;

[0032] In the process of gradual measurement update, the gradual step length is defined as , and , the asymptotic likelihood function It is expressed as:

[0033] (7)

[0034] in, Represents an exponential distribution.

[0035] Furthermore, the step 4 comprises:

[0036] According to the asymptotic likelihood function in the asymptotic measurement update process, the corresponding weight of each particle is calculated , and normalize it; then resample the particles according to the normalized weights to obtain the particle set of each human joint point at time k :

[0037] (8)

[0038] (9)

[0039] in, represents the unnormalized weight of the ith particle at the human joint point n at the time k in the jth step of the progressive process, represents the normalized weight of the ith particle at the human joint point n at time k in the jth step of the progressive process, represents the sum of all weights.

[0040] Furthermore, the step 5 comprises:

[0041] Design the cutoff condition of the gradual measurement update process. In the gradual measurement update process, determine whether the integration of the j-th post-measurement improves the estimation performance of the filter. If the result is yes, execute step 6; if the result is no, execute step 3. The cutoff condition of the gradual measurement update process is designed as:

[0042] (10)

[0043] (11)

[0044] in, is a small constant, , Human joints In the The estimation error in the sub-asymptotic measurement process, is the state estimate at the current asymptotic moment, , Human joints In the The deviation between the predicted value and the actual value during the asymptotic measurement process.

[0045] Furthermore, the step 6 comprises:

[0046] The particle set at the current asymptotic moment is taken as The particle set at the moment, and then the state estimate of the current asymptotic moment is obtained and its covariance :

[0047] (12)

[0048] (13).

[0049] The present invention also provides a human body posture estimation device based on progressive particle filtering, comprising one or more processors for implementing the human body posture estimation method based on progressive particle filtering as described above.

[0050] The present invention also provides a readable storage medium on which a program is stored. When the program is executed by a processor, the method for estimating human body posture based on progressive particle filtering as described above is implemented.

[0051] Compared with the prior art, the invention has the following beneficial effects: the invention improves the particle degradation problem by updating the particle weights at the progressive moment through progressive measurement, thereby improving the filtering accuracy. Moreover, the invention determines whether the current progressive moment is affected by an outlier through a cutoff condition, thereby effectively improving the robustness of the filter. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of the present invention.

[0053] Figure 2 Schematic diagram of the joints of the human upper limbs.

[0054] Figure 3 It is a structural schematic diagram of a human posture estimation device based on progressive particle filtering of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] See also Figure 1 and Figure 2 , a human body posture estimation method based on progressive particle filtering, comprising the following steps:

[0057] Step 1: Establish a nonlinear state space model of the human posture estimation system, which includes the kinematic model of each joint of the human body and the measurement model based on the visual sensor; determine the process noise covariance , measurement noise covariance , the initial state of the human joints and covariance And the initial particle set , where n is the serial number of the human joint point, is the number of particles. Figure 2 As shown, it describes the joints of the human body and the serial numbers of each joint, including 8 joints of the human body: hip center, head joint, shoulder center, left shoulder joint, left elbow joint, left wrist joint, right shoulder joint, right elbow joint, and right wrist joint.

[0058] The kinematic model of each joint of the human body is established as follows:

[0059] (1)

[0060] in, is a discrete time series, is the state vector of the human body posture estimation system at human joint point n at time k, , , and Respectively represent the human joint point n in the visual sensor coordinate system axis, Axis and The position on the axis, T is the transposed matrix, For human joint point n Time has come The state function at time t, process noise It follows a Gaussian distribution with a mean of zero and a covariance of ;

[0061] The measurement model based on visual sensors can be described as:

[0062] (2)

[0063] in, is the three-dimensional identity matrix, is the measurement information of human joint point n at time k, is the measurement information of human joint point n at time k, , and They are the human joint point n on the visual sensor axis, Axis and Measurement values ​​on axis, measurement noise It follows a Gaussian distribution with a mean of zero and a covariance of , the initial state and covariance of the human joint point n are and , and is independent of each noise.

[0064] Step 2: In the progressive particle filter, the state transfer function As a proposal distribution in particle filtering Importance sampling is performed; according to the kinematic model of each joint of the human body (1) and All particles related to the human joint point n at the moment , the importance sampling process is expressed as:

[0065] (3)

[0066] Thus, we get Particles related to human joint point n at moment According to the obtained particles and The weights of all particles related to the human joint point n at the moment , the state prediction value of human joint point n can be calculated and its corresponding covariance :

[0067] (4)

[0068] (5).

[0069] Step 3: Decompose the original measurement update process into A gradual measurement update process is used to construct the covariance of the measurement noise of the human joint point n Asymptotic progression from infinity to the true covariance:

[0070] (6)

[0071] in, is the asymptotic covariance of the measurement noise of human joint point n in the jth step of the asymptotic measurement update process, is a small constant, is a progressive parameter that controls the integration of measurement information. , an extreme case is , indicating that there is no measurement information at present; the other extreme case is , indicating that all current measurement information has been integrated;

[0072] In the process of gradual measurement update, the gradual step length is defined as , and , the asymptotic likelihood function It is expressed as:

[0073] (7)

[0074] in, is the state vector of human joint point n, is the measured value of human joint point n, Represents an exponential distribution.

[0075] Step 4: Calculate the corresponding weight of each particle according to the asymptotic likelihood function in the asymptotic measurement update process , and normalize it; then resample the particles according to the normalized weights to obtain the particle set of each human joint point at time k :

[0076] (8)

[0077] (9)

[0078] in, represents the unnormalized weight of the ith particle at the human joint point n at the time k in the jth step of the progressive process, represents the normalized weight of the ith particle at the human joint point n at time k in the jth step of the progressive process, represents the sum of all weights.

[0079] Step 5: Considering that the measurement may contain wild values, the cutoff condition of the gradual measurement update process is designed. In the gradual measurement update process, it is determined whether the integration of the j-th measurement improves the estimation performance of the filter. If the result is yes, then execute step 6; if the result is no, then execute step 3; the cutoff condition of the gradual measurement update process is designed as:

[0080] (10)

[0081] (11)

[0082] in, is a small constant, , Human joints In the The estimation error in the sub-asymptotic measurement process, is the state estimate at the current asymptotic moment, , Human joints In the The deviation between the predicted value and the actual measurement in the incremental measurement process. When the two judgment conditions are met at the same time, it means that there is no need to perform incremental iteration.

[0083] Step 6: Take the particle set at the current asymptotic moment as The particle set at the moment, and then the state estimate of the current asymptotic moment is obtained and its covariance :

[0084] (12)

[0085] (13).

[0086] Step 7, repeat steps 2 to 6 to complete the state estimation of each joint of the human body, and obtain the human body posture estimation based on progressive particle filtering.

[0087] See also Figure 3 An embodiment of the present invention provides a human body posture estimation device based on progressive particle filtering, including one or more processors, which are used to implement a human body posture estimation method based on progressive particle filtering in the above embodiment.

[0088] An embodiment of a human posture estimation device based on progressive particle filtering of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From a hardware perspective, if Figure 3 As shown in FIG. 1 , a hardware structure diagram of a human posture estimation device based on progressive particle filtering according to the present invention is provided in any device with data processing capability, except Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in the embodiments may also include other hardware, usually based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0089] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0090] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] An embodiment of the present invention further provides a readable storage medium on which a program is stored. When the program is executed by a processor, a human body posture estimation method based on progressive particle filtering in the above embodiment is implemented.

[0092] The readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0093] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A human body posture estimation method based on progressive particle filtering, characterized in that: include: Step 1, construct a human posture estimation system, initialize the parameters of the human posture estimation system, initialize the particle set, and determine the state vector of the human posture estimation system; Step 2, taking the state transition function in the state space model as the proposed distribution and sampling the predicted particles; Step 3: Progressive measurement update, calculate the weights in the progressive process; Step 4, resampling to obtain a new particle set; Step 5, determine whether the integration of the measurement improves the estimation performance of the filter, if yes, go to step 6, otherwise go back to step 3; Step 6, obtain the state vector and its covariance according to the particle set; Step 7, repeat steps 2 to 6 to complete the state estimation of each joint of the human body, and obtain the human body posture estimation based on progressive particle filtering.

2. A method for estimating human posture based on progressive particle filtering according to claim 1, characterized in that: The step 1 comprises: Establish a nonlinear state space model for the human posture estimation system, which includes the kinematic model of each joint of the human body and the measurement model based on visual sensors; determine the process noise covariance , measurement noise covariance , the initial state of the human joints and covariance And the initial particle set , where n is the serial number of the human joint point, is the number of particles.

3. A method for estimating human posture based on progressive particle filtering according to claim 2, characterized in that: The kinematic model of each joint point of the human body established in step 1 is: (1) in, is a discrete time series, is the state vector of the human body posture estimation system at human joint point n at time k, , , and Respectively represent the human joint point n in the visual sensor coordinate system axis, Axis and The position on the axis, T is the transposed matrix, For human joint point n Time has come The state function at time t, process noise It follows a Gaussian distribution with a mean of zero and a covariance of ; The measurement model based on visual sensors is described as: (2) in, is the three-dimensional identity matrix, is the measurement information of human joint point n at time k, is the measurement information of human joint point n at time k, , and They are the human joint point n on the visual sensor axis, Axis and Measurement value on axis, measurement noise It follows a Gaussian distribution with a mean of zero and a covariance of , the initial state and covariance of the human joint point n are and , and is independent of each noise.

4. A method for estimating human posture based on progressive particle filtering according to claim 2, characterized in that: The step 2 comprises: In the progressive particle filter, the state transfer function As a proposal distribution in particle filtering Importance sampling is performed; according to the kinematic model of each joint of the human body (1) and All particles related to the human joint point n at the moment , the importance sampling process is expressed as: (3) get Particles related to human joint point n at moment , according to the obtained particles and The weights of all particles related to the human joint point n at the moment , calculate the state prediction value of human joint point n and its corresponding covariance : (4) (5)。 5. A method for estimating human posture based on progressive particle filtering according to claim 1, characterized in that: The step 3 comprises: Decompose the original measurement update process into A gradual measurement update process is used to construct the covariance of the measurement noise of the human joint point n Asymptotic progression from infinity to the true covariance: (6) in, is the asymptotic covariance of the measurement noise of human joint point n in the jth step of the asymptotic measurement update process, is a small constant, is a progressive parameter that controls the integration of measurement information. , an extreme case is , indicating that there is no measurement information at present; the other extreme case is , indicating that all current measurement information has been integrated; In the process of gradual measurement update, the gradual step length is defined as , and , the asymptotic likelihood function It is expressed as: (7) in, Represents an exponential distribution.

6. A method for estimating human posture based on progressive particle filtering according to claim 1, characterized in that: The step 4 comprises: According to the asymptotic likelihood function in the asymptotic measurement update process, the corresponding weight of each particle is calculated , and normalize it; then resample the particles according to the normalized weights to obtain the particle set of each human joint point at time k : (8) (9) in, represents the unnormalized weight of the i-th particle at the human joint point n at the time k in the j-th step of the progressive process, represents the normalized weight of the ith particle at the human joint point n at time k in the jth step of the progressive process, represents the sum of all weights.

7. A method for estimating human posture based on progressive particle filtering according to claim 1, characterized in that: The step 5 comprises: Design the cutoff condition of the gradual measurement update process. In the gradual measurement update process, determine whether the integration of the j-th post-measurement improves the estimation performance of the filter. If the result is yes, execute step 6; if the result is no, execute step 3. The cutoff condition of the gradual measurement update process is designed as: (10) (11) in, is a small constant, , Human joints In the The estimation error in the sub-asymptotic measurement process, is the state estimate at the current asymptotic moment, , Human joints In the The deviation between the predicted value and the actual value during the asymptotic measurement process.

8. A method for estimating human posture based on progressive particle filtering according to claim 1, characterized in that: The step 6 comprises: The particle set at the current asymptotic moment is taken as The particle set at the moment, and then the state estimate of the current asymptotic moment is obtained and its covariance : (12) (13)。 9. A human body posture estimation device based on progressive particle filtering, characterized in that: It comprises one or more processors, and is used to implement a human posture estimation method based on progressive particle filtering as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a human body posture estimation method based on progressive particle filtering as described in any one of claims 1 to 8 is implemented.

Images