A pose estimation method, apparatus, device and medium

Abstract

The present disclosure provides a pose estimation method, device, equipment and medium, the method comprises: determining the target feature point that the target object in the current frame image successfully tracks the original feature point according to the original feature point of the target object in the upper frame image of the current frame image; determining the effective feature point meeting the preset screening condition from the target feature point, generating a motion estimation model according to the effective feature point; generating the first pose of the target object in the current frame image according to the motion estimation model and the pose estimation of the target object in the upper frame image, and extracting the second pose of the target object in the current frame image through the preset deep learning model; the first pose and the second pose are smoothed to generate the third pose of the target object in the current frame image. The embodiment of the present disclosure can generate the third pose according to the first pose and the second pose, so that the pose estimation has good stability and precision at the same time, thereby improving the experience and satisfaction of the user.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to an attitude estimation method, apparatus, device, and medium. Background Technology

[0002] With the development of computer technology, the application scenarios of pose estimation technology are becoming more and more widespread, such as virtual try-on based on pose estimation and driving the fitting of some virtual materials.

[0003] However, current attitude estimation methods cause some jitter, resulting in a somewhat floating effect in the application of attitude estimation. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides an attitude estimation method, apparatus, device, and medium.

[0005] In a first aspect, this disclosure provides a pose estimation method, the method comprising:

[0006] Based on the original feature points of the target object in the previous frame image of the current frame image, determine the target feature points in the current frame image where the original feature points were successfully tracked;

[0007] Valid feature points that meet preset screening conditions are determined from the target feature points, and a motion estimation model is generated based on the valid feature points;

[0008] The first pose of the target object in the current frame image is generated based on the motion estimation model and the pose estimation of the target object in the previous frame image, and the second pose of the target object in the current frame image is extracted by a preset deep learning model.

[0009] The first pose and the second pose are smoothed to generate the third pose of the target object in the current frame image.

[0010] In one optional implementation, determining the target feature points in the current frame image where the original feature points have been successfully tracked includes:

[0011] The original feature points are tracked using a sparse optical flow method to obtain the target feature points that have been successfully tracked.

[0012] In one optional implementation, effective feature points that meet preset screening conditions are determined from the target feature points, and a motion estimation model is generated based on the effective feature points, including:

[0013] Analyze the changes in the target feature points from the previous frame image to the current frame image;

[0014] Based on the changes, the target feature points are clustered into different sets of feature points with different directions of motion.

[0015] Compare the number of each set of feature points and obtain the target set with the largest number. Determine the feature points in the target set as valid feature points that meet the preset filtering conditions.

[0016] A motion transformation matrix is ​​generated based on the effective feature points.

[0017] In one optional implementation, generating the first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image includes:

[0018] The motion transformation matrix and the pose estimate of the target object in the previous frame image are multiplied by a dot product to generate the first pose of the target object in the current frame image.

[0019] In one optional implementation, the step of smoothing the first pose and the second pose to generate a third pose of the target object in the current frame image includes:

[0020] Determine a first smoothing parameter corresponding to the first posture, and determine a second smoothing parameter corresponding to the second posture;

[0021] The first pose and the second pose are smoothed according to the first smoothing parameter and the second smoothing parameter to generate the third pose of the target object in the current frame image.

[0022] In one optional implementation, determining the first smoothing parameter corresponding to the first pose and the second smoothing parameter corresponding to the second pose includes:

[0023] Determine whether the target feature point satisfies a preset first valid condition;

[0024] When the target feature point satisfies the first valid condition, it is determined whether the valid feature point satisfies the preset second valid condition.

[0025] When the effective feature point satisfies the second effective condition, the first smoothing parameter is determined to be the first coefficient and the second smoothing parameter is determined to be the second coefficient, wherein the first coefficient is greater than the second coefficient and the difference between the first coefficient and the second coefficient is greater than a preset threshold.

[0026] In one optional implementation, it further includes:

[0027] If the target feature point does not meet the first valid condition, then the first smoothing parameter is determined to be the third coefficient and the second smoothing parameter is determined to be the fourth coefficient, wherein the third coefficient is less than the fourth coefficient, and the difference between the fourth coefficient and the third coefficient is greater than a preset threshold.

[0028] In one optional implementation, it further includes:

[0029] When the target feature point satisfies the first valid condition, and when the valid feature point does not satisfy the second valid condition, the first smoothing parameter is determined to be the fifth coefficient, and the second smoothing parameter is determined to be the sixth coefficient, wherein the absolute value of the difference between the fifth coefficient and the sixth coefficient is less than a preset threshold.

[0030] In one optional implementation, determining whether the target feature point satisfies a preset first valid condition includes:

[0031] The number of the target feature points is compared with a preset first threshold.

[0032] If the number of target feature points is greater than or equal to the first threshold, then the first valid condition is determined to be met.

[0033] If the number of target feature points is less than the first threshold, then the first valid condition is not met.

[0034] In one optional implementation, determining whether the target feature point satisfies a preset first valid condition includes:

[0035] Calculate a first ratio between the number of target feature points and the number of original feature points, and compare the first ratio with a preset second threshold.

[0036] If the first ratio is greater than or equal to the second threshold, then the first valid condition is determined to be met;

[0037] If the first ratio is less than the second threshold, then it is determined that the first valid condition is not met.

[0038] In one optional implementation, determining whether the valid feature point satisfies a preset second valid condition includes:

[0039] The number of valid feature points is compared with a preset third threshold.

[0040] If the number of valid feature points is greater than or equal to the third threshold, then the second valid condition is determined to be satisfied.

[0041] If the number of valid feature points is less than the third threshold, then the second valid condition is determined not to be met.

[0042] In one optional implementation, determining whether the valid feature point satisfies a preset second valid condition includes:

[0043] Calculate a second ratio between the number of effective feature points and the number of target feature points, and compare the second ratio with a preset fourth threshold.

[0044] If the second ratio is greater than or equal to the fourth threshold, then the second valid condition is determined to be satisfied.

[0045] If the second ratio is less than the fourth threshold, then the second valid condition is determined not to be met.

[0046] Secondly, this disclosure also provides an attitude estimation apparatus, the apparatus comprising:

[0047] The determination module is used to determine the target feature points in the current frame image where the target object has successfully tracked the original feature points, based on the original feature points of the target object in the previous frame image of the current frame image;

[0048] The filtering module is used to determine the effective feature points that meet the preset filtering conditions from the target feature points, and generate a motion estimation model based on the effective feature points;

[0049] The first generation module is used to generate a first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image, and to extract a second pose of the target object in the current frame image through a preset deep learning model.

[0050] The second generation module is used to smooth the first pose and the second pose to generate a third pose of the target object in the current frame image.

[0051] Thirdly, this disclosure provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the above-described method.

[0052] Fourthly, this disclosure provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described above.

[0053] Fifthly, this disclosure provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the method described above.

[0054] The technical solution provided in this disclosure has at least the following advantages compared with the prior art:

[0055] The pose estimation method provided in this disclosure generates a first pose based on a motion estimation model, which exhibits good stability; a second pose is generated based on a deep learning model, which demonstrates high accuracy. A third pose is then generated based on the first and second poses, which simultaneously possesses good stability and accuracy, thereby enhancing user experience and satisfaction. Attached Figure Description

[0056] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0057] Figure 1 A flowchart illustrating an attitude estimation method provided in an embodiment of this disclosure;

[0058] Figure 2 A flowchart illustrating another attitude estimation method provided in this embodiment of the disclosure;

[0059] Figure 3a A schematic diagram of the previous frame image for a pose estimation method provided in an embodiment of this disclosure;

[0060] Figure 3b A schematic diagram of the current frame image for a pose estimation method provided in an embodiment of this disclosure;

[0061] Figure 3c A schematic diagram of the current frame image for another pose estimation method provided in an embodiment of this disclosure;

[0062] Figure 4a A schematic diagram of the previous frame image of the current frame image for another pose estimation method provided in this embodiment of the present disclosure;

[0063] Figure 4b A schematic diagram of the current frame image for another pose estimation method provided in an embodiment of this disclosure;

[0064] Figure 4c A schematic diagram of the current frame image for another pose estimation method provided in an embodiment of this disclosure;

[0065] Figure 5This is a schematic diagram of the structure of an attitude estimation device provided in an embodiment of the present disclosure;

[0066] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0067] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0068] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0069] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0070] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0071] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0072] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0073] To address the aforementioned problems, this disclosure provides an attitude estimation method, which will be described below with reference to specific embodiments.

[0074] Figure 1This is a flowchart illustrating an attitude estimation method provided in an embodiment of the present disclosure. The method can be executed by an attitude estimation device, which can be implemented in software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method includes:

[0075] Step 101: Based on the original feature points of the target object in the previous frame image of the current frame image, determine the target feature points in the current frame image where the original feature points of the target object have been successfully tracked.

[0076] Video is captured of the target object, and the pose of the target object in the video frames is estimated. Then, relevant applications are applied based on the estimated pose. For example, if the target object is a person's foot, virtual shoe try-on can be performed based on the foot's pose estimation; if the target object is a person's hand, special effects can be added, and so on. Therefore, to reduce pose estimation jitter and improve the application's fit, more accurate pose estimation of the target object is needed. It should be noted that the target object can be selected according to the application scenario; this embodiment does not impose any restrictions.

[0077] For the current frame image in a video stream where pose estimation is to be performed, the pose estimation method provided in this application processes the current frame image and the adjacent previous frame image to estimate the pose of the target object in the current frame image. The specific process is described as follows:

[0078] In some embodiments of this disclosure, original feature points can be extracted from the target object in the previous frame image. Various methods can be used to extract these original feature points, and the appropriate method can be selected based on different application scenarios. This embodiment does not impose any limitations. Examples include the Harris corner method, the Shi-Tomas corner method, and the FAST (Features From Accelerated Segment Test) method. It is understood that the extracted original feature points may differ depending on the target object. For instance, if the target object is a human hand, the extracted original feature points may include the knuckles of each finger and the wrist joint.

[0079] Then, the original feature points of the target object in the previous frame of the current frame image are tracked, and the target feature points corresponding to the original feature points of the target object in the current frame image are determined by tracking. There are various methods for tracking the original feature points, which can be selected according to the application scenario. This embodiment does not impose any restrictions, such as sparse optical flow method, dense optical flow method, etc.

[0080] Understandably, during the tracking of original feature points, it is possible that all original feature points are successfully tracked; it is also possible that all original feature points are tracked but not tracked; or it is possible that some original feature points are tracked but not tracked, while some original feature points are tracked successfully. Therefore, the target feature points can include the feature points corresponding to the successfully tracked original feature points in the current frame image.

[0081] For example, if the target object has 100 original feature points in the previous frame of the current frame image, and all original feature points are successfully tracked, then the target object has 100 corresponding target feature points in the current frame image. If some original feature points are successfully tracked, then the target object has 100 corresponding target feature points in the current frame image. For example, if 80 original feature points are successfully tracked, then the target object has 80 target feature points.

[0082] In some embodiments of this disclosure, it should be noted that the target object's motion amplitude is large, resulting in a significant change in the target object in the current frame image compared to the target object in the previous frame image. This reduces the probability of the current frame image successfully tracking the original feature points from the previous frame image, leading to an insufficient number of target feature points. In this case, feature points can be re-extracted from the target object in the current frame image, and these re-extracted feature points can be used as the original feature points. These original feature points can then be used to generate target feature points for the target object in the next frame image.

[0083] Step 102: Determine the effective feature points that meet the preset screening conditions from the target feature points, and generate a motion estimation model based on the effective feature points.

[0084] It should be noted that in some embodiments of this disclosure, there are some target feature points with large tracking errors. In order to accurately generate the motion estimation model, these target feature points with large tracking errors are removed, and target feature points with better tracking performance are retained.

[0085] In one optional implementation, target feature points can be filtered using preset screening conditions to obtain more accurate target feature points, which are then considered valid feature points. Understandably, in some application scenarios, if all target feature points meet the preset screening conditions, then the valid feature points can be all target feature points in the current frame image.

[0086] For example, if the target object has 100 target feature points in the current frame image, and all target feature points meet the preset filtering conditions, then the target object has 100 valid feature points in the current frame image; if 80 of the 100 target feature points meet the preset filtering conditions, then the target object has 80 valid feature points in the current frame image.

[0087] Optionally, there are various filtering conditions for selecting valid feature points, which can be selected according to the application scenario. This embodiment does not impose any restrictions, and examples are illustrated below:

[0088] Example 1:

[0089] A distance threshold can be preset to calculate the distance between each original feature point and its corresponding target feature point. The distance is then filtered by the distance threshold. If the distance is less than the preset distance threshold, the target feature point corresponding to that distance can be considered a valid feature point.

[0090] Example 2:

[0091] Calculate the direction of change of each target feature point from the previous frame image to the current frame image, and determine the effective feature points based on the direction of change.

[0092] In some embodiments of this disclosure, a corresponding motion estimation model can be generated based on the acquired valid feature points. There are various types of motion estimation models, which can be selected according to the specific application scenario. This embodiment does not impose any restrictions, such as motion change matrices, functions, etc.

[0093] Step 103: Generate the first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image, and extract the second pose of the target object in the current frame image through a preset deep learning model.

[0094] In one optional implementation, after obtaining the motion estimation model, the motion estimation model can be used to process the pose estimation of the target object in the previous frame image to generate the first pose of the target object in the current frame image. Understandably, the motion estimation model can process the pose estimation of the target object in the previous frame image according to the motion rules of effective feature points, thereby generating the first pose of the target object in the current frame image.

[0095] Furthermore, a second pose of the target object in the current frame image is generated using a preset deep learning model. Various preset deep learning models are available and can be selected according to different application scenarios; this embodiment does not impose any limitations.

[0096] Step 104: Smooth the first and second poses to generate the third pose of the target object in the current frame image.

[0097] In some embodiments of this disclosure, based on the generation of a first pose and a second pose, a smoothing process can be applied to the first and second poses to generate a third pose of the target object in the current frame image. Various smoothing processes can be used, and the appropriate one can be selected based on the application scenario. This embodiment does not impose any limitations; examples are provided below:

[0098] Example 1: A preset neural network model can be trained using training samples. The trained neural network model can smooth the first and second poses. When the model is applied, the first and second poses are input, and the model output is the third pose of the current frame image after smoothing.

[0099] Example 2: Smoothing parameters can be set according to the application scenario, and the first and second poses can be smoothed according to the smoothing parameters.

[0100] According to the pose estimation method in this embodiment, a first pose of the target object is generated based on a motion estimation model. This first pose has a high correlation with the target object in the previous frame image, thus exhibiting good stability. This good stability avoids jitter in pose estimation, resulting in stable application effects generated based on pose estimation. A second pose of the target object is generated based on a deep learning model, and this second pose has high accuracy. A third pose is generated based on the first and second poses. The third pose combines the advantages of both the first and second poses, possessing both good stability and accuracy. This allows for more appropriate placement of the application effects generated based on pose estimation, improving user experience and satisfaction.

[0101] Figure 2 This is a flowchart illustrating another pose estimation method provided in this embodiment. Based on the above embodiments, this embodiment more clearly describes the specific tracking method and smoothing method as follows:

[0102] Step 201: Track the original feature points using the sparse optical flow method to obtain the target feature points that have been successfully tracked.

[0103] It's important to note that optical flow can be used to represent changes in an image. If a target object in an image is moving, the optical flow can contain information about that motion. Therefore, optical flow can be used to represent the correspondence between a target object in the current frame and the previous frame. Sparse optical flow methods can be used to calculate optical flow, and there are several options available, such as the pyramid Lucas-Kanade optical flow method.

[0104] Therefore, the sparse optical flow method is used to track the original feature points extracted from the previous frame image, and the target feature points of the target object successfully tracked in the current frame image are obtained. The sparse optical flow method can quickly complete the tracking of the original feature points, and when applied to mobile devices, it can ensure low latency in generating pose estimates.

[0105] Step 202: Analyze the changes of target feature points from the previous frame image to the current frame image, and cluster the target feature points into different feature point sets with different motion directions based on the changes.

[0106] In some embodiments of this disclosure, the changes of target feature points from the previous frame to the current frame can be analyzed. These changes include, but are not limited to, the movement direction of the target feature points: upward, downward left, and clockwise. The target feature points can be clustered into different sets of feature points with different movement directions based on the changes. It should be noted that a target feature point can exist in multiple feature point sets. For example, a target feature point can simultaneously satisfy the conditions of leftward and counterclockwise movement; this target feature point can exist in both the set of feature points with a leftward movement direction and the set of feature points with a counterclockwise movement direction.

[0107] Step 203: Compare the number of each feature point set and obtain the target set with the largest number. Determine the feature points in the target set as valid feature points that meet the preset screening conditions, and generate a motion transformation matrix based on the valid feature points.

[0108] Then, the number of target feature points in different feature point sets corresponding to different motion directions is compared, and the target set with the largest number of target feature points is obtained. The feature points in the target set are determined as valid feature points that meet the preset screening conditions. For example, assuming there are a total of 80 target feature points showing changes from the previous frame to the current frame, the target feature points are clustered into a first feature point set moving from bottom to top, a second feature point set moving from top to bottom, and a third feature point set moving from left to right. The first feature point set contains 75 feature points, the second feature point set contains 2 feature points, and the third feature point set contains 3 feature points. Therefore, the first feature point set is determined as the target set, and the 75 feature points in it are considered valid feature points. After obtaining the valid feature points, a motion change matrix can be generated based on these valid feature points.

[0109] Step 204: Perform dot product processing on the motion change matrix and the pose estimation of the target object in the previous frame image to generate the first pose of the target object in the current frame image, and extract the second pose of the target object in the current frame image through a preset deep learning model.

[0110] In some embodiments of this application, after obtaining the motion transformation matrix, a dot product method can be used to process the pose estimation of the target object in the previous frame image according to the motion matrix, generating the first pose of the target object in the current frame image. Furthermore, a second pose of the target object in the current frame image can be extracted further using a preset deep learning model.

[0111] Step 205: Determine the first smoothing parameter corresponding to the first pose and the second smoothing parameter corresponding to the second pose, and smooth the first pose and the second pose according to the first smoothing parameter and the second smoothing parameter to generate the third pose of the target object in the current frame image.

[0112] Understandably, the first smoothing parameter can be used to determine the weight of the first pose in the process of generating the third pose; the second smoothing parameter can be used to determine the weight of the second pose in the process of generating the third pose.

[0113] In some embodiments of this disclosure, the first smoothing parameter can be determined by whether the target feature points satisfy a first valid condition, and the second smoothing parameter can be determined by whether the valid feature points satisfy a second valid condition. The first pose and the second pose can be smoothed based on the obtained first and second smoothing parameters. For example, the first smoothing parameter can be used as the weight of the first pose, and the second smoothing parameter can be used as the weight of the second pose. Based on the first pose and its weight, and the second pose and its weight, a third pose of the current frame image is generated.

[0114] It should be noted that when a target feature point meets the first valid condition, it means that the number of target feature points is sufficient; when a valid feature point meets the second valid condition, it means that the number of valid feature points is sufficient. Understandably, there are multiple methods to determine whether a target feature point meets the preset first condition, and there are also multiple methods to determine whether a valid feature point meets the preset second condition.

[0115] For example, methods for determining whether a target feature point meets a preset first condition include, but are not limited to, the following two:

[0116] Method 1 involves comparing the number of target feature points with a preset first threshold. If the number of target feature points is greater than or equal to the first threshold, the first valid condition is satisfied; if the number of target feature points is less than the first threshold, the first valid condition is not satisfied. The first threshold can be set according to different application scenarios, and this embodiment does not impose any restrictions. For example, the first threshold can be preset to 300. When the number of target feature points is 500, if the number of target feature points is greater than the first threshold, the first valid condition is satisfied.

[0117] Method 2: Calculate the first ratio between the number of target feature points and the number of original feature points, and compare the first ratio with a preset second threshold. If the first ratio is greater than or equal to the second threshold, the first valid condition is satisfied; if the first ratio is less than the second threshold, the first valid condition is not satisfied.

[0118] The target feature points are selected from the original feature points. Understandably, the smaller the proportion of removed feature points in the original feature points, the higher the confidence level of the target feature points. The second threshold can be set according to different application scenarios; this embodiment does not impose any restrictions. For example, the second threshold can be preset to 0.7. With 625 original feature points and 500 target feature points, the first ratio is 0.8. Since the first ratio is greater than the second threshold, the first valid condition is determined to be met.

[0119] For example, methods for determining whether a valid feature point satisfies a preset second condition include, but are not limited to, the following two:

[0120] Method 1: Compare the number of valid feature points with a preset third threshold. If the number of valid feature points is greater than or equal to the third threshold, the second valid condition is satisfied; if the number of valid feature points is less than the third threshold, the second valid condition is not satisfied.

[0121] The third threshold can be set according to different application scenarios, and this embodiment does not impose any restrictions. For example, the third threshold can be preset to 300. If the number of valid feature points is greater than the third threshold when the number of valid feature points is 500, then the second valid condition is determined to be met.

[0122] Method 2: Calculate the second ratio between the number of valid feature points and the number of target feature points, and compare the second ratio with a preset fourth threshold. If the second ratio is greater than or equal to the fourth threshold, the second valid condition is satisfied; if the second ratio is less than the fourth threshold, the second valid condition is not satisfied.

[0123] In this context, valid feature points are selected from target feature points. Understandably, the smaller the proportion of removed feature points in the target feature points, the higher the confidence level of the valid feature points. The fourth threshold can be set according to different application scenarios; this embodiment does not impose any restrictions. For example, the fourth threshold can be preset to 0.7. With 400 valid feature points and 500 target feature points, the second ratio is 0.8. Since the second ratio is greater than the fourth threshold, the second valid condition is determined to be met.

[0124] Optionally, the first smoothing parameter and the second smoothing parameter can be determined based on whether the first valid condition and / or the second valid condition are met in the application scenario. There are various methods for determining these parameters, and the appropriate method can be selected based on the application scenario. This embodiment does not impose any restrictions. An example is illustrated below:

[0125] Example 1:

[0126] If the target feature point satisfies the first valid condition and the valid feature point satisfies the second valid condition, then the first smoothing parameter is determined as the first coefficient and the second smoothing parameter is determined as the second coefficient. The first coefficient is greater than the second coefficient, and the difference between the first coefficient and the second coefficient is greater than a preset threshold.

[0127] Understandably, in Example 1, the number of target feature points and effective feature points is sufficient. Therefore, the first pose generated by the motion estimation model and the pose estimation from the previous frame is sufficiently accurate, and its stability is also high. Consequently, in this scenario, the first pose accounts for a larger proportion in the generation of the third pose. Therefore, the first coefficient corresponding to the first smoothing parameter is greater than the second coefficient corresponding to the second smoothing parameter. To make the difference between the first and second coefficients sufficiently large, a preset threshold can be set so that the difference between the first and second coefficients is greater than the preset threshold. The preset threshold can be set according to the application scenario; this example does not impose any restrictions, for example: 0.5, 0.7.

[0128] In one optional implementation, the preset threshold can be 0.5, the target feature point satisfies the first valid condition, and the valid feature point satisfies the second valid condition. The first coefficient can be 1, then the first smoothing parameter is the first coefficient 1; the second coefficient can be 0, then the second smoothing parameter is the second coefficient 0, and the third pose of the current frame image is the same as the first pose of the current frame image.

[0129] Example 2:

[0130] If the target feature point does not meet the first valid condition, the first smoothing parameter is determined as the third coefficient and the second smoothing parameter is determined as the fourth coefficient. The third coefficient is less than the fourth coefficient, and the difference between the fourth coefficient and the third coefficient is greater than a preset threshold.

[0131] Understandably, in Example 2, the number of target feature points is insufficient. Therefore, the first pose generated by the motion estimation model and the pose estimation from the previous frame is not accurate enough. Consequently, in this scenario, the second pose accounts for a larger proportion during the generation of the third pose. Therefore, the third coefficient corresponding to the first smoothing parameter is smaller than the fourth coefficient corresponding to the second smoothing parameter. To make the difference between the third and fourth coefficients sufficiently large, a preset threshold can be set so that the difference between the fourth and third coefficients is greater than the preset threshold. The preset threshold can be set according to the application scenario; this example does not impose any restrictions, for example, 0.5 or 0.7.

[0132] In one optional implementation, the preset threshold can be 0.5, meaning the target feature point does not meet the first valid condition. If the third coefficient is 0, then the first smoothing coefficient is the third coefficient 0; if the fourth coefficient is 1, then the second smoothing coefficient is the fourth coefficient 1, and the third pose of the current frame image is the same as the second pose of the current frame image.

[0133] Example 3:

[0134] When the target feature point meets the first valid condition, and when the valid feature point does not meet the second valid condition, the first smoothing parameter is determined to be the fifth coefficient, and the second smoothing parameter is determined to be the sixth coefficient, wherein the absolute value of the difference between the fifth coefficient and the sixth coefficient is less than a preset threshold.

[0135] Understandably, in Example 3, the number of target feature points is sufficient, but the number of effective feature points is insufficient. Therefore, the first pose generated by the motion estimation model and the pose estimation from the previous frame is not accurate enough. A second pose can be used to smooth the first pose and generate the third pose. Thus, in this scenario, the proportions of the first and second poses are not significantly different during the generation of the third pose. Therefore, the values ​​of the fifth coefficient corresponding to the first smoothing parameter and the sixth coefficient corresponding to the second smoothing parameter are not significantly different. To make the difference between the fifth and sixth coefficients sufficiently small, a preset threshold can be set so that the absolute value of the difference between the fifth and sixth coefficients is less than the preset threshold. The value of the preset threshold can be set according to the application scenario; this example does not impose any restrictions, for example: 0.05, 0.01.

[0136] According to the pose estimation method of this disclosure, the original feature points are tracked using a sparse optical flow method, which can quickly complete the tracking of the original feature points. When the application scenario is a mobile device, the real-time performance of the pose estimation method on the mobile device can be guaranteed. Valid feature points are obtained based on the motion direction of the target feature points, and inaccurate target feature points are eliminated, thereby improving the performance of the first pose. The first and second poses are smoothed using a first smoothing coefficient and a second smoothing coefficient to generate a third pose. Using smoothing coefficients to efficiently and concisely process the first and second poses improves the generation efficiency of the third pose, enhances the user experience, and increases user satisfaction.

[0137] Based on the above embodiments, in order to more clearly illustrate the application effect of the attitude estimation method provided in this disclosure, through... Figure 3a , 3b The virtual try-on shown in 3C and Figure 4a , 4b The specific application of the virtual assets shown in 4c will be explained below:

[0138] Figure 3a This is a schematic diagram of the previous frame image for a pose estimation method provided in an embodiment of this disclosure, wherein the target object is a human head, and the application effect is to virtually place a hat on the human head. Figure 3a As shown, the hat is positioned appropriately at this point.

[0139] In some embodiments of this application, the target feature points of the head in the current frame image can be determined based on the original feature points of the head in the previous frame image, and the target feature points can be filtered to obtain effective feature points. A motion estimation model can be generated based on the effective feature points, a first pose can be generated based on the motion estimation model and the pose estimation in the previous frame image, a second pose can be generated through a deep learning model, and a third pose can be generated based on the first pose and the second pose.

[0140] Figure 3b This is a schematic diagram of the current frame image of a pose estimation method provided in an embodiment of this disclosure. The application effect added based on the third pose is as follows: Figure 3b As shown, the head has been moved clockwise, and the hat has been adjusted accordingly. The hat is now in a more appropriate position.

[0141] If the third pose is ineffective, the application effect will be as follows: Figure 3c As shown, Figure 3c The hat is positioned incorrectly.

[0142] Figure 4a This is a schematic diagram of the previous frame image of the current frame image for another pose estimation method provided in this embodiment of the disclosure, wherein the target object is a human hand, and the application effect is to virtually add a heart to the human hand. Figure 4a As shown, the heart is positioned appropriately at this point.

[0143] In some embodiments of this application, the target feature points of the hand in the current frame image can be determined based on the original feature points of the hand in the previous frame image, and the target feature points can be filtered to obtain effective feature points. A motion estimation model can be generated based on the effective feature points, a first pose can be generated based on the motion estimation model and the pose estimation in the previous frame image, a second pose can be generated through a deep learning model, and a third pose can be generated based on the first pose and the second pose.

[0144] Figure 4b This is a schematic diagram of the current frame image of another pose estimation method provided in this embodiment of the disclosure, showing the application effect added based on the third pose, as follows. Figure 4b As shown, the hand moved clockwise, and the heart was adjusted accordingly. At this point, the heart's position is more appropriate.

[0145] If the third pose is ineffective, the application effect will be as follows: Figure 4c As shown, Figure 4c The heart is not positioned correctly in the picture.

[0146] The pose estimation method according to the embodiments of this application is used to add special effects to target objects in videos. This method can improve the stability and accuracy of the applied special effects and improve the user experience.

[0147] According to embodiments of this disclosure, this disclosure also provides an attitude estimation device.

[0148] Figure 5 This is a schematic diagram of an attitude estimation device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware, and is generally integrated into an electronic device. Figure 5 As shown, the attitude estimation device 500 may include:

[0149] The determining module 501 determines the target feature points in the current frame image where the target object has successfully tracked the original feature points based on the original feature points of the target object in the previous frame image of the current frame image;

[0150] The filtering module 502 is used to determine the effective feature points that meet the preset filtering conditions from the target feature points, and generate a motion estimation model based on the effective feature points;

[0151] The first generation module 503 generates a first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image, and extracts a second pose of the target object in the current frame image through a preset deep learning model.

[0152] The second generation module 504 performs smoothing processing on the first pose and the second pose to generate a third pose of the target object in the current frame image.

[0153] Optionally, the determining module 501 is configured to:

[0154] The original feature points are tracked using a sparse optical flow method to obtain the target feature points that have been successfully tracked.

[0155] Optionally, the filtering module 502 is used for:

[0156] Analyze the changes in the target feature points from the previous frame image to the current frame image;

[0157] Based on the changes, the target feature points are clustered into different sets of feature points with different directions of motion.

[0158] Compare the number of each set of feature points and obtain the target set with the largest number. Determine the feature points in the target set as valid feature points that meet the preset filtering conditions.

[0159] A motion transformation matrix is ​​generated based on the effective feature points.

[0160] Optionally, the first generation module 503 is configured to:

[0161] The motion transformation matrix and the pose estimate of the target object in the previous frame image are multiplied by a dot product to generate the first pose of the target object in the current frame image.

[0162] Optionally, the second generation module 504 includes:

[0163] The determining unit is used to determine a first smoothing parameter corresponding to the first posture and to determine a second smoothing parameter corresponding to the second posture.

[0164] The generation unit is used to smooth the first pose and the second pose according to the first smoothing parameter and the second smoothing parameter to generate the third pose of the target object in the current frame image.

[0165] Optionally, the determining unit is configured to:

[0166] Determine whether the target feature point satisfies a preset first valid condition;

[0167] When the target feature point satisfies the first valid condition, it is determined whether the valid feature point satisfies the preset second valid condition.

[0168] When the effective feature point satisfies the second effective condition, the first smoothing parameter is determined to be the first coefficient and the second smoothing parameter is determined to be the second coefficient, wherein the first coefficient is greater than the second coefficient and the difference between the first coefficient and the second coefficient is greater than a preset threshold.

[0169] Optionally, the determining unit is further configured to:

[0170] If the target feature point does not meet the first valid condition, then the first smoothing parameter is determined to be the third coefficient and the second smoothing parameter is determined to be the fourth coefficient, wherein the third coefficient is less than the fourth coefficient, and the difference between the fourth coefficient and the third coefficient is greater than a preset threshold.

[0171] Optionally, the determining unit is further configured to:

[0172] When the target feature point satisfies the first valid condition, and when the valid feature point does not satisfy the second valid condition, the first smoothing parameter is determined to be the fifth coefficient, and the second smoothing parameter is determined to be the sixth coefficient, wherein the absolute value of the difference between the fifth coefficient and the sixth coefficient is less than a preset threshold.

[0173] Optionally, the determining unit is configured to:

[0174] The number of the target feature points is compared with a preset first threshold.

[0175] If the number of target feature points is greater than or equal to the first threshold, then the first valid condition is determined to be met.

[0176] If the number of target feature points is less than the first threshold, then the first valid condition is not met.

[0177] Optionally, the determining unit is configured to:

[0178] Calculate a first ratio between the number of target feature points and the number of original feature points, and compare the first ratio with a preset second threshold.

[0179] If the first ratio is greater than or equal to the second threshold, then the first valid condition is determined to be met;

[0180] If the first ratio is less than the second threshold, then it is determined that the first valid condition is not met.

[0181] Optionally, the determining unit is configured to:

[0182] The number of valid feature points is compared with a preset third threshold.

[0183] If the number of valid feature points is greater than or equal to the third threshold, then the second valid condition is determined to be satisfied.

[0184] If the number of valid feature points is less than the third threshold, then the second valid condition is determined not to be met.

[0185] Optionally, the determining unit is configured to:

[0186] Calculate a second ratio between the number of effective feature points and the number of target feature points, and compare the second ratio with a preset fourth threshold.

[0187] If the second ratio is greater than or equal to the fourth threshold, then the second valid condition is determined to be satisfied.

[0188] If the second ratio is less than the fourth threshold, then the second valid condition is determined not to be met.

[0189] The attitude estimation device provided in this disclosure can execute the information editing and processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method execution.

[0190] This disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the information editing and processing method provided in any embodiment of this disclosure.

[0191] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0192] The following is a detailed reference. Figure 6 The diagram illustrates a structural schematic suitable for implementing the electronic device 600 in the embodiments of this disclosure. The electronic device 600 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0193] like Figure 6As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0194] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0195] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the information editing processing method of embodiments of this disclosure.

[0196] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0197] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0198] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0199] The aforementioned computer-readable medium carries one or more programs. When the electronic device executes these programs, the electronic device causes the following actions: Based on the original feature points of the target object in the previous frame image, it determines the target feature points in the current frame image where the original feature points have been successfully tracked; it determines valid feature points from the target feature points that meet preset filtering conditions, and generates a motion estimation model based on the valid feature points; it generates a first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image, and extracts a second pose of the target object in the current frame image using a preset deep learning model; it then smooths the first and second poses to generate a third pose of the target object in the current frame image. This embodiment of the present disclosure can generate a third pose based on the first and second poses, enabling the pose estimation to simultaneously possess good stability and accuracy, thereby improving user experience and satisfaction.

[0200] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0201] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0202] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0203] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0204] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0205] According to one or more embodiments of this disclosure, this disclosure provides an attitude estimation method, including:

[0206] Based on the original feature points of the target object in the previous frame image of the current frame image, determine the target feature points in the current frame image where the original feature points were successfully tracked;

[0207] Valid feature points that meet preset screening conditions are determined from the target feature points, and a motion estimation model is generated based on the valid feature points;

[0208] The first pose of the target object in the current frame image is generated based on the motion estimation model and the pose estimation of the target object in the previous frame image, and the second pose of the target object in the current frame image is extracted by a preset deep learning model.

[0209] The first pose and the second pose are smoothed to generate the third pose of the target object in the current frame image.

[0210] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, determining the target feature points in the current frame image where the original feature points have been successfully tracked by the target object includes:

[0211] The original feature points are tracked using a sparse optical flow method to obtain the target feature points that have been successfully tracked.

[0212] According to one or more embodiments of this disclosure, a pose estimation method provided by this disclosure includes determining effective feature points that meet preset screening conditions from the target feature points, and generating a motion estimation model based on the effective feature points, comprising:

[0213] Analyze the changes in the target feature points from the previous frame image to the current frame image;

[0214] Based on the changes, the target feature points are clustered into different sets of feature points with different directions of motion.

[0215] Compare the number of each set of feature points and obtain the target set with the largest number. Determine the feature points in the target set as valid feature points that meet the preset filtering conditions.

[0216] A motion transformation matrix is ​​generated based on the effective feature points.

[0217] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, generating a first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image includes:

[0218] The motion transformation matrix and the pose estimate of the target object in the previous frame image are multiplied by a dot product to generate the first pose of the target object in the current frame image.

[0219] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, the step of smoothing the first pose and the second pose to generate a third pose of the target object in the current frame image includes:

[0220] Determine a first smoothing parameter corresponding to the first posture, and determine a second smoothing parameter corresponding to the second posture;

[0221] The first pose and the second pose are smoothed according to the first smoothing parameter and the second smoothing parameter to generate the third pose of the target object in the current frame image.

[0222] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, determining a first smoothing parameter corresponding to the first pose and determining a second smoothing parameter corresponding to the second pose includes:

[0223] Determine whether the target feature point satisfies a preset first valid condition;

[0224] When the target feature point satisfies the first valid condition, it is determined whether the valid feature point satisfies the preset second valid condition.

[0225] When the effective feature point satisfies the second effective condition, the first smoothing parameter is determined to be the first coefficient and the second smoothing parameter is determined to be the second coefficient, wherein the first coefficient is greater than the second coefficient and the difference between the first coefficient and the second coefficient is greater than a preset threshold.

[0226] According to one or more embodiments of this disclosure, the attitude estimation method provided by this disclosure further includes:

[0227] If the target feature point does not meet the first valid condition, then the first smoothing parameter is determined to be the third coefficient and the second smoothing parameter is determined to be the fourth coefficient, wherein the third coefficient is less than the fourth coefficient, and the difference between the fourth coefficient and the third coefficient is greater than a preset threshold.

[0228] According to one or more embodiments of this disclosure, the attitude estimation method provided by this disclosure further includes:

[0229] When the target feature point satisfies the first valid condition, and when the valid feature point does not satisfy the second valid condition, the first smoothing parameter is determined to be the fifth coefficient, and the second smoothing parameter is determined to be the sixth coefficient, wherein the absolute value of the difference between the fifth coefficient and the sixth coefficient is less than a preset threshold.

[0230] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, determining whether the target feature point satisfies a preset first valid condition includes:

[0231] The number of the target feature points is compared with a preset first threshold.

[0232] If the number of target feature points is greater than or equal to the first threshold, then the first valid condition is determined to be met.

[0233] If the number of target feature points is less than the first threshold, then the first valid condition is not met.

[0234] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, determining whether the target feature point satisfies a preset first valid condition includes:

[0235] Calculate a first ratio between the number of target feature points and the number of original feature points, and compare the first ratio with a preset second threshold.

[0236] If the first ratio is greater than or equal to the second threshold, then the first valid condition is determined to be met;

[0237] If the first ratio is less than the second threshold, then it is determined that the first valid condition is not met.

[0238] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, determining whether the effective feature point satisfies a preset second effective condition includes:

[0239] The number of valid feature points is compared with a preset third threshold.

[0240] If the number of valid feature points is greater than or equal to the third threshold, then the second valid condition is determined to be satisfied.

[0241] If the number of valid feature points is less than the third threshold, then the second valid condition is determined not to be met.

[0242] According to one or more embodiments of this disclosure, in a pose estimation method provided by this disclosure, determining whether the effective feature point satisfies a preset second effective condition includes:

[0243] Calculate a second ratio between the number of effective feature points and the number of target feature points, and compare the second ratio with a preset fourth threshold.

[0244] If the second ratio is greater than or equal to the fourth threshold, then the second valid condition is determined to be satisfied.

[0245] If the second ratio is less than the fourth threshold, then the second valid condition is determined not to be met.

[0246] According to one or more embodiments of this disclosure, this disclosure provides an attitude estimation apparatus, comprising:

[0247] The determination module is used to determine the target feature points in the current frame image where the target object has successfully tracked the original feature points, based on the original feature points of the target object in the previous frame image of the current frame image;

[0248] The filtering module is used to determine the effective feature points that meet the preset filtering conditions from the target feature points, and generate a motion estimation model based on the effective feature points;

[0249] The first generation module generates a first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image, and extracts a second pose of the target object in the current frame image through a preset deep learning model.

[0250] The second generation module performs smoothing processing on the first pose and the second pose to generate a third pose of the target object in the current frame image.

[0251] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining module is configured to:

[0252] The original feature points are tracked using a sparse optical flow method to obtain the target feature points that have been successfully tracked.

[0253] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the screening module is used for:

[0254] Analyze the changes in the target feature points from the previous frame image to the current frame image;

[0255] Based on the changes, the target feature points are clustered into different sets of feature points with different directions of motion.

[0256] Compare the number of each set of feature points and obtain the target set with the largest number. Determine the feature points in the target set as valid feature points that meet the preset filtering conditions.

[0257] A motion transformation matrix is ​​generated based on the effective feature points.

[0258] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the first generation module is configured to:

[0259] The motion transformation matrix and the pose estimate of the target object in the previous frame image are multiplied by a dot product to generate the first pose of the target object in the current frame image.

[0260] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the second generation module includes:

[0261] The determining unit is used to determine a first smoothing parameter corresponding to the first posture and to determine a second smoothing parameter corresponding to the second posture.

[0262] The generation unit is used to smooth the first pose and the second pose according to the first smoothing parameter and the second smoothing parameter to generate the third pose of the target object in the current frame image.

[0263] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is configured to:

[0264] Determine whether the target feature point satisfies a preset first valid condition;

[0265] When the target feature point satisfies the first valid condition, it is determined whether the valid feature point satisfies the preset second valid condition.

[0266] When the effective feature point satisfies the second effective condition, the first smoothing parameter is determined to be the first coefficient and the second smoothing parameter is determined to be the second coefficient, wherein the first coefficient is greater than the second coefficient and the difference between the first coefficient and the second coefficient is greater than a preset threshold.

[0267] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is further configured to:

[0268] If the target feature point does not meet the first valid condition, then the first smoothing parameter is determined to be the third coefficient and the second smoothing parameter is determined to be the fourth coefficient, wherein the third coefficient is less than the fourth coefficient, and the difference between the fourth coefficient and the third coefficient is greater than a preset threshold.

[0269] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is further configured to:

[0270] When the target feature point satisfies the first valid condition, and when the valid feature point does not satisfy the second valid condition, the first smoothing parameter is determined to be the fifth coefficient, and the second smoothing parameter is determined to be the sixth coefficient, wherein the absolute value of the difference between the fifth coefficient and the sixth coefficient is less than a preset threshold.

[0271] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is configured to:

[0272] The number of the target feature points is compared with a preset first threshold.

[0273] If the number of target feature points is greater than or equal to the first threshold, then the first valid condition is determined to be met.

[0274] If the number of target feature points is less than the first threshold, then the first valid condition is not met.

[0275] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is configured to:

[0276] Calculate a first ratio between the number of target feature points and the number of original feature points, and compare the first ratio with a preset second threshold.

[0277] If the first ratio is greater than or equal to the second threshold, then the first valid condition is determined to be met;

[0278] If the first ratio is less than the second threshold, then it is determined that the first valid condition is not met.

[0279] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is configured to:

[0280] The number of valid feature points is compared with a preset third threshold.

[0281] If the number of valid feature points is greater than or equal to the third threshold, then the second valid condition is determined to be satisfied.

[0282] If the number of valid feature points is less than the third threshold, then the second valid condition is determined not to be met.

[0283] According to one or more embodiments of this disclosure, in the attitude estimation apparatus provided by this disclosure, the determining unit is configured to:

[0284] Calculate a second ratio between the number of effective feature points and the number of target feature points, and compare the second ratio with a preset fourth threshold.

[0285] If the second ratio is greater than or equal to the fourth threshold, then the second valid condition is determined to be satisfied.

[0286] If the second ratio is less than the fourth threshold, then the second valid condition is determined not to be met.

[0287] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0288] processor;

[0289] Memory for storing processor-executable instructions;

[0290] The processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the attitude estimation methods provided in this disclosure.

[0291] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium storing a computer program for performing any of the attitude estimation methods provided in the present disclosure.

[0292] According to one or more embodiments of this disclosure, this disclosure provides a computer program product comprising a computer program / instructions that, when executed by a processor, implement any of the attitude estimation methods provided in this disclosure.

[0293] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0294] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0295] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A pose estimation method, characterized in that, include: Based on the original feature points of the target object in the previous frame image of the current frame image, determine the target feature points in the current frame image where the original feature points were successfully tracked; Valid feature points that meet preset screening conditions are determined from the target feature points, and a motion estimation model is generated based on the valid feature points; The first pose of the target object in the current frame image is generated based on the motion estimation model and the pose estimation of the target object in the previous frame image, and the second pose of the target object in the current frame image is extracted by a preset deep learning model. A third pose of the target object in the current frame image is generated by smoothing the first pose and the second pose; wherein the smoothing is performed based on smoothing parameters that are weights corresponding to the first pose and the second pose, respectively.

2. The method according to claim 1, characterized in that, The step of determining the target feature points in the current frame image where the original feature points of the target object have been successfully tracked includes: The original feature points are tracked using a sparse optical flow method to obtain the target feature points that have been successfully tracked.

3. The method according to claim 1, characterized in that, Determine valid feature points that meet preset screening conditions from the target feature points, and generate a motion estimation model based on the valid feature points, including: Analyze the changes in the target feature points from the previous frame image to the current frame image; Based on the changes, the target feature points are clustered into different sets of feature points with different directions of motion. Compare the number of each set of feature points and obtain the target set with the largest number. Determine the feature points in the target set as valid feature points that meet the preset filtering conditions. A motion transformation matrix is ​​generated based on the effective feature points.

4. The method according to claim 3, characterized in that, The step of generating the first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image includes: The motion transformation matrix and the pose estimate of the target object in the previous frame image are multiplied by a dot product to generate the first pose of the target object in the current frame image.

5. The method according to claim 1, characterized in that, The step of smoothing the first pose and the second pose to generate the third pose of the target object in the current frame image includes: Determine a first smoothing parameter corresponding to the first posture, and determine a second smoothing parameter corresponding to the second posture; The first pose and the second pose are smoothed according to the first smoothing parameter and the second smoothing parameter to generate the third pose of the target object in the current frame image.

6. The method according to claim 5, characterized in that, The determination of the first smoothing parameter corresponding to the first posture and the determination of the second smoothing parameter corresponding to the second posture include: Determine whether the target feature point satisfies a preset first valid condition; When the target feature point satisfies the first valid condition, it is determined whether the valid feature point satisfies the preset second valid condition. When the effective feature point satisfies the second effective condition, the first smoothing parameter is determined to be the first coefficient and the second smoothing parameter is determined to be the second coefficient, wherein the first coefficient is greater than the second coefficient and the difference between the first coefficient and the second coefficient is greater than a preset threshold.

7. The method according to claim 6, characterized in that, Also includes: If the target feature point does not meet the first valid condition, then the first smoothing parameter is determined to be the third coefficient and the second smoothing parameter is determined to be the fourth coefficient, wherein the third coefficient is less than the fourth coefficient, and the difference between the fourth coefficient and the third coefficient is greater than a preset threshold.

8. The method according to claim 6, characterized in that, Also includes: When the target feature point satisfies the first valid condition, and when the valid feature point does not satisfy the second valid condition, the first smoothing parameter is determined to be the fifth coefficient, and the second smoothing parameter is determined to be the sixth coefficient, wherein the absolute value of the difference between the fifth coefficient and the sixth coefficient is less than a preset threshold.

9. The method according to any one of claims 6-8, characterized in that, The step of determining whether the target feature point satisfies a preset first valid condition includes: The number of the target feature points is compared with a preset first threshold. If the number of target feature points is greater than or equal to the first threshold, then the first valid condition is determined to be met. If the number of target feature points is less than the first threshold, then the first valid condition is not met.

10. The method according to any one of claims 6-8, characterized in that, The step of determining whether the target feature point satisfies a preset first valid condition includes: Calculate a first ratio between the number of target feature points and the number of original feature points, and compare the first ratio with a preset second threshold. If the first ratio is greater than or equal to the second threshold, then the first valid condition is determined to be met; If the first ratio is less than the second threshold, then it is determined that the first valid condition is not met.

11. The method according to any one of claims 6-8, characterized in that, The step of determining whether the valid feature point satisfies the preset second valid condition includes: The number of valid feature points is compared with a preset third threshold. If the number of valid feature points is greater than or equal to the third threshold, then the second valid condition is determined to be satisfied. If the number of valid feature points is less than the third threshold, then the second valid condition is determined not to be met.

12. The method according to any one of claims 6-8, characterized in that, The step of determining whether the valid feature point satisfies the preset second valid condition includes: Calculate a second ratio between the number of effective feature points and the number of target feature points, and compare the second ratio with a preset fourth threshold. If the second ratio is greater than or equal to the fourth threshold, then the second valid condition is determined to be satisfied. If the second ratio is less than the fourth threshold, then the second valid condition is determined not to be met.

13. An attitude estimation device, characterized in that, The device includes: The determination module is used to determine the target feature points in the current frame image where the target object has successfully tracked the original feature points, based on the original feature points of the target object in the previous frame image of the current frame image; The filtering module is used to determine the effective feature points that meet the preset filtering conditions from the target feature points, and generate a motion estimation model based on the effective feature points; The first generation module is used to generate a first pose of the target object in the current frame image based on the motion estimation model and the pose estimation of the target object in the previous frame image, and to extract a second pose of the target object in the current frame image through a preset deep learning model. The second generation module is used to perform smoothing processing on the first pose and the second pose to generate a third pose of the target object in the current frame image; wherein the smoothing processing is performed based on smoothing parameters corresponding to the first pose and the second pose as weights.

14. An electronic device, characterized in that, The electronic device includes: processor; Memory for storing processor-executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the attitude estimation method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to implement the attitude estimation method as described in any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the attitude estimation method as described in any one of claims 1-12.

Images