A virtual image jitter optimization method for 3D object tracking

Abstract

The present application relates to a kind of 3D object tracking virtual image jitter optimization method, belong to computer vision and intelligent manufacturing technical field.The method uses ICG (Iterative Corresponding Geometry) tracking algorithm, in combination with grey prediction model, the optimization of tracking algorithm parameter and Kalman filtering to realize jitter optimization, specifically as follows: create sample data used as grey prediction model;Sample data is carried out level ratio test;Train the model for solving predicted pose;According to residual test, the prediction accuracy of model is judged, and the predicted pose of next frame is obtained from the model;The change size of predicted pose and current frame pose is calculated to obtain ADDS gray ;ADDS gray And the mapping relationship between learning rate a is fitted, and the learning rate of next frame is controlled by ADDS gray ;Jitter is optimized using Kalman filtering, and the final optimized pose is obtained.The present application improves the use experience of AR glasses by optimizing pose oscillation, avoids dizziness, eye fatigue and other phenomena.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent manufacturing technology, and relates to a virtual image jitter optimization method for 3D object tracking. Background Technology

[0002] In on-site assisted assembly scenarios, personnel wearing AR glasses can collect real-time images, videos, and audio data, which are transmitted to the on-site assisted assembly system via a 5G network. The system analyzes and processes the data, generating production support information, which is then distributed to on-site terminals via the 5G network. This enables enhanced image overlay of operation steps and visualization of assembly processes, assisting on-site personnel in assembling complex or precision equipment. Furthermore, remote expert guidance, equipment operation manuals, drawings, and documents can also be synchronized to on-site terminals in real-time via the 5G network. On-site assembly personnel can be trained quickly and immediately begin work, effectively improving their assembly skills, achieving intelligent assembly processes, and increasing assembly efficiency.

[0003] In such assisted assembly systems, pose estimation algorithms are often used to obtain the coordinate position and rotation direction of parts. These positions and directions are then assigned to the 3D model of the part, allowing the 3D model to follow the target part in corresponding pose transformations. However, current pose estimation algorithms often prioritize the accuracy of pose estimation while neglecting pose oscillation. Pose oscillation causes the virtual image of the 3D part model to constantly shake in AR glasses. This constant shaking can easily cause dizziness, eye fatigue, and other adverse reactions in AR glasses wearers, significantly impacting the user experience and reducing assembly success rates. Therefore, there is an urgent need to propose corresponding solutions to address the pose oscillation problem and optimize the virtual image shaking caused by pose oscillation. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a virtual image jitter optimization method for 3D object tracking, so as to optimize the oscillation and jitter problem that occurs when the 3D model undergoes pose transformation and improve the user experience of AR glasses.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A virtual image jitter optimization method for 3D object tracking is disclosed. This method employs the ICG tracking algorithm, combined with a gray prediction model, optimization of tracking algorithm parameters, and Kalman filtering to achieve jitter optimization. Specifically, it includes the following steps:

[0007] S1. Create sample data to be used as a grey prediction model;

[0008] S2. Perform a grade ratio test on the sample data to verify the feasibility of using the grey prediction model for the data;

[0009] S3. Train the model used to solve the predicted pose;

[0010] S4. Determine the prediction accuracy of the model in step S3 based on the residual test, and obtain the predicted pose of the next frame from the model.

[0011] S5. Calculate the change between the predicted pose and the current frame pose to obtain ADDS. gray ;

[0012] S6, Fitting ADDS gray The mapping relationship between the learning rate 'a' and the learning rate 'a' is achieved through ADDS. gray Control the learning rate for the next frame;

[0013] S7. Use Kalman filtering to optimize jitter and obtain the optimized final pose.

[0014] Furthermore, step S1 specifically involves inputting the RGB-D image and the target object model into the 3D tracking algorithm to obtain the output pose G. i The sample data is as follows:

[0015] x (0) =G i (i = n - s + 1, n - s + 2, ..., n) = (x (0) (1),x (0) (2),...,x (0) (s))

[0016] In the formula, i represents the number of frames and s represents the number of samples.

[0017] Furthermore, step S2 specifically involves calculating the grade ratio λ(k) of the sample data:

[0018]

[0019] In the formula, x (0) (k) represents the sample data, and s represents the number of samples;

[0020] If λ(k) is in the interval Within the range, using the grey prediction model is feasible; if λ(k) is in the interval... In addition, an arbitrary constant c is added to each sample data, and it is determined whether the level ratio after adding the constant is within the interval. If the data is within the specified range, then using the grey prediction model is feasible; otherwise, discard the current frame data.

[0021] Furthermore, step S3 specifically involves summing the original sample data to obtain new sample data:

[0022] x (1)=(x (1) (1),x (1) (1)+x (1) (2),...,x (1) (1)+x (1) (2) + ... + x (1) (s))

[0023] A curve that fits the new sample data is constructed, and the functional expression of the curve is solved by first-order ordinary differential equations to serve as a model for predicting pose.

[0024] Furthermore, step S5 specifically involves: predicting the pose G... gray ADDS is obtained by performing ADD-S precision calculation with the current frame pose. gray ADDS gray It represents the magnitude of the change between the predicted pose and the current frame pose.

[0025] Furthermore, step S6 specifically involves defining ADDS. gray The effective range of variation is (ADDS) low ADDS high The learning rate *a* varies within the range of (a... low ,a high Using y = cos n Using the x function to fit ADDS gray The mapping relationship between the learning rate 'a' and the learning rate 'a' is used to obtain the learning rate 'a' for the next frame. s+1 :

[0026]

[0027] Furthermore, step S7 specifically involves: based on the obtained predicted pose G... gray and measurement pose G n The pose is optimized through two steps: prediction and update, to obtain the optimized final pose G. k .

[0028] The specific prediction steps are as follows:

[0029] (1) Solve for the estimated pose of the current frame.

[0030]

[0031] In the formula, G gray This represents the pose predicted for the k-th frame at frame (k-1).

[0032] (2) Solve for the covariance of the prior estimate

[0033]

[0034] In the formula, F is the state transition variable, which is a 7th-order identity matrix; P k-1 denoted by , where represents the covariance of the posterior estimate; Q represents the prediction noise covariance matrix.

[0035] The specific update steps are as follows:

[0036] (1) Update Kalman gain K k :

[0037]

[0038] In the formula, H represents the covariance of the prior estimate; H represents the observation matrix, which is a 7th order diagonal identity matrix; R represents the measurement noise covariance matrix.

[0039] (2) Correcting the optimal estimate

[0040]

[0041] In the formula, z k This represents the measurement value from the ICG tracking algorithm;

[0042] (3) Update the covariance P of the posterior estimate k :

[0043]

[0044] In the formula, I represents a 7th-order identity matrix.

[0045] The beneficial effects of this invention are as follows: This invention adopts the ICG tracking algorithm and combines gray prediction + tracking algorithm parameter optimization + Kalman filtering to achieve jitter optimization, which improves the problem of pose data oscillation when the 3D model changes pose according to the target part, enhances the user experience of AR glasses, and avoids dizziness, eye fatigue and other phenomena when using AR glasses.

[0046] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0048] Figure 1 Flowchart of a method for optimizing virtual image jitter in 3D object tracking. Detailed Implementation

[0049] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0050] Virtual image jitter is caused by excessively large changes in the estimated inter-frame pose of the real object by the pose estimation algorithm. This jitter is particularly noticeable to the user when the object is stationary. Therefore, this invention provides a jitter optimization method for virtual images in 3D object tracking. When the real object is relatively stationary, the optimization system prioritizes the stability of the virtual image; while when the real object begins to move, it prioritizes the accuracy of virtual image tracking. The pose estimation algorithm of this invention uses the ICG tracking algorithm and employs a combination of gray prediction, tracking algorithm parameter optimization, and Kalman filtering to achieve jitter optimization. The specific process is as follows: Figure 1 As shown:

[0051] I. Defining the Samples for the Grey Prediction GM(1,1) Model

[0052] In 3D space, the pose of an object is represented by its position and orientation; therefore, in this invention, G = [T, Q] is used. T The pose is represented by G, where T represents coordinates, T = [x, y, z], where x, y, and z represent the spatial coordinate axes; Q is a quaternion, Q = [w, a, b, c], where w, a, b, and c represent the coordinate axes of the quaternion Q, representing the object's orientation in space. Quaternions are used instead of Euler angles to represent the object's orientation because of their superior interpolation properties. The 3D tracking algorithm takes an RGB-D image and a target object model as input and outputs the pose G. i (i = 1, 2, ..., n), where the pose of the current frame is P. n Define the number of gray prediction samples s (in this embodiment, let s = 5), and take the latest s-frame pose as the sample. Then the gray prediction GM(1,1) pose sample is:

[0053] x (0) =G i (i = n - s + 1, n - s + 2, ..., n) = (x (0) (1),x (0) (2),...,x (0) (s)) (1)

[0054] II. Ratio Test of Data

[0055] To determine the feasibility of using the GM(1,1) model on the data, a tier test needs to be performed on the original data at the beginning, as follows:

[0056] calculate Determine whether λ(k) is in the interval If λ(k) is within the interval, then the GM(1,1) model can be used. If λ(k) is outside the interval, a translation transformation can be tried, that is, adding an arbitrary constant c to each data point and seeing if λ(k) of the data after adding the constant c is within the interval. If the data is within the specified interval, the grey prediction model is feasible; otherwise, the current frame data is discarded. If a constant c is added to the original data, the constant is subtracted from the output data obtained from the model solution.

[0057] III. Training the Solution Model

[0058] For the original sequence x (0) The new sequence x is obtained by summing the results. (1) :

[0059] x (1) =(x (1) (1),x (1) (1)+x (1) (2),...,x (1) (1)+x (1) (2) + ... + x (1) (s)) (2)

[0060] This new sequence can be approximated by an exponential curve or even a straight line. A first-order ordinary differential equation can be constructed to solve for the functional expression of the fitted curve.

[0061]

[0062] Equation (3) contains unknowns a and u. Solving for a and u yields the prediction equation, which is then used as the model for predicting the pose. The least squares method can be used to solve for a and u. According to equation (3), the matrix form of the equation Y = BU is obtained, which gives the following equation:

[0063]

[0064] Find (Y-BU) T Find the minimum value of U (Y-BU) and estimate the value of U:

[0065]

[0066] The parameters a and u can be obtained from equation (5), and then substituted into the original differential equation to solve the original differential equation.

[0067] IV. Model Validation

[0068] Model validation is performed to determine how much the fitted values ​​obtained from the prediction model in the previous step differ from the actual values. This step uses residual testing:

[0069]

[0070] In the formula, The original data is obtained by predicting it using a predictive model.

[0071] Determine the model accuracy based on the table below:

[0072] Table 1 Prediction Accuracy Levels

[0073]

[0074]

[0075] If the posterior error ratio is less than or equal to 0.65, the prediction model is acceptable, and the predicted pose for the next frame can be obtained based on the prediction model.

[0076] V. Calculate the magnitude of the change between the predicted pose and the current pose.

[0077] In attitude estimation, ADD-S accuracy is often used as a criterion for judging the accuracy of the predicted attitude. ADD-S first calculates the distance between two point sets whose attitudes have been transformed by the predicted and ground truth poses, and then extracts the average distance. When the object has symmetric pose ambiguity, the average distance is calculated from the nearest point between the two transformed sets. ADD-S accuracy is defined as the percentage of examples whose calculated average distance is less than 10% of the model diameter.

[0078]

[0079] Therefore, the predicted pose G gray Performing ADD-S accuracy calculations with the current frame pose yields the magnitude of the change between the predicted pose and the current frame pose. gray .

[0080] VI. Using ADDS gray The learning rate a controls the next frame.

[0081] Define ADDS gray The effective range of variation is (ADDS) low ADDS high When ADDS gray>ADDS high When the object is considered to be at rest, take ADDS. gray =ADDS high When ADDS gray <ADDS low If the object's pose change is considered too large, then ADDS is used. gray =ADDS low The range of the learning rate *a* is defined as (a... low ,a high In the ICG algorithm, when the number of iterations is kept to 3, a lower learning rate 'a' improves the jitter optimization, but also reduces accuracy significantly. Since users perceive virtual object jitter most noticeably when the object is stationary, in ADDS... gray When the learning rate is large, a smaller learning rate 'a' should be set for the next frame. At the same time, in order to ensure accuracy, the learning rate 'a' should be rapidly increased when the object starts to move and leaves the stationary state.

[0082] Therefore, adopting the approach of noise reduction through fusion connection of different audio inputs at both ends in industrial applications and the Brin-Von der Lune illumination model in computer graphics for specular angle processing, we use y = cos n Using the x function to fit ADDS gray The mapping relationship between the learning rate and the learning rate determines the learning rate 'a' for the next frame. s+1 for:

[0083]

[0084] VII. Optimize jitter using Kalman filtering

[0085] Based on the predicted pose G gray and measurement pose G n (Obtained via 3D tracking algorithm), jitter is optimized using Kalman filtering, including prediction and update steps.

[0086] (1) Prediction Steps

[0087] ① Solve for the estimated pose of the current frame.

[0088]

[0089] ② Solve for the covariance of the prior estimates

[0090]

[0091] In the formula, F is the state transition scalar, which is a 7th order identity matrix; P k-1The covariance is the posterior estimate obtained from the update step of the previous frame. P0 in the first frame can be initialized automatically, with an initial value set to a 7th-order identity matrix, and will be automatically updated thereafter. Q represents the prediction noise covariance matrix, which can be set manually. A smaller Q value indicates higher confidence in the predicted value, but an excessively small Q can also cause system divergence; a larger Q value indicates higher confidence in the measured value.

[0092] (2) Update steps

[0093] ① Update Kalman gain K k :

[0094]

[0095] In the formula, H represents the observation matrix, indicating which state variables in the state matrix are included in the measured values. A 7th-order diagonal identity matrix is ​​used to represent all observations. R represents the measurement noise covariance matrix, which can be set by the user. A smaller R value indicates a larger initial gain of the system and faster convergence, but introducing too much noise in steady-state conditions can easily lead to oscillations and non-convergence. A larger R value indicates a lower confidence in the measured values, and the response will also be slower.

[0096] ② Correcting the optimal estimate

[0097]

[0098] In the formula z k This represents the measurement value from the ICG tracking algorithm.

[0099] ③ Update the covariance P of the posterior estimate k :

[0100]

[0101] In the formula, I represents a 7th-order identity matrix.

[0102] 8. Output the optimized pose G for each frame. k ,

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for virtual image dithering optimization for 3D object tracking, the method comprising: The method adopts an ICG tracking algorithm, combines a grey prediction model, optimization of tracking algorithm parameters, and Kalman filtering to realize jitter optimization, and specifically includes the following steps. ​ S1, create sample data used as a gray prediction model; input the RGB-D image and the target object model into the 3D tracking algorithm to obtain the output pose G i Then the sample data is: In the formula, i represents the number of frames, s represents the number of samples S2, performing level ratio test on the sample data to verify the feasibility of using the grey prediction model for the data; Computing the order of the sample data : In the formula, represents sample data, s represents the number of samples; like In the interval Within this range, using a grey prediction model is feasible; if In the interval In addition, an arbitrary constant is added to each sample data. c Determine whether the series ratio after adding the constant is within the interval. If the data is within the specified range, then the grey prediction model is feasible; if it is not within the specified range, then the current frame data should be discarded. S3, training a model for solving the predicted pose; S4, judging the prediction accuracy of the model in step S3 according to a residual test, and obtaining the predicted pose of the next frame from the model; S5, calculating a change size of the predicted pose and the current frame pose to obtain ADDS gray ; S6, fitting ADDS gray a mapping relationship between the learning rate a ADDS gray controls the learning rate of the next frame;​ Definition ADDS gray The effective variation range of , the learning rate a The variation range of , using The function to fit ADDS gray The mapping relationship between the learning rate a , the next frame learning rate : S7, optimizing the jitter by using Kalman filtering to obtain the final optimized pose.

2. The method of claim 1, wherein: Step S3 is specifically as follows: performing accumulation summation on the original sample data to obtain new sample data: A curve fitted with the new sample data is constructed, and a function expression of the curve is solved through a first-order ordinary differential equation, to serve as a model of the predicted pose.

3. The method of claim 1, wherein: Step S5 is specifically: obtaining the predicted pose G gray and the current frame pose ADD-S to calculate the accuracy ADDS gray , to ADDS gray characterize the change size between the predicted pose and the current frame pose.

4. The method of claim 1, wherein: Step S7 is specifically: according to the obtained predicted pose G gray and the measured pose G n , the pose is optimized through the two steps of prediction and updating to obtain the optimized final pose G k .

5. The method of claim 4, wherein: The prediction step is specifically as follows: (1) solving an estimated pose of a current frame : In the formula, G gray represents the pose of the kth frame predicted at the k-1th frame; (2) Solving the covariance of the prior estimate : wherein F is a state transition variable, taking a 7x7 identity matrix; denotes the covariance of the posterior estimate; Q denotes the predicted noise covariance matrix.

6. The method of claim 4, wherein: The update step is specifically as follows: (1) update the Kalman gain K k : wherein represents the covariance of the prior estimate; H represents the observation matrix, taken as a 7th order diagonal unit matrix; R denotes the measurement noise covariance matrix; (2) Correcting the optimal estimate : In the formula, ICG tracking algorithm measurements; (3) updating the covariance of the posterior estimate P k : In the formulae, I denotes a 7th order identity matrix.

Images