Real-time six-degree-of-freedom pose tracking method for weak discrimination image sequence

Abstract

The invention provides a real-time six-degree-of-freedom pose tracking method for a weak-discrimination image sequence, and the method comprises the steps: generating a plurality of virtual camera viewpoints at an offline stage, and constructing a viewpoint model at each viewpoint position; in the tracking stage, dense optical flow of an image is calculated, and an object sparse internal corresponding relation between a current frame and a previous frame is calculated by using internal sampling points pre-generated in the off-line stage; proposing a motion probability model according to the obtained sparse flow, and carrying out the random edge filtering based on the motion probability model; aiming at the color probability graph and the filtering edge graph, performing contour point search by adopting a fan-shaped search strategy; constructing a contour energy function with robust weight for the contour point corresponding relation; aiming at the internal mode, constructing a block-by-block corresponding optical flow confidence coefficient and a weighted internal energy function; the contour energy function and the internal energy function are fused, the multi-modal energy function is constructed for joint pose optimization, and the method has the advantages of being high in precision, real-time and low in hardware dependence.

Description

Technical Field

[0001] The invention belongs to the fields of visual measurement, computer vision and robotics technology, and in particular relates to a real-time six-degree-of-freedom posture tracking method for weak-discrimination image sequences. Background Art

[0002] In the fields of mixed reality, robotic grasping, medical surgical navigation, and human-computer interaction, calculating the six-degree-of-freedom pose of a three-dimensional object with a given model in a continuous image sequence is a classic task. After years of in-depth research, many methods for six-degree-of-freedom object tracking have been proposed. These methods are mainly divided into feature point-based, direct method, edge-based, region-based, and deep learning-based strategies. Although single-modality methods have achieved remarkable results in performance, due to the complementarity between different modalities, multimodal tracking strategies that combine multiple modalities can often better improve the overall performance.

[0003] In recent years, contour-based methods have achieved significant improvements in accuracy and efficiency. To address the problems of partial occlusion and high similarity of foreground and background colors, some studies have proposed contour point search strategies and pixel-by-pixel weighted energy functions that consider distance and color constraints. In addition, to more effectively track inconspicuous objects, researchers have proposed a contour component model that identifies the correspondence of contour points through template matching based on gradient directions. To improve efficiency, a study introduced a corresponding line model and a new smooth step function, derived a posterior probability with Gaussian characteristics, and introduced a sparse viewpoint model to improve computational efficiency and avoid rendering three-dimensional models during tracking. At the same time, in order to cope with the tracking of large displacements of objects, a study proposed a non-local contour point search strategy and a hybrid pose optimization method that combines local and non-local optimization.

[0004] However, current advanced contour-based technologies still face important challenges. When the color distinction between the object and the background is low, the color difference between the foreground and the background cannot accurately segment the object contour, resulting in large errors when finding the correspondence between the object projection contour and the object image contour. This is particularly true in low-light environments, seriously affecting the stability of tracking. In addition, some cameras only record the intensity of light and generate grayscale images, resulting in unclear color differences between the object and the background, so tracking methods based on color statistics for contour segmentation are almost ineffective.

[0005] Edge images can make up for the problem of unclear object and background contour segmentation caused by relying solely on color probability. However, although commonly used edge extraction algorithms can obtain relatively clear contours, edge images are often disturbed by cluttered edges inside the object and in the background. Existing edge-based methods usually use the statistical difference in color between the object and the background as a criterion for removing cluttered edges, but the effectiveness of this strategy is limited when color discrimination is low. Another strategy is to calculate the corner points on the object contour and match them with the contour corner points in the image to reduce noise interference, but not all contours have obvious corner points. Therefore, how to effectively filter out cluttered edges in weakly discriminable image sequences has become an urgent problem to be solved. Summary of the invention

[0006] In order to solve the problem that low-cost and easy-to-deploy monocular cameras are difficult to track objects in six degrees of freedom when the distinction between the object and the background is low, the present invention provides a real-time six-degree-of-freedom posture tracking method for weak-distinction image sequences, which can achieve a very high tracking success rate even under extremely weak foreground and background distinction conditions such as low-light images or even grayscale images. The method is a low-cost and easy-to-deploy monocular camera six-degree-of-freedom object posture tracking method that can be used in scenes with weak foreground and background color distinction.

[0007] A real-time six-degree-of-freedom posture tracking method for a weakly distinguishable image sequence comprises the following steps:

[0008] S1: In the offline preprocessing stage, according to the 3D CAD model of a given object, N uniformly generated spherical surfaces centered at the origin of the 3D object coordinate system are generated. v sparse virtual camera viewpoint positions, and use virtual camera rendering to calculate the mask of a given object at each viewpoint position, and build a viewpoint model;

[0009] The information stored in the viewpoint model includes: the direction vector of the virtual camera optical center at the current viewpoint position pointing to the origin of the three-dimensional object coordinate system, the depth of the origin of the three-dimensional object coordinate system from the virtual camera optical center at the current viewpoint position, the three-dimensional object point coordinates corresponding to the two-dimensional contour points randomly sampled on the edge of the object mask, the direction vector in the three-dimensional object coordinate system corresponding to the normal vector of the two-dimensional contour points randomly sampled on the edge of the object mask, the three-dimensional object point coordinates corresponding to the two-dimensional internal points randomly sampled inside the object mask, and the average pixel area of ​​each internal sampling point in the object mask;

[0010] S2: In the tracking stage, for the current image frame I, determine the sparse virtual camera viewpoint direction v that is closest to the z-axis direction of the actual camera coordinate system i , where i ranges from 1 to N v; Project the three-dimensional object points corresponding to all the two-dimensional contour points stored in the i-th viewpoint model onto the current image frame I acquired by the actual camera, and obtain the object projection contour point x on the current image frame I cnt,m , where m is the serial number of the sampled two-dimensional contour point; at the same time, the direction vectors represented by the normal vectors of all two-dimensional contour points in the three-dimensional object coordinate system are projected onto the current image frame I acquired by the actual camera to obtain the two-dimensional contour normal vector n on the current image frame I cnt,m ; Project the three-dimensional object points corresponding to all two-dimensional internal points onto the current image frame I acquired by the actual camera, and obtain the object projection internal point x on the current image frame I in,n , where n is the serial number of the two-dimensional internal point obtained by sampling;

[0011] S3: Project the contour point x on the object cnt,m Fill the periphery with pixels to obtain the bounding box of the given object in the current image frame I, and use it as the region of interest R; cut out the area selected by the bounding box of the given object on the previous image frame I' acquired by the actual camera as the region of interest R'; obtain the internal points x of each projection of the given object by calculating the dense optical flow between R' and R in,n and the corresponding pixel x' in the region of interest R in,n The matching point pairs (x in,n ,x' in,n );

[0012] S4: Based on the internal matching point pairs (x in,n ,x' in,n ), using a filtering algorithm to filter the cluttered edges of the region of interest R, to obtain a region of interest F with the cluttered edges filtered out;

[0013] S5: Background color probability map P based on the region of interest R b The region of interest F of the filtered edges is obtained by adopting the fan-shaped contour point search strategy of multi-directional search lines to obtain the projection contour points x of each object on the current image frame I. cnt,m , and the data association between the object image contour points based on color probability segmentation and the object image contour points based on filtering edges. The data association includes: the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m mean, profile shape uncertainty, and noise uncertainty of the one-dimensional distribution of directions;

[0014] S6: Based on the projection contour points x of each object cnt,mThe average value of the one-dimensional distribution of each group of potential contour points, the contour shape uncertainty and the uncertainty, obtain the contour energy E based on color probability segmentation of a given object corresponding to the current image frame I scnt and the contour energy E based on the filtered edge ecnt ;

[0015] S7: Based on the internal matching point pair (x in,n ,x' in,n ), obtain the internal energy E of a given object corresponding to the current image frame I in ;

[0016] S8: According to the contour energy E scnt 、E ecnt and the internal energy E in A multimodal energy function is constructed, and according to the mapping relationship between the multimodal energy function and the six-degree-of-freedom posture, a nonlinear optimization algorithm is used to obtain the six-degree-of-freedom posture of a given object in the current image frame I.

[0017] Further, in step S4, based on the matching point pair (x in,n ,x' in,n ), the filtering algorithm is used to filter the cluttered edges of the region of interest R, and the region of interest F with filtered cluttered edges is obtained as follows:

[0018] S41: Calculate an edge map E of the region of interest R using an edge operator;

[0019] S42: Determine whether each edge point x on the edge graph E is a control point, and add the edge point x with a determination result of yes to the control point set; wherein the determination process of any edge point x is as follows:

[0020] S42a: Calculate the rough motion probability P at the edge point x on the edge map E crs (x):

[0021]

[0022] Among them, N r is the region of interest R with the edge point x as the center and the radius r c The internal point x of the object projection in the circular area in,n The corresponding point x' in,n The number of max The radius r of the circular area is the maximum cumulative number defined. c The calculation method is:

[0023]

[0024] in, is the average pixel area of ​​each internal sampling point in the object mask When constructing the viewpoint model, Z is the depth of the origin of the given object from the optical center of the i-th virtual camera. C is the depth value of the origin of the three-dimensional object coordinate system from the actual camera optical center, s c is the set adjustment factor;

[0025] S42b: Determine P crs (x) is greater than the preset low threshold and less than the preset high threshold. If the judgment result is yes, the fine motion probability P at the edge point x is calculated. fne (x):

[0026]

[0027] Among them, the standard deviation σ of the bivariate Gaussian distribution f The calculation method is as follows:

[0028]

[0029] Among them, s f is the set adjustment factor;

[0030] S42c: Determine P fne (x) is greater than a preset low threshold and less than a preset high threshold. If the judgment result is yes, the edge point x is regarded as a control point;

[0031] S43: Using an edge connection algorithm, connect all control points to obtain a region of interest F after filtering out the jumbled edges.

[0032] Further, in step S5, based on the background color probability map P of the region of interest R b , get the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The method for the mean value of a one-dimensional distribution in the direction is:

[0033] S51: Calculate the background color probability map P of the region of interest R in the current image frame I b ;

[0034] S52: For background color probability map P b Each object projection contour point x cnt,m , respectively construct each x cnt,m is the origin, each x cnt,m The corresponding two-dimensional contour normal vector n cnt,m is the axis of symmetry, and the angle between the left and right sides is [-θ max ,θ max ] sector search area, and l s,m(θ) represents the contour curve equation of a given object located in the fan-shaped search area, where θ max is the set angle value, l s,m (θ) is the projection contour point x of the given object on the mth object cnt,m The equation of the curve in the corresponding fan-shaped search area, θ∈[-θ max ,θ max ];

[0035] S53: For any sector search area, construct cnt,m As the origin, divide [-θ max ,θ max ] sam Search lines, N sam is an odd number; calculate the points on each search line on P b The directional gradient g along the search line m,i,j , and i represents the number of the search line, ranging from 1 to N sam , j represents the point number on the i-th search line; the points on each search line whose directional gradient is greater than 0 are respectively taken as candidate object image contour points on each search line, and the points with the largest directional gradient are taken as potential object image contour points on each search line;

[0036] S54: For any sector search area, obtain N sam N calculated on the search line sam The potential object image contour points are projected onto n cnt,m The average value of the one-dimensional position distribution obtained in the direction is as follows:

[0037]

[0038] Among them, n cnt,m Project the contour point x for the mth object cnt,m The corresponding normal vector, μ scnt,m Project the contour point x for the mth object cnt,m The corresponding multi-directional search for potential object image contour points along the contour normal vector n cnt,m The average value of the one-dimensional distribution in the direction, T represents the transpose, l s,m (θ i ) is the projection contour point x of the mth object cnt,m The direction angle θ of the i-th search line in the corresponding fan-shaped search area i The corresponding curve equation coordinates, θ i ∈[-θ max ,θ max ].

[0039] Furthermore, based on the background color probability map P of the region of interest R b, the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The uncertainty σ of the one-dimensional distribution of the direction scnt,m The calculation is as follows:

[0040] σ scnt,m 2 =σ sshp,m 2 σ snoi,m 2

[0041] Among them, σ sshp,m 2 Project the contour point x for the mth object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The variance of the one-dimensional distribution of the direction, σ snoi,m 2 Project the contour point x for the mth object cnt,m The corresponding noise uncertainty is calculated as follows:

[0042]

[0043] Among them, g m,j Project the contour point x for the mth object cnt,m The corresponding search area along the contour normal vector n cnt,m The jth point on the search line of the direction is at P b Upper edge n cnt,m The directional gradient value of the direction, G is the set of directional gradients corresponding to all candidate object image contour points on the search line.

[0044] Furthermore, the contour energy E based on color probability segmentation scnt The calculation method is:

[0045]

[0046] Among them, the weight β scnt,m is the set external point suppression coefficient, the residual r scnt,m Defined as x cnt,m and the average value μ scnt,m Between along n cnt,m One-dimensional projection distance in the direction, N cnt is the total number of object projection contour points.

[0047] Further, in step S5, based on the region of interest F of the filtered edges, the projection contour points x of each object are obtained. cnt,mThe corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The method for the mean value of a one-dimensional distribution in the direction is:

[0048] S51: For each object in the region of interest F, project the contour point x cnt,m , respectively construct each x cnt,m is the origin, each x cnt,m The corresponding two-dimensional contour normal vector n cnt,m is the axis of symmetry, and the angle between the left and right sides is [-θ max ,θ max ] sector search area, and l e,m (θ) represents the contour curve equation of a given object located in the fan-shaped search area, where θ max is the set angle value, l e,m (θ) is the projection contour point x of the given object on the mth object cnt,m The equation of the curve in the corresponding fan-shaped search area, θ∈[-θ max ,θ max ];

[0049] S52: For any sector search area, construct cnt,m As the origin, divide [-θ max ,θ max ] sam Search lines, N sam is an odd number; the edge points on each search line are taken as candidate object image contour points, and the distance from x to each search line is cnt,m The nearest candidate object image contour point is used as the potential object image contour point on each search line;

[0050] S53: For any sector search area, obtain N sam N calculated on the search line sam The potential object image contour points are projected onto n cnt,m The average value of the one-dimensional position distribution obtained in the direction is as follows:

[0051]

[0052] Among them, n cnt,m Project the contour point x for the mth object cnt,m The corresponding normal vector, μ ecnt,m Project the contour point x for the mth object cnt,m The corresponding multi-directional search for potential object image contour points along the contour normal vector n cnt,m The average value of the one-dimensional distribution in the direction, T represents the transpose, l e,m (θ i ) is the projection contour point x of the mth objectcnt,m The direction angle θ of the i-th search line in the corresponding fan-shaped search area i The corresponding curve equation coordinates, θ i ∈[-θ max ,θ max ].

[0053] Furthermore, based on the region of interest F of the filtered edges, the contour points x of each object are projected cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The uncertainty σ of the one-dimensional distribution of the direction ecnt,m The calculation is as follows:

[0054] σ ecnt,m 2 =σ eshp,m 2 σ enoi,m 2

[0055] Among them, σ eshp,m 2 Project the contour point x for the mth object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The variance of the one-dimensional distribution of the direction, σ enoi,m 2 Project the contour point x for the mth object cnt,m The corresponding noise uncertainty is calculated as follows:

[0056] σ enoi,m =f·|{c m,k}|

[0057] Among them, |{c m,k}| Projection contour point x for the mth object cnt,m The normal vector n of the corresponding search area along the object projection contour point cnt,m The number of candidate object image contour points on the search line in the direction, and f is the adjustment factor.

[0058] Furthermore, based on the contour energy E of the filtered edges ecnt The calculation method is:

[0059]

[0060] Among them, the weight β ecnt,m is the set external point suppression coefficient, the residual r ecnt,m Defined as x cnt,m and mean μ ecnt,m Between along n cnt,mOne-dimensional projection distance in the direction, N cnt is the total number of object projection contour points.

[0061] Furthermore, the internal energy E of a given object in the current image frame I in The calculation method is:

[0062]

[0063] Among them, the weight γ is the set external point suppression coefficient, c in,n is the optical flow confidence based on pixel-by-pixel blocks, and the residual r in,n Defined as the internal sampling point pair (x in,n ,x' in,n ) reprojection error, N in The total number of points inside the object projection.

[0064] Furthermore, according to the contour energy E scnt 、E ecnt and the internal energy E in The constructed multimodal energy function E is:

[0065] E=λE in +(1-λ)(ηE ecnt +(1-η)E scnt )

[0066] Among them, λ and η are both set weight factors.

[0067] Beneficial effects:

[0068] The present invention provides a real-time six-degree-of-freedom posture tracking method for a weak-discrimination image sequence. In an offline stage, a plurality of virtual camera viewpoints are generated, and a viewpoint model is constructed at each viewpoint position to store pre-calculated quantities. In a tracking stage, dense optical flow of the image is calculated, and the sparse internal correspondence of the object between the current frame and the previous frame is calculated using the internal sampling points pre-generated in the offline stage. According to the obtained sparse flow, a motion probability model is proposed, which includes a coarse model and a fine model, and cluttered edge filtering is performed based on the motion probability model. For a color probability map and a filtered edge map, a fan-shaped search strategy is used to search for contour points. A contour energy function with robust weights is constructed for the correspondence of contour points. For an internal mode, a block-by-block corresponding optical flow confidence and a weighted internal energy function are constructed. The contour energy function and the internal energy function are fused to construct a multi-modal energy function for joint posture optimization. It can be seen that the present invention is a high-precision, real-time, low-hardware-dependence object six-degree-of-freedom posture tracking method. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1A schematic diagram of an offline sparse viewpoint model provided by the present invention;

[0070] Figure 2 A schematic diagram of edge filtering based on sparse internal point correspondence of an object provided by the present invention;

[0071] Figure 3 A schematic diagram of the contour mode based on color probability segmentation and edge filtering provided by the present invention;

[0072] Figure 4 A schematic diagram of the internal mode based on the sparse correspondence relationship inside the object in the image provided by the present invention;

[0073] Figure 5 Schematic diagram of the multi-modal joint posture optimization provided by the present invention. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0075] The current dense optical flow algorithm can maintain high accuracy in grayscale images, and can approximately calculate the pixel-level correspondence between internal points of objects between adjacent frames even when the color distinction is not obvious; based on this, the present invention proposes a probabilistic method for object motion segmentation using the correspondence between sparse internal points of objects, thereby filtering out redundant edges in edge images; in order to improve the robustness of the tracking algorithm, a strategy for fusing multiple modalities for joint pose optimization is proposed; for contour modalities based on color statistical segmentation and contour modalities based on cluttered edge filtering, a fan-shaped contour point search strategy is constructed, and a robust contour energy function is derived; at the same time, for sparse internal point correspondences based on optical flow calculation, an internal energy function based on block-by-block corresponding optical flow confidence is proposed; by fusing internal modalities and contour modalities, this method can achieve high-precision, real-time six-degree-of-freedom object pose tracking in weakly distinguishable image sequences.

[0076] Specifically, a real-time six-degree-of-freedom posture tracking method for a weakly distinguishable image sequence comprises the following steps:

[0077] S1: In the offline preprocessing stage, according to the 3D CAD model of a given object, N uniformly generated spherical surfaces centered at the origin of the 3D object coordinate system are generated. v sparse virtual camera viewpoint positions, and use virtual camera rendering to calculate the mask of a given object at each viewpoint position, and construct a viewpoint model, such as Figure 1 As shown;

[0078] The information stored in the viewpoint model includes: the direction vector of the virtual camera optical center at the current viewpoint position pointing to the origin of the three-dimensional object coordinate system, the depth of the origin of the three-dimensional object coordinate system from the virtual camera optical center at the current viewpoint position, the three-dimensional object point coordinates corresponding to the two-dimensional contour points randomly sampled on the edge of the object mask, the direction vector in the three-dimensional object coordinate system corresponding to the normal vector of the two-dimensional contour points randomly sampled on the edge of the object mask, the three-dimensional object point coordinates corresponding to the two-dimensional internal points randomly sampled inside the object mask, and the average pixel area of ​​each internal sampling point in the object mask;

[0079] S2: In the tracking stage, for the current image frame I, determine the sparse virtual camera viewpoint direction v that is closest to the z-axis direction of the actual camera coordinate system i , where i ranges from 1 to N v ; Project the three-dimensional object points corresponding to all the two-dimensional contour points stored in the i-th viewpoint model onto the current image frame I acquired by the actual camera, and obtain the object projection contour point x on the current image frame I cnt,m , where m is the serial number of the sampled two-dimensional contour point; at the same time, the direction vectors represented by the normal vectors of all two-dimensional contour points in the three-dimensional object coordinate system are projected onto the current image frame I acquired by the actual camera to obtain the two-dimensional contour normal vector n on the current image frame I cnt,m ; Project the three-dimensional object points corresponding to all two-dimensional internal points onto the current image frame I acquired by the actual camera, and obtain the object projection internal point x on the current image frame I in,n , where n is the serial number of the two-dimensional internal point obtained by sampling;

[0080] S3: Project the contour point x on the object cnt,m Fill the periphery with pixels to obtain the bounding box of the given object in the current image frame I, and use it as the region of interest R; cut out the area selected by the bounding box of the given object on the previous image frame I' acquired by the actual camera as the region of interest R'; obtain the internal points x of each projection of the given object by calculating the dense optical flow between R' and R in,n and the corresponding pixel x' in the region of interest R in,n The matching point pairs (x in,n ,x' in,n );

[0081] S4: Based on the internal matching point pairs (x in,n ,x' in,n ), using a filtering algorithm to filter the cluttered edges of the region of interest R, to obtain a region of interest F with the cluttered edges filtered, specifically comprising the following steps:

[0082] S41: Calculate an edge map E of the region of interest R using an edge operator;

[0083] S42: Determine whether each edge point x on the edge graph E is a control point, and add the edge point x with a determination result of yes to the control point set; wherein the determination process of any edge point x is as follows:

[0084] S42a: Calculate the rough motion probability P at the edge point x on the edge map E crs (x):

[0085]

[0086] Among them, N r is the region of interest R with the edge point x as the center and the radius r c The internal point x of the object projection in the circular area in,n The corresponding point x' in,n The number of max The radius r of the circular area is the maximum cumulative number defined. c The calculation method is:

[0087]

[0088] in, is the average pixel area of ​​each internal sampling point in the object mask When constructing the viewpoint model, Z is the depth of the origin of the given object from the optical center of the i-th virtual camera. C is the depth value of the origin of the three-dimensional object coordinate system from the actual camera optical center, s c is the set adjustment factor;

[0089] S42b: Determine P crs (x) is greater than the preset low threshold and less than the preset high threshold. If the judgment result is yes, the fine motion probability P at the edge point x is calculated. fne (x):

[0090]

[0091] Among them, the standard deviation σ of the bivariate Gaussian distribution f The calculation method is as follows:

[0092]

[0093] Among them, s f is the set adjustment factor;

[0094] S42c: Determine P fne (x) is greater than a preset low threshold and less than a preset high threshold. If the judgment result is yes, the edge point x is regarded as a control point;

[0095] S43: Using an edge connection algorithm, connect all control points to obtain a region of interest F after filtering out the jumbled edges.

[0096] S5: Background color probability map P based on the region of interest R b The region of interest F of the filtered edges is obtained by adopting the fan-shaped contour point search strategy of multi-directional search lines to obtain the projection contour points x of each object on the current image frame I. cnt,m , and the data association between the object image contour points based on color probability segmentation and the object image contour points based on filtering edges. The data association includes: the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m mean, profile shape uncertainty, and noise uncertainty of the one-dimensional distribution of directions;

[0097] It should be noted that the present invention provides two methods to obtain the projection contour points x of each object: cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m Mean, profile shape uncertainty, and noise uncertainty of the one-dimensional distribution of directions.

[0098] The first one is based on the background color probability map P of the region of interest R. b , get the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The method for the mean value of a one-dimensional distribution in the direction is:

[0099] S51: Calculate the background color probability map P of the region of interest R in the current image frame I b ;

[0100] S52: For background color probability map P b Each object projection contour point x cnt,m , respectively construct each x cnt,m is the origin, each x cnt,m The corresponding two-dimensional contour normal vector n cnt,m is the axis of symmetry, and the angle between the left and right sides is [-θ max ,θ max ] sector search area, and l s,m (θ) represents the contour curve equation of a given object located in the fan-shaped search area, where θ max is the set angle value, l s,m (θ) is the projection contour point x of the given object on the mth object cnt,m The equation of the curve in the corresponding fan-shaped search area, θ∈[-θ max ,θmax ]; that is, the contour curve equation of the object located in the fan-shaped search area is l s,m (θ) indicates;

[0101] S53: For any sector search area, construct cnt,m As the origin, divide [-θ max ,θ max ] sam Search lines, N sam is an odd number; calculate the points on each search line on P b The directional gradient g along the search line m,i,j , and i represents the number of the search line, ranging from 1 to N sam , j represents the point number on the i-th search line, and the value range of j is related to the actual number of points on the search line; the points with directional gradient greater than 0 on each search line are respectively regarded as candidate object image contour points on each search line, and the points with the largest directional gradient are regarded as potential object image contour points on each search line;

[0102] S54: For any sector search area, obtain N sam N calculated on the search line sam The potential object image contour points are projected onto n cnt,m The average value of the one-dimensional position distribution obtained in the direction is as follows:

[0103]

[0104] Among them, n cnt,m Project the contour point x for the mth object cnt,m The corresponding normal vector, μ scnt,m Project the contour point x for the mth object cnt,m The corresponding multi-directional search for potential object image contour points along the contour normal vector n cnt,m The average value of the one-dimensional distribution in the direction, T represents the transpose, l s,m (θ i ) is the projection contour point x of the mth object cnt,m The direction angle θ of the i-th search line in the corresponding fan-shaped search area i The corresponding curve equation coordinates, θ i ∈[-θ max ,θ max ].

[0105] At the same time, based on the background color probability map P of the region of interest R b , the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The uncertainty σ of the one-dimensional distribution of the directionscnt,m The calculation is as follows:

[0106] σ scnt,m 2 =σ sshp,m 2 σ snoi,m 2

[0107] Among them, σ sshp,m 2 Project the contour point x for the mth object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The variance of the one-dimensional distribution of the direction, which reflects the shape uncertainty of the object image contour, σ snoi,m 2 Project the contour point x for the mth object cnt,m The corresponding noise uncertainty is calculated as follows:

[0108]

[0109] Among them, g m,j Project the contour point x for the mth object cnt,m The normal vector n of the corresponding search area along the object projection contour point cnt,m The jth point on the search line of the direction is at P b Upper edge n cnt,m The directional gradient value of the direction, G is the set of directional gradients corresponding to all candidate object image contour points on the search line.

[0110] The second method is to obtain the projection contour points x of each object based on the region of interest F of the filtered edges. cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The method for the mean value of a one-dimensional distribution in the direction is:

[0111] S51: For each object in the region of interest F, project the contour point x cnt,m , respectively construct each x cnt,m is the origin, each x cnt,m The corresponding two-dimensional contour normal vector n cnt,m is the axis of symmetry, and the angle between the left and right sides is [-θ max ,θ max ] sector search area, and l e,m (θ) represents the contour curve equation of a given object located in the fan-shaped search area, where θ max is the set angle value, l e,m (θ) is the projection contour point x of the given object on the mth object cnt,mThe equation of the curve in the corresponding fan-shaped search area, θ∈[-θ max ,θ max ]; that is, the contour curve equation of the object located in the fan-shaped search area is l e,m (θ) indicates;

[0112] S52: For any sector search area, construct cnt,m As the origin, divide [-θ max ,θ max ] sam Search lines, N sam is an odd number; the edge points on each search line are taken as candidate object image contour points, and the distance from x to each search line is cnt,m The nearest candidate object image contour point is used as the potential object image contour point on each search line;

[0113] S53: For any sector search area, obtain N sam N calculated on the search line sam The potential object image contour points are projected onto n cnt,m The average value of the one-dimensional position distribution obtained in the direction is as follows:

[0114]

[0115] Among them, n cnt,m Project the contour point x for the mth object cnt,m The corresponding normal vector, μ ecnt,m Project the contour point x for the mth object cnt,m The corresponding multi-directional search for potential object image contour points along the contour normal vector n cnt,m The average value of the one-dimensional distribution in the direction, T represents the transpose, l e,m (θ i ) is the projection contour point x of the mth object cnt,m The direction angle θ of the i-th search line in the corresponding fan-shaped search area i The corresponding curve equation coordinates, θ i ∈[-θ max ,θ max ].

[0116] At the same time, based on the region of interest F of the filtered edges, the contour points x of each object are projected cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The uncertainty σ of the one-dimensional distribution of the direction ecnt,m The calculation is as follows:

[0117] σ ecnt,m 2 =σ eshp,m2 σ enoi,m 2

[0118] Among them, σ eshp,m 2 Project the contour point x for the mth object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The variance of the one-dimensional distribution of the direction, σ enoi,m 2 Project the contour point x for the mth object cnt,m The corresponding noise uncertainty is calculated as follows:

[0119] σ enoi,m =f·|{c m,k}|

[0120] Among them, |{c m,k}| Projection contour point x for the mth object cnt,m The normal vector n of the corresponding search area along the object projection contour point cnt,m The number of candidate object image contour points on the search line in the direction, and f is the adjustment factor.

[0121] S6: Based on the projection contour points x of each object cnt,m The average value of the one-dimensional distribution of each group of potential object image contour points, contour shape uncertainty and noise uncertainty are calculated to obtain the contour energy E of the given object based on color probability segmentation corresponding to the current image frame I. scnt and the contour energy E based on the filtered edge ecnt ;

[0122] Specifically, the contour energy E based on color probability segmentation scnt The calculation method is:

[0123]

[0124] Among them, the weight β scnt,m is the set external point suppression coefficient, the residual r scnt,m Defined as x cnt,m and the average value μ scnt,m Between along n cnt,m One-dimensional projection distance in the direction, N cnt is the total number of object projection contour points.

[0125] Furthermore, based on the contour energy E of the filtered edges ecnt The calculation method is:

[0126]

[0127] Among them, the weight β ecnt,m is the set external point suppression coefficient, the residual r ecnt,m Defined as x cnt,m and mean μ ecnt,m Between along n cnt,m One-dimensional projection distance in the direction, N cnt is the total number of object projection contour points.

[0128] S7: Based on the internal matching point pair (x in,n ,x' in,n ), obtain the internal energy E of a given object corresponding to the current image frame I in , as follows:

[0129]

[0130] Among them, the weight γ is the set external point suppression coefficient, c in,n is the optical flow confidence based on pixel-by-pixel blocks, and the residual r in,n Defined as the internal sampling point pair (x in,n ,x' in,n ) reprojection error, N in The total number of points inside the object projection.

[0131] S8: According to the contour energy E scnt 、E ecnt and the internal energy E in A multimodal energy function is constructed, and according to the mapping relationship between the multimodal energy function and the six-degree-of-freedom posture, a nonlinear optimization algorithm is used to obtain the six-degree-of-freedom posture of a given object in the current image frame I.

[0132] Specifically, according to the contour energy E scnt 、E ecnt and the internal energy E in The constructed multimodal energy function E is:

[0133] E=λE in +(1-λ)(ηE ecnt +(1-η)E scnt )

[0134] Among them, λ and η are both set weight factors.

[0135] The implementation modes of the present invention are further described in detail below with reference to specific parameters.

[0136] Step 1: In the offline preprocessing stage, according to the 3D CAD model of a given object, 2562 sparse virtual camera viewpoint positions are uniformly generated on a spherical surface with a radius of 0.8m and the origin of the 3D object coordinate system as the center, and a viewpoint model is constructed at each viewpoint position to store the pre-calculated amount. Figure 1 shown.

[0137] (1) For the i-th virtual camera viewpoint, store the direction vector v from the optical center of the virtual camera to the origin of the object coordinate system i , stores the depth of the object coordinate system origin from the camera optical center.

[0138] (2) For the i-th virtual camera viewpoint, the depth map of the 3D object is rendered using the OpenGL rendering pipeline. Figure 2 The object mask is obtained by quantizing the depth of the object mask. 200 2D contour points are randomly sampled on the edge of the object mask and their 2D normal vectors are calculated. 200 2D internal points are randomly sampled inside the object mask. The 2D sampled points are mapped to the 3D object coordinate system according to the rendered depth value and the virtual camera pose, and the coordinates of the corresponding 3D object points are stored. The 2D normal vector is converted to the 3D object coordinate system and the direction vector is stored.

[0139] (3) Calculate the pixel area occupied by the object mask and divide this area by 200 to get the average pixel area occupied by each internal sampling point

[0140] Step 2: In the tracking stage, input the current image frame I, and calculate the pose T of the object coordinate system to the camera coordinate system according to the previous frame I' CM , project the contour sampling points, contour normal vectors and internal sampling points of the three-dimensional object model onto I, and obtain other pre-calculated quantities.

[0141] (1) First, calculate the sparse viewpoint direction v that is closest to the camera z-axis direction i Reproject all contour points in the three-dimensional object coordinate system stored in the viewpoint model onto I to obtain the object projection contour point x cnt,m , fill 40 pixels around these projected contour points to obtain the bounding box of the object in the image. Reproject all normal vectors in the three-dimensional object coordinate system stored in the viewpoint model onto I to obtain the two-dimensional contour normal vector n cnt,m .

[0142] (2) Project all internal points in the three-dimensional object coordinate system stored in the viewpoint model onto I to obtain the object projection internal point x in,n Get the average pixel area of ​​the object mask occupied by the internal sampling points stored in the viewpoint model And the depth of the object origin from the camera optical center when constructing the viewpoint model

[0143] Step 3: Based on the bounding box of the object in the image calculated in step 2 (1), the regions of interest R' and R in the bounding box are cropped on the previous image frame I' and the current image frame I, respectively. The DIS dense optical flow algorithm is used to calculate the dense optical flow between the R' and R regions to obtain the internal matching point pairs of the object (x in,n ,x' in,n ).

[0144] Step 4: Based on the sparse object internal matching point pairs obtained in step 3 (x in,n ,x' in,n ), use filtering algorithm to filter out messy edges. Figure 2 shown.

[0145] (1) The Canny edge detection algorithm is used to calculate the edge map E of the region of interest R. The kernel size of the Sobel operator used in the Canny edge detection algorithm is 5×5, the high threshold is set to 1200, and the low threshold is set to 400.

[0146] (2) According to formula (1), calculate the coarse motion probability P at the edge point x on the edge graph E crs (x):.

[0147]

[0148] N r is centered at pixel x, r c The corresponding point x' is the internal projection in the circular area with radius in,n The radius of the circle is r c According to formula (2), Z C It is the depth value of the origin of the object coordinate system in the camera coordinate system.

[0149]

[0150] (3) Determine P crs (x) Whether it is greater than the preset low threshold of 0.2 and less than the preset high threshold of 0.9.

[0151] (4) According to the judgment result of step 4 (3), if the conditions are met, further calculate the fine motion probability P at x fne (x):

[0152]

[0153] Among them, the standard deviation σ of the bivariate Gaussian distribution f Calculate according to formula (4).

[0154]

[0155] (5) If P fne If (x) is greater than the preset low threshold of 0.3 and less than the preset high threshold of 0.6, x is regarded as a control point and added to the control point set.

[0156] (6) Use the 8-pixel neighborhood edge connection algorithm based on gradient consistency to obtain the segmented edges of the object after filtering out the noise in E. For each control point in the control point set, find the two points in its 8-pixel neighborhood that are most consistent with its gradient direction as the starting point of the edge search. For each starting point, search along the edge in E for points that satisfy gradient consistency in its 8-pixel neighborhood until no more points that meet the conditions are found. For segmented edges whose two endpoints are control points, directly add them to the filtered edge set; for segmented edges with only one end being a control point, first calculate the ratio r of the number of points in each edge point whose coarse motion probability satisfies the lower threshold of 0.2 and less than the upper threshold of 0.9 to the total number of edge points in this section. vld , if r vld If the value of the segment edge is greater than the preset threshold value 0.7, the segment edge is added to the edge set. These segment edges constitute the region of interest F for filtering the edges.

[0157] Step 5: Based on the fan-shaped contour point search strategy of multi-directional search lines, construct the data association between the object projection contour points and the object image contour points. Figure 3 shown.

[0158] (1) Data association between object projection contour points and object image contour points based on color probability segmentation.

[0159] ① Construct the background color probability map P of the region of interest R in the current image frame I b The calculation process is as shown in formula (5), where p f (I(x)) and p b (I(x)) represents the normalized value of the current object and background color statistical histogram when the pixel value I(x) at a given pixel x is given.

[0160] P b (x) = p b (I(x)) / (p f (I(x))+p b (I(x))) (5)

[0161] ②For each background color probability map P b Projection contour point x of the object cnt,m , construct x cnt,m As the origin, with the normal vector n cnt,m The fan-shaped search area is a symmetry axis with a left and right angle of [-30,30]. The contour curve equation of the object located in the fan-shaped search area is ls,m (θ) indicates.

[0162] ③ In this fan-shaped search area, construct cnt,m As the origin, there are seven search lines in 60° directions. m,i,x ,l m,i,y ] T The jth point x on the direction search line m,i,j =[x m,i,j ,y m,i,j ] T , according to formula (6) to calculate its b Upper edge m,i,x ,l m,i,y ] T Directional gradient g m,i,j :

[0163]

[0164] and is the color probability map P b In x m,i,j The gradient along the x and y directions at the pixel. Points with directional gradients greater than 0 are considered to be candidate object image contour points on the search line, and points with the largest directional gradients are considered to be potential object image contour points on the search line.

[0165] ④ Project the potential object image contour points calculated on the 7 search lines to n cnt,m In the direction, the average value of the one-dimensional position distribution is calculated according to formula (7):

[0166]

[0167] ⑤ Use equation (8) to calculate the shape uncertainty and limit it to be greater than 1:

[0168]

[0169] ⑥ Use equation (9) to calculate the noise uncertainty, defined as n cnt,m The sum of the directional gradients of all candidate object image contour points searched by the directional search line is divided by the directional gradient of the potential object image contour point:

[0170]

[0171] ⑦ Multiply the shape uncertainty and the noise uncertainty to obtain the uncertainty of the contour point distribution of the object image based on color probability segmentation, as shown in formula (10):

[0172] σ scnt,m 2 =σsshp,m 2 σ snoi,m 2 (10)

[0173] (2) Data association between object projection contour points and object image contour points based on filtered edges.

[0174] ① Obtain the region of interest F of the filtered edges obtained in step 4.

[0175] ② For each object projection contour point x in the region of interest F of the filtered edge cnt,m , construct x cnt,m As the origin, with the normal vector n cnt,m The fan-shaped search area is a symmetry axis with a left and right angle of [-30,30]. The contour curve equation of the object located in the fan-shaped search area is l e,m (θ) indicates.

[0176] ③ In this fan-shaped search area, construct cnt,m As the origin, there are seven search lines in 60° directions. m,i,x ,l m,i,y ] T The jth point x on the direction search line m,i,j =[x m,i,j ,y m,i,j ] T , if this point is an edge point, it is recorded as a candidate object image contour point, and the distance from x cnt,m The nearest candidate object image contour point is considered to be a potential object image contour point on this search line.

[0177] ④ Project the potential object image contour points calculated on the 7 search lines to n cnt,m In the direction, the average value of the one-dimensional position distribution is calculated according to formula (11):

[0178]

[0179] ⑤ Use equation (12) to calculate the shape uncertainty and limit it to be greater than 1:

[0180]

[0181] ⑥ Use equation (13) to calculate the noise uncertainty, defined as n cnt,m The number of all filtered edge points found by the directional search line is multiplied by 0.9:

[0182] σ enoi,m =0.9·|{c m,i}| (13)

[0183] ⑦ Multiply the shape uncertainty and the noise uncertainty to obtain the uncertainty of the final object image contour point distribution based on the filtered edges, as shown in formula (14):

[0184] σ ecnt,m 2 =σ eshp,m 2 σ enoi,m 2 (14)

[0185] Step 6: Construct a contour energy function based on robust weights. Figure 3 shown.

[0186] (1) For all projected contour points x cnt,m , use formula (15) to calculate the contour energy E based on color probability segmentation scnt :

[0187] The weight

[0188] Residual r scnt,m Defined as x cnt,m and mean μ scnt,m Between along n cnt,m The one-dimensional projection distance in the direction.

[0189] (2) For all projected contour points x cnt,m , use formula (16) to calculate the contour energy E based on the filtered edges ecnt :

[0190] The weight

[0191] Residual r ecnt,m Defined as x cnt,m and mean μ ecnt,m Between along n cnt,m The one-dimensional projection distance in the direction.

[0192] Step 7: Calculate the internal energy function of the sparse correspondence inside the object according to formula (17). Figure 4 shown.

[0193] The weight

[0194] Residual r in,n Defined as the internal corresponding point pair (x in,n ,x' in,n ) reprojection error. in,n To calculate the optical flow confidence based on pixel-by-pixel blocks, use formula (18):

[0195] c in,n =max{1-e n 2 ,0} (18)

[0196] e n For x in,n The surrounding 3×3 pixel block is related to x' in,n The image intensity error e between the surrounding 3×3 pixel blocks I,n , image gradient error e G,n and the optical flow field smoothness error e S,n The sum of is shown in formula (19):

[0197] e n =e I,n +e G,n +e S,n (19)

[0198] Step 8: Construct the multimodal energy function shown in formula (20), and use the Gauss-Newton method with regularization term to solve the pose, as follows: Figure 5 shown.

[0199] E=0.5·E in +0.5·(0.3·E ecnt +0.7·E scnt ) (20)

[0200] In summary, compared with the prior art, the technical effects achieved by the present invention are:

[0201] The present invention proposes a high-precision, high-robust, low-hardware-device-dependent real-time six-degree-of-freedom object posture tracking algorithm, which can achieve a very high tracking success rate under the condition of relying only on a monocular camera and a CPU, and under extremely weak foreground-background distinction conditions such as low-light images or even grayscale images. On the grayscale RBOT dataset, due to the decrease in foreground-background color distinction, the traditional methods based on the statistical difference of foreground-background colors have a significant performance degradation, but the method proposed by the present invention still has a very high tracking success rate, which greatly exceeds the most advanced existing algorithms, reflecting the strong robustness of the present invention in weakly distinguished image sequences, as shown in Table 1.

[0202] Table 1

[0203]

[0204] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may certainly make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A real-time six-degree-of-freedom pose tracking method for weak-resolution image sequences, characterized in that: The following steps are involved: S1: In the offline preprocessing stage, according to the 3D CAD model of a given object, N uniformly generated spherical surfaces centered at the origin of the 3D object coordinate system are generated. v sparse virtual camera viewpoint positions, and use virtual camera rendering to calculate the mask of a given object at each viewpoint position, and build a viewpoint model; The information stored in the viewpoint model includes: the direction vector of the virtual camera optical center at the current viewpoint position pointing to the origin of the three-dimensional object coordinate system, the depth of the origin of the three-dimensional object coordinate system from the virtual camera optical center at the current viewpoint position, the three-dimensional object point coordinates corresponding to the two-dimensional contour points randomly sampled on the edge of the object mask, the direction vector in the three-dimensional object coordinate system corresponding to the normal vector of the two-dimensional contour points randomly sampled on the edge of the object mask, the three-dimensional object point coordinates corresponding to the two-dimensional internal points randomly sampled inside the object mask, and the average pixel area of ​​each internal sampling point in the object mask; S2: In the tracking stage, for the current image frame I, determine the sparse virtual camera viewpoint direction v that is closest to the z-axis direction of the actual camera coordinate system i , where the value of i ranges from 1 to N v ; Project the three-dimensional object points corresponding to all the two-dimensional contour points stored in the i-th viewpoint model onto the current image frame I acquired by the actual camera, and obtain the object projection contour point x on the current image frame I cnt,m , where m is the serial number of the sampled two-dimensional contour point; at the same time, the direction vectors represented by the normal vectors of all two-dimensional contour points in the three-dimensional object coordinate system are projected onto the current image frame I acquired by the actual camera to obtain the two-dimensional contour normal vector n on the current image frame I cnt,m ; Project the three-dimensional object points corresponding to all two-dimensional internal points onto the current image frame I acquired by the actual camera, and obtain the object projection internal point x on the current image frame I in,n , where n is the serial number of the two-dimensional internal point obtained by sampling; S3: Project contour point x on the object cnt,m Fill the periphery with pixels to obtain the bounding box of the given object in the current image frame I, and use it as the region of interest R; cut out the area selected by the bounding box of the given object on the previous image frame I' acquired by the actual camera as the region of interest R'; obtain the internal points x of each projection of the given object by calculating the dense optical flow between R' and R in,n and the corresponding pixel x' in the region of interest R in,n The matching point pairs between (x in,n ,x' in,n ); S4: Based on the internal matching point pairs (x in,n ,x' in,n ), using a filtering algorithm to filter the cluttered edges of the region of interest R, to obtain a region of interest F with the cluttered edges filtered out; S5: Background color probability map P based on the region of interest R b The region of interest F of the filtered edges is obtained by adopting the fan-shaped contour point search strategy of multi-directional search lines to obtain the projection contour points x of each object on the current image frame I. cnt,m , and the data association between the object image contour points based on color probability segmentation and the object image contour points based on filtering edges. The data association includes: the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m mean, profile shape uncertainty, and noise uncertainty of the one-dimensional distribution of directions; S6: Based on the projection contour points x of each object cnt,m The average value of the one-dimensional distribution of each group of potential contour points, the contour shape uncertainty and the uncertainty, obtain the contour energy E based on color probability segmentation of a given object corresponding to the current image frame I scnt and the contour energy E based on the filtered edge ecnt ; S7: Based on the internal matching point pair (x in,n ,x' in,n ), obtain the internal energy E of a given object corresponding to the current image frame I in ; S8: According to the contour energy E scnt 、E ecnt and the internal energy E in A multimodal energy function is constructed, and according to the mapping relationship between the multimodal energy function and the six-degree-of-freedom posture, a nonlinear optimization algorithm is used to obtain the six-degree-of-freedom posture of a given object in the current image frame I.

2. A real-time six-degree-of-freedom pose tracking method for a weak-resolution image sequence as claimed in claim 1, characterized in that: In step S4, based on the matching point pairs (x in,n ,x' in,n ), the filtering algorithm is used to filter the cluttered edges of the region of interest R, and the region of interest F with filtered cluttered edges is obtained as follows: S41: Calculate an edge map E of the region of interest R using an edge operator; S42: Determine whether each edge point x on the edge graph E is a control point, and add the edge point x with a determination result of yes to the control point set; wherein the determination process of any edge point x is as follows: S42a: Calculate the rough motion probability P at the edge point x on the edge map E crs (x): Among them, N r is the region of interest R with the edge point x as the center and the radius r c The internal point x of the object projection in the circular area in,n The corresponding point x' in,n The number of max The radius r of the circular area is the maximum cumulative number defined. c The calculation method is: in, is the average pixel area of ​​each internal sampling point in the object mask When constructing the viewpoint model, Z is the depth of the origin of the given object from the optical center of the i-th virtual camera. C is the depth value of the origin of the three-dimensional object coordinate system from the actual camera optical center, s c is the set adjustment factor; S42b: Determine P crs (x) is greater than the preset low threshold and less than the preset high threshold. If the judgment result is yes, the fine motion probability P at the edge point x is calculated. fne (x): Among them, the standard deviation σ of the bivariate Gaussian distribution f The calculation method is as follows: Among them, s f is the set adjustment factor; S42c: Determine P fne (x) is greater than a preset low threshold and less than a preset high threshold. If the judgment result is yes, the edge point x is regarded as a control point; S43: Using an edge connection algorithm, connect all control points to obtain a region of interest F after filtering out the jumbled edges.

3. A real-time six-degree-of-freedom posture tracking method for a weak-resolution image sequence as claimed in claim 1, characterized in that: In step S5, based on the background color probability map P of the region of interest R b , get the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The method for the mean value of a one-dimensional distribution in the direction is: S51: Calculate the background color probability map P of the region of interest R in the current image frame I b ; S52: For background color probability map P b Each object projection contour point x cnt,m , respectively construct cnt,m is the origin, each x cnt,m The corresponding two-dimensional contour normal vector n cnt,m is the axis of symmetry, and the angle between the left and right sides is [-θ max ,θ max ] sector search area, and l s,m (θ) represents the contour curve equation of a given object located in the fan-shaped search area, where θ max is the set angle value, l s,m (θ) is the projection contour point x of the given object on the mth object cnt,m The equation of the curve in the corresponding fan-shaped search area, θ∈[-θ max ,θ max ]; S53: For any sector search area, construct cnt,m is the origin, and divides [-θ max ,θ max ] sam Search lines, N sam is an odd number; calculate the points on each search line on P b The directional gradient g along the search line m,i,j , and i represents the number of the search line, ranging from 1 to N sam , j represents the point number on the i-th search line; the points on each search line whose directional gradient is greater than 0 are respectively taken as candidate object image contour points on each search line, and the points with the largest directional gradient are taken as potential object image contour points on each search line; S54: For any sector search area, obtain N sam N calculated on the search line sam The potential object image contour points are projected onto n cnt,m The average value of the one-dimensional position distribution obtained in the direction is as follows: Among them, n cnt,m Project the contour point x for the mth object cnt,m The corresponding normal vector, μ scnt,m Project the contour point x for the mth object cnt,m The corresponding multi-directional search for potential object image contour points along the contour normal vector n cnt,m The average value of the one-dimensional distribution in the direction, T represents the transpose, l s,m (θ i ) is the projection contour point x of the mth object cnt,m The direction angle θ of the i-th search line in the corresponding fan-shaped search area i The corresponding curve equation coordinates, θ i ∈[-θ max ,θ max ].

4. A real-time six-degree-of-freedom pose tracking method for a weak-resolution image sequence as claimed in claim 3, characterized in that: Background color probability map P based on the region of interest R b , the projection contour points x of each object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The uncertainty σ of the one-dimensional distribution of the direction scnt,m The calculation is as follows: s scnt,m 2 =s sshp,m 2 s snoi,m 2 Among them, σ sshp,m 2 Project the contour point x for the mth object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The variance of the one-dimensional distribution of the direction, σ snoi,m 2 Project the contour point x for the mth object cnt,m The corresponding noise uncertainty is calculated as follows: Among them, g m,j Project the contour point x for the mth object cnt,m The corresponding search area along the contour normal vector n cnt,m The jth point on the search line of the direction is at P b Upper edge n cnt,m The directional gradient value of the direction, G is the set of directional gradients corresponding to all candidate object image contour points on the search line.

5. A real-time six-degree-of-freedom posture tracking method for a weak-resolution image sequence as claimed in claim 4, characterized in that: Contour Energy E Based on Color Probability Segmentation scnt The calculation method is: Among them, the weight β scnt,m is the set external point suppression coefficient, the residual r scnt,m Defined as x cnt,m and the average value μ scnt,m Between along n cnt,m One-dimensional projection distance in the direction, N cnt is the total number of object projection contour points.

6. A real-time six-degree-of-freedom pose tracking method for a weak-resolution image sequence as claimed in claim 1, characterized in that: In step S5, based on the region of interest F of the filtered edges, the projection contour points x of each object are obtained. cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The method for the mean value of a one-dimensional distribution in the direction is: S51: For each object in the region of interest F, project the contour point x cnt,m , respectively construct cnt,m is the origin, each x cnt,m The corresponding two-dimensional contour normal vector n cnt,m is the axis of symmetry, and the angle between the left and right sides is [-θ max ,θ max ] sector search area, and l e,m (θ) represents the contour curve equation of a given object located in the fan-shaped search area, where θ max is the set angle value, l e,m (θ) is the projection contour point x of the given object on the mth object cnt,m The equation of the curve in the corresponding fan-shaped search area, θ∈[-θ max ,θ max ]; S52: For any sector search area, construct cnt,m is the origin, and divides [-θ max ,θ max ] sam Search lines, N sam is an odd number; the edge points on each search line are taken as candidate object image contour points, and the distance from x to each search line is cnt,m The nearest candidate object image contour point is used as the potential object image contour point on each search line; S53: For any sector search area, obtain N sam N calculated on the search line sam The potential object image contour points are projected onto n cnt,m The average value of the one-dimensional position distribution obtained in the direction is as follows: Among them, n cnt,m Project the contour point x for the mth object cnt,m The corresponding normal vector, μ ecnt,m Project the contour point x for the mth object cnt,m The corresponding multi-directional search for potential object image contour points along the contour normal vector n cnt,m The average value of the one-dimensional distribution in the direction, T represents the transpose, l e,m (θ i ) is the projection contour point x of the mth object cnt,m The direction angle θ of the i-th search line in the corresponding fan-shaped search area i The corresponding curve equation coordinates, θ i ∈[-θ max ,θ max ].

7. A real-time six-degree-of-freedom posture tracking method for a weak-resolution image sequence as claimed in claim 6, characterized in that: Based on the region of interest F filtered by the mixed edges, the contour points x of each object are projected cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The uncertainty σ of the one-dimensional distribution of the direction ecnt,m The calculation is as follows: s ecnt,m 2 =s eshp,m 2 s enoi,m 2 Among them, σ eshp,m 2 Project the contour point x for the mth object cnt,m The corresponding multi-directional searched potential object image contour points along the object projection contour point normal vector n cnt,m The variance of the one-dimensional distribution of the direction, σ enoi,m 2 Project the contour point x for the mth object cnt,m The corresponding noise uncertainty is calculated as follows: σ enoi,m =f·|{c m,k }| Among them, |{c m,k }| Projection contour point x for the mth object cnt,m The normal vector n of the corresponding search area along the object projection contour point cnt,m The number of candidate object image contour points on the search line in the direction, and f is the adjustment factor.

8. A real-time six-degree-of-freedom posture tracking method for a weak-resolution image sequence as claimed in claim 7, characterized in that: Contour energy E based on filtered edges ecnt The calculation method is: Among them, the weight β ecnt,m is the set external point suppression coefficient, the residual r ecnt,m Defined as x cnt,m and mean μ ecnt,m Between along n cnt,m One-dimensional projection distance in the direction, N cnt is the total number of object projection contour points.

9. A real-time six-degree-of-freedom posture tracking method for a weak-resolution image sequence as claimed in claim 1, characterized in that: The internal energy E of a given object in the current image frame I in The calculation method is: Among them, the weight γ is the set external point suppression coefficient, c in,n is the optical flow confidence based on pixel-by-pixel blocks, and the residual r in,n Defined as the internal sampling point pair (x in,n ,x' in,n ) reprojection error, N in The total number of points inside the object projection.

10. A real-time six-degree-of-freedom posture tracking method for a weak-resolution image sequence according to claim 1, characterized in that: According to the contour energy E scnt 、E ecnt and the internal energy E in The constructed multimodal energy function E is: E=λE in +(1-λ)(ηE ecnt +(1-η)E scnt ) Among them, λ and η are both set weight factors.

Images