Dynamic target pose measurement method based on multiple exposure technology

Abstract

The present invention relates to the field of image processing technology, and in particular to a dynamic target posture measurement method based on multiple exposure technology, firstly, a cooperative marker is set on a target object to determine the feature points on the target object; a feature point coordinate group of the target object at two consecutive moments is obtained, and then the feature point change speed is calculated; a measurement image group of the target object at the next moment is continuously obtained; the first image in the measurement image group at this moment is used as a reference, and the feature point coordinates in the first image are compensated in combination with the feature point change speed, so as to obtain a compensated feature point image coordinate set; finally, the combined feature point image coordinate set is used to calculate the posture of the target object. The present invention detects target features through multiple images collected continuously, and at the same time compensates for the feature points, thereby improving the accuracy of posture solution and reducing the influence of feature misalignment between multiple images caused by target motion on the accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a dynamic target posture measurement method based on multiple exposure technology. Background Art

[0002] A cooperative marker is a special mark installed on a target object, which can be identified and tracked by a visual sensor, thereby achieving accurate positioning and measurement of the target. Cooperative marker measurement technology plays a vital role in tasks such as spacecraft docking and robotic arm operation. It not only improves the accuracy and reliability of operations, but also provides important guarantees for the safe operation of spacecraft. Cooperative markers on spacecraft generally have relatively significant passive graphic features (rings, dots, etc.) and do not have the ability to emit light autonomously. Since there is no atmospheric scattering in space, the brightness difference between the direct sunlight area and the shadow area is very large, and the light and shadow distribution on the target surface is changing all the time. Under a certain camera exposure time, the features on the cooperative marker in the image may be overexposed or underexposed, resulting in the algorithm being unable to achieve recognition and measurement. At the same time, in tasks such as spacecraft docking and robotic arm operation, there is relative motion between the measuring camera and the target being measured. The relative motion characteristics of the camera-target are also key factors to be considered in visual measurement tasks. How to reduce the impact of ambient light and improve the all-day working ability of the visual measurement system while ensuring accuracy performance is a key issue that needs to be solved in the visual measurement of space cooperative markers. Summary of the invention

[0003] In view of this, the invention aims to provide a dynamic target posture measurement method based on multiple exposure technology, which performs target feature detection on multiple acquired images, solves the problem of target feature recognition failure due to complex lighting, and improves the success rate of target feature recognition; at the same time, in order to improve the accuracy of posture solution and reduce the influence of feature misalignment between multiple images caused by target motion on accuracy, a method for compensating feature points by calculating target three-dimensional motion information is also realized. Therefore, the invention solves the problem of degradation of recognition and measurement performance of moving targets under complex lighting conditions.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0005] A dynamic target posture measurement method based on multiple exposure technology includes the following steps:

[0006] S1: setting the cooperative marker on the target object to determine the feature points on the target object;

[0007] S2: Obtain the measurement image groups of the target object at time t-2 and time t-1, and then calculate the position and posture of the target object at the two times, and then obtain the feature point coordinate groups of the target object at the two times, and the feature point coordinate offset between the two feature point coordinate groups, and then obtain the feature point change speed;

[0008] S3: obtaining a measurement image group of the target object at time t; taking the first image in the measurement image group at time t as a reference, determining feature points not in the first image according to other images in the measurement image group at time t;

[0009] S4: compensating the feature point coordinates of the feature points determined in step S3 in combination with the feature point change speed, and incorporating the compensated feature point coordinates into the feature point coordinate group in the first image to obtain a compensated feature point image coordinate set;

[0010] S5: Combine the feature point image coordinate set obtained in step S4 to calculate the position and posture of the target object at time t.

[0011] Further, step S2 includes the following steps:

[0012] S21: using a camera to continuously photograph the target object at time t-2 to obtain a measurement image group of the target object at time t-2;

[0013] S22: Calculate the position and posture of the target object at time t-2 according to the measurement image group at time t-2;

[0014] S23: Combined with the position and posture at time t-2, the feature point coordinate group at time t-2 is obtained through the perspective projection model:

[0015] S24: repeating steps S21 to S23 at time t-1 to obtain a measurement image group of the target object at time t-1 and a feature point coordinate group of the target object at time t-1;

[0016] S25: Subtract the corresponding elements of the feature point coordinate group obtained in step S23 from the feature point coordinate group obtained in step S24 to obtain the feature point coordinate offset; and then obtain the feature point change speed through the following formula:

[0017] ;

[0018] Among them, v represents the changing speed of the feature points, Indicates the feature point coordinate offset, Represents the time interval between the imaging triggering times of the measurement image group at time t-1 and time t-2.

[0019] Furthermore, the feature point change speed v includes the change speed vectors of N feature points, that is, , Represents the change speed vector of the nth feature point; the change speed vector of each feature point Includes the motion components of the feature point in both horizontal and vertical directions.

[0020] Furthermore, in step S4, the feature point image coordinate set is obtained by the following formula:

[0021] ;

[0022] in, represents the image coordinate set of feature points at time t, represents the feature point coordinate group in the first image of the measurement image group at time t, , i represents the number of images in the measurement image group at time t, I represents the total number of images in the measurement image group at time t, represents a feature point coordinate group consisting of feature points that the first image in the measurement image group at time t does not have compared with the i-th image; Represents the lag time of the i-th image relative to the first image in the measurement image group at time t.

[0023] Compared with the prior art, the invention can achieve the following beneficial effects:

[0024] In the dynamic target posture measurement method based on multiple exposure technology created by the present invention, by detecting feature identification on multiple images collected continuously, the problem of being unable to be recognized due to overexposure or underexposure of local features under complex lighting conditions is solved; at the same time, the short-term continuous invariance of target motion is used as a priori, and the pixel coordinate compensation value of the feature point of the current frame is calculated by the posture of the previous frame. This compensation method can reflect the 6D degree of freedom motion component of the target in three-dimensional space, which is better than the method of directly calculating the target motion through pixel deviation. This scheme improves the target recognition rate of cooperative markers under complex lighting conditions, and has high posture accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0026] Figure 1 A flowchart of a dynamic target posture measurement method based on multiple exposure technology according to an embodiment of the present invention;

[0027] Figure 2A schematic diagram of characteristic numbers of the cooperation marker described in the embodiment of the present invention;

[0028] Figure 3 A schematic diagram of a dynamic target posture measurement method based on multiple exposure technology according to an embodiment of the present invention;

[0029] Figure 4 A measurement timing diagram of a dynamic target posture measurement method based on multiple exposure technology described in an embodiment of the present invention.

[0030] Description of reference numerals:

[0031] 1. Camera; 2. Image plane; 3. Cooperation marker. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0033] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0034] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0035] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0036] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0037] like Figure 1~Figure 4 As shown, the dynamic target posture measurement method based on the multiple exposure technology described in the embodiment of the present invention comprises the following steps:

[0038] S1: Set the cooperative marker 3 on the target object and determine the feature points on the target object.

[0039] S2: Obtain the measurement image groups of the target object at time t-2 and time t-1, and then calculate the position and posture of the target object at the two times (position and posture represent 6 freedom metrics including translation and rotation), so as to obtain the feature point coordinate groups of the target object at the two times, and the feature point coordinate offset between the two feature point coordinate groups, and then obtain the feature point change speed. Step S2 includes the following steps:

[0040] S21: Use camera 1 to continuously shoot the target object at time t-2 to obtain a measurement image group frm of the target object at time t-2 t-2 In the embodiment of the present invention, the camera 1 is a monocular camera, and its imaging process is controlled by FPGA. The image triggering imaging time and the camera exposure time of the camera 1 can be set according to actual needs.

[0041] S22: Based on the measurement image group frm at time t-2 t-2 , calculate the position and posture of the target object at time t-2 In the present invention, the PnP (Perspective-n-Point) pose measurement technology algorithm is preferably used, which estimates the pose of the target object based on the three-dimensional coordinates of the feature points on the target object and their corresponding two-dimensional image points. Specifically, the feature points of the cooperative marker 3 are placed in the target coordinate system O T The homogeneous three-dimensional coordinates under (i.e., the three-dimensional coordinates of the feature points on the target object) are defined as pt3d={m1,m2,…,m N}, where N represents the total number of feature points on the cooperative marker 3; the projection point coordinates of the feature points on the image plane 2 (i.e., the feature point coordinates of the feature points) are pt2d={pt1,pt2,…,pt N}, target coordinate system O T To the camera coordinate system O C The transformation matrix is , is the pose of the target object.

[0042] S23: Combined with the pose at time t-2 , the feature point coordinate group pt2d at time t-2 is obtained through the perspective projection model t-2 , as follows:

[0043] ;

[0044] Where n∈{1,2,3,…,N}, n represents the number of feature points; represents the feature point coordinates of the nth feature point at time t-2; f(·) represents the perspective projection model; M and D represent the camera intrinsic parameter matrix and camera distortion coefficient of camera 1, respectively. In the embodiment of the present invention, the camera intrinsic parameter matrix M and the camera distortion coefficient D are specifically:

[0045] ;

[0046] D = [k1k2p1p2];

[0047] Among them, f x and f y They represent the ratio of the focal length of camera 1 to the pixel size, c x and c y represents the principal point coordinates of camera 1, k1 and k2 represent the radial distortion coefficients of camera 1, and p1 and p2 represent the tangential distortion coefficients of camera 1.

[0048] S24: Repeat steps S21 to S23 at time t-1 to obtain the measurement image group frm of the target object at time t-1 t-1 , and the feature point coordinate group pt2d of the target object at time t-1 is obtained by the following formula: t-1 ,Right now:

[0049] ;

[0050] in, represents the feature point coordinates of the nth feature point at time t-1, Indicates the position and posture of the target object at time t-1;

[0051] S25: Subtract the corresponding elements of the feature point coordinate group obtained in step S23 from the feature point coordinate group obtained in step S24 to obtain the feature point coordinate offset ,Right now:

[0052] ;

[0053] Then the characteristic point change speed v is obtained by the following formula:

[0054] ;

[0055] Among them, v represents the changing speed of the feature points, represents the measurement image group frm at time t-1 t-1 Imaging trigger time frm_t t-1 and the measurement image group frm at time t-2 t-2 Imaging trigger time frm_t t-2 The time interval between:

[0056] .

[0057] The feature point change speed v includes the change speed vectors of N feature points, that is, , Represents the change speed vector of the nth feature point; the change speed vector of each feature point Includes the motion components of the feature point in both horizontal and vertical directions.

[0058] S3: Get the measurement image group frm of the target object at time t t ; Take the measurement image group frm at time t t The first image im1 in is taken as the reference, and the measured image group frm at time t is used. t Other images in im i Determine the feature points that are not in the first image im1. Where i∈{2,3,…,I}, i represents the measured image group frm at time t t The number of images in, I represents the measurement image group frm at time t t The total number of images in .

[0059] S4: Compensate the feature point coordinates of the feature points determined in step S3 in combination with the feature point change speed v, and incorporate the compensated feature point coordinates into the feature point coordinate group in the first image to obtain a compensated feature point image coordinate set. The compensation process is as follows:

[0060] ;

[0061] in, represents the image coordinate set of feature points at time t, Represents the measurement image group frm at time t t The feature point coordinate group in the first image im1, Represents the measurement image group frm at time t t The first image im1 and the i-th image im i Compared with the feature point coordinate group composed of no feature points; Represents the image im in the measurement image group at time t i Lag relative to the first image im1.

[0062] S5: Combine the feature point image coordinates obtained in step S4 , calculate the position and posture of the target object at time t. In the present invention, the PnP algorithm is also preferably used to calculate the position and posture of the target object. , and record the current posture at the same time for subsequent calculation of image feature point coordinate motion compensation information.

[0063] Specific embodiment: In order to clearly demonstrate the dynamic target posture measurement method based on multiple exposure technology described in the embodiment of the invention, a specific embodiment of the method provided by the invention is provided in which the total number of feature points N=5 on double exposure and cooperative marker 3.

[0064] The feature point numbers (a1, a2, a3, a4, a5) on the cooperative marker 3 are as follows Figure 2 As shown in the figure, the left side of the target object is directly illuminated by sunlight, while the right side is blocked by shadows. Figure 3 As shown in the figure, it is difficult for camera 1 to take into account both the direct sunlight area and the shadow area when imaging at one exposure time. Feature points a1 and a4 in the direct sunlight area are prone to overexposure, feature points a2 and a5 in the shadow area are prone to underexposure, and feature point a3 in the light-shadow transition area may be exposed to both.

[0065] For the measurement image group frm at time t t ={im1,im2}, measurement image group frm t The first image im1 is a low exposure time image, and the measurement image group frm t The second image im2 is a high exposure time image. In this specific embodiment, the camera 1 is driven by FPGA, which can accurately control and record the imaging time, exposure time, and the relative imaging time interval between consecutive images. The double exposure imaging timing is as follows: Figure 4 shown.

[0066] Since the first image im1 is imaged with a low exposure time, the imaging effect of the feature points a1 and a4 in the direct sunlight area is better; similarly, the second image im2 is imaged with a high exposure time, and the imaging effect of the feature points a2 and a5 in the shadow area is better. Feature detection is performed on the two images respectively, such as Figure 3 As shown in the figure, the set of feature points detected in the first image im1 is {a1, a3, a4}, and the set of feature points detected in the second image im2 is {a2, a3, a5}. The union of the two feature point sets is {a1, a2, a3, a4, a5}, and the feature point coordinates of the corresponding feature points are pt2d={pt1, pt2, pt 3, pt4,pt5}.

[0067] The imaging time of the second image im2 lags behind the imaging time of the first image im1 Due to the relative motion between the target and camera 1, there is a certain offset between the feature point coordinates pt4 and pt5 and the feature point coordinates pt1, pt2 and pt3, which leads to errors in the PnP pose solution and even causes the PnP algorithm to not converge. Therefore, it is necessary to compensate the coordinates of the feature points according to the target motion characteristics.

[0068] To solve the above technical problems, the dynamic target posture measurement method of the multiple exposure technology described in this specific embodiment includes the following steps:

[0069] S1: Set the cooperative marker 3 on the target object and determine the feature points {a1, a2, a3, a4, a5} on the target object.

[0070] S2: Get the measurement image group frm of the target object at time t-2 and time t-1 respectively t-2 and frm t-1 , and then calculate the position and posture of the target object at two moments through the PnP algorithm and , and then use the perspective projection model to obtain the feature point coordinate group pt2d of the target object at two moments t-2 and pt2d t-1 ,Right now:

[0071] ;

[0072] ;

[0073] According to the two sets of feature point coordinates pt2d t-2 and pt2d t-1 , get the feature point coordinate offset ,Right now:

[0074] ;

[0075] Then the characteristic point change speed v is obtained by the following formula:

[0076] ;

[0077] The feature point change speed v here includes the change speed vectors of 5 feature points, namely .

[0078] S3: Get the measurement image group frm of the target object at time t t . Take the measurement image group frm at time t t The first image im1 in is taken as the reference. The set of feature points detected in the first image im1 is {a1, a3, a4}, and the set of feature points detected in the second image im2 is {a2, a3, a5}. Therefore, the feature points that are not in the first image im1 are a2 and a5. The corresponding feature point coordinate group composed of the feature points that are not in the first image im1 is , Represents the feature point coordinates of the second feature point at time t, Represents the feature point coordinates of the 5th feature point at time t.

[0079] S4: Combine the feature point change speed v to calculate the feature point coordinate group determined in step S3 Perform compensation to obtain the compensated feature point image coordinate set ,Right now:

[0080] ;

[0081] in, , Represents the measurement image group frm at time t t The feature point coordinate group in the first image im1; Represents the feature point coordinates of the first feature point at time t; Represents the feature point coordinates of the second feature point at time t The coordinates of the feature points after compensation are ; Represents the feature point coordinates of the third feature point at time t; Represents the feature point coordinates of the fourth feature point at time t; Represents the feature point coordinates of the fifth feature point at time t The coordinates of the feature points after compensation are ; Indicates the lag time of the second image im2 relative to the first image im1.

[0082] S5: Combine the feature point image coordinates obtained in step S4 , calculate the position and posture of the target object at time t.

[0083] For the first moment measurement image group frm1 and the second moment measurement image group frm2, since there is no posture information at the previous moment, the PnP algorithm is directly applied using the combined feature points; for the measurement image frame frm after the second moment t (t>2), repeat steps S2~S4 to compensate the feature point coordinates to obtain the pose at the current moment.

[0084] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0085] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A dynamic target posture measurement method based on multiple exposure technology, characterized in that: The following steps are involved: S1: setting a cooperative marker on a target object to determine feature points on the target object; S2: Obtaining measurement image groups of the target object at time t-2 and time t-1, respectively, and then calculating the position and posture of the target object at the two times, and then respectively obtaining feature point coordinate groups of the target object at the two times, and feature point coordinate offsets between the two feature point coordinate groups, and then obtaining feature point change speed; S3: obtaining a measurement image group of the target object at time t; taking the first image in the measurement image group at time t as a reference, determining feature points not in the first image according to other images in the measurement image group at time t; S4: compensating the feature point coordinates of the feature points determined in step S3 in combination with the feature point change speed, and incorporating the compensated feature point coordinates into the feature point coordinate group in the first image to obtain a compensated feature point image coordinate set; In step S4, the feature point image coordinate set is obtained by the following formula: ； in, represents the image coordinate set of feature points at time t, represents the feature point coordinate group in the first image of the measurement image group at time t, , i represents the number of images in the measurement image group at time t, I represents the total number of images in the measurement image group at time t, represents a feature point coordinate group consisting of feature points that the first image in the measurement image group at time t does not have compared with the i-th image; represents the lag time of the i-th image relative to the first image in the measurement image group at time t, and v represents the change speed of the feature point; S5: Combine the feature point image coordinate set obtained in step S4 to calculate the position and posture of the target object at time t.

2. The dynamic target posture measurement method based on multiple exposure technology according to claim 1 is characterized in that: Step S2 includes the following steps: S21: using a camera to continuously photograph the target object at time t-2 to obtain a measurement image group of the target object at time t-2; S22: calculating the position and posture of the target object at time t-2 according to the measurement image group at time t-2; S23: Combined with the position and posture at time t-2, the feature point coordinate group at time t-2 is obtained through the perspective projection model: S24: repeating steps S21 to S23 at time t-1 to obtain a measurement image group of the target object at time t-1 and a feature point coordinate group of the target object at time t-1; S25: Subtract the corresponding elements of the feature point coordinate group obtained in step S23 from the feature point coordinate group obtained in step S24 to obtain the feature point coordinate offset; and then obtain the feature point change speed through the following formula: ； Wherein, v represents the changing speed of the feature point, represents the coordinate offset of the feature point, Represents the time interval between the imaging triggering times of the measurement image group at time t-1 and time t-2.

3. The dynamic target posture measurement method based on multiple exposure technology according to claim 2 is characterized in that: The feature point change speed v includes the change speed vectors of N feature points, that is, , Represents the change speed vector of the nth feature point; the change speed vector of each feature point Includes the motion components of the feature point in both horizontal and vertical directions.

Images