Long-distance continuous pose high-speed video measurement method
By combining multi-binocular continuous observation with bundle adjustment and an adaptive variable weighting strategy based on depth and distance constraints, and dynamic inversion of position and attitude parameters through plane fitting, the problems of low accuracy and consistency in long-distance continuous pose measurement are solved, and high-precision dynamic parameter measurement is achieved.
Patent Information
- Application Number
- CN202210799215.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-06
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2042-07-06
AI Technical Summary
Existing technologies suffer from low positioning accuracy and inconsistent long-distance observation accuracy in long-distance continuous pose measurement. Traditional contact measurement methods have limited range and damage the model structure, while non-contact measurement methods such as laser technology are expensive and have low acquisition frequency.
The measurement results are optimized by using a combination of multi-binocular continuous observation and bundle adjustment, combined with an adaptive variable weighting strategy that incorporates depth, distance and adjustment residual constraints, and dynamic inversion of position and attitude parameters based on plane fitting.
It achieves high-precision continuous pose measurement of long-distance moving targets, with a measurement and positioning accuracy better than 3mm, which is more than 30% higher than that of internationally renowned vision measurement software.
Smart Images
Figure CN115311358B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of space engineering model dynamic parameter measurement, in particular to a long-distance continuous position and posture high-speed video measurement method. BACKGROUND
[0002] With the rapid development of space engineering, high-precision measurement of long-distance continuous position and posture parameters in ground verification test experiments has become a frontier research topic. Traditional contact measurement methods, such as fixing displacement meters, accelerometers, strain gauges and other contact sensors on the surface of the measurement target object to obtain displacement, acceleration, strain and other information, have defects such as limited range, small measurement area and even damage to the model structure. In order to overcome the above defects, non-contact sensors represented by cameras and laser sensors are used, among which laser technology has defects such as single measurement point, high price and low acquisition frequency. Therefore, the measurement method of using cameras to record the changing state of moving objects in detail and then accurately measuring the three-dimensional space coordinate changes and three-dimensional deformation of the target feature points through photogrammetry analysis method to obtain the dynamic parameters of the measured object is more widely used.
[0003] In the process of space engineering model dynamic parameter measurement, due to the long movement process of the target in the depth direction, the relative distance of the target to the camera is different between different observation sections, and the current common method is to use multiple cameras to observe and calculate the long-distance continuous position and posture separately, which leads to low overall positioning accuracy and inconsistent observation accuracy at long distances. SUMMARY
[0004] The purpose of the present application is to overcome the defects of the prior art and provide a long-distance continuous position and posture high-speed video measurement method.
[0005] The purpose of the present application can be achieved by the following technical solutions:
[0006] According to one aspect of the present application, a long-distance continuous position and posture high-speed video measurement method is provided, which comprises the following steps:
[0007] Step 1: using multi-biocular continuous observation joint bundle adjustment to solve the observation values;
[0008] Step 2: using a depth distance and adjustment residual constraint adaptive variable weight strategy to process the solving process of step 1) to obtain the continuous position and posture parameter measurement results;
[0009] Step 3: using a plane fitting-based position and posture parameter dynamic inversion to optimize the continuous position and posture parameter measurement results obtained in step 2).
[0010] As a preferred technical solution, the step 1, the multi-bino continuous observation joint bundle adjustment process is specifically:
[0011] 101, in photogrammetry, the least square principle is used to make the cumulative square sum of the distance residual between the back projection and the original image point minimum, which is expressed in the form of the following least square problem:
[0012]
[0013] Where (u i,j ,v i,j ) is the actual observation pixel coordinate of the jth three-dimensional point on the image plane of the ith camera; π(Cam i ,P j ) = (u (i,j)distorted ,v (i,j)distorted ), that is, the back projection pixel coordinate of the jth three-dimensional point in the object space through the internal and external matrix perspective transformation of the ith camera, and considering the offset of the camera lens distortion parameters, e, m and n are respectively the back projection residual, the number of cameras and the number of three-dimensional points;
[0014] 102, the long-distance continuous pose solving process is described as a plurality of independent bundle adjustment processes by using formula (1), and the expression is changed to:
[0015]
[0016] Where BA k represents the kth local bundle adjustment process, m k represents the camera set and the observed object space three-dimensional point set in the kth binocular vision measurement system, respectively.
[0017] 103, the joint bundle adjustment solution is improved by formula (2):
[0018]
[0019] Where m′ k-1 represents the camera observation to the repeated object space three-dimensional point set in the kth binocular vision measurement system, BA Global represents the joint bundle adjustment process.
[0020] As a preferred technical solution, the adaptive variable weight strategy of the depth distance and adjustment residual constraint in the step 2 includes introducing iterative reweighting and introducing depth change weighting strategy in the adjustment process.
[0021] As a preferred technical solution, the introduction of iterative reweighting is to continuously adjust the weight value of the observation value in the bundle adjustment optimization iterative solution.
[0022] As a preferred technical solution, the weight of the observation value is adjusted in the light beam adjustment optimization iterative solution, and the weight of the observation value is adjusted in the light beam adjustment optimization iterative solution.
[0023] Based on Huber robust loss function, before each iteration optimization, the observation value is iteratively reweighted as follows:
[0024]
[0025] Where w r(k) represents the robust weight of the kth observation value, r k represents the two norm of the back projection error of the kth observation value calculated by the error equation, that is, r k =‖e k ‖2, δ represents the threshold of abnormal points.
[0026] As a preferred technical solution, the threshold of abnormal points is 0.001.
[0027] As a preferred technical solution, the depth change weighting strategy is introduced in the adjustment process, that is, the reciprocal of the depth distance between the observation value of different depth and the baseline of binocular vision system is taken as the weight basis, so as to fully consider the objective fact that the observation accuracy is reduced due to the large depth value of target point.
[0028] As a preferred technical solution, the weight mode is specifically:
[0029]
[0030] Where w z(k) represents the depth change weight of the kth observation value, D k represents the depth distance between the kth observation value and the baseline of binocular vision system, D δ represents the depth distance threshold, and the above formula shows that only the observation outside the specific depth range is weighted.
[0031] As a preferred technical solution, the D δ is set as the depth distance between the space intersection point of the optical axis of the two cameras and the baseline of the binocular vision system, which can be easily calculated by the initial parameter intersection of the two cameras.
[0032] As a preferred technical solution, the step 3 is specifically:
[0033] Taking the three-dimensional coordinates of the four artificial marks given in the reference initial state as the reference, the rotation and translation change relationship of the plane at each time relative to the initial state can be represented by the rigid body transformation relationship between the four mark points as follows:
[0034]
[0035] wherein R k and t k are translation and rotation parameters of the model surface at the k moment relative to the reference initial state, X0, Y0, Z0, X k , Y k , Z k are three-dimensional coordinates of the reference initial state and the k moment, respectively.
[0036] Compared with the prior art, the application can realize high-precision continuous measurement of long-distance moving target dynamic position and posture parameters, and the measurement positioning precision is better than 3mm, which is improved by more than 30% compared with the precision of the international famous visual measurement software. BRIEF DESCRIPTION OF DRAWINGS
[0037] Figure 1 is a schematic diagram of a joint beam adjustment method;
[0038] Figure 2 is a schematic diagram of position and posture parameter dynamic inversion based on plane fitting;
[0039] Figure 3 is a schematic diagram of multi-biocular visual measurement system layout;
[0040] Figure 4 is a schematic diagram of a space three-dimensional control field layout method;
[0041] Figure 5 is a schematic diagram of a 20° upward pitch posture parameter solution result;
[0042] Figure 6 is a schematic diagram of a 20° right deviation posture parameter solution result;
[0043] Figure 7 is a schematic diagram of a left-right swing posture parameter solution result. DETAILED DESCRIPTION
[0044] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of, but not all of the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative work should belong to the protection scope of the application.
[0045] 1. Overview of the application
[0046] With the rapid development of aerospace engineering, high-precision measurement of dynamic pose parameters of aerospace models in ground verification test has become a frontier research topic. In order to meet the demand of high-precision non-contact measurement of long-distance continuous pose parameters, a long-distance continuous pose estimation method based on adaptive variable weight joint bundle adjustment is proposed. A multi-biocular vision measurement system using adaptive variable weight least squares adjustment strategy is used to observe long-distance continuous joint bundle adjustment method, combined with the position and attitude parameter dynamic inversion technology based on plane fitting, the accuracy of dynamic parameter measurement results is optimized. In order to verify the reliability of the proposed long-distance pose parameter measurement method, satellite rendezvous and docking ground test high-speed video pose measurement experiment is carried out, the experimental results show that the proposed measurement method can realize high-precision continuous measurement of dynamic position and attitude parameters of long-distance moving target, the measurement positioning accuracy is better than 3mm, compared with the international famous visual measurement software, the accuracy is improved by more than 30%.
[0047] 2. Long-distance continuous pose high-speed video measurement method
[0048] The long-distance continuous pose high-speed video measurement method proposed in this patent mainly includes two parts: long-distance continuous observation joint bundle adjustment method with depth distance and adjustment residual constraint weight, and position and attitude parameter dynamic inversion based on plane fitting.
[0049] 2.1 Multi-biocular continuous observation joint bundle adjustment
[0050] In photogrammetry, the least squares principle is used to minimize the cumulative square sum of the distance residuals between the back projection and the original image points, that is, the square sum of the back projection error is minimized. This method is called bundle adjustment. It can be expressed in the form of the following least squares problem:
[0051]
[0052] In the formula, (u i,j ,v i,j ) is the actual observed pixel coordinate of the jth three-dimensional point on the image plane of the ith camera with distortion; π(Cam i ,P j ) = (u (i,j)distorted ,v (i,j)distorted ), which means that the jth three-dimensional point in the object space is transformed by the internal and external matrix of the ith camera, and the offset of the camera lens distortion parameters is considered. The back projection pixel coordinates obtained after the offset are considered.
[0053] In long-distance continuous pose measurement, the traditional multi-camera segmented measurement method commonly used is to take the observation of each segment binocular vision measurement system as a separate calculation unit, and to perform local bundle adjustment in each observation range to obtain the extrinsic parameters of the stereo camera and the spatial three-dimensional information of the target point to be measured. Finally, the processing results of each segment are spliced to obtain the final calculation results of the whole process. Using formula 2.1, the above calculation process can be described as multiple independent bundle adjustment processes:
[0054]
[0055] where BA k represents the kth local bundle adjustment process, m k represents the camera set and the observed object three-dimensional point set in the kth binocular vision measurement system, respectively. Using this calculation method will cause the poses of the single independent stereo pair to be independent of each other, which cannot guarantee that they are consistent with the actual relative poses in the overall spatial coordinate system, resulting in the lack of integrity of the spatial three-dimensional information of the target point to be measured, and the discontinuity of the global result when splicing. Finally, the calculation result is not robust.
[0056] Since the binocular vision measurement system in the latter segment will overlap with the binocular vision system in the former segment at a far distance in the depth direction (as shown in Figure 1 , this patent proposes a joint global bundle adjustment calculation strategy for multi-binocular continuous observation, which fully utilizes the constraint information provided by adjacent repeated observations.
[0057] Based on the above strategy, the joint bundle adjustment calculation can be obtained by improving formula 2.2:
[0058]
[0059] where m′ k-1 represents the object three-dimensional point set observed by the camera in the kth binocular vision measurement system and repeated in the k-1th binocular vision measurement system.
[0060] 2.2 Adaptive variable weight strategy of depth distance and adjustment residual constraint
[0061] The proposed adaptive variable weight least squares adjustment strategy is based on the idea of iterative reweighting and depth variation weighting, and uses depth distance and adjustment residual constraint for adaptive weight determination. The basic idea of introducing the iterative reweighting least squares method is to continuously adjust the weight of a certain observation value in the bundle adjustment optimization iterative solution, thereby reducing the influence of abnormal points on the final global calculation result. Based on the Huber robust loss function, the following iterative reweighting operation is performed on a certain observation value before each iteration optimization:
[0062]
[0063] where w r(k) represents the robust weight of the kth observation iteration reweighting, r k represents the 2-norm of the back-projection error of the kth observation calculated by the error equation, i.e., r k =‖e k ‖2, and δ represents the threshold of abnormal points (the commonly used experience value 0.001 is selected in the patent).
[0064] In addition, the baseline B and the camera focal length f are constant values, assuming that the parallax deviation is Δd, considering the influence of the size of the depth Z itself (i.e., the depth range) on the measurement accuracy, it can be derived that:
[0065]
[0066] It is easy to know from formula 2.5 that the distance of the target from the lens is different, and the measurement accuracy is different. In the depth direction of the observation field of view, the parallax of the homonymous point located at a deeper depth in the stereo image pair is smaller, that is, a slight change in the pixel coordinates of the observation point located at a deeper depth in the stereo image pair will cause a larger offset in the depth direction.
[0067] Based on the above related observation experience and theoretical research basis, the patent proposes to introduce a depth change weighting strategy in the adjustment process, and the inverse of the depth distance between the observation value at different depths and the baseline of the binocular vision system is used as the weight basis, so as to fully consider the objective fact that the observation accuracy is reduced due to the large depth value of the target point, avoid its influence on the joint beam method adjustment iteration optimization process to a large extent, and thus make the overall adjustment result more accurate and stable.
[0068] In view of the fact that most of the observations in a specific observation task are within a higher precision observation range, only some observation points in the distance may have the above errors, the patent adopts the following weight setting method:
[0069]
[0070] where w z(k) represents the depth change weight of the kth observation, D k represents the depth distance between the kth observation and the baseline of the binocular vision system, D δ represents the depth distance threshold, and the above formula shows that only the observations outside the specific depth range are weighted. According to the measurement experience, the binocular vision measurement system has generally the best measurement accuracy at the spatial intersection point of the optical axes of the two cameras, so the patent sets D δ as the depth distance between the spatial intersection point of the optical axes of the two cameras and the baseline of the binocular vision system, which can be easily calculated by the initial parameter intersection of the two cameras.
[0071] In summary, the proposed adaptive variable weight least square adjustment strategy combines the robust weight updated in each iteration adjustment process and the depth variation weight determined by the spatial three-dimensional coordinates of all observation points, which jointly act on the above-mentioned joint beam adjustment process.
[0072] 2.3 Position and attitude parameter dynamic inversion based on plane fitting
[0073] Generally, the target model can be approximated as an ideal rigid body, and the dynamic attitude of the model at different times during the movement process can be equivalent to the rigid body rotation and translation change relationship between its attitude in the reference initial state, i.e., the rotation and translation parameters of the body coordinate system composed of the model surface at each time relative to the reference initial state are the unknowns. The model surface in the real scene is relatively flat, and can be replaced by the plane fitted by the four artificial markers pasted on its surface, so the translation and rotation attitude angle parameters of the model at each time relative to the reference initial state can be replaced by the rotation and translation relationship between the artificial marker fitting planes, i.e., the model dynamic parameter solving problem is transformed into the position and attitude parameter dynamic inversion problem based on the artificial marker fitting plane, and the principle diagram is shown in Figure 2 .
[0074] Taking the three-dimensional coordinates of the four artificial markers given in the reference initial state as the reference, the rotation and translation change relationship of the fitting plane at each time relative to the initial state can be represented by the rigid body transformation relationship between the four markers:
[0075]
[0076] In the formula, R k and t k are the translation and rotation parameters of the model surface at time k relative to the reference initial state, and there are a total of 6 unknown parameters. Based on the least square fitting principle, the three-dimensional coordinates corresponding to the four artificial markers at each time obtained by high-speed video measurement can be used to solve R k and t k , and then R k is calculated according to the “ZYX” rotation axis sequence to restore the corresponding roll angle, yaw angle and pitch angle.
[0077] 3. Experimental and experimental results
[0078] 3.1 Multi-biocular vision measurement system layout
[0079] With the satellite rendezvous and docking ground test high-speed video pose measurement as a specific experimental scene, the satellite rendezvous and docking model will slide forward along the ground test track for about 45m. To meet the needs of long-distance measurement, the measurement experiment is arranged with 3 pairs of binocular vision measurement systems every 15m on both sides of the track to record the entire experimental conditions. The baseline length between the two cameras in each individual binocular vision measurement system is calculated to be about 6.75m, and the angle between the optical axis of each camera and the baseline is set to about 67° to ensure that each individual binocular vision measurement system can clearly observe and record the motion changes of objects within the approximate 15m depth of field in the effective field of view in front. The specific measurement system arrangement is shown in Figure 3 .
[0080] To provide the coordinate system reference for camera external pose solution in multiple binocular vision measurement systems, a stable three-dimensional space control field needs to be arranged within the observation field of view. The specific arrangement is shown in Figure 4 . The true three-dimensional space coordinates of all control points are observed and obtained by a total station before the experiment (with a precision of sub-millimeter level).
[0081] 3.2 High-speed video measurement system precision analysis
[0082] To verify the measurement positioning accuracy of the high-speed video measurement pose estimation method proposed in this patent, the control points in the three-dimensional control field can be divided into two parts, one part as stable control points participating in joint bundle adjustment calculation, and the other part as check points participating in accuracy evaluation. In addition, to verify the effectiveness of the adaptive variable weight least squares strategy of the joint bundle adjustment method of the multiple binocular vision measurement system proposed in this patent, this patent also compares with the segmented solution method of the internationally renowned visual measurement software Photomodeler, and the specific results are shown in Table 1.
[0083] Table 1
[0084]
[0085] From the comparison of the measurement accuracy of the check points in the control field, it can be seen that the measurement accuracy obtained by the joint bundle adjustment strategy proposed in this patent has a root mean square error of less than 2mm in each direction, and the total positioning accuracy is about 3mm, which is equivalent to 0.2mm / m in the camera field of view of about 15m. Compared with the segmented solution method of the internationally renowned visual measurement software Photomodeler, the positioning accuracy in X, Y and Z directions is increased by 34.72%, 36.53% and 19.88% respectively, and the total positioning accuracy is increased by 31%, which proves the effectiveness of the adaptive variable weight joint bundle adjustment pose estimation method proposed in this patent, and also verifies that the high-speed video measurement can meet the accuracy requirements of long-distance pose measurement in satellite rendezvous and docking ground test.
[0086] 3.3 Model pose parameter solution analysis
[0087] The camera parameters recovered using the joint bundle adjustment strategy proposed in the present patent can be used to accurately calculate the three-dimensional coordinates of the four tracking points on the model surface at each moment in space. Then, the dynamic parameter solution is obtained using the position and attitude parameter dynamic inversion technology based on plane fitting. The present patent lists the pose parameter solutions of the satellite rendezvous and docking model under several different motion states, such as Figures 5 to 7 as shown in the figures, respectively representing the model "tilting 20°", "right deviation 20°", and "left-right nutation".
[0088] As can be seen from the above high-speed video measurement dynamic parameter solution, the position and attitude parameter dynamic inversion based on plane fitting can accurately represent the actual motion state of the model. The change amplitude of the attitude angle is consistent with the set change amplitude of 20° and shows stable periodic changes. The overall position and attitude inversion result is relatively smooth.
[0089] The above is only a specific implementation of the present patent, but the protection scope of the present patent is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present patent, and these modifications or replacements should be covered within the protection scope of the present patent. Therefore, the protection scope of the present patent should be subject to the protection scope of the claims.
Claims
1. A long-distance continuous pose high-speed video measurement method, characterized in that, The method comprises the following steps: Step 1, using multi-biocular continuous observation joint bundle adjustment to solve the observation value; Step 2, using the adaptive variable weight strategy of depth distance and adjustment residual constraint to process the solving process of step 1), to obtain the continuous pose parameter measurement result; Step 3, using the plane fitting based position and attitude parameter dynamic inversion to optimize the continuous pose parameter measurement result obtained in step 2); The adaptive variable weight strategy of depth distance and adjustment residual constraint in step 2 comprises introducing iterative reweighting and introducing depth change weighting strategy in the adjustment process; the introduction of iterative reweighting is: constantly adjusting the weight value of an observation value in the bundle adjustment optimization iterative solution; The depth change weighting strategy introduced in the adjustment process is: according to the reciprocal of the depth distance between the observation value of different depth and the baseline of the binocular vision system as the weight basis, to fully consider the objective fact that the observation accuracy is reduced due to the large depth value of the target point; The bundle adjustment process of step 1 is specifically as follows: 101, in photogrammetry, the least square principle is used to make the cumulative square sum of the distance residual between the back projection and the original image point minimum, which is expressed in the form of the following least square problem: （1） wherein is the actual observed pixel coordinate of the first three-dimensional point on the image plane captured by the i-th camera; i.e. represents the back-projected pixel coordinate of the first three-dimensional point after perspective transformation by the intrinsic and extrinsic matrix of the i-th camera and after considering the shift of the camera lens distortion parameters, , respectively the back-projection residual, the number of cameras and the number of three-dimensional points. 102, the long distance continuous pose solving process is described as a plurality of independent bundle adjustment processes by using formula (1), and the expression is changed to: （2） wherein denotes the segment local bundle adjustment process, denotes the set of cameras and observed object space 3D points in the binocular vision measurement system of segment 103, the joint bundle adjustment solution is improved by formula (2): （3） wherein represents the first segment binocular vision measurement system, the camera observes the first segment binocular vision measurement system, the repeated object-side three-dimensional point set, represents the joint bundle adjustment process.
2. The long-distance continuous pose high-speed video measurement method according to claim 1, characterized in that, The constantly adjusting the weight value of an observation value in the bundle adjustment optimization iterative solution is specifically as follows: Based on Huber robust loss function, before each iteration optimization, the iteration reweighting operation of a certain observation value is as follows: （4） wherein represents the robust weight of the i-th observation value iteration re-addition, represents the robust weight of the i-th observation value iteration re-addition, represents the two-norm of the back-projection error of the i-th observation value calculated by the error equation, that is, , represents the abnormal point threshold value. 3. The long distance continuous pose high-speed video measurement method according to claim 2, characterized in that, The threshold value of the abnormal point is 0.
001.
4. The long-distance continuous pose high-speed video measurement method according to claim 1, characterized in that, The weight setting mode is specifically as follows: （5） wherein represents the depth change weight of the th observation, represents the depth distance between the th observation and the baseline of the binocular vision system, represents a depth distance threshold, the above equation indicates that the weight correction is only performed on the observations outside a certain depth range.
5. The long distance continuous pose high-speed video measurement method according to claim 4, characterized in that, The The depth distance is set as the distance between the space intersection of the optical axes of the two cameras and the baseline of the binocular vision system, which can be easily calculated by the initial parameters of the two cameras.
6. The long distance continuous pose high-speed video measurement method according to claim 1, characterized in that, Step 3 is specifically as follows: Taking the three-dimensional coordinates of the four artificial markers given in the reference initial state as the reference, the rotation and translation change relationship of each time plane relative to the initial state can be represented by the rigid body transformation relationship between the four markers as: （6） wherein and are respectively translation and rotation parameters of the surface of the model at time k relative to the reference initial state, , , , , , are respectively three-dimensional coordinates of the reference initial state and of the surface of the model at time k.