Dynamic attitude measurement method based on BP neural network

Through the BP neural network, the data of laser tracking, vision and inertial measurement units are integrated, the limitations of existing attitude measurement systems are solved during dynamic measurement, and the dynamics and stability, adaptability and accuracy of attitude measurement are improved, making up for the vision The problem of the field of view of the measurement unit being blocked and high frequency requirements.

CN119085633APending Publication Date: 2024-12-06HUBEI UNIV OF TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411213353.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing attitude measurement system has limitations in dynamic measurement, especially the laser tracking equipment is difficult to match in terms of light-off detection, fast and high-precision measurement. Vision sensors and PSD sensors have cross-view requirements when used, and when the object to be measured The posture accuracy cannot be guaranteed when the movement speed increases. The non-equal-period Kalman filter data fusion method has high requirements for the synchronization of the measurement system and the stability of the measurement frequency, and is not very adaptable.

Method used

Using the laser tracking attitude dynamic measurement method based on BP neural network, the laser tracking measurement unit, visual measurement unit and inertial measurement unit are used to fuse data from different coordinate systems to solve and predict the final measurement object. Attitude angle to improve dynamic measurement performance.

Benefits of technology

This method improves the overall dynamics, stability and robustness of the attitude measurement system, makes up for the defects that the measurement system cannot measure when the visual measurement unit's field of view is blocked, and improves the measurement speed and accuracy when the motion speed of the measured object is increased. The guarantee of the measurement frequency of the measuring equipment does not require strict requirements, which improves adaptability and universality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119085633A_ABST
    Figure CN119085633A_ABST
Patent Text Reader

Abstract

The invention provides a laser tracking attitude dynamic measurement method based on a BP neural network. Hardware of the method comprises a laser tracking measurement unit, a vision measurement unit, an inertial measurement unit (IMU) and a cooperative target. Establishing an attitude dynamic measurement model by comprehensively utilizing the high dynamic nature of the inertial measurement unit and the high precision of visual attitude measurement; according to the method, the dynamic nature, stability and robustness of a laser tracking attitude measurement system are improved by utilizing the advantage of a complex nonlinear relation of BP neural network modeling, the defect that the measurement system cannot perform measurement when the field of view of a vision measurement unit is shielded is made up, and the defect that the measurement system cannot perform measurement when the movement speed of a measured object is increased is improved. And the precision, the adaptability and the universality of the measurement system are effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of precision measurement technology, and more specifically, to a posture dynamic measurement method based on BP neural network. Background Art

[0002] With the rapid development of high-end intelligent manufacturing and large-scale scientific engineering in my country, as well as the updating and iteration of computer and sensor technology, fields such as large-scale industrial equipment manufacturing, aerospace, automobiles and ships have increasingly higher requirements for attitude measurement accuracy, dynamics, and real-time performance. Improving attitude measurement accuracy and dynamics has become one of the important research directions in the field of attitude measurement technology.

[0003] At present, both at home and abroad, research in the field of attitude measurement technology is being carried out. Foreign countries are relatively advanced in this field and have mature products for commercial use. Among them, there are three representative companies: Leica in Germany, FARO in the United States, and API. Leica combines laser trackers and high-speed cameras to form a dynamic attitude measurement system. With the T-Mac accessory installed on the target to be measured, the reflective prism on the accessory receives the laser beam emitted by the laser tracker to obtain the three-dimensional spatial coordinates of the target to be measured. The visual measurement system measures the attitude of the object to be measured by identifying the optical feature points on the accessory. The system has a high measurement response frequency, a wide measurement range, and good dynamic performance. However, the high-speed zoom camera is complex and difficult to manufacture, and there is a problem that the laser tracking device cannot track the attitude information of the object to be measured due to light interruption. API uses a laser tracker and a SmartTrack accessory equipped with a PSD (position sensitive detector) sensor and an inclinometer to measure the attitude angle. Due to the low update frequency of the inclinometer, the system dynamics is limited. FARO has built a laser tracking attitude measurement system with a laser tracking device as the base station and measurement accessories. The system uses a joint arm and an encoder for measurement, has high repeatability and can perform collaborative measurements with multiple machines. It can also measure hidden points and internal features, but manual operation is required throughout the process. The Institute of Microelectronics of the Chinese Academy of Sciences and Hubei University of Technology have successively studied laser tracking attitude measurement systems based on vision and PSD sensors, making full use of the non-contact, high-precision of visual measurement and the high resolution of PSD sensors to achieve static attitude measurement of objects within a large size range. In order to further improve the dynamic measurement performance, Wu Chuqi and others from Hubei University of Technology proposed an attitude measurement system that combines vision with IMU. The system uses a non-equiperiodic Kalman filter data fusion method to achieve dynamic attitude measurement.

[0004] The above measurement systems and methods are not well matched in terms of laser tracking equipment light failure detection, fast, high-precision measurement, and dynamic measurement. Visual sensors and PSD sensors require line of sight when in use. Once the laser tracking equipment and the corresponding corners are broken, the posture information of the object being measured will be lost; in addition, when the movement speed of the object being measured increases, the posture accuracy cannot be guaranteed. The non-equiperiodic Kalman filter data fusion method fuses the visual and IMU measurement data, and realizes dynamic posture measurement to a certain extent. However, this method has high requirements on the synchronization of the measurement system and the stability of the measurement frequency of each measurement unit, and is not adaptable to different measurement backgrounds. Summary of the invention

[0005] In view of the limitations of the above-mentioned posture measurement system and method in dynamic measurement, the present invention provides a laser tracking posture dynamic measurement method based on BP neural network.

[0006] The hardware components required for the measurement method provided by the present invention include: a laser tracking measurement unit, a visual measurement unit, a cooperative target, and an inertial measurement unit.

[0007] The laser tracking measurement unit and the visual measurement unit are respectively fixedly placed at preset positions, and the cooperative target is fixedly placed on the object to be measured; a plurality of non-coplanar feature marks are arranged outside the cooperative target, and a far-field and near-field target design is adopted, and an inertial measurement unit is installed inside the cooperative target;

[0008] The laser tracking measurement unit serves as a base station of the attitude dynamic measurement system, and provides point coordinate information required by the six-degree-of-freedom measurement system; the first coordinate system is the own coordinate system of the laser tracking measurement unit, and the origin is located at the laser transmitter head;

[0009] The visual measurement unit is used to obtain the two-dimensional pixel coordinates of the feature identifier on the cooperative target in the camera pixel coordinate system; the self coordinate system of the visual measurement unit is defined as the second coordinate system, and the origin is located at the optical center of the camera;

[0010] The inertial measurement unit is used to obtain dynamic motion data of the cooperative target in a third coordinate system, where the third coordinate system is the coordinate system of the inertial measurement unit itself, and the origin is located at the sensitive center of the inertial measurement unit;

[0011] The self coordinate system of the cooperative target is the fourth coordinate system, the origin of which is located at the geometric center of the target surface of the cooperative target; the distance between the origin of the fourth coordinate system and the origin of the third coordinate system is L, and the three axes are parallel to the three axes of the third coordinate system;

[0012] Through calibration experiments, the conversion relationship between the first coordinate system and the second coordinate system, the second coordinate system and the third coordinate system, and the third coordinate system and the fourth coordinate system is obtained; according to the rotation matrix obtained by solving the measurement data of the inertial measurement unit and the visual measurement unit, and the conversion relationship between different coordinate systems, the first attitude angle and the second attitude angle of the cooperative target in the first coordinate system are obtained; through fusion, the final attitude angle of the object under test is obtained;

[0013] Furthermore, the laser tracking measurement unit is a total station or a laser tracker, and the visual measurement unit is a CCD industrial camera.

[0014] Furthermore, the plurality of non-coplanar feature markers are made of infrared LED light tubes.

[0015] The present invention provides a posture dynamic measurement method based on BP neural network, which is characterized by comprising the following steps:

[0016] (1) Determine the relative position relationship between different coordinate systems through calibration experiments and determine the camera's intrinsic parameter matrix;

[0017] (2) measuring the rotation matrix between the fourth coordinate system and the second and third coordinate systems;

[0018] (3) Calculate the first and second attitude angles;

[0019] (4) Determine the BP neural network structure;

[0020] (5) Solve the fusion attitude angle.

[0021] The step (1) of determining the relative position relationship between the coordinate systems through a calibration experiment means obtaining the translation vectors between the first coordinate system and the second coordinate system, between the second coordinate system and the third coordinate system, and between the third coordinate system and the fourth coordinate system through a calibration external parameter experiment before the experiment begins; and obtaining the intrinsic parameter matrix of the visual measurement unit using the Zhang Zhengyou camera calibration method.

[0022] The measured rotation matrix between the fourth coordinate system and the second coordinate system in step (2) refers to solving the rotation matrix of the fourth coordinate system in the second coordinate system based on the visually captured feature point image; the measured rotation matrix between the fourth coordinate system and the third coordinate system refers to solving the rotation matrix of the fourth coordinate system in the third coordinate system based on the data measured by the inertial measurement unit.

[0023] The calculation of the first and second attitude angles in step (3) refers to solving the rotation matrices of the two fourth coordinate systems in the first coordinate system based on the two rotation matrices obtained in step (2) and the relative position relationship between the different coordinate systems obtained in step (1); based on these two rotation matrices, the attitude angles of the two groups of cooperative targets in the first coordinate system, i.e., the first attitude angle and the second attitude angle, can be obtained.

[0024] Determining the BP neural network structure in step (4) refers to preprocessing the first posture angle and the second posture angle in step (3), using the processed first posture angle and the second posture angle as the input layer of the BP neural network, and using the final posture angle as the output layer. The structure of the hidden layer is designed through simulation analysis, and the weight matrix and activation function between each layer are determined.

[0025] The solution of the fused attitude angle described in step (5) refers to the data fusion of the first attitude angle and the second attitude angle obtained in step (3) after preprocessing according to the BP neural network structure described in step (4), so as to obtain the final attitude angle of the object under test in the base station coordinate system, i.e., the first coordinate system.

[0026] Furthermore, step (1) is specifically as follows: the laser tracking measurement unit measures the spatial three-dimensional coordinates of the feature identifier in the first coordinate system, the visual measurement unit captures the cooperative target image with the feature identifier, and obtains the two-dimensional pixel coordinates of the feature identifier through image processing. The rotation matrix between the laser tracking measurement unit and the visual measurement unit can be solved by combining the PNP concept, that is, the conversion relationship between the first coordinate system and the second coordinate system; the rotation matrix between the inertial measurement unit and the cooperative target is calibrated by using the double vector orthogonal method, that is, the conversion relationship between the third coordinate system and the fourth coordinate system; the conversion relationship between the second coordinate system and the third coordinate system can be obtained by combining the rotation matrix obtained in step (2).

[0027] Furthermore, step (2) is specifically as follows: using a visual measurement unit to capture an image of a cooperative target with feature points, using an image processing algorithm to obtain the two-dimensional pixel coordinates of the feature points, performing two-dimensional to three-dimensional matching of the feature point coordinates according to a PNP algorithm, and obtaining a rotation matrix of the fourth coordinate system in the second coordinate system:

[0028]

[0029] Where K is the intrinsic parameter matrix of the camera, t is the rotation matrix and translation vector between the second coordinate system and the fourth coordinate system, f is the focal length of the camera, dx and dy are the actual physical sizes of a single pixel in the camera image plane in the u and v directions, (u, v) represents the pixel coordinates of the feature point, (x w,y w ,z w ) represents the coordinates of the feature point in the second coordinate system; the gyroscope sensor in the inertial measurement unit measures the angular velocity data of the cooperative target during its movement, obtains the angle increment through integration, and then obtains the rotation vector of the coordinate system. The change quaternion can be calculated from the rotation vector, and the rotation matrix of the fourth coordinate system in the third coordinate system is represented by the quaternion.

[0030] Assuming that the cooperative target moves from position A to position B, the change in its coordinate system can be expressed by the following quaternion:

[0031]

[0032] Where θ represents the rotation vector calculated based on the output data of the inertial measurement unit The rotation angle, r x 、r y and r z They are defined The components on the x, y, and z axes of coordinate system A, θ represents the rotation angle of coordinate system B relative to coordinate system A.

[0033] Transformation matrix between coordinate systems A and B It can be expressed as:

[0034]

[0035] Since the inertial measurement unit measures the rotation of the cooperative target relative to the IMU coordinate system, the rotation matrix between the inertial measurement unit and the cooperative target can be expressed as:

[0036] Furthermore, step (3) is specifically as follows: in order to meet the six-degree-of-freedom measurement requirements of the measurement system, the calculated attitude angle data needs to be converted to the base station coordinate system, that is, the first coordinate system. The conversion method of different coordinate systems can be based on:

[0037]

[0038]

[0039] In the formula, They are the rotation matrices of the fourth coordinate system in the first coordinate system solved by the measurement data of the inertial measurement unit and the visual measurement unit, Obtained from step (1), they represent the transformation matrices between the third coordinate system and the fourth coordinate system, the first coordinate system and the second coordinate system, and the second coordinate system and the third coordinate system respectively.

[0040] According to the obtained rotation matrix Solve the attitude angle and express the two as:

[0041]

[0042]

[0043] The attitude angle can be obtained:

[0044] α IMU =arcsin(-2(q3q4-q1q2))

[0045]

[0046]

[0047] α C = -arcsin(r 23 )

[0048]

[0049]

[0050] In the formula, α C , β C , γ C is the attitude angle obtained according to the measurement results of the visual measurement unit; α IMU , β IMU , γ IMU It is the attitude angle obtained based on the measurement results of the inertial measurement unit.

[0051] Furthermore, the step (4) is specifically as follows: designing the structure of a BP neural network. The structure of the BP neural network used in this method includes an input layer, a hidden layer, and an output layer; each layer is connected by a weight matrix and an activation function.

[0052] Furthermore, the input vector of the neural network is set as: X = (x1, x2, ..., x n ); Hidden layer output: Y = (y1, y2, y3, ..., y m ); Output result: O = (o1, o2, o3, ..., o t ); Expected output: D = (d1, d2, d3, ..., d t ); Weight matrix: V = (v1, v2, v3, ..., v m ); W=(w1,w2,w3,...,w t );

[0053] Among them, the connection weight from the input layer to the hidden layer is V; and the connection weight from the hidden layer to the output layer is W; there are two directions of signal propagation inside the BP neural network, signal forward propagation and error reverse propagation. In the process of signal forward propagation, the calculation formula from the input layer to the hidden layer is expressed as:

[0054]

[0055] In the formula, f h (·) is the activation function between the input layer and the hidden layer of the BP neural network, θ is the bias of the hidden layer, n represents the threshold of the input layer neurons, and m represents the threshold of the hidden layer neurons;

[0056] Furthermore, the calculation formula from the hidden layer to the output layer can be expressed as:

[0057]

[0058] In the formula, f o (·) is the activation function between the hidden layer and the output layer of the BP neural network, λ is the bias of the output layer, m represents the threshold of the neurons in the hidden layer, and t represents the threshold of the neurons in the input layer.

[0059] In the error back propagation process, since there is an error between the output result of the forward propagation and the expected output result, the error is fed back to the hidden layer to adjust the weights of each node in the hidden layer. The error definition formula between the final result and the expected result is expressed as:

[0060]

[0061] The formula of the error function from the output layer to the hidden layer is expressed as:

[0062]

[0063] The formula of the error function expanded to the input layer is expressed as:

[0064]

[0065] Furthermore, the process of error reverse conduction is to solve the gradient of each weight. Therefore, according to the gradient descent strategy, the error signal is defined as:

[0066]

[0067] The error signal of the hidden layer is defined as:

[0068]

[0069] The iterative update of weights is expressed as:

[0070]

[0071]

[0072] In the above formula, the negative sign represents the gradient descent, μ is a constant, and the range is (0,1) indicating the proportional coefficient.

[0073] Furthermore, the input layer receives data features, the hidden layer performs nonlinear transformation and feature extraction, and the output layer gives the network results, representing classification or regression values. The training process of the BP neural network is as follows:

[0074] (1) Initialize weights: Randomly initialize the weights of all connections.

[0075] (2) Forward propagation: pass data from the input layer to the output layer and calculate the network output.

[0076] (3) Calculate loss: Compare the network output with the true label and calculate the loss function (such as mean square error or cross entropy).

[0077] (4) Back propagation: Use the gradient information of the loss function to calculate the impact of each weight on the loss, and update the weight to reduce the loss.

[0078] (5) Update weights: Use an optimization algorithm (such as gradient descent) to update the network weights based on the back-propagated gradients.

[0079] (6) Iterative training: Repeat forward propagation, loss calculation, back propagation, and weight update until the network converges or reaches the stopping condition.

[0080] The BP neural network learns the mapping relationship between the features of the input data and the output by continuously adjusting the weights to minimize the loss function.

[0081] Furthermore, before using the BP neural network for data fusion, the first attitude angle and the second attitude angle data need to be preprocessed; during the measurement process, the data acquisition frequency of the inertial measurement unit is high, and the data acquisition frequency of the visual measurement unit is low, so the collected visual data needs to be interpolated to achieve time alignment;

[0082] Assuming that the cooperative target is moving at a uniform speed, the change of attitude angle and time are linear characteristics. In this case, linear interpolation is used to fill in the missing data. Vi , β Vi , γ Vi ) and (α Cj , β Cj , γ Cj ) are the visual measurement units at t i ,t jThe linear interpolation principle of the attitude angle data collected at all times can be expressed as:

[0083]

[0084] In the formula, (α Vk , β Vk , γ Vk ) is the filled t k After completing the interpolation and data alignment, the BP neural network is used to predict the final attitude angle; the data Q measured by the interpolated visual measurement unit is V and the data Q measured by the inertial measurement unit I As the input x of BP neural network in , which can be expressed as:

[0085] x in =[Q V Q I ]

[0086] BP neural network outputs the fused attitude angle y out It can be expressed as:

[0087] y out =[αβγ]

[0088] In the attitude information fusion system based on the visual measurement unit and the inertial measurement unit described in the present invention, a BP neural network including an input layer, a hidden layer and an output layer is constructed; the input layer includes six nodes, which respectively correspond to the three attitude angles α calculated by the visual measurement unit in step (3) C , β C , γ C And the three attitude angles α calculated by the inertial measurement unit IMU , β IMU , γ IMU The output layer includes three nodes, which are the three-axis attitude information corresponding to the final attitude angle.

[0089] The scale of a neural network is determined by the depth of the hidden layer and the number of neurons in each layer, which directly affects the performance of the network. Therefore, determining the number of neurons in the hidden layer is crucial for building a training model. In general, the formula for determining the number of neurons in the hidden layer is expressed as:

[0090]

[0091] Where count input 、count output Represents the number of neurons in the input layer and the output layer, and a is an adjustment parameter between 0 and 10.

[0092] Furthermore, the step (5) is specifically as follows: using the BP neural network designed in step (4) to perform data fusion and calculate the final attitude angle.

[0093] The cooperative target is set to perform a uniaxial motion of 5° / s, and the training target value determined by the neighborhood search method is used as the target output value for training and learning. The hidden layer input is:

[0094] Q input =w1*x+b1

[0095] Where x is the attitude angle Q measured by vision and IMU V and Q I . Hidden layer output:

[0096] Q output =sigmoid(Q input )

[0097] Output layer input:

[0098] U input =w2*Q output +b2

[0099] Output layer output:

[0100] U output =U input

[0101] Among them, w1 and b1 are the weight matrix and bias vector between the input layer and the hidden layer, w2 and b2 are the weight matrix and bias vector between the hidden layer and the output layer, and sigmoid is the activation function. During the BP neural network training process, the weight matrix and bias vector will be continuously learned and adjusted so that the model can better fit the training data.

[0102] In the neural network training process, the selection of data set size is crucial. The present invention selects a sampling time of 50 seconds and a sampling frequency of 100 Hz to obtain a data set containing 5000 test data points. By inputting the entire data set into the neural network, the gradient descent algorithm is used for iterative training, and the weight matrices w1, w2 and bias vectors b1, b2 are continuously updated to minimize the loss function and optimize the model performance. Ensure that the model makes full use of all sample information, while ensuring memory usage, improve training speed and model stability, and the trained neural network can accurately predict the attitude angle of the target object. The fusion processing of the attitude angle data of vision and IMU is realized through simulation.

[0103] The present invention provides a method for dynamic posture measurement based on BP neural network. By using BP neural network data fusion method, the high dynamics of inertial measurement unit and the high precision of visual measurement unit are comprehensively utilized to complete the measurement of posture angle based on the conversion relationship between different coordinate systems. Since BP neural network can model complex nonlinear relationships, dynamic posture measurement of the measured object can also be realized when the measured object performs complex motion, so the data fusion method based on BP neural network can be used to perform posture measurement in most application scenarios. This method improves the overall dynamics, stability, and robustness of the laser tracking posture measurement system, makes up for the defect that the measurement system cannot measure when the visual measurement unit field of view is blocked, and improves the problem that the measurement speed and accuracy cannot be guaranteed when the moving speed of the measured object increases. Compared with the strict requirements of the non-equiperiodic Kalman filter data fusion method on the stability and accuracy of the measurement frequency of the measuring device, the data fusion method based on BP neural network does not need to strictly require the measurement frequency of the measuring device, which effectively improves the adaptability and versatility of the measuring method. And compared with the non-equiperiodic Kalman filter data fusion method, the dynamic measurement accuracy of the data fusion method based on BP neural network is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0105] Figure 1 A schematic diagram of a laser tracking attitude dynamic combined measurement system provided by an embodiment of the present invention;

[0106] Figure 2 A schematic diagram of the installation position of an inertial measurement unit provided by an embodiment of the present invention in a cooperative target;

[0107] Figure 3 A flow chart of a method for performing attitude angle measurement using the above-mentioned laser tracking attitude dynamic combined measurement system provided by an embodiment of the present invention;

[0108] Figure 4 A comparison diagram of simulation results of neural network training with different numbers of hidden layer neurons when other conditions are the same is provided for an embodiment of the present invention.

[0109] Figure 5 The embodiment of the present invention provides a schematic diagram of the structure of the BP neural network designed by the present invention based on the above simulation results.

[0110] Figure 6 A comparison chart of three solution results of the azimuth and pitch angles of the cooperative target installed on a precision two-dimensional turntable at 3m provided in an embodiment of the present invention, namely, solution by the inertial measurement unit after calibration, solution by the visual measurement unit, and fusion value of the solutions by the inertial measurement unit and the visual measurement unit.

[0111] Figure 7 A comparison chart of three solution results of the azimuth and pitch angles of the cooperative target installed on a precision two-dimensional turntable at 8m provided in an embodiment of the present invention, namely, solution by the inertial measurement unit after calibration, solution by the visual measurement unit, and fusion value of the solutions by the inertial measurement unit and the visual measurement unit.

[0112] Figure 8 A comparison chart of three solution results provided in an embodiment of the present invention, namely, solution of the azimuth and pitch angles of the cooperative target installed on the robotic arm after calibration by the inertial measurement unit, solution by the visual measurement unit, and fusion value of the solutions by the inertial measurement unit and the visual measurement unit, at a distance of 3 meters.

[0113] Fig. 9 A comparison chart of three solution results provided in an embodiment of the present invention, namely, solution of the azimuth and pitch angles of the cooperative target installed on the robotic arm after calibration by the inertial measurement unit, solution by the visual measurement unit, and fusion value of the solutions by the inertial measurement unit and the visual measurement unit, at a distance of 3 meters. DETAILED DESCRIPTION

[0114] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0115] Figure 1 A laser tracking attitude dynamic combined measurement system is provided for an example of the present invention, comprising: a laser tracking measurement unit 1, a visual measurement unit 2, an inertial measurement unit 3 and a cooperative target 4.

[0116] Among them, the laser tracking measurement unit 1 and the visual measurement unit 2 are respectively fixed at preset positions, the inertial measurement unit is fixedly placed at a preset position inside the cooperative target, the two are rigidly connected, the cooperative target 4 is installed on the object to be measured, and a number of non-coplanar feature markers are arranged on the outside of the cooperative target 4.

[0117] The laser tracking measurement unit's own coordinate system is defined as the first coordinate system; the visual measurement unit's own coordinate system is defined as the second coordinate system; the inertial measurement unit's own coordinate system is defined as the third coordinate system; and the cooperative target's own coordinate system is defined as the fourth coordinate system.

[0118] According to the measurement data of the inertial measurement unit and the visual measurement unit and the conversion relationship between different coordinate systems obtained through calibration, the motion of the fourth coordinate system is converted to the first coordinate system, and the final attitude angle data of the object under test is obtained through the BP neural network data fusion algorithm.

[0119] Among them, the cooperative target coordinate system corresponds to the fourth coordinate system. When the cooperative target (i.e., the object to be measured) moves, the fourth coordinate system will also move accordingly. The change of the coordinate system during the movement can be represented by a rotation matrix. This rotation matrix is ​​the rotation matrix of the fourth coordinate system in the second coordinate system. The first attitude angle of the cooperative target in the first coordinate system is obtained by transferring the coordinate system and solving the attitude angle. The inertial measurement unit and the cooperative target are rigidly connected. The cooperative target drives the inertial measurement unit to move. The rotation matrix solved by the inertial measurement unit output data is the rotation matrix of the fourth coordinate system in the third coordinate system. According to the conversion relationship between the second coordinate system and the third coordinate system and the first coordinate system and the second coordinate system, the rotation matrix of the fourth coordinate system in the first coordinate system is obtained, and the second attitude angle of the cooperative target in the first coordinate system is obtained by solving the attitude angle. The first attitude angle and the second attitude angle are fused through the BP neural network data to obtain the final attitude angle of the cooperative target in the laser tracking measurement unit coordinate system.

[0120] In the above embodiment, the laser tracking measurement unit is a total station or a laser tracker, and the visual measurement unit is an industrial CCD camera.

[0121] In the above embodiment, the plurality of non-coplanar feature markers are made of infrared LED light tubes.

[0122] Figure 3 A flow chart of a method for measuring attitude angle using the above laser tracking attitude dynamic combined measurement system is provided as an example of the present invention, such as Figure 3 As shown, including:

[0123] S301, fixing the laser tracking measurement unit and the visual measurement unit at preset positions respectively, fixing the cooperative target on the object to be measured, rigidly connecting the inertial measurement unit and the cooperative target together, and obtaining the conversion relationship between the coordinate systems and the internal parameter matrix of the visual measurement unit through a calibration experiment;

[0124] S302, using the visual measurement unit to obtain a rotation matrix of the fourth coordinate system in the second coordinate system through the plurality of non-coplanar feature identifiers; and calculating the rotation matrix of the fourth coordinate system in the third coordinate system according to output data of the inertial measurement unit;

[0125] S303, according to the conversion relationship between the different coordinate systems calibrated in S301 and the two rotation matrices obtained in S302, solving two sets of rotation matrices of the fourth coordinate system in the first coordinate system, and solving the first attitude angle and the second attitude angle;

[0126] S304, according to the vector dimensions of the first posture angle, the second posture angle, and the final posture angle obtained in S303, determine the number of nodes in the input layer and the output layer of the BP neural network, and select the most appropriate number of hidden layer nodes by comparing the training time and the accuracy of the training results through simulation; then determine the appropriate activation function, and finally determine the structure of the BP neural network.

[0127] S305, fusing the first posture angle and the second posture angle through a BP neural network data fusion algorithm to obtain a final posture angle of the cooperative target in the first coordinate system;

[0128] The present invention provides a method for dynamic posture measurement based on BP neural network. By using BP neural network data fusion method, the high dynamics of inertial measurement unit is comprehensively utilized, but there will be cumulative errors in long-term measurement, and the high precision of visual measurement unit, but the measurement frequency is low, and the posture angle measurement is completed based on the conversion relationship between different coordinate systems.

[0129] Since the BP neural network can model complex nonlinear relationships, it can also realize dynamic posture measurement of the measured object when the measured object performs complex movements. Therefore, the data fusion method based on the BP neural network can be used to perform posture measurement in most application scenarios. This method improves the overall dynamics, stability, and robustness of the laser tracking posture measurement system, makes up for the defect that the measurement system cannot measure when the visual measurement unit field of view is blocked, and improves the problem that the measurement speed and accuracy cannot be guaranteed when the movement speed of the measured object increases. Compared with the strict requirements of the non-equiperiodic Kalman filter data fusion method on the stability and accuracy of the measurement frequency of the measuring equipment, the data fusion method based on the BP neural network does not need to strictly require the measurement frequency of the measuring equipment, which effectively improves the adaptability and versatility of the measurement method. In addition, compared with the non-equiperiodic Kalman filter data fusion method, the dynamic measurement accuracy of the data fusion method based on the BP neural network is also improved.

[0130] In the above embodiment, using the visual measurement unit to obtain the rotation matrix between the second coordinate system and the fourth coordinate system through the plurality of non-coplanar feature identifiers specifically includes:

[0131] Specifically, the visual measurement unit captures an image of a cooperative target with a feature identifier, obtains the two-dimensional pixel coordinates of the feature identifier after image processing, and completes the two-dimensional to three-dimensional matching of the feature identifier in combination with the PNP algorithm based on the known coordinates of the feature identifier in the cooperative target coordinate system. After solving, the relative relationship between the cooperative target coordinate system and the visual measurement unit coordinate system is obtained, that is, the rotation matrix of the fourth coordinate system in the second coordinate system; the PNP algorithm is not described in detail here, and reference is made to the prior art.

[0132] In the above embodiment, calculating the rotation matrix between the third coordinate system and the fourth coordinate system according to the output data of the inertial measurement unit specifically includes:

[0133] The output data of the gyroscope in the inertial measurement unit is angular velocity. The available angle change is obtained by integration and error calibration. The quaternion is obtained by calculating the rotation vector to represent the rotation matrix, which is the rotation matrix between the third coordinate system and the fourth coordinate system. The quaternion attitude solution algorithm is not described here, and reference is made to the prior art. According to the quaternion attitude solution algorithm conversion matrix, the rotation matrix of the cooperative target relative to the initial coordinate system of the inertial measurement unit is obtained:

[0134]

[0135] In the above embodiment, according to the conversion relationship between different coordinate systems obtained by calibration, and the two rotation matrices obtained by the visual measurement unit and the inertial measurement unit, the rotation matrices of two sets of the fourth coordinate system in the first coordinate system are solved, and the first attitude angle and the second attitude angle are calculated, specifically including:

[0136] According to the conversion relationship between the first coordinate system and the second coordinate system, the rotation matrix of the fourth coordinate system in the second coordinate system is transferred to the first coordinate system; according to the conversion relationship between the first coordinate system and the second coordinate system, and between the second coordinate system and the third coordinate system, the rotation matrix of the fourth coordinate system in the third coordinate system is transferred to the first coordinate system; the first attitude angle and the second attitude angle of the cooperative target in the laser tracking measurement unit coordinate system can be obtained by solving the calculation.

[0137] Figure 4 The simulation results of neural network training with different numbers of hidden layer neurons under the same conditions are compared for the embodiments of the present invention. For the BP neural network described in the present invention, the input layer has 6 nodes and the output layer has 3 nodes. The number of neurons in the hidden layer of the BP neural network can be obtained as counthidden The value range of should be between 3 and 15. Considering that the number of neurons in the hidden layer is usually greater than the number of neurons in the input layer, the number of neurons count hidden The value range of is [6,15]. Under the condition of setting the other error terms, the neural network training is carried out by simulating different numbers of neurons one by one. Figure 4 From the results of (a) and (b), it can be seen that when the number of neurons is 11 and 15, the standard error of the attitude angle predicted by the BP neural network is small; however, as the number of neurons increases, the time required for training will also be correspondingly extended. When the number of hidden layer neurons is 11, its training time is significantly less than the training time when the number of neurons is 15. After comprehensively considering the impact of the number of neurons on the training time and accuracy, the present invention sets the number of hidden layer nodes of the BP neural network to 11.

[0138] Figure 5 The embodiment of the present invention provides a schematic diagram of the structure of the BP neural network designed by the present invention based on the above simulation results. The BP neural network structure of the present invention is determined to be 6-11-3, that is, the BP neural network input layer has 6 neurons, the hidden layer has 11 neurons, and the output layer has 3 neurons; the Sigmoid function that is closer to the signal output form of the neuron is selected as the activation function from the input layer to the hidden layer, and the linear function is selected as the activation function from the hidden layer to the output layer.

[0139] The initial weights will affect the stability of the training results. Selecting a smaller initial weight can not only ensure that the initial value of the input neuron is smaller, but also effectively prevent the unreasonable growth of the weight after multiple continuous training and learning. In the invention, the initial weights of the BP neural network are set as random numbers between (-1, 1).

[0140] In the above embodiment, the simulation training of the BP neural network can use the theoretical value of the posture as the target value for training, but in practical applications, the theoretical value of the posture is difficult to obtain, so the present invention uses a training target value determination method based on neighborhood search to determine the target value that should be used when training the neural network by analyzing the temporal change trend of the sensor data.

[0141] In the above embodiment, the trained BP neural network is experimentally verified, and the first posture angle and the second posture angle are fused to obtain the final posture angle of the cooperative target through the BP neural network data fusion algorithm, which specifically includes:

[0142] The first attitude angle α of the cooperative target in the first coordinate system C , β C , γ C , the second attitude angle α IMU , β IMU, γ IMU Substituting into the BP neural network data fusion algorithm, the final attitude angles α, β, and γ can be obtained.

[0143] The experiments of the present invention can be further illustrated by the following experimental results:

[0144] The laser tracking measurement unit is set to use Leica TM50 total station, with a measurement range of 1.5 to 1000m and a measurement accuracy of 2mm+1mm / m; the visual measurement unit uses Basler acA2500-20gm industrial camera, with a resolution of 2590×2048pixel, a pixel size of 4.8um, a frame rate of up to 21FPS, and a matching lens using KOWA's LM75HC 75mm fixed-focus lens, with an aperture range of F1.8 to F16 and a distortion of less than 0.1%. The inertial measurement unit (IMU) uses ADIS16488 inertial sensor; the cooperative target adopts a three-dimensional coplanar design, and the target has 24 infrared LED light tube feature points on the outside to adapt to the measurement requirements of different distances. The present invention only needs to use 9 LED light tubes to meet the measurement requirements.

[0145] In order to verify that the BP neural network dynamic posture measurement method can realize the measurement of the posture of the object to be measured, the present invention adopts a precision two-dimensional turntable as the object to be measured. The measurement accuracy of the precision two-dimensional turntable is 2″, and the indicated angular velocity is 5° / s; the azimuth angle can be rotated in the range of 50° to 330°, and the pitch angle can be rotated in the range of 0° to 360°.

[0146] In building Figure 1 After the measurement system is shown, the conversion relationship between the coordinate systems and the internal parameter matrix of the visual measurement unit are obtained through calibration experiments.

[0147] The precision two-dimensional turntable is controlled by the host computer to perform uniform single-axis rotation, and the pitch angle of the turntable is set to 50°, and the azimuth angle of the turntable is set to 55°; the data acquisition frequency of the inertial measurement unit is set to 100Hz, the inertial measurement unit communicates with the single-chip microcomputer through the SPI communication protocol, and the single-chip microcomputer communicates with the host computer through the serial port; because the BP neural network data fusion algorithm has a relatively loose requirement on the stability of the visual measurement frequency, the measurement frequency of the visual measurement unit is set to about 1Hz, and the visual measurement unit transmits signals with the host computer through GigE Vision;

[0148] During the experiment, the inertial measurement unit and the visual measurement unit are first turned on for signal collection, and then the precision two-dimensional turntable is controlled to rotate on a single axis. After the precision two-dimensional turntable stops rotating, the inertial measurement unit and the visual measurement unit stop collecting signals. At this time, the output value of the gyroscope of the inertial measurement unit, the image signal collected by the visual measurement unit, and the time corresponding to each frame of the image are obtained. By performing quaternion attitude calculation on the output value of the gyroscope collected by the inertial measurement unit, processing the image collected by the visual measurement unit, and performing the above-mentioned coordinate system conversion, the above-mentioned first attitude angle and second attitude angle can be obtained.

[0149] Since the precision two-dimensional turntable used in the experiment rotates at a uniform speed, the time axis of the data collected by the inertial measurement unit and the visual measurement unit is aligned, and the second attitude angle solved by the visual measurement unit is linearly interpolated; then the data collected by the inertial measurement unit and the processed data of the visual measurement unit are brought into the above-mentioned BP neural network input layer, and the final attitude angle data is output through the BP neural network fusion algorithm. Some experimental data of a single-axis rotation experiment with a measurement distance of 3m, a rotation angle of pitch angle, and a rotation angle of 50° are shown in Table 1.

[0150] Table 1 3m-pitch angle-50° experimental data

[0151]

[0152]

[0153] It can be seen from Table 1 that the BP neural network fusion algorithm can realize the tracking measurement of the posture of the object being measured. Figure 6 and Figure 7 It can be seen more intuitively that when the object being measured is rotating at a single-axis uniform speed of 5° / s, the attitude dynamic measurement method based on the BP neural network of the present invention can realize the attitude dynamic measurement of the object being measured. Figure 8 and Fig. 9 It can be seen that when the object to be measured performs a single-axis rotation at a speed of 15° / s, the dynamic measurement method of the present invention can also achieve tracking measurement of the posture of the object to be measured.

[0154] It can be seen from the last five rows of Table 1 that when the precision two-dimensional turntable stops rotating, the inertial measurement unit will continue to generate cumulative errors as the measurement time increases. Since the visual measurement unit has no cumulative errors and has relatively high accuracy, after the two sets of attitude angle data are fused through the BP neural network, the final attitude angle data obtained has both high-frequency characteristics and avoids the phenomenon of large errors in long-term measurement.

[0155] The dynamic posture measurement method based on BP neural network described in the present invention has improved accuracy compared with the existing method based on non-equiperiodic Kalman filter fusion algorithm. Taking azimuth measurement as an example, the measurement results are shown in Table 2.

[0156] Table 2 Comparison of measurement accuracy

[0157]

[0158] It can be seen from Table 2 that at a measurement distance of 3 m, the error of the dynamic posture measurement method of the present invention is significantly improved in accuracy compared with the non-equiperiodic Kalman filter fusion method.

[0159] The present invention compares the measurement results of the attitude angle calculated by visual measurement, the attitude angle calculated after the inertial measurement unit is calibrated, and the fusion attitude angle of the visual measurement unit and the inertial measurement unit at the positions of 3m and 8m, respectively. The results show the measurement results of the cooperative target installed on the precision two-dimensional turntable and the mechanical arm for single-axis rotation, such as Figures 6 to 9 As shown in the figure, it can be seen that the posture dynamic measurement method described in the present invention can track and measure the dynamic characteristics of the objects under test with different movement speeds and different movement modes, and when the cooperative target is blocked and the CCD cannot be seen, the posture can also be tracked, which can adapt to a variety of application scenarios.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A laser tracking posture dynamic combined measurement method, comprising the following steps: (1) Determine the relative position relationship between different coordinate systems through calibration experiments and determine the camera's intrinsic parameter matrix; (2) measuring the rotation matrix between the fourth coordinate system and the second and third coordinate systems; (3) Calculate the first and second attitude angles; (4) Determine the BP neural network structure; (5) Solve the fusion attitude angle.

2. The measuring method according to claim 1, characterized in that: Determining the relative position relationship between the coordinate systems through a calibration experiment in step (1) means obtaining the translation vectors between the first coordinate system and the second coordinate system, between the second coordinate system and the third coordinate system, and between the third coordinate system and the fourth coordinate system through a calibration external parameter experiment before the experiment begins; And the Zhang Zhengyou camera calibration method is used to obtain the intrinsic parameter matrix of the visual measurement unit.

3. The measuring method according to claim 1, characterized in that: The measured rotation matrix between the fourth coordinate system and the second coordinate system in step (2) refers to solving the rotation matrix of the fourth coordinate system in the second coordinate system based on the visually captured feature point image; the measured rotation matrix between the fourth coordinate system and the third coordinate system refers to solving the rotation matrix of the fourth coordinate system in the third coordinate system based on the data measured by the inertial measurement unit.

4. The measuring method according to claim 1, characterized in that: The step (3) of calculating the first and second posture angles refers to solving the rotation matrices of the two fourth coordinate systems in the first coordinate system based on the two rotation matrices obtained in the step (2) and the relative position relationship between the different coordinate systems obtained in the step (1); According to the two rotation matrices, the attitude angles of the two groups of cooperative targets in the first coordinate system, namely, the first attitude angle and the second attitude angle, can be obtained.

5. The measuring method according to claim 1, characterized in that: Determining the BP neural network structure in step (4) refers to preprocessing the first posture angle and the second posture angle in step (3), using the processed first posture angle and the second posture angle as the input layer of the BP neural network, and using the final posture angle as the output layer. The structure of the hidden layer is designed through simulation analysis, and the weight matrix and activation function between each layer are determined.

6. The measuring method according to claim 1, characterized in that: The solution of the fused attitude angle described in step (5) refers to the data fusion of the first attitude angle and the second attitude angle obtained in step (3) after preprocessing according to the BP neural network structure described in step (4), so as to obtain the final attitude angle of the object under test in the base station coordinate system, i.e., the first coordinate system.

Citation Information

Cited By

  • IMU camera calibration system and method based on neural network

    CN119919504A

  • A neural network-based imu camera calibration system and method

    CN119919504B