A method for measuring the pan-tilt tracking performance based on machine vision technology

Abstract

The present invention discloses a method for measuring the tracking performance of a pan-tilt based on machine vision technology. The camera is placed facing the display screen directly. The display screen plays a pre-set video of the target movement trajectory. The camera captures images and tracks the target in the display screen in real time, and acquires all the image sequences during the tracking process. Then, the image sequences are analyzed and processed to obtain the measurement result of the pan-tilt tracking performance. The present invention can be realized under simple devices in daily life, without other devices such as turntables. It combines the target tracking algorithm with the pan-tilt well. On the already built structure, if it is necessary to unidirectionally measure the tracking performance of the algorithm or the pan-tilt, it can also be realized on this structure by controlling variables.

Description

Technical Field

[0001] The present invention belongs to the machine vision related processing methods in the technical field of unmanned aerial vehicles, and particularly relates to a pan-tilt tracking performance measurement method based on machine vision technology. Background Art

[0002] When traditionally detecting and evaluating the pan-tilt tracking performance, the target detection and tracking algorithm and the pan-tilt control algorithm can only be separated, and the quality of the algorithms can only be detected and evaluated unilaterally. The pan-tilt control algorithm is also based on online simulation and debugging. However, after most simulations are debugged and applied in practice, there are still significant differences in the effects.

[0003] Traditional pan-tilt tracking performance measurement cannot take into account the response speed and tracking accuracy of the pan-tilt itself, and cannot well integrate the target tracking algorithm with the pan-tilt. Even when applying the pan-tilt control algorithm to the pan-tilt, it can only qualitatively evaluate the quality, and cannot quantitatively give a measurement and evaluation data.

[0004] Due to the combined effects of various internal and external disturbances such as carrier perturbation, friction, mass imbalance, airflow perturbation, output torque fluctuation, and complex frame structure and coupling relationship, it is difficult for the system to achieve stable control. Therefore, whether researching the pan-tilt or improving the target algorithm, a tracking measurement system for improving the pan-tilt is particularly important. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to complete a pan-tilt tracking accuracy measurement system for embedding face recognition in an unmanned aerial vehicle under daily limited living conditions.

[0006] To achieve the following technologies, the present invention adopts the following technical solutions:

[0007] The present invention places the camera facing the display screen directly. The display screen plays a pre-set video of the target moving trajectory. The camera captures images and tracks the target in the display screen in real time, and collects all the image sequences during the tracking process. Then, the image sequences are analyzed and processed to obtain the measurement result of the pan-tilt tracking performance.

[0008] For each image in the image sequence, the position of the target in the image is identified, the difference between the position of the target in the image and the center point of the image is obtained to get the deviation data, and then the deviation data of each image changing with time are combined into two-dimensional complex data. In this way, a two-dimensional complex data is formed by using the time dimension and the distance dimension, which serves as the data source for measuring the pan-tilt tracking accuracy; then the two-dimensional complex data is normalized to obtain the measurement result.

[0009] The time dimension is in frames, the total number of frames is N, the frame period is T, the distance dimension is in pixel units, and the observation range is [1, L], where L represents the horizontal width of the camera image.

[0010] The target moving trajectory video is divided into three stage parts:

[0011] The first stage part is a one-dimensional step motion process. Specifically, the target is set to reciprocate along a horizontal straight line / vertical straight line trajectory on the display screen;

[0012] The second stage part is a two-dimensional ramp motion process. Specifically, the target is set to reciprocate along an inclined straight line trajectory on the display screen;

[0013] The third stage part is a two-dimensional trajectory motion process. Specifically, the target is set to circularly move along a closed-loop crossable curve trajectory on the display screen.

[0014] The target is set as a marker on the display screen.

[0015] During the process of the target moving along the straight line trajectory, from one end of the trajectory to the other end, it is completed with uniformly accelerated motion in the first half of the trajectory and uniformly decelerated motion in the second half of the trajectory.

[0016] The closed-loop crossable curve can specifically be an 8-shaped curve, and the target moves along the 8-shaped curve. However, it is not limited to this.

[0017] For the image sequences collected in the first stage part and the second stage part, a discrete time model with a fixed sampling period t is established. According to the two-dimensional complex data processing calculation, the overshoot, residual error, oscillation times, and decay ratio of each object are obtained. The tracking performance of the pan-tilt is judged by using the overshoot, residual error, oscillation times, and decay ratio. Specifically, a performance evaluation model shown by the following formula is established for the above data:

[0018] SUM = m1×w1 + m2×w2 + m3×w3 + m4×w4

[0019] Among them, SUM is the performance evaluation parameter, m1, m2, m3, m4 are the overshoot, residual error, oscillation times, and decay ratio, and w1, w2, w3, w4 are the influence factors of each overshoot, residual error, oscillation times, and decay ratio.

[0020] The overshoot ranges from 0% to 10%, and any value beyond the limit is 0. The value of m1 ranges from 1 to 0; the residual error is the remaining deviation at the end of the transient process. The smaller the deviation value, the higher the value. Specifically, the range of the residual error is 0% to 5%, and the value of m2 ranges from 1 to 0; the number of oscillations outside the threshold of 5% is 2 to 5 times, and the value of m3 ranges from 1 to 0; the decay ratio refers to the ratio of the two consecutive peak values. The optimal ratio for specific implementation is 4:1. When the ratio is the optimal ratio, the score value of m4 is 1. When the ratio is less than 4:1 but greater than 6:1, m4 evenly distributes the score values from 1 to 0; similarly, when the ratio is greater than 4:1 and less than 2:1, m4 also evenly distributes the score values from 1 to 0.

[0021] Based on the performance evaluation parameters obtained from the performance evaluation model, the following judgments are made:

[0022] When SUM is greater than 0.8, the pan-tilt tracking performance is excellent;

[0023] When 0.6 <= SUM < 0.8, the pan-tilt tracking performance is good;

[0024] When 0.4 <= SUM < 0.6, the pan-tilt tracking performance is poor;

[0025] When 0 <= SUM < 0.4, the pan-tilt tracking performance is extremely poor.

[0026] The above overshoot and residual error are calculated as follows:

[0027] Overshoot = [x max - s] / s × 100%

[0028] Residual error = x ∞ - s

[0029] where x ∞ is the point where the image center finally stabilizes during tracking, x max is the maximum distance by which an object exceeds s, and s represents the position coordinate of the target.

[0030] The processes of the first-stage part and the second-stage part are equivalent to an oscillation process, such as forms of divergent oscillation, equal-amplitude oscillation, decaying oscillation, monotonic process, etc. The tracking performance during its step change is determined by parameters such as overshoot, residual error, and number of oscillations.

[0031] For the image sequence collected in the third-stage part, a discrete-time model with a fixed sampling period t is established. By calculating the deviation between the target position and the image center point through two-dimensional complex data processing, and then calculating the determination coefficient and uncertainty, the tracking performance of the pan-tilt is obtained by making a judgment using the determination coefficient and uncertainty.

[0032] Under the action of a step input, the instantaneous maximum deviation value of the controlled variable (X maxThe ratio to the steady-state value (X(∞)); the offset is the residual deviation at the end of the transient process; the number of oscillations is the number of oscillations of the regulated parameter during the transient time; the height of the first peak is four times that of the second peak (such a curve is also called a 4:1 decay curve). A 4:1 decay transient process is the best.

[0033] The measurement of the sub-uncertainty is specifically as follows:

[0034] S1. First, obtain the type A sub-uncertainty u in the following manner A :

[0035] The deviation data collected in the second stage is [e1, e2, e3,...], and the calculation is as follows:

[0036] e i = x - s i

[0037] where x is the coordinate of the center point of the image, and s i is the coordinate of the target position of the i-th frame, and e i is the deviation of the i-th frame;

[0038] Furthermore, calculate the experimental standard deviation S A (e) as follows:

[0039]

[0040] Then, calculate the type A sub-uncertainty u of the evaluation azimuth angle according to the following formula based on the experimental standard deviation S A (e): A :

[0041]

[0042] where n represents the amount of data in the two-dimensional complex data, that is, the number of frames of the image;

[0043] S2. First, obtain the type B sub-uncertainty u in the following manner B :

[0044] Calculate the type B sub-uncertainty of each component according to the deviation data of each frame of the image by the following formula:

[0045] u Bi = e i / k

[0046] where k represents the coverage factor, and u Bi represents the type B sub-uncertainty of the i-th frame;

[0047] The final type B sub-uncertainty is:

[0048]

[0049] Among them, u B Indicates the Class B sub-uncertainty;

[0050] S3, according to the uncertainty u of type A A and the Class B sub-uncertainty u B The total uncertainty u is calculated by the following formula c :

[0051]

[0052] The method of the present invention digitizes the pan / tilt tracking performance, which could only be roughly estimated, and visualizes the pan / tilt tracking performance.

[0053] The beneficial effects of the present invention are:

[0054] Traditionally, when evaluating pan-tilt tracking performance, the target detection and tracking algorithm is separated from the pan-tilt. This unilateral evaluation of the algorithm fails to consider the pan-tilt's own response speed and tracking accuracy. This invention can be implemented with simple everyday equipment, eliminating the need for a turntable or other equipment.

[0055] The present invention changes the traditional method of detecting and evaluating the pan-tilt tracking performance, which can only separate the target tracking algorithm from the pan-tilt, and unilaterally detects and evaluates the quality of the algorithm without considering the response speed and tracking accuracy of the pan-tilt itself.

[0056] The present invention integrates the target tracking algorithm and the pan-tilt head very well. On the established structure, if it is necessary to unilaterally measure the tracking performance of the algorithm or the pan-tilt head, it can also be achieved on this structure by controlling variables. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of the present invention;

[0058] Figure 2 It is the target trajectory diagram along the X direction in the first stage of the target trajectory;

[0059] Figure 3 It is the target trajectory diagram along the Y direction in the first stage of the target trajectory;

[0060] Figure 4 It is the target oblique motion trajectory diagram of the first stage of the target trajectory;

[0061] Figure 5 is the target motion trajectory in the second stage of the target trajectory;

[0062] Figure 6 It is the overall structure diagram of the tracking performance measurement system;

[0063] Figure 7 This is a schematic diagram of a three-axis gimbal.

[0064] In the figure: pitching motor rotating mechanism 1, rolling motor rotating mechanism 2, horizontal motor rotating mechanism 3. Specific implementation manner

[0065] The following illustrates the implementation manners of the present invention through specific examples. The following will further elaborate on the specific working process of the present invention in conjunction with the accompanying drawings of the specification.

[0066] As Figure 6 and Figure 7 shown, in specific implementation, the present invention constructs the following system. The system includes a pan-tilt head, a camera, and a display screen. The camera is rotatably mounted on the pan-tilt head through a multi-axis motor, and the camera is placed facing the display screen. Specifically, the camera is rotatably mounted on the pan-tilt head through the pitching motor rotating mechanism 1, the rolling motor rotating mechanism 2, and the horizontal motor rotating mechanism 3, and can rotate in three mutually orthogonal rotation directions.

[0067] As Figure 7 shown, the pan-tilt head adopts a three-axis pan-tilt head, and this three-axis adjustment structure is composed of mutually perpendicular rotating mechanisms; the first rotating mechanism 1 is formed by the first motor and the upper section of the rod, and the upper section of the rod is connected to the pan-tilt head control board to control the heading angle of the pan-tilt head; the second rotating mechanism 2 is composed of the second motor and the second motor bracket, and is used to control the roll angle of the pan-tilt head. Mainly when the unmanned aerial vehicle is flying, the pan-tilt head controls the camera to be parallel to the ground to ensure stable shooting of the camera interface; the third rotating mechanism 3 is composed of the third motor and the third motor bracket, and is used to control the pitching angle of the pan-tilt head.

[0068] The pan-tilt head can be fixed through a tripod, and the display screen can be connected to a computer to receive display information, and then be controlled to play a pre-set target movement trajectory video.

[0069] The pan-tilt head internally has a tracking algorithm. The tracking algorithm controls each of the pitching motor rotating mechanism 1, the rolling motor rotating mechanism 2, and the horizontal motor rotating mechanism 3 to drive the camera to rotate and move, and then track the target in the display screen. In this process, the method of the present invention is used for tracking to test the performance and accuracy of the overall tracking algorithm, the pan-tilt head, and each motor rotating mechanism.

[0070] As Figure 1 shown, the examples of the present invention and their implementation processes are as follows:

[0071] First, place the camera facing the display screen. The display screen plays a pre-set target movement trajectory video. The camera captures images and tracks the target in the display screen in real time, and collects all the image sequences during the tracking process, and then analyzes and processes the image sequences to obtain the measurement results of the pan-tilt head tracking performance.

[0072] For each image in the image sequence, identify the position of the target in the image, calculate the deviation data by subtracting the position of the target in the image from the center point of the image, and then form the deviation data of each image changing with time into two-dimensional complex data. In this way, the two-dimensional complex data is composed of the time dimension and the distance dimension, serving as the data source for measuring the tracking accuracy of the pan-tilt head; then perform normalization processing on the two-dimensional complex data to obtain the measurement result.

[0073] The time dimension is in frames, the total number of frames is N, the frame period is T, the distance dimension is in pixel units, and the observation range is [1, L], where L represents the horizontal width of the camera image.

[0074] In the specific implementation, set the position of the target in the image as the percentage of the coordinates of the target in the image relative to the width and height of the image. Set the target in the image as a landmark with a fixed shape.

[0075] Divide the target movement trajectory into three stage parts:

[0076] The first stage part is a one-dimensional step motion process, as Figure 2 and Figure 3 shown. Specifically, set the target to move back and forth along a horizontal / vertical straight line trajectory on the display screen.

[0077] The second stage part is a two-dimensional ramp motion process, as Figure 4 shown. Specifically, set the target to move back and forth along an inclined straight line trajectory on the display screen;

[0078] The third stage part is a two-dimensional trajectory motion process, as [[ID=2i5]] Figure 5 shown. Specifically, set the target to move in a loop along a closed-loop crossable curve trajectory on the display screen. The specific closed-loop cross curve can be an 8-shaped curve, and the target moves along the 8-shaped curve, but it is not limited to this, and it can also be other fixed trajectories. l>

[0079] For the image sequences collected in the first stage part and the second stage part, establish a discrete time model with a fixed sampling period t, and calculate the overshoot, oscillation period, residual error, oscillation times, and decay ratio of each object according to the two-dimensional complex data processing.

[0080] The best transition process of a PID is that the transition process can stabilize after oscillating twice, and there is a decay ratio of nearly 4:1 after oscillating twice. Specifically, it is best to have two oscillation times, and the shorter the oscillation period, the better; in terms of the response speed, the smaller the oscillation times, the better, indicating little fluctuation; the smaller the residual error, the better, indicating small steady-state error; the overshoot is sometimes also called the maximum deviation, which represents the degree of deviation of the controlled variable from the set value. The decay ratio is the difference between the first wave peak value and the second wave peak value divided by the first wave peak value.

[0081] The overshoot and residual error are calculated as follows:

[0082] Overshoot = [(x max – s) / s] × 100%

[0083] Residual error = x ∞ - s

[0084] Where x ∞ is the point where the center of the last image stabilizes during tracking, x max is the maximum distance by which an object exceeds s, and s represents the position coordinate of the target.

[0085] The processes of the first-stage part and the second-stage part are equivalent to an oscillation process, such as forms of divergent oscillation, equal-amplitude oscillation, damped oscillation, monotonic process, etc. The tracking performance during its step change is determined by parameters such as overshoot, residual error, and the number of oscillations.

[0086] For the image sequence collected in the third-stage part, a discrete-time model with a fixed sampling period t is established. By calculating the deviation between the target position and the center point of the image through two-dimensional complex data processing, the determination coefficient and uncertainty are then calculated, and the tracking performance of the pan-tilt is judged using the determination coefficient and uncertainty.

[0087] Among them, the target motion trajectory in the first stage is as Figures 2 - 4 shown. In the first stage, the target performs a one-way reciprocating motion. The sampling of each frame of the target point is artificially changed. After the target is tracked to the first target, the target is moved to the next target point, and the motion trajectory is as Figure 3 shown.

[0088] After tracking and extracting the data, their determination coefficient and uncertainty are calculated. The determination coefficient is obtained from the distance between the center point of the camera and the center point of the target.

[0089] Regarding the calculation of uncertainty, first obtain the type A sub-uncertainty u A as follows:

[0090] The deviation data obtained during the second-stage acquisition is [e1, e2, e3,...], and the calculation is:

[0091] e i = x - s i

[0092] Where x is the coordinate of the center point of the image, s i is the coordinate of the target position in the i-th frame, and e i is the deviation in the i-th frame;

[0093] Furthermore, calculate the experimental standard deviation S A (e) as:

[0094]

[0095] Then, according to the following formula based on the experimental standard deviation S A (e), calculate the type A sub-uncertainty u of the evaluation azimuth angle A :

[0096]

[0097] Among them, n represents the amount of data in the two-dimensional complex data, that is, the number of frames of the image;

[0098] S42. First, obtain the type B sub-uncertainty u in the following manner B :

[0099] Calculate the type B sub-uncertainty of each component according to the deviation data of each frame of the image according to the following formula:

[0100] u Bi = e i / k

[0101] Among them, k represents the coverage factor, and u Bi represents the type B sub-uncertainty of the i-th frame;

[0102] The final type B sub-uncertainty is:

[0103]

[0104] Among them, u B represents the type B sub-uncertainty;

[0105] S43. According to the type A sub-uncertainty u A and the type B sub-uncertainty u B Comprehensively calculate the total uncertainty u according to the following formula c :

[0106]

[0107] The target tracking performance of the pan-tilt obtained by using the determination coefficient and uncertainty for judgment is specifically as follows in the table:

[0108] Performance level Coefficient of determination Uncertainty (k = 2) [[ID= ≥0.6 ≥0.50 ​ ≥0.6 <0.50 ​ <0.6 ≥0.35 ​ <0.6 ≥0.35 ​ ≥0.6 <0.35 ​ <0.6 <0.35

[0109] As can be seen from the above table, when the determination coefficient is greater than or equal to 0.6 and the uncertainty is greater than or equal to 0.5, the performance of the pan-tilt is extremely poor; when the determination coefficient is greater than or equal to 0.6 and the uncertainty is less than 0.5, the state performance is poor; if the determination coefficient is less than 0.6 and the uncertainty is greater than or equal to 0.35, the performance of the pan-tilt is also poor. Under the condition of ensuring that the PID control remains unchanged, this table shows the tracking performance of the pan-tilt. If the pan-tilt remains unchanged and the tracking algorithm remains unchanged, this table shows the PID control performance. To achieve excellent performance, it is necessary to first ensure that the PID parameters are appropriate and restore the controller performance. When the determination coefficient is less than 0.6 and the uncertainty is less than 0.35, the tracking performance of the pan-tilt is excellent.

[0110] The method of the present invention quantifies the originally rough estimable tracking performance data of the pan-tilt and visualizes the tracking performance of the pan-tilt. This method can more comprehensively characterize the tracking performance of the pan-tilt and improve the accuracy and practicality of the measurement of the tracking performance of the pan-tilt.

Claims

1. A method for measuring the tracking performance of a pan-tilt based on machine vision technology, characterized in that The method includes: Placing the camera facing the display screen directly. The display screen plays a pre-set video of the target movement trajectory. The camera captures images and tracks the target in the display screen in real time, collects all the image sequences during the tracking process, and then analyzes and processes the image sequences to obtain the measurement results of the pan-tilt tracking performance. For each image in the image sequence, identify the position of the target in the image, calculate the difference between the position of the target in the image and the center point of the image to obtain deviation data, and then form two-dimensional complex data from the deviation data of each image changing with time. Then perform normalization processing on the two-dimensional complex data to obtain the measurement results. The target movement trajectory video is divided into three stage parts: The first stage part is a one-dimensional step motion process, specifically setting the target to reciprocally move along a horizontal straight line / vertical straight line trajectory in the display screen. The second stage part is a two-dimensional ramp motion process, specifically setting the target to reciprocally move along an inclined straight line trajectory in the display screen. The third stage part is a two-dimensional trajectory motion process, specifically setting the target to circularly move along a closed-loop crossable curve trajectory in the display screen. For the image sequences collected in the first stage part and the second stage part, calculate the overshoot, residual error, oscillation times, and decay ratio of each object according to the two-dimensional complex data processing. Use the overshoot, residual error, oscillation times, and decay ratio to judge the tracking performance of the pan-tilt. Specifically, establish a performance evaluation model as shown in the following formula for the above data: SUM = m1×w1 + m2×w2 + m3×w3 + m4×w4 Where, SUM is the performance evaluation parameter, m1, m2, m3, m4 are the overshoot, residual error, oscillation times, and decay ratio, and w1, w2, w3, w4 are the influence factors of each overshoot, residual error, oscillation times, and decay ratio. Make the following judgments according to the performance evaluation parameter obtained from the performance evaluation model: When SUM is greater than 0.8, the pan-tilt tracking performance is excellent. When 0.6 <= SUM < 0.8, the pan-tilt tracking performance is good. When 0.4 <= SUM < 0.6, the pan-tilt tracking performance is poor. When 0 <= SUM < 0.4, the pan-tilt tracking performance is extremely poor. For the image sequences collected in the third stage part, calculate the deviation between the target position and the center point of the image according to the two-dimensional complex data processing, and then calculate the determination coefficient and the sub-uncertainty. Use the determination coefficient and the sub-uncertainty to judge the tracking performance of the pan-tilt. The determination coefficient is obtained from the distance between the center point of the camera and the center point of the target. The measurement of the sub-uncertainty is specifically as follows: S1. First, obtain the type A sub-uncertainty u in the following manner A :[[]]END]] The deviation data collected in the second stage is [e1, e2, e3,...], and the calculation is: e i = x - s i where x is the coordinate of the center point of the image, and s i is the coordinate of the target position of the i-th frame, and e i is the deviation of the i-th frame; Furthermore, calculate the experimental standard deviation S A (e) is as follows: Then, according to the following formula, based on the experimental standard deviation S A (e) Calculate the type A sub-uncertainty u A : Where, n represents the amount of data in the two-dimensional complex data, that is, the number of image frames. S2. First, obtain the type B sub-uncertainty u in the following manner B :[[]]END]] Calculate the type B sub-uncertainty of each component according to the deviation data of each frame of image according to the following formula: u Bi = e i / k where k represents the coverage factor, and u Bi represents the class B sub-uncertainty of the i-th frame; The final type B sub-uncertainty is: where, u B represents the type B sub-uncertainty; S3. Calculate the total uncertainty \(u\) comprehensively according to the type-A sub-uncertainty \(u\) A and the type-B sub-uncertainty \(u\) B by the following formula: c :

2. The method for measuring the pan-tilt tracking performance based on machine vision technology according to claim 1, wherein: The target is set as a marker on the display screen.

3. A method for measuring the pan-tilt tracking performance based on machine vision technology according to claim 1, characterized in that: During the movement of the target along a linear trajectory, from one end of the trajectory to the other end, it is completed by uniformly accelerating in the first half of the trajectory and uniformly decelerating in the second half of the trajectory.

4. A method for measuring the pan-tilt tracking performance based on machine vision technology according to claim 1, characterized in that: The described closed-loop intersectable curve can specifically be an 8-shaped curve, and the target moves along the 8-shaped curve.

Images