Target tracking trajectory optimization method imitating human eye vision based on rotating biprisms

Abstract

The invention relates to a human-vision-imitating target tracking trajectory optimization method based on rotary biprisms, and belongs to the technical field of photoelectric target tracking. The method comprises the steps that a data set is collected and trained, and position information of a target in a view field is acquired and recorded; according to a non-paraxial ray tracing method, constructing a mathematical model of the moving speed of a biprism pointing beam and the rotating speed of the biprism; the speed and direction of the center of the imaging view field pointing to the tracked target are calculated, the optimal rotating speed of the rotating biprism is calculated through a particle swarm optimization algorithm, the center of the imaging view field moves towards the target, and the optimal rotating speed needs to be calculated again every time the center of the imaging view field moves by a certain distance; the rotating biprism is controlled to rotate at the optimal rotating speed, and the effectiveness of the method is verified through the position change track of the tracked target. The method has good performance in the pointing test of the static target and the tracking test of the moving target, and improves the pointing precision while realizing the optimization of the tracking trajectory.

Description

Technical Field

[0001] The invention belongs to the technical field of photoelectric target tracking, and in particular relates to a human eye vision target tracking trajectory optimization method based on a rotating double prism. Background Art

[0002] Target pointing and tracking is an important target observation technology. It can detect and track moving targets in the field of view in time and obtain the position of the target in the field of view in real time. It has a wide range of applications in monitoring security, vehicle automatic driving, traffic positioning systems and human-computer interaction. At present, the beam pointing mechanisms commonly used in visual tracking systems include universal bogies, fast reflectors and rotating dual prisms, but the universal bogies are large in size, heavy in mass, have poor dynamic performance and slow response time; the fast reflector has a small beam polarization angle; compared with these structures, the dual prism control system usually has a more compact structure, occupies relatively small space, and responds quickly. This makes it more suitable for some space-constrained application scenarios, such as small drones, mobile robots, etc.

[0003] When a rotating dual prism is used for target tracking, it is necessary to drive and control the rotation of the two prisms to control the direction of the light beam or the imaging visual axis so that it can aim at and track the target. Although the existing target tracking method based on a rotating dual prism makes the target finally be at the center of the imaging field of view, due to the nonlinear relationship between the direction of the output light beam of the rotating dual prism and the rotation angle position of the two prisms, it is impossible to control the center of the dual prism imaging field of view to track the target, resulting in the target being easily lost in the field of view. Summary of the invention

[0004] The purpose of the present invention is to overcome the problem that the existing pointing method based on rotating bi-prism cannot control the center of the bi-prism imaging field of view to track the target, and to provide a human eye vision target tracking trajectory optimization method based on rotating bi-prism.

[0005] To achieve the above object, the technical solution of the present invention is: a human eye vision target tracking trajectory optimization method based on a rotating dual prism, comprising:

[0006] Collect data sets and conduct training to obtain and record the position information of the target in the field of view;

[0007] According to the non-paraxial ray tracing method, a mathematical model of the moving speed of the light beam pointed by the rotating biprism and the rotating speed of the rotating biprism is constructed;

[0008] Calculate the speed and direction of the center of the imaging field of view pointing to the tracked target, and use the particle swarm optimization algorithm to solve the optimal rotation speed of the rotating dual prism so that the center of the imaging field of view moves toward the target;

[0009] The rotating dual prism is controlled to rotate at the optimal speed to record the position change of the tracked target.

[0010] In one embodiment of the present invention, the optimal rotation speed needs to be recalculated every time the center of the imaging field of view moves a certain distance.

[0011] In one embodiment of the present invention, the specific implementation method of collecting data sets and performing training, and obtaining and recording the position information of the target in the field of view is:

[0012] S11, using a collection system equipped with a visible light camera to collect visible light images, where the collected visible light images contain a variety of scene information;

[0013] S12. Mark and train the targets in the collected visible light images, and use a multi-target tracking algorithm to obtain and record the position information of the targets to be tracked in the field of view.

[0014] In one embodiment of the present invention, the specific method of constructing a mathematical model of the moving speed of the rotating double prism pointing to the light beam and the rotating speed of the rotating double prism is:

[0015] S21. Based on the non-paraxial ray tracing method, the deflection velocity of the rotating biprism pointing to the light beam is decomposed into the radial deflection velocity w r and the tangential deflection velocity w t , derive the radial deflection velocity w r and the tangential deflection velocity w t Relationship between deflection angle and azimuth angle of rotating biprism;

[0016] S22, according to the exit position of the rotating double prism pointing to the light beam, the moving speed is decomposed into v along the x-axis and y-axis respectively. x and v y , obtain the relationship between the deflection speed and the moving speed of the light beam pointed by the rotating biprism;

[0017] S23. Use the chain derivation rule to further obtain the relationship between the deflection speed of the pointing light beam and the rotational speed of the rotating biprism, and substitute it back into the relationship between the deflection speed of the pointing light beam and the moving speed of the rotating biprism obtained in step S22 to obtain the relationship between the moving speed of the pointing light beam of the rotating biprism and the rotational speed of the rotating biprism.

[0018] In one embodiment of the present invention, the specific method of using the particle swarm optimization algorithm to calculate the optimal rotation speed of the rotating dual prism is:

[0019] S31, calculating the moving speed of the light beam pointed by the rotating bi-prism according to the position of the center of the imaging field of view and the position information of the tracked target, and determining each known quantity in the relationship between the moving speed of the light beam and the rotating speed of the rotating bi-prism;

[0020] S32, constructing a fitness function of a particle swarm optimization algorithm, setting initial parameters of the optimization algorithm, and solving an optimal speed that meets the conditions;

[0021] S33. The moving speed of the light beam pointed by the rotating biprism is related to the current angle of the rotating biprism. Therefore, the optimal rotation speed needs to be recalculated every time the rotating biprism rotates 0.5°.

[0022] In one embodiment of the present invention, in step S21, the radial deflection speed w r and the tangential deflection velocity w t The relationship between the deflection angle and azimuth angle of the rotating biprism is derived as follows:

[0023]

[0024] Wherein, θ1 and θ2 are the angles of prism 1 and prism 2 in the rotating biprism, Φ and Θ are the deflection angle and azimuth angle of the rotating biprism, and (K, L, M) are the direction cosines of the beam pointed by the rotating biprism;

[0025]

[0026] Where n1 and n2 are the refractive indices of prism 1 and prism 2 respectively, and the deflection angle and azimuth angle of the rotating biprism are expressed as:

[0027] Φ=arccos(-M)

[0028]

[0029] Therefore, the radial deflection velocity w of the beam pointed by the rotating biprism is r and the tangential deflection velocity w t for:

[0030]

[0031] In one embodiment of the present invention, in step S22, the moving speed of the rotating biprism pointing to the light beam is decomposed into v along the x-axis and the y-axis respectively. x and v y , the expressions of the deflection speed and movement speed of the light beam pointed by the rotating biprism are:

[0032]

[0033] f1 and f2 are v x and v y The functional expression of includes radial deflection velocity and tangential deflection velocity.

[0034] In one embodiment of the present invention, step S23 is specifically implemented as follows:

[0035] The relationship between the deflection speed of the light beam pointed by the rotating biprism and the rotation speed of the rotating biprism is derived as follows:

[0036]

[0037] Where ω1 and ω2 are the rotation speeds of prism 1 and prism 2 respectively. Substituting the expression of the deflection speed of the light beam back, the expression of the moving speed of the rotating biprism pointing to the light beam and the rotation speed of the biprism is obtained as follows:

[0038]

[0039] Where g1 and g2 are function expressions including the rotation speed of the prism respectively.

[0040] In one embodiment of the present invention, the fitness function of the optimization algorithm is expressed as:

[0041]

[0042] Among them, (v cx ,v cy ) is the theoretical moving speed of the center of the camera field of view along the X and Y axes, v x and v y are the moving speeds of the rotating biprism pointing light beam along the x-axis and y-axis respectively.

[0043] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention improves the pointing accuracy of static targets, and has obvious improvements in indicators such as maximum pointing error, average pointing error, and root mean square error;

[0046] 2. The present invention performs well in the tracking test of moving targets and can ensure that the target always remains within a distance of 25 pixels from the center of the dual prism field of view.

[0047] 3. The present invention quantitatively analyzes the relationship between the tracking speed and the rotation speed of the dual prism, and proposes an effective solution to the nonlinear control problem existing in the target tracking application of the rotating dual prism imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is an overall flow chart of the human eye vision target tracking trajectory optimization method based on rotating dual prisms according to an embodiment of the present invention;

[0049] Figure 2A structural diagram of a rotating dual prism system according to an embodiment of the present invention;

[0050] Figure 3 This is a static target tracking test result diagram of an embodiment of the present invention;

[0051] Figure 4 This is a diagram of the dynamic target tracking test results of an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0053] The present invention provides a human eye vision target tracking trajectory optimization method based on a rotating dual prism, comprising:

[0054] Collect data sets and conduct training to obtain and record the position information of the target in the field of view;

[0055] According to the non-paraxial ray tracing method, a mathematical model of the moving speed of the rotating biprism pointing to the light beam and the rotating speed of the rotating biprism is constructed; the optimal speed needs to be recalculated every time the center of the imaging field of view moves a certain distance;

[0056] Calculate the speed and direction of the center of the imaging field of view pointing to the tracked target, and use the particle swarm optimization algorithm to solve the optimal rotation speed of the rotating dual prism so that the center of the imaging field of view moves toward the target;

[0057] The rotating dual prism is controlled to rotate at the optimal speed to record the position change of the tracked target.

[0058] The following is the specific implementation process of the present invention.

[0059] Example 1

[0060] like Figure 1 As shown, the present invention provides a human eye vision target tracking trajectory optimization method based on a rotating dual prism, the method comprising:

[0061] S1, collect data sets and conduct training, obtain and record the position information of the target in the field of view;

[0062] Specifically, the data set used in the present invention is a data set collected by a synchronous acquisition system using a visible light camera, including visible light images of ship targets under various environmental backgrounds. The data set is trained for 200 rounds using yolov5 to obtain a trained small model, which is used in combination with the multi-target tracking algorithm DeepSort to recognize the images collected by the imaging system and obtain the image position information of the target to be tracked.

[0063] S2, constructing a mathematical model of the moving speed of the light beam pointed by the dual prism and the rotation speed of the dual prism;

[0064] Specifically, the specific process of constructing the mathematical model of the present invention is:

[0065] S21. Based on the non-paraxial ray tracing method, the deflection velocity of the pointing beam is decomposed into the radial deflection velocity w r and the tangential deflection velocity w t , derive the relationship between the two and the deflection angle and azimuth angle of the prism;

[0066] Specifically, the present invention derives the relationship between the two and the deflection angle and azimuth angle of the prism as follows:

[0067]

[0068] Among them, Figure 2 The θ1 and θ2 shown are the angles of prism 1 and prism 2 respectively, Φ and Θ are the deflection angle and azimuth angle of the prism respectively, and (K, L, M) are the direction cosines of the pointing light beam.

[0069]

[0070] The deflection angle and azimuth angle of the prism are expressed as:

[0071] Φ=arccos(-M)

[0072]

[0073] Therefore, the radial deflection velocity w of the pointing beam is r and the tangential deflection velocity w t for:

[0074]

[0075] S22, according to the emission position of the pointing beam, its moving speed is decomposed into v along the x-axis and y-axis respectively. x and v y , obtain the relationship between the deflection speed and the moving speed of the pointing light beam;

[0076] Specifically, the relationship between the deflection speed and the moving speed of the pointing light beam is obtained as follows:

[0077]

[0078] S23. Using the chain derivation rule, further obtain the relationship between the deflection speed of the pointing light beam and the rotation speed of the dual prism, and substitute it back into the relationship of S22 to obtain the relationship between the moving speed of the pointing light beam and the rotation speed of the dual prism.

[0079] Specifically, the derivation process is as follows:

[0080]

[0081] Substituting the expression of the deflection speed of the light beam back, we can obtain the relationship between the moving speed of the pointing light beam and the rotation speed of the biprism:

[0082]

[0083] S3: Calculate the speed and direction of the imaging field center pointing to the tracked target, and use the particle swarm optimization algorithm to solve the optimal rotation speed of the rotating dual prism so that the imaging field center moves toward the target. The optimal rotation speed needs to be recalculated every time the imaging field center moves a certain distance.

[0084] Specifically, the process of obtaining the optimal rotation speed using the particle swarm optimization algorithm in the present invention is as follows:

[0085] S31, calculating the moving speed of the pointing light beam according to the position of the center of the imaging field of view and the position information of the tracked target, and determining the known quantity in the relationship between the moving speed of the light beam and the rotation speed of the dual prism;

[0086] S32, constructing a fitness function of a particle swarm optimization algorithm, setting initial parameters of the optimization algorithm, and solving an optimal speed that meets the conditions;

[0087] Specifically, the fitness function of the optimization algorithm used in the present invention is expressed as:

[0088]

[0089] Where (v cx ,v cy ) is the theoretical moving speed of the center of the camera's field of view along the X and Y axes, the sum of which should point to the target.

[0090] Specifically, the initial parameters of the optimization algorithm used in the present invention are:

[0091] The particle swarm size is 50, the inertia weight decreases linearly from 0.8 to 0.5, the individual learning factor is 2, the group learning factor is 2, the particle dimension is 2, the maximum number of iterations is 200, the particle search range is [-20, 20], and the particle search speed is [-1, 1].

[0092] S33. The moving speed of the pointing light beam is related to the current angle of the dual prism, so the optimal rotation speed needs to be recalculated every time the dual prism rotates by 0.5°.

[0093] In order to evaluate the pointing error of this method in actual equipment application, the distance between the target point and the center of the field of view after the prism is rotated into place is defined as d e ,Right now

[0094]

[0095] Where: (x, y) is the position of the target point in the camera's field of view; (x c ,y c ) is the coordinate of the center of the field of view. Further, the trajectory optimization method of the present invention and the two-step method are used to perform a pointing test on a static target, and 60 groups of test results are randomly calculated to calculate the deviation distance d e The experimental comparison results are shown in Table 1.

[0096] Table 1

[0097] <![CDATA[p max ↓]]> <![CDATA[p mean ↓]]> <![CDATA[p rms ↓]]> Trajectory Optimization Methods 13.45 7.01 7.67 Two-step method 17.72 9.06 9.96

[0098] Table p max There are 60 groups of deviation distances d e The maximum value reflects the lowest pointing accuracy of the two methods. In this indicator, this method is 24.1% lower than the two-step method; p mean There are 60 groups of deviation distances d e The average pointing accuracy of the two methods reflects that the average pointing accuracy of this method is 22.6% lower than that of the two-step method; p rms There are 60 groups of deviation distances d e The root mean square error is used to evaluate d e The fluctuation of the method is reduced by 23.0% compared with the two-step method.

[0099] S4. Control the rotating dual prism to rotate at an optimal speed and record the position change of the tracked target.

[0100] Specifically, as the center of the imaging field of view gradually approaches the target, the position of the tracked target in the imaging field of view changes, and its coordinates are recorded and an image is drawn to analyze the difference between it and the predetermined trajectory. The test results of trajectory optimization for static targets in different quadrants are shown in the figure below: Figure 3 As shown in the figure, the first column indicates that the target is fixed at different quadrants in the camera field of view at the beginning of the experiment; the second column reflects the longitudinal error d of the randomly selected trajectory point y The third and fourth columns respectively show the angle changes of prism 1 and prism 2 during the rotating dual prism tracking process. The trajectory optimization results of the moving target are shown in Figure 4 As shown, Figure 4 (a) is the initial position of the target during the test, and the target moving area is not at the control singularity position; Figure 4 (b) The Euclidean distance between the target position and the center of the dual-prism camera's field of view changes over time. In the first 10 seconds of the experiment, the target and the camera's field of view moved simultaneously, and the distance between the center of the camera's field of view and the target position continued to decrease. Within 50 seconds after the experiment, the target position remained within 25 pixels of the center of the camera's field of view. Figure 4 (c) with Figure 4(d) respectively shows the change of the angle of prism 1 and prism 2 over time during the tracking process. As can be seen from the figure, this method can enable the human eye vision target tracking system based on rotating dual prisms to achieve directional tracking with higher pointing accuracy.

[0101] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0102] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A human-eye-like vision target tracking trajectory optimization method based on a rotating dual prism, characterized in that: include: Collect data sets and conduct training to obtain and record the position information of the target in the field of view; According to the non-paraxial ray tracing method, a mathematical model of the moving speed of the light beam pointed by the rotating biprism and the rotating speed of the rotating biprism is constructed; Calculate the speed and direction of the center of the imaging field of view pointing to the tracked target, and use the particle swarm optimization algorithm to solve the optimal rotation speed of the rotating dual prism so that the center of the imaging field of view moves toward the target; The rotating dual prism is controlled to rotate at the optimal speed to record the position change of the tracked target.

2. The method for optimizing the target tracking trajectory based on rotating bi-prisms according to claim 1, characterized in that: The optimal rotation speed needs to be recalculated every time the center of the imaging field of view moves a certain distance.

3. The method for optimizing the target tracking trajectory based on rotating bi-prisms according to claim 1, characterized in that: The specific implementation method of collecting data sets and training, obtaining and recording the position information of the target in the field of view is: S11, using a collection system equipped with a visible light camera to collect visible light images, where the collected visible light images contain a variety of scene information; S12. Mark and train the targets in the collected visible light images, and use a multi-target tracking algorithm to obtain and record the position information of the targets to be tracked in the field of view.

4. The method for optimizing the target tracking trajectory based on rotating dual prisms according to claim 1, characterized in that: The specific method of constructing the mathematical model of the moving speed of the rotating biprism pointing to the light beam and the rotating speed of the rotating biprism is: S21. Based on the non-paraxial ray tracing method, the deflection velocity of the rotating biprism pointing to the light beam is decomposed into the radial deflection velocity w r and the tangential deflection velocity w t , derive the radial deflection velocity w r and the tangential deflection velocity w t Relationship between deflection angle and azimuth angle of rotating biprism; S22, according to the exit position of the rotating double prism pointing to the light beam, the moving speed is decomposed into v along the x-axis and y-axis respectively. x and v y , obtain the relationship between the deflection speed and the moving speed of the light beam pointed by the rotating biprism; S23. Use the chain derivation rule to further obtain the relationship between the deflection speed of the pointing light beam and the rotational speed of the rotating biprism, and substitute it back into the relationship between the deflection speed of the pointing light beam and the moving speed of the rotating biprism obtained in step S22 to obtain the relationship between the moving speed of the pointing light beam of the rotating biprism and the rotational speed of the rotating biprism.

5. The method for optimizing the target tracking trajectory based on rotating dual prisms according to claim 1, characterized in that: The specific method of using the particle swarm optimization algorithm to solve the optimal rotation speed of the rotating biprism is: S31, calculating the moving speed of the light beam pointed by the rotating bi-prism according to the position of the center of the imaging field of view and the position information of the tracked target, and determining each known quantity in the relationship between the moving speed of the light beam and the rotating speed of the rotating bi-prism; S32, constructing a fitness function of a particle swarm optimization algorithm, setting initial parameters of the optimization algorithm, and solving an optimal speed that meets the conditions; S33. The moving speed of the light beam pointed by the rotating biprism is related to the current angle of the rotating biprism. Therefore, the optimal rotation speed needs to be recalculated every time the rotating biprism rotates 0.5°.

6. The method for optimizing the target tracking trajectory based on rotating bi-prisms according to claim 4, characterized in that: In step S21, the radial deflection speed w r and the tangential deflection velocity w t The relationship between the deflection angle and azimuth angle of the rotating biprism is derived as follows: Wherein, θ1 and θ2 are the angles of prism 1 and prism 2 in the rotating biprism, Φ and Θ are the deflection angle and azimuth angle of the rotating biprism, and (K, L, M) are the direction cosines of the beam pointed by the rotating biprism; Where n1 and n2 are the refractive indices of prism 1 and prism 2 respectively, and the deflection angle and azimuth angle of the rotating biprism are expressed as: Φ=arccos(-M) Therefore, the radial deflection velocity w of the beam pointed by the rotating biprism is r and the tangential deflection velocity w t for:

7. The method for optimizing the target tracking trajectory based on rotating bi-prisms according to claim 6, characterized in that: In step S22, the moving speed of the rotating biprism pointing to the light beam is decomposed into v and x and v y , the expressions of the deflection speed and movement speed of the light beam pointed by the rotating biprism are: f1 and f2 are v x and v y The functional expression of includes radial deflection velocity and tangential deflection velocity.

8. The method for optimizing the target tracking trajectory based on rotating dual prisms according to claim 7, characterized in that: Step S23 is specifically implemented as follows: The relationship between the deflection speed of the light beam pointed by the rotating biprism and the rotation speed of the rotating biprism is derived as follows: Where ω1 and ω2 are the rotation speeds of prism 1 and prism 2 respectively; Substituting the expression of the deflection speed of the light beam back, we can obtain the expression of the moving speed of the rotating biprism pointing to the light beam and the rotation speed of the biprism: Where g1 and g2 are function expressions including the rotation speed of the prism respectively.

9. The method for optimizing the target tracking trajectory based on rotating bi-prisms according to claim 5, characterized in that: The fitness function of the optimization algorithm is expressed as: Among them, (v cx ,v cy ) is the component of the theoretical moving speed of the center of the camera field of view along the X and Y axes, vx and vy are the moving speeds of the light beam pointed by the rotating biprism along the x and y axes respectively.

10. A computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 9 can be implemented.

Images