Rapid capturing and stable tracking method and system for large-angle click target of photoelectric pod

Abstract

The invention provides a photoelectric pod large-angle click target rapid capturing and stable tracking method and system, and the method comprises the steps: obtaining a ground station click target instruction and an automatic tracking instruction, and converting the position of a click target into a specific image coordinate value; searching a set pixel region by taking the position of the click target as a center to carry out target feature extraction; generating a tracking template according to the extracted target features, and storing and updating the template; calculating miss distance of the target relative to the center of the field of view, and sending the miss distance and the automatic tracking instruction to a servo system; a timing and amplitude limiting control strategy responded by the servo system is configured according to the miss distance, the angle deviation control quantity is solved, and automatic tracking of pulling the target to the center of the field of view is carried out according to the angle deviation control quantity. According to the invention, the method combines the manual target clicking, tracking algorithm and servo control method in the loop, avoids the situation that the target cannot be detected due to excessive overshoot and the capture fails, and improves the target capture speed and tracking precision of the photoelectric pod.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, system, device and medium for rapid capture and stable tracking of a target selected at a large angle by an optoelectronic pod. Background Art

[0002] As one of the important payloads of UAVs, optoelectronic pods have been widely used in military reconnaissance, border patrol, disaster monitoring, environmental investigation and other fields. One of the basic functions of optoelectronic pods mounted on UAVs is to conduct reconnaissance and tracking of targets. Optoelectronic pods are usually equipped with high-resolution imaging sensors, which can collect video images of the target area in real time and transmit the image data to the ground control station through a communication link. When the operator at the ground station identifies a specific target of interest, it is hoped that the optoelectronic pod will keep tracking the target and pull it to the center of the field of view after the target is locked, so as to observe all or part of the details of the target with a certain field of view angle, which requires rapid capture of the target and stable tracking.

[0003] When performing the above tasks, the following limitations exist:

[0004] (1) The operator needs to repeatedly control the direction and pitch angle of the optoelectronic pod to track the target, which is not only time-consuming but also requires high professional skills of the operator, especially when dealing with fast-moving targets.

[0005] (2) For targets that move at high speed or change direction suddenly, due to the limitation of the servo system response time, it is easy for the target to move out of the field of view in a short period of time, thereby increasing the difficulty of tracking or even completely losing the ability to monitor the target.

[0006] (3) In order to achieve rapid capture of targets within a wide angle range, it is often necessary to increase the proportional gain of the servo system to speed up the response. However, this may lead to excessive overshoot, causing the target to deviate too much from the center of the field of view and become difficult to reposition. At the same time, if attempts are made to enhance dynamic characteristics by improving the servo bandwidth, the power consumption, weight and cost of the entire system will increase, which conflicts with the demand for lightweight design of aircraft, especially small drones.

[0007] (4) The PID control commonly used in servo systems is itself a post-correction processing mechanism. In addition, due to factors such as inertia, friction, and time delay in the servo system, when a large error occurs, the servo system starts moving quickly, which can easily cause the target to move out of the processing area of ​​the target detection algorithm of the image processing unit, resulting in target capture failure. Summary of the invention

[0008] The present invention provides a method, system, device and medium for rapid capture and stable tracking of a target by a photoelectric pod at a large angle, which solves the problem of how to improve the capture speed and tracking accuracy of the photoelectric pod for the target.

[0009] In order to achieve the above objectives, this application adopts the following technical solutions:

[0010] In a first aspect, a method for rapid capture and stable tracking of a target by a photoelectric pod at a large angle is provided, comprising:

[0011] Obtaining a target selection instruction and an automatic tracking instruction at a ground station, converting the position of the selected target into a specific image coordinate value; searching a set pixel area with the position of the selected target as the center to extract target features; generating a tracking template based on the extracted target features and storing and updating the template;

[0012] Calculating the miss distance of the target relative to the center of the field of view and sending it to the servo system together with the automatic tracking instruction;

[0013] The timing and amplitude limiting control strategies of the servo system response are configured according to the miss amount, and the angle deviation control amount is solved, and the target is automatically tracked to the center of the field of view according to the angle deviation control amount.

[0014] In a first possible implementation manner of the first aspect, configuring the timing and limiting control strategy of the servo system response according to the miss amount, solving the angle deviation control amount, and automatically tracking the target to the center of the field of view according to the angle deviation control amount, includes:

[0015] When the miss distance is greater than the set value, the limit control is adopted within the set time, and the miss distance is gradually increased in proportion until the set time is reached and the control is performed according to the actual miss distance;

[0016] According to the miss amount, the PID parameters are adjusted; when the miss amount is greater than the set number of pixels, PD control is adopted; when the miss amount is less than or equal to the set number of pixels, PID control is adopted;

[0017] After the target is pulled to the center of the field of view, the PID parameters are adjusted to maintain control accuracy and stable tracking.

[0018] Based on the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the method for rapid capture and stable tracking of a target by a large-angle point selection of an optoelectronic pod further includes:

[0019] If the target is blocked during the process of being pulled to the center of the field of view, the moving state of the target before being blocked is maintained for a set time, and if the target reappears in the field of view, tracking is continued; otherwise, the timing and limiting control strategies for configuring the servo system response according to the miss amount are repeated, and the angle deviation control amount is solved, and the automatic tracking step of pulling the target to the center of the field of view is performed according to the angle deviation control amount.

[0020] Based on any possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, if the target is blocked or target tracking fails, it is allowed to re-acquire the target selection instruction.

[0021] Based on any possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, if the target is located within a specific rectangular area, a response is directly performed according to an actual miss distance.

[0022] Based on any possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the specific image is configured as a visible light television or an infrared thermal imager with 1920×1080 pixels;

[0023] The origin coordinates of each frame image are the (0,0) pixel position in the upper left corner of the image. The first digit of the coordinate value represents the pixel value in the azimuth direction, and the second digit of the coordinate value represents the pixel value in the elevation direction.

[0024] The specific rectangular area is configured as: a rectangular area formed by four points of pixel coordinates A(479,269), B(1439,269), C(479,809), and D(1439,809);

[0025] When the field of view angle is greater than 12 degrees, limit control is adopted within the set time;

[0026] When the miss amount is greater than 480 pixels, PD control is used; when the miss amount is less than 480 pixels, PID control is used.

[0027] In the second aspect, a photoelectric pod large-angle point-selection target rapid capture and stable tracking system is provided, comprising:

[0028] An image processing module is used to obtain a target selection instruction and an automatic tracking instruction from a ground station, convert the position of the selected target into a specific image coordinate value; search a set pixel area with the position of the selected target as the center to extract target features; generate a tracking template based on the extracted target features and store and update the template;

[0029] A miss distance calculation module is used to calculate the miss distance of the target relative to the center of the field of view, and send it to the servo system together with the automatic tracking instruction;

[0030] The servo control module is used to configure the timing and limiting control strategy of the servo system response according to the miss amount, and to solve the angle deviation control amount, and to automatically track the target to the center of the field of view according to the angle deviation control amount.

[0031] In a first possible implementation manner of the second aspect, the servo control module is specifically configured to:

[0032] When the miss distance is greater than the set value, the limit control is adopted within the set time, and the miss distance is gradually increased in proportion until the set time is reached and the control is performed according to the actual miss distance;

[0033] According to the miss amount, the PID parameters are adjusted; when the miss amount is greater than the set number of pixels, PD control is adopted; when the miss amount is less than or equal to the set number of pixels, PID control is adopted;

[0034] After the target is pulled to the center of the field of view, the PID parameters are adjusted to maintain control accuracy and stable tracking.

[0035] According to a third aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for rapid capture and stable tracking of targets at large angles by an optoelectronic pod are implemented as described in the first aspect.

[0036] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method for rapid capture and stable tracking of targets at large angles by an optoelectronic pod are implemented as described in the first aspect.

[0037] The present application provides a low-cost solution that does not require changes to the servo system or image tracking algorithm hardware, and does not increase the technical difficulty. It only requires the use of timing and limiting control strategies and methods for large miss amounts, so that the capture and tracking of the target of interest can be quickly achieved and pulled to the center of the field of view, which can effectively solve practical needs.

[0038] This application combines human-in-the-loop artificial target selection, tracking algorithms, and servo control methods. The control strategy arranges a transition process for the servo system in the process of gradually reducing the error of the servo system, avoiding excessive overshoot caused by rapid and large-angle movement of the servo system, resulting in the image processing unit failing to detect the target and capturing failure; and the length of the timing segmented limit control time can be determined according to the specific performance of different servo systems, and the size of the miss amount, that is, the specific value of the corresponding large angle, can also be appropriately adjusted according to the actual situation. The method is highly applicable and easy to promote. At the same time, this application also has memory tracking and PID parameter adaptive adjustment functions, giving full play to their respective advantages, learning from each other's strengths and weaknesses, not only has the ability to resist target occlusion, but also ensures that the target of interest will be captured and pulled to the center of the field of view as soon as possible, laying the foundation for further analysis of the target or guiding other systems.

[0039] The application is simple and efficient to operate, especially for moving targets. The operator does not need to estimate the target's direction of movement, speed of movement, or switch to the appropriate field of view. At the same time, the operator does not need to manually move the target to the center of the field of view through the servo system and then lock and track it, which reduces the operator's reaction ability, control proficiency and other professional skill requirements. In addition, the application does not need to use large drive power to improve the servo system's acceleration and deceleration capabilities and improve the system's dynamic response characteristics. It can meet actual needs only through appropriate control strategies and adaptive parameter adjustment based on the miss amount, reduce the production cost of the optoelectronic pod, and improve the optoelectronic pod's adaptability to the carrier aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a method for rapid capture and stable tracking of a target at a large angle by an optoelectronic pod provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of coordinate values ​​of a frame of image provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of a timed amplitude limiting input of a miss amount provided in an embodiment of the present application;

[0043] Figure 4 A structural block diagram of a photoelectric pod large-angle point-selection target rapid capture and stable tracking system provided in an embodiment of the present application;

[0044] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the technical solutions in the embodiments of the present application are clearly described. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present application.

[0046] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0047] The description of the method flow in the specification of the present application and the steps of the flowchart in the drawings of the present specification do not have to be strictly executed according to the step numbers, and the method steps can be executed in a different order. Moreover, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0048] In the following, in conjunction with the accompanying drawings and preferred embodiments, the method, device and medium for rapid capture and stable tracking of large-angle point-selection targets for an optoelectronic pod provided in the embodiments of the present application are described in detail as follows.

[0049] First, the application scenarios of the method for rapid capture and stable tracking of targets at large angles by an optoelectronic pod in an embodiment of the present application are described in detail.

[0050] There is relative motion between the optoelectronic pod and the ground target. Further observation can only be carried out after the optoelectronic pod is controlled to capture and track the target. Otherwise, the target will soon move out of the field of view, especially for moving targets. At the same time, there is also a delay in the transmission of the pixel coordinates of the relevant position to the optoelectronic pod image processing unit after clicking the target. For targets near the center of the field of view, as long as the tracking algorithm is appropriate, it is easy to capture the target. For targets far away from the center of the field of view in the video, fast target capture places high demands on the servo bandwidth, rapidity, servo stability and target tracking algorithm. Otherwise, it is not easy to capture the target, let alone effectively track and pull the target to the center of the field of view. Even if the servo system is repeatedly controlled to move, the target may move out of the field of view and the opportunity to further observe and analyze the target is lost.

[0051] For the problem of large-angle target capture: When capturing a target at a large angle, it is a large step input response process for the servo system. If the control output is too large, it may cause the servo system to overshoot and cause the target to be lost. If the servo system bandwidth is improved, the system's drive system power, weight and cost will increase. Specifically:

[0052] Under the action of input signal, the time response of servo system consists of two parts: dynamic process and steady-state process. The dynamic response process of an actual servo system must be attenuated, that is, the system must be stable, but the servo system has factors such as inertia, friction, and time delay, and the system output cannot completely reproduce the change of input. In the dynamic response process, there are generally overshoot and dynamic adjustment processes; the steady-state process is also called steady-state response, which characterizes the degree to which the system output finally reproduces the input.

[0053] It is generally believed that step input is the most severe working condition for the servo system. If the dynamic performance of the servo system under step input meets the requirements, then the dynamic performance of the servo system under other forms of input signals will also be good. For stable performance, if the output of the system is not equal to the input or a definite function of the input when time tends to infinity, the system has a steady-state error. For the servo tracking system, if the position and speed of the external tracking target cannot be directly obtained or predicted, that is, the control input signal is random, and the servo system is a continuous dynamic output with a certain inertia, and when tracking the external target, the miss amount is a typical discontinuous, multiple continuous step signals with a jump nature, and the servo tracking system is prone to overshoot and tracking errors.

[0054] Regarding the limitations of PID control:

[0055] PID control is a linear servo control method, which forms a control deviation based on the given value r(t) and the actual output value c(t):

[0056] e(t)=r(t)-c(t)

[0057] The proportion (P), integral (I) and differential (D) of the deviation are linearly combined to form the control quantity to control the controlled object. The digital PID controller is used in computers or embedded control systems, and its discrete PID control law expression is:

[0058]

[0059] Where:

[0060] k is the sampling number, k = 0, 1, 2, ...; u(k) is the PID output value at the kth sampling time; e(k) is the deviation value input at the kth sampling time; e(k-1) is the deviation value input at the k-1th sampling time; K P is the integral gain; K I is the integral gain; K D is the differential gain.

[0061] The gain functions of each link of PID are as follows:

[0062] 1) Proportional gain, which proportionally reflects the deviation signal of the control system. Once the deviation occurs, the controller immediately takes control action to reduce the deviation;

[0063] 2) Integral gain is mainly used to eliminate static error and improve system tracking accuracy. There is a phase delay in the integral link, and the integral action can cause the error to accumulate, resulting in excessive control. Excessive integral gain can easily cause system oscillation;

[0064] 3) Differential gain can reflect the changing trend of the deviation signal and introduce a correction signal before the deviation signal value becomes too large, thereby speeding up the system response speed and reducing the adjustment time; at the same time, the differential action will increase the system damping, which can help reduce overshoot, but the differential action can easily introduce noise signals into the system, which will have an adverse effect on system stability.

[0065] PID control eliminates errors through error feedback, and it itself has a correction delay problem. At the same time, when the PID gains remain unchanged, if there is a large mutation in the tracking error, the control amount will increase, which will cause the servo system to overshoot. In other words, there is a contradiction between the rapidity and overshoot of the servo system.

[0066] Therefore, when the target is in a large-angle area, directly pulling the target to the center of the field of view may cause the servo system to overshoot due to excessively fast movement, causing the target to leave the search area, making it impossible for the image processing unit to detect the target, and ultimately causing the target to be lost. In response to the above technical problems, the present application uses a large-angle target capture and tracking servo control strategy and method to arrange a step transition process for the servo system. Without changing the servo system and the image processing unit hardware and tracking algorithm, only a low-cost servo control strategy and method can be used to achieve stable capture and tracking of the target, and the target can be stably controlled in the center of the field of view, so as to effectively meet practical applications.

[0067] See also Figure 1 The embodiment of the present application provides a method for fast capture and stable tracking of a target by an optoelectronic pod at a large angle, such as Figure 1 As shown, the method of the embodiment of the present application includes:

[0068] Step S1, obtain the ground site target selection instruction and automatic tracking instruction, convert the position of the selected target into a specific image coordinate value; search the set pixel area with the position of the selected target as the center to extract the target feature; generate a tracking template based on the extracted target feature and store and update the template.

[0069] For example: an optoelectronic pod based on azimuth and pitch two-axis gyro stabilization control, equipped with a visible light TV with a resolution of 1920×1080 pixels and a frame rate of 50 Hz is described as an example.

[0070] Visible light television is used to detect, capture and track targets. The origin coordinates of each frame of the image are the (0,0) pixel position in the upper left corner of the image. The first digit of the coordinate value represents the pixel value in the azimuth direction, and the second digit of the coordinate value represents the pixel value in the elevation direction.

[0071] See also Figure 2 The wide angle area referred to in this application is defined as the area outside the rectangle formed by the four points of pixel coordinates A(479,269), B(1439,269), C(479,809), and D(1439,809), where F is the center of the field of view. Figure 2 The oblique line area in the figure, that is, there is a target of interest in the oblique line area that needs to be locked and pulled to the center of the field of view for tracking, which is the large-angle point selection target area referred to in this application.

[0072] In the specific implementation process, when selecting a target at the ground station, the ground station control software will convert the selected position into Figure 2 The pixel coordinate value centered on point F is sent to the image processing unit of the optoelectronic pod together with the automatic tracking instruction and other information. The image processing unit extracts target features in the target detection algorithm search area centered on the point selected position. The search area size can be set to 128×128 pixels. After the target features are extracted, the tracking template is obtained and the tracking template is stored and updated.

[0073] Step S2, calculate the miss distance of the target relative to the center of the field of view, and send it to the servo system together with the automatic tracking instruction.

[0074] The miss distance of the target relative to the center of the field of view is the pixel coordinate value in the azimuth and pitch directions from the center of the field of view. The miss distance of the captured target and the automatic tracking instruction are sent to the optoelectronic pod servo system.

[0075] Step S3, configuring the timing and amplitude limiting control strategy of the servo system response according to the miss amount, and solving the angle deviation control amount, and automatically tracking the target to the center of the field of view according to the angle deviation control amount.

[0076] After receiving the miss distance and automatic tracking instructions, the servo system calculates the angle deviation control amount of the optoelectronic pod to pull the target to the center of the field of view according to the current field of view angle, resolution, azimuth / pitch angle of the optoelectronic pod and other information, and performs automatic tracking. At this time, for the servo system, eliminating the tracking error is a large step response process. If a large proportional control is used to ensure that the system has a higher dynamic response, the following situations will occur: First, the servo system moves too fast, and overshoot is inevitable. The target in the next frame or subsequent frames of images is easy to move out of the search area, and then the target cannot be detected, resulting in target loss; second, if the size of the target detection search area is increased, the image processing unit will have too much calculation or even impossible to achieve; at the same time, if the target detection search time is too long, the target may also move out of the field of view and lose the target; third, if the servo moves too fast in the case of large error, it will cause image blur, and the target detection algorithm cannot detect the target; fourth, due to the bandwidth limitation of the servo system, the instantaneous large step input can cause the control amount to be too large and cannot be achieved. If the dynamic response performance of the servo system is improved, the servo system drive capability and bandwidth have higher requirements. Therefore, the timing and limiting control strategies of the servo system response are configured according to the miss distance, and the servo system response process under large miss distance is gradually completed; specifically, the following are included:

[0077] Step S301, when the miss distance is greater than the set value, the limit control is adopted within the set time, and the miss distance is gradually increased in proportion until the set time is reached and the control is performed according to the actual miss distance.

[0078] Based on the above examples, the horizontal field of view angle range of the visible light television in this application is 4.3 to 63 degrees.

[0079] When the field of view angle is greater than 12 degrees, 3-second limiting control is achieved; that is, when the field of view angle of the imaging sensor is greater than 12 degrees, after selecting the target, the miss distance is proportionally limited within 3 seconds.

[0080] When the field of view angle is less than or equal to 12 degrees, the servo motion deviation angle is relatively small, and the time to complete the capture and tracking process is relatively short. Therefore, the timing limit control strategy is not adopted.

[0081] It is worth noting that the above-mentioned field of view angle, time and ratio are set according to the system of this application, and different systems can be determined according to specific circumstances. By judging the size of the miss amount, a transition process is arranged for the servo system, so that the servo system can smoothly complete the step response process under large deviations in sequence, ensuring that the servo system converges and responds smoothly, avoiding a large control amount causing the target to jump out of the detection search area of ​​the image processing, so that the target features can still be continuously detected in the next image frame, and then the miss amount is continuously sent to the servo system.

[0082] In some possible implementations, the above timing limit control strategy can be found in Figure 3 , Figure 3 A schematic diagram of the proportional limiting input of the miss amount is given. If the miss amount is E, the 1st, 2nd and 3rd seconds are respectively subject to step-by-step limiting control at 50%, 70% and 90% of the miss amount. After 3 seconds, the servo system responds and controls according to the actual miss amount without limiting the amount.

[0083] Step S302, adjusting PID parameters according to the miss amount; adopting PD control when the miss amount is greater than a set number of pixels; adopting PID control when the miss amount is less than or equal to a set number of pixels.

[0084] On the basis of the above-mentioned timing limiting strategy, in order to smoothly pull the target to the center of the field of view, a position tracking loop PID parameter adaptive control strategy based on the miss distance is adopted. The specific adaptive control strategy is as follows:

[0085] When the miss distance is greater than 480 pixels, in order to achieve a fast response of the servo system, the servo system position tracking loop adopts a large proportional gain and an appropriate differential combination, namely PD control. Removing the integral link can achieve a fast response of the servo system and avoid the system overshoot or oscillation caused by the integral accumulation reaching the limit control amount. Increasing the appropriate differential gain can increase the damping effect and reduce overshoot. When the miss distance is less than or equal to 480 pixel error, the position tracking loop adopts PID control to obtain higher tracking accuracy. The position tracking loop PID parameter adjustment form is expressed as follows:

[0086] Multiply the integral term by the coefficient α, which is calculated as follows:

[0087]

[0088] Step S303: After the target is pulled to the center of the field of view, the PID parameters are adjusted to maintain control accuracy and stable tracking.

[0089] After the target is pulled to the vicinity of the center of the field of view, in order to maintain control accuracy and stable tracking, at this time, appropriate adjustments are made based on the original parameters, maintaining the original proportional gain and differential gain, increasing the integral gain, and also selecting the integral gain according to the error size. In the integral term, it is adjusted by the coefficient β. The integral gain is taken as follows:

[0090]

[0091] When the error is less than or equal to 2 pixels, the generated PID control amount is relatively small, and a suitable integral gain can be selected to achieve stable tracking. Through testing, the integral gain is selected as twice the integral gain when the error is greater than 2 pixels, and PID control is adopted. The differential D can increase the control speed and reduce the adjustment time.

[0092] In the specific implementation of the above steps, if the target pitch error exceeds 480 pixels and the azimuth error is within ±2 pixels when the target is selected, that is, Figure 1 In the middle upper oblique line area, the position near the WF connecting line or the corresponding area on the symmetrical lower side of the image is pulled to the center of the field of view. The pitch axis position tracking loop PID adopts the above control method, and the azimuth axis position tracking loop PID adopts the servo control method in step S303; similarly, if the azimuth direction error exceeds 480 pixels before the target of interest is selected, and the pitch direction error is within ±2 pixels, that is, Figure 1 The position near the HF line in the left oblique line area or the corresponding area on the symmetrical right side of the image is pulled to the center of the field of view. The azimuth axis position tracking loop PID adopts the control method in step S301 and step S302, and the pitch axis position tracking loop PID adopts the servo control method in step S303.

[0093] In some possible implementations, a target occlusion processing step is also included, namely:

[0094] Step S4, if the target is blocked during the process of being pulled to the center of the field of view, the motion state of the target before being blocked is maintained for a set time, and if the target reappears in the field of view, tracking is continued; otherwise, step S3 is repeated, and the timing and limiting control strategies of the servo system response are configured according to the miss amount, and the angle deviation control amount is solved, and the target is automatically tracked when it is pulled to the center of the field of view according to the angle deviation control amount.

[0095] In specific implementation, if the target is blocked during the process of being pulled to the center of the field of view, the image processing unit sends a flag to the servo system and starts the memory tracking mode, that is, the servo system continues to move for 5 seconds according to the movement speed and direction of the target before it is blocked (this time can be determined according to actual conditions. When the movement ends in this mode, the servo system stops at the current position). During the operation of the memory tracking mode, if the target reappears and is detected and captured, the image processing unit continues to send the miss amount and clears the memory tracking mode flag, the memory tracking movement mode is automatically terminated, and the servo system continues to execute the above-mentioned control strategy and method according to the miss amount.

[0096] Furthermore, in some possible implementations, if the target is blocked or target tracking fails, it is allowed to re-acquire the target selection instruction.

[0097] In specific implementation, if the target is blocked or target tracking fails, even if the memory tracking mode has been activated, as long as the target is still in the field of view, the advantage of human-in-the-loop can be used to manually re-select the target, and the memory tracking mode will automatically terminate. The control strategy and method are the same as the above execution process; at the same time, when the servo system enters the memory tracking mode, a motion control operation can be performed in any direction of the servo system in azimuth or pitch to end the current memory tracking mode and enter the waiting state for target capture again.

[0098] Furthermore, in some possible implementations, if the target is located within a specific rectangular area, a response is directly made according to the actual miss distance.

[0099] If the selected target is within the rectangle formed by the four points A(479,269), B(1439,269), C(479,809), and D(1439,809), the servo system will not perform timing limiting regardless of the size of the field of view, that is, it will respond according to the actual miss distance, and the position tracking loop PID control parameter adjustment will still be executed according to the control method of steps S302 and S303.

[0100] In some possible implementations, the present application can implement a timing segmented limiting input control strategy and method according to the actual system. Other specific servo systems can subdivide the miss amount into two intervals, four intervals, etc. according to actual conditions. The time can also be different time periods such as 1.5 seconds and 2 seconds to achieve stable target capture and tracking.

[0101] Based on the above technical solution, this application has the following technical effects and advantages:

[0102] The present application provides a low-cost solution that does not require changes to the servo system or image tracking algorithm hardware, and does not increase the technical difficulty. It only requires the use of timing and limiting control strategies and methods for large miss amounts, so that the capture and tracking of the target of interest can be quickly achieved and pulled to the center of the field of view, which can effectively solve practical needs.

[0103] This application combines human-in-the-loop artificial target selection, tracking algorithms, and servo control methods. The control strategy arranges a transition process for the servo system in the process of gradually reducing the error of the servo system, avoiding excessive overshoot caused by rapid and large-angle movement of the servo system, resulting in the image processing unit failing to detect the target and capturing failure; and the length of the timing segmented limit control time can be determined according to the specific performance of different servo systems, and the size of the miss amount, that is, the specific value of the corresponding large angle, can also be appropriately adjusted according to the actual situation. The method is highly applicable and easy to promote. At the same time, this application also has memory tracking and PID parameter adaptive adjustment functions, giving full play to their respective advantages, learning from each other's strengths and weaknesses, not only has the ability to resist target occlusion, but also ensures that the target of interest will be captured and pulled to the center of the field of view as soon as possible, laying the foundation for further analysis of the target or guiding other systems.

[0104] Compared with related technologies, the present application is simple and efficient to operate, especially for moving targets. The operator does not need to estimate the target's direction of movement, speed of movement, or switch to the appropriate field of view. At the same time, the operator does not need to manually move the target to the center of the field of view through the servo system and then lock and track it, which reduces the requirements for the operator's professional skills such as reaction ability and control proficiency. In addition, the present application does not need to use large drive power to improve the acceleration and deceleration capabilities of the servo system and improve the dynamic response characteristics of the system. It can meet actual needs only through appropriate control strategies and adaptive parameter adjustment based on the miss amount, reduce the production cost of the optoelectronic pod, and improve the adaptability of the optoelectronic pod to the carrier aircraft.

[0105] In terms of specific applications, the continuous zoom infrared thermal imager detector carried by the optoelectronic pod used in this application has a resolution of 640×512, and the output video image resolution by interpolation or stretching is also 1920×1080, and the field of view is between 2 degrees and 26 degrees. Therefore, the above method is still applicable to target selection, capture and tracking under the infrared thermal imager channel.

[0106] See also Figure 4 , corresponding to the above-mentioned optoelectronic pod large-angle point-selection target rapid capture and stable tracking method embodiment, the present application embodiment provides an optoelectronic pod large-angle point-selection target rapid capture and stable tracking system, the system comprising:

[0107] The image processing module 1001 is used to obtain the ground site target selection instruction and the automatic tracking instruction, convert the position of the selected target into a specific image coordinate value; search for a set pixel area with the position of the selected target as the center to extract the target feature; generate a tracking template according to the extracted target feature and store and update the template;

[0108] The miss distance calculation module 1002 is used to calculate the miss distance of the target relative to the center of the field of view, and send it to the servo system together with the automatic tracking instruction;

[0109] The servo control module 1003 is used to configure the timing and limiting control strategy of the servo system response according to the miss amount, and to solve the angle deviation control amount, and to automatically track the target to the center of the field of view according to the angle deviation control amount.

[0110] Furthermore, the servo control module is specifically used for:

[0111] When the miss distance is greater than the set value, the limit control is adopted within the set time, and the miss distance is gradually increased in proportion until the set time is reached and the control is performed according to the actual miss distance;

[0112] According to the miss amount, the PID parameters are adjusted; when the miss amount is greater than the set number of pixels, PD control is adopted; when the miss amount is less than or equal to the set number of pixels, PID control is adopted;

[0113] After the target is pulled to the center of the field of view, the PID parameters are adjusted to maintain control accuracy and stable tracking.

[0114] The above-mentioned optoelectronic pod large-angle point-selected target rapid capture and stable tracking system implements the steps and various processes of the above-mentioned optoelectronic pod large-angle point-selected target rapid capture and stable tracking method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0115] See also Figure 5 , corresponding to the above-mentioned optoelectronic pod large-angle point-selected target rapid capture and stable tracking method embodiment, the embodiment of the present application provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps and various processes of the above-mentioned optoelectronic pod large-angle point-selected target rapid capture and stable tracking method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0116] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0117] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1010.

[0118] Corresponding to the above-mentioned embodiment of the method for rapid capture and stable tracking of targets at large angles by an optoelectronic pod, the embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps and various processes of the embodiment of the above-mentioned method for rapid capture and stable tracking of targets at large angles by an optoelectronic pod are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0119] The processor is the processor in the electronic device described in the above embodiment of the present application. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0120] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0122] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for rapid capture and stable tracking of targets by optoelectronic pods at large angles, characterized in that: include: Obtaining ground site target selection instructions and automatic tracking instructions, and converting the position of the selected target into a specific image coordinate value; Searching a set pixel area with the position of the selected target as the center to extract target features; generating a tracking template according to the extracted target features and storing and updating the template; Calculating the miss distance of the target relative to the center of the field of view and sending it to the servo system together with the automatic tracking instruction; The timing and amplitude limiting control strategies of the servo system response are configured according to the miss amount, and the angle deviation control amount is solved, and the target is automatically tracked to the center of the field of view according to the angle deviation control amount.

2. The method for rapid capture and stable tracking of a target by a photoelectric pod at a large angle according to claim 1 is characterized in that: The method of configuring the timing and limiting control strategy of the servo system response according to the miss amount, solving the angle deviation control amount, and automatically tracking the target to the center of the field of view according to the angle deviation control amount includes: When the miss distance is greater than the set value, the limit control is adopted within the set time, and the miss distance is gradually increased in proportion until the set time is reached and the control is performed according to the actual miss distance; According to the miss amount, the PID parameters are adjusted; when the miss amount is greater than the set number of pixels, PD control is adopted; when the miss amount is less than or equal to the set number of pixels, PID control is adopted; After the target is pulled to the center of the field of view, the PID parameters are adjusted to maintain control accuracy and stable tracking.

3. The method for rapid capture and stable tracking of a target by a photoelectric pod at a large angle according to claim 2 is characterized in that: The method for rapid capture and stable tracking of a target by a large-angle point selection of an optoelectronic pod also includes: If the target is blocked during the process of being pulled to the center of the field of view, the motion state of the target before being blocked is maintained for a set time, and if the target reappears in the field of view, tracking is continued; otherwise, the timing and limiting control strategies for configuring the servo system response according to the miss amount are repeated, and the angle deviation control amount is solved, and the automatic tracking step of pulling the target to the center of the field of view is performed according to the angle deviation control amount.

4. The method for rapid capture and stable tracking of a target by a photoelectric pod at a large angle according to any one of claims 1 to 3, characterized in that: If the target is blocked or target tracking fails, it is allowed to re-acquire the target selection command.

5. The method for rapid capture and stable tracking of a target by a photoelectric pod at a large angle according to any one of claims 1 to 3, characterized in that: If the target is within a specific rectangular area, the system responds directly according to the actual miss distance.

6. The method for rapid capture and stable tracking of a target by a photoelectric pod at a large angle according to any one of claims 1 to 5, characterized in that: Visible light television or infrared thermal imager with a specific image configuration of 1920×1080 pixels; The origin coordinates of each frame image are the (0,0) pixel position in the upper left corner of the image. The first digit of the coordinate value represents the pixel value in the azimuth direction, and the second digit of the coordinate value represents the pixel value in the elevation direction. The specific rectangular area is configured as: a rectangular area formed by four points of pixel coordinates A(479,269), B(1439,269), C(479,809), and D(1439,809); When the field of view angle is greater than 12 degrees, limit control is adopted within the set time; When the miss amount is greater than 480 pixels, PD control is used; when the miss amount is less than 480 pixels, PID control is used.

7. The optoelectronic pod has a large-angle point-selection target rapid capture and stable tracking system, which is characterized by: include: An image processing module is used to obtain ground site target selection instructions and automatic tracking instructions, and convert the position of the point selected target into a specific image coordinate value; Searching a set pixel area with the position of the selected target as the center to extract target features; generating a tracking template according to the extracted target features and storing and updating the template; A miss distance calculation module is used to calculate the miss distance of the target relative to the center of the field of view, and send it to the servo system together with the automatic tracking instruction; The servo control module is used to configure the timing and limiting control strategy of the servo system response according to the miss amount, and to solve the angle deviation control amount, and to automatically track the target to the center of the field of view according to the angle deviation control amount.

8. The optoelectronic pod large-angle point-selection target rapid capture and stable tracking system according to claim 7 is characterized in that: The servo control module is specifically used for: When the miss distance is greater than the set value, the limit control is adopted within the set time, and the miss distance is gradually increased in proportion until the set time is reached and the control is performed according to the actual miss distance; According to the miss amount, the PID parameters are adjusted; when the miss amount is greater than the set number of pixels, PD control is adopted; when the miss amount is less than or equal to the set number of pixels, PID control is adopted; After the target is pulled to the center of the field of view, the PID parameters are adjusted to maintain control accuracy and stable tracking.

9. An electronic device, characterized in that: The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for rapid capture and stable tracking of large-angle point-selection targets by an optoelectronic pod as described in any one of claims 1 to 6 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the method for rapid capture and stable tracking of a target at a large angle by an optoelectronic pod are implemented as described in any one of claims 1 to 6.

Images