A system for searching, identifying, and tracking water surface targets and its usage method

Abstract

This invention relates to the field of target detection and tracking technology, specifically to a system for searching, identifying, and tracking water surface targets and its usage method. The system includes an optoelectronic pod, a positioning module, a display module, a power module, and a perception processing module. The optoelectronic pod includes a two-axis gimbal, a zoom lens, a laser ranging unit, and an imaging sensor. The two-axis gimbal provides pitch and azimuth rotation. The positioning module acquires the current geographical location information, and the display module displays processed data and images. In operation, when the perception processing module controls the rotation of the optoelectronic pod, it needs to identify the water surface line and the target. A water area segmentation algorithm is used to find the dividing line between the water area and the sky as the water surface line. If the water surface line is too high, the pitch angle needs to be lowered; if it is too low, the pitch angle needs to be raised to ensure the water surface line remains within the field of view and that the water surface target is not lost.

Description

Technical Field

[0001] This invention relates to the field of target detection and tracking technology, specifically to a system for searching, identifying, and tracking water surface targets and its usage method. Background Technology

[0002] Surface target search, identification, and tracking technologies have wide applications in maritime patrol, search and rescue, waterway monitoring, and the protection of maritime rights. Existing shipborne electro-optical search and tracking systems typically consist of a two-axis gimbal equipped with visible light and infrared imaging sensors, which scan and track surface targets through the azimuth and pitch movements of the gimbal.

[0003] In actual maritime operations, ships are affected by wind and waves, resulting in continuous roll, pitch, and heave movements, which cause significant disturbances in the line-of-sight of the electro-optical pod. Current technologies for two-axis gimbals mostly employ inertial measurement units (IMUs) for mechanical horizon maintenance. However, due to the randomness and non-stationary nature of wave motion, relying solely on the physical stability of the IMU is insufficient to effectively compensate for surface field-of-sight drift at the image level. When the ship is violently rocking, the horizon line in the image frequently shifts vertically or even moves out of the field of view, significantly reducing the continuity of surface target search and easily leading to missed scans and target loss.

[0004] In target recognition, current mainstream solutions rely on deep convolutional neural networks (such as YOLO and Faster R-CNN) for end-to-end target detection and classification of images. While these methods have achieved good results in general target recognition tasks, they suffer from high task switching costs and insufficient adaptability. When it is necessary to change the search target from one type (such as ships) to another specific type (such as a specific type of buoy or a vessel with a specific paint scheme), it is usually necessary to recollect a large number of labeled samples and retrain or fine-tune the neural network. This process is time-consuming and difficult to complete on-site in embedded environments at sea where there is no network support and computing resources are limited, severely restricting the system's flexible deployment capabilities.

[0005] Furthermore, in the tracking control stage, existing systems mostly use proportional-integral-derivative (PID) control laws to drive the gimbal to track the target. Its control parameters are fixed, making it difficult to achieve high-precision, low-latency stable tracking under the complex dynamic conditions of target maneuvering and ship's own swaying. It is prone to tracking overshoot and lag.

[0006] In summary, existing surface target search, identification and tracking systems still have significant technical shortcomings in areas such as maintaining visibility under adverse sea conditions, rapid task switching for target identification, and tracking control under complex disturbances. There is an urgent need to propose a new system and method that can overcome these problems. Summary of the Invention

[0007] The purpose of this invention is to provide a system for searching, identifying, and tracking water surface targets and a method for using the same. This system has the advantages of simple structure and reasonable design, and solves the problems mentioned in the above-mentioned technical background.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a water surface target search, identification, and tracking system, comprising an optoelectronic pod, a positioning module, a display module, a power supply module, and a sensing and processing module. The optoelectronic pod includes a two-axis gimbal, a zoom lens, a laser ranging unit, and an imaging sensor. The two-axis gimbal provides pitch and azimuth rotation. The positioning module is used to acquire current geographical location information. The display module is used to display processed data and images. The power supply module provides operating power to the optoelectronic pod, the positioning module, the display module, and the sensing and processing module. The perception processing module includes a non-volatile memory, an external data interface, and an embedded AI computing unit. It is electrically connected to the photoelectric pod, the positioning module, and the display module, respectively, and is used to acquire image data, pod attitude data, and position data, and perform processing. The workflow of the perception processing module is as follows: ① Control the photoelectric pod to perform horizontal rotation search in sequence with multiple increasing optical magnifications, with different optical magnifications corresponding to different horizontal rotation angular velocities; ②Real-time identification of the hydro-antenna in the image, and calculation of the pitch angle correction based on the height position of the hydro-antenna in the image, thereby controlling the pitch angle of the optoelectronic pod to keep the hydro-antenna within the preset image height range; ③ Use a pre-trained deep convolutional neural network to perform class recognition on the target in the image. When a target is identified as belonging to a specific preset class, the candidate target region is matched with multiple template images that are pre-stored and collected at predetermined angle intervals around the target of that class. If the match is successful, it is determined to be the target of interest. ④ After identifying the target of interest, adjust the optical magnification of the photoelectric pod to maintain the pixel size of the target in the image at a preset ratio; ⑤ The target distance information is obtained by using the laser ranging unit and the relative position of the target is calculated by combining the rotation angle of the two-axis gimbal. At the same time, the target's moving speed is estimated by image analysis. Kalman filtering is used to fuse the laser ranging data and the image estimation data to obtain the target's movement state estimate. Based on the movement state estimate, the model predictive control algorithm is used to generate the rotation angle and magnification control commands of the optoelectronic pod to achieve target tracking.

[0009] Preferably, the steps of the sensing and processing module in identifying the waterline and calculating the elevation angle correction are as follows: ① For the current frame image Binary water area masks are generated using a water area segmentation algorithm. ; ② Extract the water area mask The equation for the water antenna is obtained by Hough transform linear fitting at the boundary edge with the sky region. And take the column coordinates of the image center. Calculate the pixel coordinates of the waterline in the vertical direction of the image. , where W is the total number of columns in the image; ③ Set the desired position of the waterline to a preset reference value of the vertical center line of the image. Where H is the total number of rows in the image, and the positional error is calculated. ; ④ Set dead zone threshold ,when At that time, based on the current focal length f and the height of the sensor target surface Calculate the vertical field of view This allows us to obtain the angular resolution corresponding to each pixel. ; ⑤ Calculate the pitch angle correction amount ,in The preset proportional control gain has a value range of [value range missing]. ; ⑥ Set the current pitch angle command value Revised to The command is then sent to the pitch axis servo motor of the optoelectronic pod for execution.

[0010] Preferably, when the sensing and processing module identifies the hydrofoil and calculates the pitch angle correction, the pitch angle output by the optoelectronic pod... When the target approaches the preset mechanical limit threshold, a pitch limit warning signal is issued through the display module, and the current optical magnification is automatically switched to a lower magnification to expand the search field of view.

[0011] Preferably, the step of the sensing and processing module controlling the photoelectric pod to perform horizontal rotation search with increasing optical magnification is as follows: ① Initialize the optical magnification index Set the total number of magnification levels (N) and the focal length set as follows: And satisfy The range of values ​​is ; ② Calculate the horizontal rotational angular velocity of the optoelectronic pod's azimuth axis at the corresponding optical magnification level i. Its value is determined by the constraints. Determined, where W is the image width. The angular resolution corresponding to a unit pixel. The processing time for a single frame of image. The reference focal length; ③ With angular velocity Control the optoelectronic pod to complete a 360° horizontal rotation scan; ④ Determine the current gear index ,like If no target of interest is found, then... Switch the optical magnification to And return to step ② to continue execution; if If a target of interest has been found, the search process ends. The optical magnification starts from the minimum magnification, and the higher the optical magnification, the higher the corresponding horizontal rotational angular velocity. The smaller the value, the better, to ensure that the overlap rate of adjacent frames is not lower than a preset threshold.

[0012] Preferably, the specific steps for the perception processing module to perform category recognition and multi-angle template matching are as follows: ① Input the image frames acquired by the photoelectric pod into a pre-trained deep convolutional neural network. The deep convolutional neural network is a YOLO series object detection network, which outputs at least one candidate object region and its corresponding category label C. ② Determine whether the category label C belongs to the preset set of interest categories. If it belongs to the category, then execute ③; otherwise, ignore the candidate target. ③ Extract the candidate target region image patch P and compare it with the template image set pre-stored in the non-volatile memory of the perception processing module. Normalized cross-correlation matching was performed one by one to obtain the correlation coefficient sequence. , where M is the total number of template images; ④ Take the maximum correlation coefficient ,like ,in The preset matching threshold and its value range is [value range missing]. If the candidate target is identified as a confirmed target of interest, it is determined to be a false alarm or a target of non-interest and is excluded. The template image set T is constructed as follows: before target acquisition, the carrier platform carrying the system is manipulated to travel around the target once, and the photoelectric pod keeps the target locked. An image is taken every 10° azimuth angle. After target segmentation processing, a total of 36 template images covering a 360° range are obtained.

[0013] Preferably, the external data interface is used to receive newly added or updated template image sets. The perception processing module stores the received template image sets in a non-volatile memory and directly calls the template image sets for matching calculations in the subsequent target recognition process, thereby completing the recognition configuration of new target types without retraining the deep convolutional neural network.

[0014] Preferably, the step of the perception processing module controlling the magnification factor to maintain the target pixel size at a predetermined ratio is as follows: ① Obtain the pixel dimensions of the target bounding box in the current image and calculate the total area of ​​the target pixels. ; ② Set the desired pixel ratio threshold range ,in The corresponding total area of ​​the image ; ③When When, the optical zoom mechanism is increased to increase the current focal length f until the target pixel area enters the aforementioned threshold range; when At that time, the optical zoom mechanism is controlled to reduce the current focal length f until the target pixel area falls back to the above threshold range.

[0015] Preferably, the specific steps of the perception processing module in fusing laser ranging and image estimation data to estimate the target's movement state are as follows: ① The target slant range R is obtained through the laser rangefinder unit built into the optoelectronic pod, combined with the azimuth angle currently output by the two-axis gimbal. and pitch angle The calculation formula is as follows: The measured position of the target in the local coordinate system with this system as the origin is calculated. ; ② The Lucas-Kanade optical flow method is used to calculate the pixel velocity (u, v) of the target in the image plane for two consecutive frames, and the pixel velocity is converted into the velocity observation value in the world coordinate system based on the current target distance estimate. ; ③ Define the target state vector Construct a Kalman filter with the following state transition matrix: ; in, The filter period; ④ Perform asynchronous Kalman filter update if the received laser ranging measurement value Then, perform position observation updates; when the image velocity estimate is received... When the time comes, perform a velocity observation update; after fusion, output the target's current optimal state estimate. .

[0016] Preferably, the sensing and processing module controls the rotation angle and magnification of the photoelectric pod based on a model predictive control algorithm, and the specific steps are as follows: ① Estimation based on target state Using the motion model of the tracked target as input, and combining it with the carrier motion prediction information output by the inertial navigation module on this system, a prediction time domain is constructed. The target future trajectory sequence ; ② Define control variables Establish the cost function, the specific formula of which is as follows: ; in, Let C be the ground projection coordinates corresponding to the line-of-sight of the electro-optical pod, Q be the output matrix, and Q and R be positive definite weight matrices. To control the increment; ③ While satisfying the maximum angular velocity constraint of the photoelectric pod , Under the constraints of the field of view boundary, the above cost function minimization problem is solved using a quadratic programming solver to obtain the optimal control sequence. ,in To control the time domain; ④ Only the first optimal control quantity As the current cycle, the command output is sent to the azimuth and pitch axis motor drivers of the optoelectronic pod (1).

[0017] Preferably, the positioning module includes a differential global satellite navigation system receiver and an inertial measurement unit, and the sensing and processing module uses the roll and pitch angles output by the inertial measurement unit to perform feedforward compensation on the pitch angle command of the electro-optical pod.

[0018] Preferably, the imaging sensor of the optoelectronic pod includes either an infrared thermal imager or a visible light camera, and the sensing and processing module can automatically switch the imaging sensor according to the ambient lighting conditions to obtain the optimal image quality.

[0019] Preferably, the weight parameters of the deep convolutional neural network are fixed in read-only memory, and the template image set is stored in an erasable flash memory partition, with the two physical storage areas being independent of each other.

[0020] A method for using the above-mentioned water surface target search, identification, and tracking system includes the following steps: Search phase: The photoelectric pod starts from the minimum optical magnification and performs horizontal rotation scanning at an angular velocity inversely proportional to the current magnification. After each revolution, the optical magnification is increased and the angular velocity is adjusted to continue scanning until the target of interest is found or all preset magnification levels are completed. Water antenna holding phase: The position of the water antenna in the image is identified in real time. When the water antenna deviates from the preset image area, the pitch angle correction is calculated and the pitch axis of the two-axis gimbal is driven to maintain the water antenna within the preset image area and compensate for the impact of the carrier sway on the water surface view. Target recognition stage: The deep convolutional neural network is used to detect large categories of targets in the image. For the detected candidate targets of the preset category, normalized cross-correlation matching is performed one by one with 36 template images taken at 10° intervals around the target of the preset category. If the maximum correlation coefficient is greater than the preset threshold, it is confirmed as the target of interest; otherwise, it is excluded. Target tracking phase: The target distance is obtained through the laser ranging unit, and the relative position of the target is calculated by combining the rotation angle of the two-axis gimbal. The target image motion velocity is estimated by using the optical flow method. The target state is estimated by fusing the laser ranging data and the image velocity estimation data using asynchronous Kalman filtering. Then, the pitch angle, azimuth angle and magnification control commands of the electro-optical pod are generated based on the model predictive control algorithm to achieve continuous and stable tracking of the water surface target. During the target tracking phase, the field of view boundary of the optoelectronic pod is used as a hard constraint condition in the process of solving the model predictive control algorithm to ensure that the target is always inside the image field of view in the prediction time domain, thus avoiding target loss.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention provides a water surface target search, identification, and tracking system and its usage method. The water surface target search, identification, and tracking system includes an optoelectronic pod, a positioning module, a display module, a power supply module, and a sensing and processing module. The overall structure is simple and reasonably designed. The sensing and processing module is electrically connected to the optoelectronic pod, the positioning module, and the display module. When the sensing and processing module controls the rotation of the optoelectronic pod, it needs to identify the water surface line and the target. The identification of the water surface line uses a water area segmentation algorithm to find the dividing line between the water area and the sky as the water surface line. When the water surface line is too high, the elevation angle needs to be lowered; when the water surface line is too low, the elevation angle needs to be raised. This ensures that the water surface line remains within the field of view and that the water surface target is not lost. This system is suitable for widespread use.

[0022] 2. In operation, when the general category of the target is known, this invention only requires configuring relevant parameters and matching templates to quickly identify the target without retraining the neural network. It can accurately identify the target value and control the pixel size of the target to one-quarter of the size of the entire image by controlling the magnification factor, making it highly practical.

[0023] 3. When a target value is identified, the present invention measures the distance information of the target by laser measurement, calculates its relative position by combining the rotation angle of the photoelectric pod, estimates the moving speed of the target by using image information, and then uses the Kalman filter algorithm to fuse the laser ranging and image estimation data to obtain the target's movement state. Based on this movement state, the model predictive control algorithm is used to accurately control the rotation angle and magnification of the photoelectric pod. Attached Figure Description

[0024] Figure 1 This is a block diagram of the system structure of the present invention.

[0025] The reference numerals and names in the figure are as follows: 1. Optoelectronic pod; 101. Two-axis gimbal; 102. Zoom lens; 103. Laser ranging unit; 104. Imaging sensor; 2. Positioning module; 201. Differential global satellite navigation system receiver; 202. Inertial measurement unit; 3. Display module; 4. Power supply module; 5. Sensing processing module; 501. Non-volatile memory; 502. External data interface; 503. Embedded AI computing unit. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In the description of the embodiments of the present invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] In the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention according to the specific circumstances.

[0029] Please see Figure 1 The present invention provides an embodiment of a water surface target search, identification, and tracking system, which includes an optoelectronic pod 1, a positioning module 2, a display module 3, a power supply module 4, and a perception processing module 5. The optoelectronic pod 1 includes a two-axis gimbal 101, a zoom lens 102, a laser ranging unit 103, and an imaging sensor 104. The two-axis gimbal 101 provides pitch and azimuth rotation. The positioning module 2 is used to acquire current geographical location information. The display module 3 is used to display processed data and images. The power supply module 4 provides operating power to the optoelectronic pod 1, the positioning module 2, the display module 3, and the perception processing module 5.

[0030] The perception processing module 5 includes a non-volatile memory 501, an external data interface 502, and an embedded AI computing unit 503. The perception processing module 5 is electrically connected to the photoelectric pod 1, the positioning module 2, and the display module 3, respectively, and is used to acquire image data, pod attitude data, and position data and perform processing. The workflow of the perception processing module 5 is as follows: ① Control the photoelectric pod 1 to perform horizontal rotation search in sequence with multiple increasing optical magnifications, with different optical magnifications corresponding to different horizontal rotation angular velocities.

[0031] ②Real-time identification of the water antenna in the image, and calculation of the pitch angle correction based on the height position of the water antenna in the image, thereby controlling the pitch angle of the optoelectronic pod 1 to keep the water antenna within the preset image height range.

[0032] ③ Use a pre-trained deep convolutional neural network to perform broad category recognition on the target in the image. When a target is identified as belonging to a specific preset category, the candidate target region is matched with multiple template images that are pre-stored and collected at predetermined angle intervals around the target of that category. If the match is successful, it is determined to be the target of interest.

[0033] ④ After identifying the target of interest, adjust the optical magnification of the photoelectric pod 1 to maintain the pixel size of the target in the image at a preset ratio.

[0034] ⑤ The laser ranging unit 103 is used to obtain the target distance information and the relative position of the target is calculated by combining the rotation angle of the two-axis gimbal 101. At the same time, the target's moving speed is estimated by image analysis. Kalman filtering is used to fuse the laser ranging data and the image estimation data to obtain the target's movement state estimate. Based on the movement state estimate, the model predictive control algorithm is used to generate the rotation angle and magnification control commands of the optoelectronic pod 1 to achieve target tracking.

[0035] Furthermore, the steps of the sensing and processing module 5 in identifying the waterline and calculating the elevation angle correction are as follows: ① For the current frame image Binary water area masks are generated using a water area segmentation algorithm. ; ② Extract the water area mask The equation for the water antenna is obtained by Hough transform linear fitting at the boundary edge with the sky region. And take the column coordinates of the image center. Calculate the pixel coordinates of the waterline in the vertical direction of the image. , where W is the total number of columns in the image; ③ Set the desired position of the waterline to a preset reference value of the vertical center line of the image. Where H is the total number of rows in the image, and the positional error is calculated. ; ④ Set dead zone threshold ,when At that time, based on the current focal length f and the height of the sensor target surface Calculate the vertical field of view This allows us to obtain the angular resolution corresponding to each pixel. ; ⑤ Calculate the pitch angle correction amount ,in The preset proportional control gain has a value range of [value range missing]. ; ⑥ Set the current pitch angle command value Revised to The command is then sent to the pitch axis servo motor of the optoelectronic pod 1 for execution.

[0036] Furthermore, when the sensing and processing module 5 identifies the hydrofoil and calculates the pitch angle correction, the pitch angle output by the optoelectronic pod 1... When the target approaches the preset mechanical limit threshold, the display module 3 issues a pitch limit warning signal and automatically switches the current optical magnification to a lower magnification to expand the search field of view.

[0037] Furthermore, the sensing and processing module 5 controls the photoelectric pod 1 to perform a horizontal rotation search with increasing optical magnification as follows: ① Initialize the optical magnification index Set the total number of magnification levels (N) and the focal length set as follows: And satisfy The range of values ​​is ; ② Calculate the horizontal rotational angular velocity of the azimuth axis of the photoelectric pod 1 at the corresponding optical magnification level i. Its value is determined by the constraints. Determined, where W is the image width. The angular resolution corresponding to a unit pixel. The processing time for a single frame of image. The reference focal length; ③ With angular velocity Control the photoelectric pod 1 to complete a 360° horizontal rotation scan; ④ Determine the current gear index ,like If no target of interest is found, then... Switch the optical magnification to And return to step ② to continue execution; if If a target of interest has been found, the search process ends. The optical magnification starts from the minimum magnification, and the higher the optical magnification, the higher the corresponding horizontal rotational angular velocity. The smaller the value, the better, to ensure that the overlap rate of adjacent frames is not lower than a preset threshold.

[0038] Furthermore, the specific steps of the perception processing module 5 in performing category recognition and multi-angle template matching are as follows: ① Input the image frames acquired by the photoelectric pod 1 into a pre-trained deep convolutional neural network. The deep convolutional neural network is a YOLO series object detection network, which outputs at least one candidate object region and its corresponding category label C. ② Determine whether the category label C belongs to the preset set of interest categories. If it belongs to the category, then execute ③; otherwise, ignore the candidate target. ③ Extract the candidate target region image block P and compare it with the template image set pre-stored in the non-volatile memory 501 of the perception processing module 5. Normalized cross-correlation matching was performed one by one to obtain the correlation coefficient sequence. , where M is the total number of template images; ④ Take the maximum correlation coefficient ,like ,in The preset matching threshold and its value range is [value range missing]. If the candidate target is identified as a confirmed target of interest, it is determined to be a false alarm or a target of non-interest and is excluded. The template image set T is constructed as follows: before target acquisition, the carrier platform carrying the system is manipulated to travel around the target once, and the photoelectric pod 1 keeps the target locked. An image is taken every 10° azimuth angle. After target segmentation processing, a total of 36 template images covering a 360° range are obtained.

[0039] Furthermore, the external data interface 502 is used to receive newly added or updated template image sets. The perception processing module 5 stores the received template image sets in the non-volatile memory 501 and directly calls the template image sets for matching calculations in the subsequent target recognition process, thereby completing the recognition configuration of new target types without retraining the deep convolutional neural network.

[0040] Furthermore, the steps of the perception processing module 5 in controlling the magnification factor to maintain the target pixel size at a predetermined ratio are as follows: ① Obtain the pixel dimensions of the target bounding box in the current image and calculate the total area of ​​the target pixels. ; ② Set the desired pixel ratio threshold range ,in The corresponding total area of ​​the image ; ③When When, the optical zoom mechanism is increased to increase the current focal length f until the target pixel area enters the aforementioned threshold range; when At that time, the optical zoom mechanism is controlled to reduce the current focal length f until the target pixel area falls back to the above threshold range.

[0041] Furthermore, the specific steps of the perception processing module 5 in fusing laser ranging and image estimation data to estimate the target's movement state are as follows: ① The target slant range R is obtained by the laser ranging unit 103 built into the optoelectronic pod (1), and combined with the azimuth angle currently output by the two-axis gimbal 101. and pitch angle The calculation formula is as follows: The measured position of the target in the local coordinate system with this system as the origin is calculated. ; ② The Lucas-Kanade optical flow method is used to calculate the pixel velocity (u, v) of the target in the image plane for two consecutive frames, and the pixel velocity is converted into the velocity observation value in the world coordinate system based on the current target distance estimate. ; ③ Define the target state vector Construct a Kalman filter with the following state transition matrix: ; in, The filter period; ④ Perform asynchronous Kalman filter update if the received laser ranging measurement value Then, perform position observation updates; when the image velocity estimate is received... When the time comes, perform a velocity observation update; after fusion, output the target's current optimal state estimate. .

[0042] Furthermore, the perception processing module 5 uses a model predictive control algorithm to control the rotation angle and magnification of the photoelectric pod 1. The specific steps are as follows: ① Estimation based on target state Using the motion model of the tracked target as input, and combining it with the carrier motion prediction information output by the inertial navigation module on this system, a prediction time domain is constructed. The target future trajectory sequence ; ② Define control variables Establish the cost function, the specific formula of which is as follows: ; in, Let C be the ground projection coordinates corresponding to the line-of-sight of the optoelectronic pod 1, Q be the output matrix, and Q and R be positive definite weight matrices. To control the increment; ③ While satisfying the maximum angular velocity constraint of photoelectric pod 1 , Under the constraints of the field of view boundary, the above cost function minimization problem is solved using a quadratic programming solver to obtain the optimal control sequence. ,in To control the time domain; ④ Only the first optimal control quantity As the current cycle, the command output is sent to the azimuth and pitch axis motor drivers of the optoelectronic pod (1).

[0043] Furthermore, the positioning module 2 includes a differential global satellite navigation system receiver 201 and an inertial measurement unit 202. The sensing and processing module 5 uses the roll and pitch angles output by the inertial measurement unit 202 to perform feedforward compensation on the pitch angle command of the optoelectronic pod 1.

[0044] Furthermore, the imaging sensor 104 of the optoelectronic pod 1 includes either an infrared thermal imager or a visible light camera, and the perception processing module 5 can automatically switch the imaging sensor 104 according to the ambient lighting conditions to obtain the optimal image quality.

[0045] Furthermore, the weight parameters of the deep convolutional neural network are fixed in read-only memory, while the template image set is stored in an erasable flash memory partition, and the two physical storage areas are independent of each other.

[0046] Please refer to it again. Figure 1 The present invention provides an embodiment of a method for using the above-mentioned water surface target search, identification, and tracking system, comprising the following steps: Search phase: The photoelectric pod 1 starts from the minimum optical magnification and performs horizontal rotation scanning at an angular velocity inversely proportional to the current magnification. After each revolution, the optical magnification is increased and the angular velocity is adjusted to continue scanning until the target of interest is found or all preset magnification levels are scanned.

[0047] Water antenna holding phase: The position of the water antenna in the image is identified in real time. When the water antenna deviates from the preset image area, the pitch angle correction is calculated and the pitch axis of the two-axis gimbal 101 is driven to maintain the water antenna within the preset image area and compensate for the impact of the carrier sway on the water surface view.

[0048] Target recognition stage: A deep convolutional neural network is used to detect large-class targets in the image. For the detected candidate targets of the preset categories, normalized cross-correlation matching is performed one by one with 36 pre-stored template images taken at 10° intervals around the target of that category. If the maximum correlation coefficient is greater than a preset threshold, it is confirmed as a target of interest; otherwise, it is excluded. Thus, when the category of the target is known, only the relevant parameters and matching templates need to be configured to quickly identify the target without retraining the neural network. The target value can be accurately identified. By controlling the magnification factor, the pixel size of the target is controlled to be one-quarter of the size of the entire image.

[0049] Target tracking stage: The target distance is obtained through the laser ranging unit 103, and the relative position of the target is calculated by combining the rotation angle of the two-axis gimbal 101. The target image motion velocity is estimated by using the optical flow method. The target state is estimated by fusing the laser ranging data and the image velocity estimation data using asynchronous Kalman filtering. Then, the pitch angle, azimuth angle and magnification control commands of the photoelectric pod 1 are generated based on the model predictive control algorithm to achieve continuous and stable tracking of the water surface target.

[0050] During the target tracking phase, the field of view boundary of the photoelectric pod 1 is used as a hard constraint condition in the process of solving the model prediction control algorithm to ensure that the target is always inside the image field of view in the prediction time domain, thus avoiding target loss.

[0051] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A system for searching, identifying, and tracking water surface targets, characterized in that: It includes an optoelectronic pod (1), a positioning module (2), a display module (3), a power supply module (4), and a sensing and processing module (5), wherein: The optoelectronic pod (1) includes a two-axis gimbal (101), a zoom lens (102), a laser ranging unit (103), and an imaging sensor (104). The two-axis gimbal (101) provides pitch and azimuth rotation. The positioning module (2) is used to obtain the current geographical location information; The display module (3) is used to display the processed data and images; The power module (4) provides working power for the photoelectric pod (1), positioning module (2), display module (3) and sensing processing module (5); The perception processing module (5) includes a non-volatile memory (501), an external data interface (502), and an embedded AI computing unit (503). The perception processing module (5) is electrically connected to the photoelectric pod (1), the positioning module (2), and the display module (3) respectively, and is used to acquire image data, pod attitude data, and position data and perform processing. The workflow of the perception processing module (5) is as follows: ① Control the photoelectric pod (1) to perform horizontal rotation search in sequence with multiple increasing optical magnifications, with different optical magnifications corresponding to different horizontal rotation angular velocities; ②Real-time identification of the water antenna in the image, and calculation of the pitch angle correction based on the height position of the water antenna in the image, controlling the pitch angle of the optoelectronic pod (1) so that the water antenna is kept within the preset image height range; ③ Use a pre-trained deep convolutional neural network to perform class recognition on the target in the image. When a target is identified as belonging to a specific preset class, the candidate target region is matched with multiple template images that are pre-stored and collected at predetermined angle intervals around the target of that class. If the match is successful, it is determined to be the target of interest. ④ After identifying the target of interest, adjust the optical magnification of the photoelectric pod (1) to maintain the pixel size of the target in the image at a preset ratio; ⑤ The target distance information is obtained by using the laser ranging unit (103) and the relative position of the target is calculated by combining the rotation angle of the two-axis gimbal (101). At the same time, the target's moving speed is estimated by image analysis. Kalman filtering is used to fuse the laser ranging data and the image estimation data to obtain the target's moving state estimate. Based on the moving state estimate, the rotation angle and magnification control command of the optoelectronic pod (1) are generated by the model predictive control algorithm to achieve target tracking.

2. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The steps of the sensing and processing module (5) in identifying the waterline and calculating the elevation angle correction are as follows: generating a binary water mask using a water segmentation algorithm ;​ ii. extracting the water mask from the image The intersection edge with the sky region, using Hough transform line fitting to obtain the water-sky line equation And take the image center column coordinate Calculate the pixel coordinate of the water-sky line in the vertical direction of the image Where W is the total number of columns of the image; ③ Set the desired position of the waterline to a preset reference value of the vertical center line of the image. Where H is the total number of rows in the image, and the positional error is calculated. ; ④ Set dead zone threshold ,when At that time, based on the current focal length f and the height of the sensor target surface Calculate the vertical field of view This allows us to obtain the angular resolution corresponding to each pixel. ; ⑤ Calculate the pitch angle correction amount ,in The preset proportional control gain has a value range of [value range missing]. ; ⑥ Set the current pitch angle command value Revised to And send it to the pitch axis servo motor of the optoelectronic pod (1) for execution.

3. The water surface target search, identification, and tracking system according to claim 2, characterized in that: When the sensing and processing module (5) identifies the water antenna and calculates the pitch angle correction, the pitch angle output by the optoelectronic pod (1) is... When the target is close to the preset mechanical limit threshold, the display module (3) issues a pitch limit warning signal and automatically switches the current optical magnification to a smaller magnification to expand the search field of view.

4. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The sensing and processing module (5) controls the photoelectric pod (1) to perform horizontal rotation search with increasing optical magnification as follows: ① Initialize the optical magnification index Set the total number of magnification levels (N) and the focal length set as follows: And satisfy The range of values ​​is ; ② Calculate the horizontal rotational angular velocity of the azimuth axis of the photoelectric pod (1) at the corresponding optical magnification level (i). Its value is determined by the constraints. Determined, where W is the image width. The angular resolution corresponding to a unit pixel. The processing time for a single frame of image. The reference focal length; ③ With angular velocity Control the photoelectric pod (1) to complete a 360° horizontal rotation scan; ④ Determine the current gear index ,like If no target of interest is found, then... Switch the optical magnification to And return to step ② to continue execution; if If a target of interest has been found, the search process ends. The optical magnification starts from the minimum magnification, and the higher the optical magnification, the higher the corresponding horizontal rotational angular velocity. The smaller the value, the better, to ensure that the overlap rate of adjacent frames is not lower than a preset threshold.

5. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The specific steps of the perception processing module (5) in performing category recognition and multi-angle template matching are as follows: ① Input the image frames acquired by the photoelectric pod (1) into a pre-trained deep convolutional neural network. The deep convolutional neural network is a YOLO series target detection network, which outputs at least one candidate target region and its corresponding category label C. ② Determine whether the category label C belongs to the preset set of interest categories. If it belongs to the category, then execute ③; otherwise, ignore the candidate target. ③ Extract the candidate target region image block P and compare it with the template image set pre-stored in the non-volatile memory (501) of the perception processing module (5). Normalized cross-correlation matching was performed one by one to obtain the correlation coefficient sequence. , where M is the total number of template images; ④ Take the maximum correlation coefficient ,like ,in The preset matching threshold and its value range is [value range missing]. If the candidate target is identified as a confirmed target of interest, it is determined to be a false alarm or a target of non-interest and is excluded. The template image set T is constructed as follows: before target acquisition, the carrier platform carrying the system is manipulated to sail around the target once, and the photoelectric pod (1) keeps the target locked. An image is taken every 10° azimuth angle. After target segmentation processing, a total of 36 template images covering a 360° range are obtained.

6. The water surface target search, identification, and tracking system according to claim 5, characterized in that: The external data interface (502) of the perception processing module (5) is used to receive new or updated template image sets, and the perception processing module (5) stores the received template image sets in a non-volatile memory (501), and directly calls the template image sets for matching calculation in the subsequent target recognition process, so as to complete the recognition configuration of new target types without retraining the deep convolutional neural network.

7. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The steps of the perception processing module (5) in controlling the magnification factor to maintain the target pixel size at a predetermined ratio are as follows: ① Obtain the pixel dimensions of the target bounding box in the current image and calculate the total area of ​​the target pixels. ; ② Set the desired pixel ratio threshold range ,in The corresponding total area of ​​the image ; ③When When, the optical zoom mechanism is increased to increase the current focal length f until the target pixel area enters the aforementioned threshold range; when At that time, the optical zoom mechanism is controlled to reduce the current focal length f until the target pixel area falls back to the above threshold range.

8. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The specific steps of the perception processing module (5) in fusing laser ranging and image estimation data to estimate the target's movement state are as follows: ① The target slant range R is obtained through the laser ranging unit (103) built into the optoelectronic pod (1), and combined with the azimuth angle currently output by the two-axis gimbal (101). and pitch angle The calculation formula is as follows: The measured position of the target in the local coordinate system with this system as the origin is calculated. ； ② The Lucas-Kanade optical flow method is used to calculate the pixel velocity (u, v) of the target in the image plane for two consecutive frames, and the pixel velocity is converted into the velocity observation value in the world coordinate system based on the current target distance estimate. ; ③ Define the target state vector Construct a Kalman filter with the following state transition matrix: ; in, The filter period; ④ Perform asynchronous Kalman filter update if the received laser ranging measurement value Then, perform position observation updates; when the image velocity estimate is received... When the time comes, perform a velocity observation update; after fusion, output the target's current optimal state estimate. .

9. The water surface target search, identification, and tracking system according to claim 8, characterized in that: The sensing and processing module (5) controls the rotation angle and magnification of the photoelectric pod (1) based on the model predictive control algorithm. The specific steps are as follows: ① Estimation based on target state Using the motion model of the tracked target as input, and combining it with the carrier motion prediction information output by the inertial navigation module on this system, a prediction time domain is constructed. The target future trajectory sequence ; ② Define control variables Establish the cost function, the specific formula of which is as follows: ； in, Let C be the ground projection coordinates corresponding to the line of sight of the optoelectronic pod (1), Q and R be the output matrix, and Q and R be the positive definite weight matrices. To control the increment; ③ Under the condition of satisfying the maximum angular velocity constraint of the photoelectric pod (1) , Under the constraints of the field of view boundary, the above cost function minimization problem is solved using a quadratic programming solver to obtain the optimal control sequence. ,in To control the time domain; ④ Only the first optimal control quantity As the current cycle, the command output is sent to the azimuth and pitch axis motor drivers of the optoelectronic pod (1).

10. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The positioning module (2) includes a differential global satellite navigation system receiver (201) and an inertial measurement unit (202). The sensing and processing module (5) uses the roll and pitch angles output by the inertial measurement unit (202) to feedforward compensation for the pitch angle command of the optoelectronic pod (1).

11. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The imaging sensor (104) of the optoelectronic pod (1) includes either an infrared thermal imager or a visible light camera. The sensing and processing module (5) can automatically switch the imaging sensor (104) according to the ambient light conditions to obtain the optimal image quality.

12. The water surface target search, identification, and tracking system according to claim 1, characterized in that: The weight parameters of the deep convolutional neural network are stored in read-only memory, while the template image set is stored in an erasable flash memory partition. The two physical storage areas are independent of each other.

13. A method of using the water surface target search, identification, and tracking system according to any one of claims 1-12, characterized in that, Includes the following steps: Search phase: Control the photoelectric pod (1) to start from the minimum optical magnification and perform horizontal rotation scanning at an angular velocity inversely proportional to the current magnification. After each revolution, increase the optical magnification and adjust the angular velocity to continue scanning until the target of interest is found or all preset magnification levels are scanned. Water antenna holding phase: Real-time identification of the position of the water antenna in the image. When the water antenna deviates from the preset image area, calculate the pitch angle correction and drive the pitch axis movement of the two-axis gimbal (101) to keep the water antenna within the preset image area and compensate for the impact of the carrier sway on the water surface view. Target recognition stage: The deep convolutional neural network is used to detect large categories of targets in the image. For the detected candidate targets of the preset category, normalized cross-correlation matching is performed one by one with 36 template images taken at 10° intervals around the target of the preset category. If the maximum correlation coefficient is greater than the preset threshold, it is confirmed as the target of interest; otherwise, it is excluded. Target tracking stage: The target distance is obtained through the laser ranging unit (103), the relative position of the target is calculated by combining the rotation angle of the two-axis gimbal (101), the target image motion speed is estimated by using the optical flow method, the target state is estimated by using asynchronous Kalman filtering to fuse the laser ranging data and the image speed estimation data, and then the pitch angle, azimuth angle and magnification control command of the optoelectronic pod (1) are generated based on the model predictive control algorithm to realize continuous and stable tracking of the water surface target; In the target tracking stage, during the solution process of the model prediction control algorithm, the field of view boundary of the photoelectric pod (1) is used as a hard constraint condition to ensure that the target is always inside the image field of view in the prediction time domain, thus avoiding target loss.

Images