A multi-photoelectric intelligent tracking method in a water area detection system

By splitting the intelligent tracking solution in the water detection system, using the coordinated work of the central station intelligent identification device and the photoelectric smooth tracking controller, the problem of inappropriate deployment of artificial intelligence identification software and hardware is solved, and the accurate tracking of multiple optoelectronic devices is achieved. Low cost control reduces delays and lags.

CN114299107BActive Publication Date: 2024-11-22MSKYEYE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111259859.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-28
Publication Date
2024-11-22
Estimated Expiration
2041-10-28

AI Technical Summary

Technical Problem

In the existing water detection system, artificial intelligence identification software and hardware deployment is inappropriate, resulting in intelligent tracking signal delay and high energy consumption, increasing costs, and long-term tracking targets lead to wasted computing power.

Method used

The intelligent tracking solution is split into a central station intelligent ship recognition device and an optical-electrical smooth tracking controller. Through the intelligent recognition computing power distribution algorithm and the KCF Tracker algorithm, it works together to reduce delays and lags and achieve accurate tracking of multiple optoelectronic devices.

Benefits of technology

It reduces the delay and lag of optoelectronic equipment, reduces costs, and realizes smooth tracking control of multiple gimbal cameras under low power conditions, ensuring the accuracy and smoothness of tracking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114299107B_ABST
    Figure CN114299107B_ABST
Patent Text Reader

Abstract

The present invention discloses a multi-photoelectric intelligent tracking method in a water area detection system, which splits the existing intelligent tracking scheme into two parts: first, the central station intelligent ship recognition device identifies the ship and extracts the feature vector A to send it to the photoelectric tracking controller, and the intelligent recognition computing power allocation algorithm is used to allow the intelligent recognition computing power to support as many photoelectric devices as possible; second, the photoelectric smooth tracking controller receives the video signal, obtains the moving target feature vector through the ship intelligent recognition algorithm and matches it with the A vector, triggering the KCF Tracker tracking algorithm to track the target. Through the collaborative work of the two parts, the jamming and delay caused by the photoelectric device when tracking the target for a long time are reduced, ensuring that the photoelectric device can accurately track the ship.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field related to radar detection, and in particular to a multi-photoelectric intelligent tracking method in a water area detection system. Background Art

[0002] The most important function of the water radar-photoelectric detection system is to use radar to detect ships and guide the optoelectronic equipment to track the ship for a long time. At present, the long-term smooth tracking of target ships through artificial intelligence algorithms is widely used in water radar-photoelectric detection systems. Common intelligent tracking solutions on the market are: 1. Radar guides optoelectronics to aim at the target; 2. Optoelectronics sends the image to the artificial intelligence device, intelligently identifies the ship, and calculates the pixel displacement vector from the center of gravity of the ship to the center of the image; 3. According to the radar coordinates of the ship, the coordinates of the current optoelectronic device, the rotation angle, the pitch angle and other information, the pixel displacement vector is converted into the optoelectronic PTZ control signal. Automatically control the optoelectronic device to smoothly track the target, keep the target in the center of the picture and enlarge it to the appropriate size.

[0003] However, the artificial intelligence recognition software and hardware in the target water detection system directly use the mature algorithms and hardware equipment in vehicle recognition and pedestrian recognition, and do not conduct in-depth research on the unique characteristics of ship tracking and recognition in water detection, resulting in the existing intelligent recognition software and hardware equipment cannot be well used in water detection.

[0004] In the existing water detection system, one radar is often equipped with 4-5 heavy-duty PTZ optoelectronic devices. Radars and optoelectronics are deployed in the wild, where the power supply and network bandwidth are extremely limited. The central station is deployed in the urban computer room, far away from radar, optoelectronics and other detection equipment. If the artificial intelligence recognition software and hardware are deployed at the central station, the network delay will cause a large delay in the control signal sent by the intelligent recognition software, and the optoelectronic equipment will not be able to accurately track the ship. If the artificial intelligence software and hardware are placed on the optoelectronic device side, the artificial intelligence software and hardware consume a lot of energy, and the power that can be provided in the wild is usually very tight and cannot support high-power equipment. In addition, artificial intelligence software and hardware are expensive, and the cost of equipping each optoelectronic with an artificial intelligence device will increase greatly.

[0005] In addition, ships usually sail very slowly, and the monitoring radius of heavy-duty gimbal optoelectronic equipment is 10-20 kilometers. Optoelectronic equipment will usually continue to track and monitor the same ship for more than 10 minutes. Using artificial intelligence recognition algorithms to continuously identify the same target for a long time is a waste of computing power. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a method for intelligent identification of multiple optoelectronic devices in a water area detection system, which splits the existing intelligent tracking scheme into two parts: first, the intelligent ship identification device of the central station identifies the ship and extracts the feature vector A and sends it to the optoelectronic tracking controller, and the intelligent identification computing power allocation algorithm is used to allow the intelligent identification computing power to support as many optoelectronic devices as possible; second, the optoelectronic smooth tracking controller receives the video signal, obtains the moving target feature vector through the ship intelligent identification algorithm and matches it with the A vector, triggering the KCF Tracker tracking algorithm to track the target. Through the collaborative work of the two parts, the jamming and delay caused by the optoelectronic device when tracking the target for a long time are reduced, ensuring that the optoelectronic device can accurately track the ship.

[0007] To achieve the above purpose, the technical solution of the present invention is as follows:

[0008] A multi-photoelectric intelligent tracking method in a water area detection system, the specific steps are as follows:

[0009] S1. Radar guides optoelectronic equipment to aim at the target vessel: if the vessel enters the warning zone, the radar is triggered to continuously send the target coordinates; if the target deviates from the route, the radar is triggered to continuously send the target coordinates;

[0010] S2, the intelligent ship recognition device at the central station receives the video stream sent by the optoelectronic device, the intelligent recognition algorithm recognizes the target in the video stream, extracts feature points of the recognized ship and forms a feature vector A, and sends the target feature vector A;

[0011] Through the intelligent recognition computing power allocation algorithm, the intelligent recognition computing power can support multiple optoelectronic devices at the same time;

[0012] S3, the optoelectronic tracking controller receives the target feature vector A, tracks the target vessel through the KCF Tracker tracking algorithm, and calculates the pixel distance between the target center and the video center;

[0013] S4, the optoelectronic tracking controller calculates the control signal of the optoelectronic device according to the radar coordinates of the target ship and the current PTZ information of the optoelectronic device through the optoelectronic PTZ control algorithm and sends it to the optoelectronic device;

[0014] S5. The photoelectric device adjusts the rotation angle, pitch angle, and focal length based on the control signal, automatically controls the photoelectric device to smoothly track the target, keep the target in the center of the picture, and enlarge it to a suitable size.

[0015] Furthermore, in step S3, the photoelectric tracking controller receives the video signal of the photoelectric device, obtains the moving target by the sparse optical flow method, calculates the feature vector of the moving target, and performs feature matching with the feature vector A, and tracks the successfully matched target by using the KCF Tracker tracking algorithm.

[0016] Furthermore, in step S4, the pixel displacement vector is converted into an optoelectronic PTZ control signal according to the radar coordinates of the ship, the coordinates of the current optoelectronic device, the rotation angle, the pitch angle and other information.

[0017] Furthermore, the central station intelligent ship identification device identifies the video stream sent by each optoelectronic device in turn; identifies the ship in the video, and then marks the target ship in the video in combination with the PTZ information of the optoelectronic device and the radar coordinates of the target, and sends the extracted feature vector A of the ship to the corresponding optoelectronic tracking controller to help the optoelectronic tracking controller "manually select the target".

[0018] Furthermore, the photoelectric tracking controller can successfully trigger the KCF Tracker algorithm to achieve target tracking by relying on the feature vector obtained by the ship intelligent recognition algorithm; when the photoelectric tracking controller fails to track, it sends a signal to the central station to trigger the ship intelligent recognition algorithm to "manually select the target" again.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] 1. The present invention installs an intelligent recognition computing power allocation algorithm on the central station intelligent ship recognition device, which can realize intelligent recognition of video streams sent by multiple optoelectronic devices at a lower cost.

[0021] 2. Installing the target tracking algorithm and the photoelectric PTZ control algorithm in the photoelectric tracking controller can greatly ensure the factuality and smoothness of the photoelectric PTZ control, reduce the jamming and delay caused by the photoelectric tracking of the target for a long time, and realize the smooth tracking control of multiple PTZ cameras under low cost conditions.

[0022] 3. When the central station directly controls the camera, there is usually a delay of 2-3 seconds. In the present invention, there is no delay in camera control under the condition of automatic identification and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a composition diagram of the multi-photoelectric intelligent tracking method of the present invention;

[0024] Figure 2 Provide a flow chart for the intelligent recognition computing power allocation algorithm;

[0025] Figure 3 This is the flow chart of the target tracking algorithm;

[0026] Figure 4 Provide a flow chart for the intelligent identification algorithm of ships;

[0027] Figure 5 This is the flow chart of the photoelectric pan / tilt control algorithm. DETAILED DESCRIPTION

[0028] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0029] like Figure 1-5 As shown, a multi-photoelectric intelligent tracking method in a water area detection system, the specific steps are as follows:

[0030] S1. Radar guides optoelectronic equipment to aim at the target vessel: if the vessel enters the warning zone, the radar is triggered to continuously send the target coordinates; if the target deviates from the route, the radar is triggered to continuously send the target coordinates;

[0031] S2, the intelligent ship recognition device at the central station receives the video stream sent by the optoelectronic device, the intelligent recognition algorithm recognizes the target in the video stream, extracts feature points of the recognized ship and forms a feature vector A, and sends the target feature vector A;

[0032] Through the intelligent recognition computing power allocation algorithm, the intelligent recognition computing power can support multiple optoelectronic devices at the same time;

[0033] S3, the optoelectronic tracking controller receives the target feature vector A, tracks the target vessel through the KCF Tracker tracking algorithm, and calculates the pixel distance between the target center and the video center;

[0034] S4, the optoelectronic tracking controller calculates the control signal of the optoelectronic device according to the radar coordinates of the target ship and the current PTZ information of the optoelectronic device through the optoelectronic PTZ control algorithm and sends it to the optoelectronic device;

[0035] S5. The photoelectric device adjusts the rotation angle, pitch angle, and focal length based on the control signal, automatically controls the photoelectric device to smoothly track the target, keep the target in the center of the picture, and enlarge it to a suitable size. Specific embodiment:

[0037] The present invention divides the existing intelligent tracking solution into two parts: a central station intelligent ship identification device and a photoelectric smooth tracking controller.

[0038] like Figure 2 As shown, the intelligent ship recognition device at the central station is responsible for identifying the ship, extracting feature points from the identified ship and forming a feature vector, and transmitting the feature vector A of the ship to the optoelectronic smooth tracking controller through the Internet. Through the intelligent recognition computing power allocation algorithm, the intelligent recognition computing power can support as many optoelectronic devices as possible.

[0039] like Figure 3 As shown in the figure, the photoelectric tracking controller receives the photoelectric video signal and obtains the moving target through the sparse optical flow method. The feature vector of the moving target is calculated and matched with the A vector. The KCF Tracker tracking algorithm is used to track the successfully matched target.

[0040] The pixel displacement vector is converted into the photoelectric PTZ control signal according to the radar coordinates of the ship, the coordinates of the current photoelectric device, the rotation angle, the pitch angle and other information. Figure 5 As shown, the optoelectronic device is automatically controlled by the optoelectronic PTZ control algorithm to smoothly track the target, keep the target in the center of the picture and enlarge it to a suitable size.

[0041] The present invention installs both the target tracking algorithm and the photoelectric pan-tilt control algorithm in the photoelectric smooth tracking controller. The advantage of this solution is that it can greatly ensure the factuality and smoothness of the photoelectric pan-tilt control, and reduce the jamming and delay caused by the photoelectric tracking of the target for a long time. However, in order to control costs, reduce power, and adapt to the high-humidity outdoor installation environment, the photoelectric tracking controller needs to select a fully enclosed small industrial computer.

[0042] Due to hardware limitations, the target tracking algorithm needs to choose the KCF Tracker tracking algorithm with low computational complexity and high speed. The KCF Tracker tracking algorithm, like other non-intelligent tracking algorithms, has the following defects: 1) The target needs to be selected manually; 2) As the target moves, the shooting angle changes, and the target is partially or completely blocked, the tracking success rate will drop rapidly.

[0043] In order to solve these two problems, the central station of the urban command center will identify the video stream sent by each optoelectronic device in turn through an intelligent recognition algorithm. It will identify the ship in the video, and then mark the target ship in the video by combining the optoelectronic PTZ information and the radar coordinates of the target, and send the feature vector A of the extracted ship to the corresponding optoelectronic tracking controller to help the optoelectronic tracking controller "manually select the target".

[0044] There is usually a 2-3 second delay between the central station and the optoelectronic tracking controller (the distance is too far and the network bandwidth is limited). Since the speed of a ship is usually very slow, the characteristics of the ship will not change much in 3 seconds. Figure 4 As shown, the optoelectronic tracking controller can successfully trigger the KCF Tracker algorithm to achieve target tracking by relying on the feature vector obtained by the ship intelligent recognition algorithm. When the optoelectronic tracking controller fails to track, it sends a signal to the central station to trigger the ship intelligent recognition algorithm to "manually select the target" again.

[0045] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A multi-photoelectric intelligent tracking method in a water area detection system, characterized in that: The specific steps are as follows: S1, radar guides the optoelectronic device to aim at the target ship; S2, the intelligent ship recognition device at the central station receives the video stream sent by the optoelectronic device, the intelligent recognition algorithm recognizes the target in the video stream, extracts feature points of the recognized ship and forms a feature vector A, and sends the target feature vector A; Through the central station intelligent ship identification device allocation algorithm, the central station intelligent ship identification device can support multiple optoelectronic devices at the same time; S3, the optoelectronic tracking controller receives the target feature vector A, tracks the target vessel through the KCF Tracker tracking algorithm, and calculates the pixel distance between the target center and the video center; The photoelectric tracking controller receives the video signal of the photoelectric device, obtains the moving target by the sparse optical flow method, calculates the feature vector of the moving target and performs feature matching with the target feature vector A, and tracks the target with the KCF Tracker tracking algorithm for the successfully matched target; S4, the optoelectronic tracking controller calculates the control signal of the optoelectronic device according to the radar coordinates of the target ship and the current PTZ information of the optoelectronic device through the optoelectronic PTZ control algorithm and sends it to the optoelectronic device; S5. The photoelectric device adjusts the rotation angle, pitch angle, and focal length based on the control signal, automatically controls the photoelectric device to smoothly track the target, keep the target in the center of the picture, and enlarge it to a suitable size.

2. The multi-photoelectric intelligent tracking method in a water area detection system according to claim 1 is characterized in that: In step S4, the pixel displacement vector is converted into the current PTZ information of the optoelectronic device according to the current coordinates, rotation angle, and pitch angle information of the optoelectronic device of the ship.

3. The multi-photoelectric intelligent tracking method in a water area detection system according to claim 1 is characterized in that: The central station intelligent ship recognition device identifies the video stream sent by each optoelectronic device in turn; identifies the ship in the video, and then marks the target ship in the video in combination with the PTZ information of the optoelectronic device and the radar coordinates of the target, and sends the feature vector A of the extracted ship to the corresponding optoelectronic tracking controller to help the optoelectronic tracking controller "manually select the target".

4. The multi-photoelectric intelligent tracking method in a water area detection system according to claim 3 is characterized in that: The photoelectric tracking controller can successfully trigger the KCF Tracker algorithm to achieve target tracking by relying on the feature vector obtained by the ship intelligent recognition algorithm; when the photoelectric tracking controller fails to track, it sends a signal to the central station to trigger the ship intelligent recognition algorithm to "manually select the target" again.