Photoelectric pod target detection system based on abnormal motion detection

Abstract

The invention relates to the technical field of photoelectric pods, in particular to a photoelectric pod target detection system based on abnormal motion detection. Comprising an optical acquisition module, an abnormal motion detection module, a data fusion decision module and a target locking tracking module. The optical acquisition module is used for acquiring optical information of a target area; the abnormal motion detection module is used for detecting a potential abnormal motion target which is different from the background motion direction and speed according to the image data, tracking the abnormal motion target and outputting target data; the data fusion decision module is used for carrying out fusion processing on the target data and the thermal radiation data and judging the threat level of the target; the target locking and tracking module is used for adjusting the direction of the optical acquisition module according to the position, size and motion direction parameters of the target, so that the target is always in the center of the visual field; by means of the mode, the effect that the target can be found in a complex environment without presetting the target type is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of optoelectronic pods, and particularly to an optoelectronic pod target detection system based on abnormal motion detection. Background Art

[0002] Optoelectronic pods are usually installed on aircraft or other aviation platforms and are mainly mission payload devices for collecting light. Currently, in the target detection of optoelectronic pods, deep learning target detection is mainly used, and the detection is trained through a large amount of labeled data and requires preset target features. For targets not labeled in the training data, it is difficult for the model to recognize them. Therefore, it is difficult for preset target features to cover the characteristics of all potential targets, and new or camouflaged targets are likely to be missed. At the same time, due to the flexible characteristics of aviation platforms such as unmanned aerial vehicles, targets are usually observed from a variety of different angles, such as top view, front view, oblique view, close range, long range, etc. However, the target detection algorithm model usually can only recognize one or some of these perspectives and cannot cover all perspectives.

[0003] Therefore, it is very necessary to propose an optoelectronic pod target detection system that does not require preset target types and can detect targets in complex environments. Summary of the Invention

[0004] The purpose of the present invention is to provide an optoelectronic pod target detection system based on abnormal motion detection, aiming to achieve the effect of detecting targets in complex environments without preset target types.

[0005] To achieve the above purpose, an optoelectronic pod target detection system based on abnormal motion detection adopted by the present invention includes an optical acquisition module, an abnormal motion detection module, a data fusion and decision-making module, and a target locking and tracking module. The optical acquisition module is respectively connected to the abnormal motion detection module and the data fusion and decision-making module, and the data fusion and decision-making module is respectively connected to the abnormal motion detection module and the target locking and tracking module;

[0006] The optical acquisition module is used to obtain optical information of the target area, including image data and thermal radiation data;

[0007] The abnormal motion detection module is used to detect potential abnormal motion targets with different motion directions and speeds from the background according to the image data, track the abnormal motion targets, and output target data;

[0008] The data fusion and decision-making module is used to fuse and process the target data and the thermal radiation data to judge the threat level of the target;

[0009] The target locking and tracking module is used to control the stable platform of the optoelectronic pod and adjust the pointing of the optical acquisition module according to the position, size, and movement direction parameters of a target with a high threat level, so that the target is always at the center of the field of view.

[0010] The optoelectronic pod target detection system based on abnormal motion detection further includes a human-machine interaction module, which is respectively connected to the data fusion decision-making module and the target locking and tracking module.

[0011] The human-machine interaction module is used to provide an operation platform for the operator.

[0012] The optical acquisition module includes a visible light camera unit and an infrared camera unit. The visible light camera unit is connected to the abnormal motion detection module, and the infrared camera unit is connected to the data fusion decision-making module.

[0013] The visible light camera unit is used to obtain high-resolution color images;

[0014] The infrared camera unit is used to capture the thermal radiation characteristics of the target.

[0015] The abnormal motion detection module includes a potential target unit and an abnormal target determination unit. The potential target unit is connected to the visible light camera unit, and the abnormal target determination unit is respectively connected to the potential target unit and the data fusion decision-making module.

[0016] The potential target unit is used to preprocess the continuously acquired two-frame image data, obtain the optical flow field, analyze the optical flow field, obtain the movement situation of each pixel in the image, classify similar and adjacent pixels according to the movement situation and position relationship of each pixel, and obtain potential abnormal motion targets with movement directions and speeds different from those of the background.

[0017] The abnormal target determination unit is used to track and predict the abnormal motion target, model the motion trajectory, motion speed, and motion acceleration characteristics of the target, and identify the target with abnormal motion by comparing with the normal motion pattern library.

[0018] An optoelectronic pod target detection system based on abnormal motion detection according to the present invention uses the optical acquisition module to obtain optical information of a target area, including image data and thermal radiation data; the abnormal motion detection module is used to detect potential abnormal motion targets with different motion directions and speeds from the background according to the image data, and track the abnormal motion targets to output target data; the data fusion decision module is used to perform fusion processing on the target data and the thermal radiation data to judge the threat level of the target; the target locking and tracking module is used to control the stable platform of the optoelectronic pod according to the position, size, and motion direction parameters of the target with a high threat level, and adjust the pointing of the optical acquisition module so that the target is always at the center of the field of view; the effect of being able to discover targets in a complex environment without presetting the target type is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a schematic diagram of the principle of the optoelectronic pod target detection system based on abnormal motion detection of the present invention.

[0021] Figure 2 is a schematic diagram of the process of the optoelectronic pod target detection system based on abnormal motion detection of the present invention.

[0022] 100 - Optical acquisition module, 101 - Visible light camera unit, 102 - Infrared camera unit, 200 - Abnormal motion detection module, 201 - Potential target unit, 202 - Abnormal target determination unit, 300 - Data fusion decision module, 400 - Target locking and tracking module, 500 - Human - machine interaction module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Please refer to Figure 1 and Figure 2 where Figure 1 is a schematic diagram of the principle of the optoelectronic pod target detection system based on abnormal motion detection, Figure 2 is a schematic diagram of the process of the optoelectronic pod target detection system based on abnormal motion detection.

[0024] The present invention provides an optoelectronic pod target detection system based on abnormal motion detection, which includes an optical acquisition module 100, an abnormal motion detection module 200, a data fusion and decision-making module 300, and a target locking and tracking module 400. The optical acquisition module 100 is respectively connected to the abnormal motion detection module 200 and the data fusion and decision-making module 300, and the data fusion and decision-making module 300 is respectively connected to the abnormal motion detection module 200 and the target locking and tracking module 400;

[0025] The optical acquisition module 100 is configured to acquire optical information of a target area, including image data and thermal radiation data;

[0026] The abnormal motion detection module 200 is configured to detect potential abnormal motion targets with different motion directions and speeds from the background according to the image data, track the abnormal motion targets, and output target data;

[0027] The data fusion and decision-making module 300 is configured to perform fusion processing on the target data and the thermal radiation data to judge the threat level of the target;

[0028] The target locking and tracking module 400 is configured to, for targets with a high threat level, control the stable platform of the optoelectronic pod according to the position, size, and motion direction parameters of the target, adjust the pointing of the optical acquisition module 100, and keep the target always at the center of the field of view.

[0029] In this embodiment, the optical acquisition module 100 acquires optical information of the target area, including image data and thermal radiation data. The abnormal motion detection module 200 detects potential abnormal motion targets with different motion directions and speeds from the background according to the image data, tracks the abnormal motion targets, and outputs target data. The data fusion and decision-making module 300 performs fusion processing on the target data and the thermal radiation data to judge the threat level of the target; comprehensively analyzes multi-dimensional information such as the appearance characteristics, thermal characteristics, and motion states of the target in multiple frames of images to judge the threat level of the target. According to the threat level, an alarm is sent to the operator, and the target information is transmitted to the command and control system to assist subsequent decision-making.

[0030] Among them, the appearance features are obtained by cropping the target and then classifying it using a multi-modal pre-trained classification model based on contrastive learning. This model is trained using the method of contrastive learning. During the training process, the correctly matched image and text pairs are used as positive samples, and the unmatched image and text pairs are used as negative samples. By maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs, the model learns the correlation between images and texts. For example, given a picture of a dog and the text description "a dog" as a positive sample, while using the picture of this dog and the text description "a car" as a negative sample. When classifying a new category, it is not necessary to retrain the data of the new category. Just input the text description and image of the new category into the model, and the model can determine whether the image belongs to this category based on the knowledge learned from pre-training.

[0031] The process of obtaining appearance features is as follows: First, according to the task requirements, set in advance the target types that need special attention or the target types that do not need to be concerned about. Crop the abnormal motion target and perform preprocessing, including operations such as scaling and normalization. Then input the processed image into the classification model to determine whether it is a category that needs attention or a category that does not need attention.

[0032] The thermal features are provided by the optical acquisition module 100. For some targets with known temperatures, such as the human body, this feature can be used to assist in judgment.

[0033] The target locking and tracking module 400, for high-threat-level targets, according to the position, size, and motion direction parameters of the target, controls the stable platform of the optoelectronic pod and adjusts the pointing of the optical acquisition module 100 so that the target is always at the center of the field of view; for high-threat targets determined by the data fusion and decision-making module, immediately start the target locking program. According to parameters such as the position, size, and motion direction of the target, control the stable platform of the optoelectronic pod and precisely adjust the pointing of the optical acquisition module 100 to keep the target always at the center of the field of view. At the same time, use a tracking algorithm based on deep learning to track and predict the target, and combine the subsequent collected data to correct the target trajectory in real time to ensure that even if the target has a short-term occlusion or maneuver, it can still be continuously locked and tracked. Through the above method, the effect of being able to discover targets in complex environments without presetting target types is obtained.

[0034] Furthermore, the optoelectronic pod target detection system based on abnormal motion detection further includes a human-computer interaction module 500, and the human-computer interaction module 500 is respectively connected to the data fusion and decision-making module 300 and the target locking and tracking module 400.

[0035] Furthermore, the human-computer interaction module 500 is used to provide an operation platform for the operator.

[0036] In this embodiment, the human-machine interaction module 500 provides an operation platform for the operator, and real-time displays the vision image of the optoelectronic pod, target annotation information, system status, alarm prompts, etc. The operator can manually intervene in the system as needed, such as adjusting detection parameters, switching sensor modes, and performing secondary confirmation on suspected targets, to achieve human-machine collaborative operation and improve the flexibility and reliability of the system.

[0037] Further, the optical acquisition module 100 includes a visible light imaging unit 101 and an infrared imaging unit 102. The visible light imaging unit 101 is connected to the abnormal motion detection module 200, and the infrared imaging unit 102 is connected to the data fusion decision module 300.

[0038] Further, the visible light imaging unit 101 is used to obtain high-resolution color images;

[0039] The infrared imaging unit 102 is used to capture the thermal radiation characteristics of the target.

[0040] In this embodiment, the visible light imaging unit 101 obtains high-resolution color images, providing rich appearance details of the target; the infrared imaging unit 102 captures the thermal radiation characteristics of the target, realizing detection under night and low-light conditions; the visible light imaging unit 101 and the infrared imaging unit 102 work together to collect optical information of the target area in all directions.

[0041] Further, the abnormal motion detection module 200 includes a potential target unit 201 and an abnormal target determination unit 202. The potential target unit 201 is connected to the visible light imaging unit 101, and the abnormal target determination unit 202 is respectively connected to the potential target unit 201 and the data fusion decision module 300.

[0042] Further, the potential target unit 201 is used to preprocess the continuously acquired two-frame image data, obtain the optical flow field, analyze the optical flow field, obtain the motion situation of each pixel in the image, and classify similar and adjacent pixels according to the motion situation and position relationship of each pixel to obtain potential abnormal motion targets with different background motion directions and speeds;

[0043] The abnormal target determination unit 202 is used to track and predict the abnormal motion target, model the motion trajectory, motion speed, and motion acceleration characteristics of the target, and identify the abnormally moving targets by comparing with the normal motion mode library.

[0044] In this embodiment, the potential target unit 201 incorporates an optical flow-based motion detection algorithm based on deep learning. Optical flow refers to the motion of pixels caused by the movement of objects in a scene or the movement of a camera between two consecutive frames of images.

[0045] In the training stage, the algorithm is trained by taking two annotated images as input and the corresponding optical flow field as output.

[0046] In the inference stage, first, for two consecutive frames collected, including the current frame and the previous frame, such as Figure 2 the first frame image and the second frame image shown, the image data is preprocessed, including operations such as image denoising, enhancement, and data normalization. Then, the processed images are fed into the motion detection algorithm to obtain the output optical flow field. After denoising the optical flow field and then analyzing it, the motion situation of each pixel in the image can be obtained. Then, based on the motion situation and position relationship of each pixel, similar and adjacent pixels are classified, and finally, potential abnormal motion targets with different motion directions and speeds from the background are obtained.

[0047] The abnormal target determination unit 202 uses a deep learning-based tracking algorithm to track and predict all potential abnormal targets that appear, and models features such as the motion trajectory, motion speed, and motion acceleration of the target in subsequent video frames, such as Figure 2 the third frame image to the Nth frame image shown, and identifies the targets with abnormal motion by comparing with the normal motion pattern library. For various complex scenarios, the algorithm can adapt to complex motion changes in different scenarios by updating the normal motion pattern library.

[0048] For the key item speed in the target motion mode, it can usually be calculated from the motion speed of the flight platform. For example: The drone flies from south to north at a speed of 5M / S and looks down at 90°. At this time, the reference objects on the ground, such as trees, small buildings, etc., move from north to south relative to the drone at a speed of 5M / S. By analyzing the optical flow field, assuming that the optical flow intensity of the background is 1 at this time, the speed of the optical flow intensity of unit 1 is 5M / S. If it is assumed that a target with a motion speed exceeding 10M / S is an abnormal target, then in the optical flow field, a target with an optical flow intensity greater than 2 is an abnormal motion target.

[0049] The above-disclosed are only one or more preferred embodiments of the present invention. Of course, the scope of the rights of the present invention cannot be limited by this. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. An optoelectronic pod target detection system based on abnormal motion detection, characterized in that: It includes an optical acquisition module, an abnormal motion detection module, a data fusion decision module and a target locking and tracking module, wherein the optical acquisition module is connected to the abnormal motion detection module and the data fusion decision module respectively, and the data fusion decision module is connected to the abnormal motion detection module and the target locking and tracking module respectively; The optical acquisition module is used to obtain optical information of the target area, including image data and thermal radiation data; The abnormal motion detection module is used to detect potential abnormal motion targets with different motion directions and speeds from the background according to the image data, and to track the abnormal motion targets and output target data; The data fusion decision module is used to fuse the target data and the thermal radiation data to determine the threat level of the target; The target locking and tracking module is used to control the stable platform of the optoelectronic pod and adjust the direction of the optical acquisition module according to the position, size and movement direction parameters of the target with a high threat level, so that the target is always in the center of the field of view.

2. The optoelectronic pod target detection system based on abnormal motion detection according to claim 1, characterized in that: The optoelectronic pod target detection system based on abnormal motion detection also includes a human-computer interaction module, which is connected to the data fusion decision module and the target locking and tracking module respectively.

3. The optoelectronic pod target detection system based on abnormal motion detection as claimed in claim 2, characterized in that: The human-computer interaction module is used to provide an operating platform for operators.

4. The optoelectronic pod target detection system based on abnormal motion detection according to claim 1, characterized in that: The optical acquisition module includes a visible light camera unit and an infrared camera unit. The visible light camera unit is connected to the abnormal motion detection module, and the infrared camera unit is connected to the data fusion decision module.

5. The optoelectronic pod target detection system based on abnormal motion detection as claimed in claim 4, characterized in that: The visible light imaging unit is used to obtain a high-resolution color image; The infrared camera unit is used to capture the thermal radiation characteristics of the target.

6. The photoelectric pod target detection system based on abnormal motion detection as claimed in claim 4, characterized in that: The abnormal motion detection module includes a potential target unit and an abnormal target determination unit. The potential target unit is connected to the visible light camera unit, and the abnormal target determination unit is respectively connected to the potential target unit and the data fusion decision module.

7. The optoelectronic pod target detection system based on abnormal motion detection according to claim 6, characterized in that: The potential target unit is used to pre-process the collected two consecutive frames of image data, obtain the optical flow field, analyze the optical flow field, obtain the motion of each pixel in the image, and classify similar and close pixels according to the motion and position relationship of each pixel to obtain potential abnormal motion targets with different motion directions and speeds from the background; The abnormal target determination unit is used to track and predict abnormal moving targets, model the target's motion trajectory, motion speed, and motion acceleration characteristics, and identify the abnormal moving targets by comparing them with a normal motion pattern library.

Images