A high-speed small target extraction method using an infrared event stream

Abstract

The application relates to a high-speed small target extraction method using an infrared event stream, which obtains infrared event stream information by processing adjacent frames of images acquired by a high-frame-frequency infrared camera, and detects high-speed moving target information in the infrared event stream information by using the characteristic that the infrared event stream information is sensitive to moving targets. The original infrared images are subjected to frame extraction processing, the target position in the original image frame is located by using the detected moving target information, local target characteristics are extracted for processing, and the target is confirmed and recognized. The method avoids the problem that the calculation complexity is high when target detection is performed on the original high-frame-frequency infrared images, uses the obtained event frame image which only contains the moving target information of the coefficient, removes a large amount of redundant useless background and static information, reduces the calculation amount, improves the detection sensitivity of high-speed infrared moving small targets, simultaneously reduces the energy consumption of the system, and can be widely applied to infrared detection systems with limited processing resources.

Description

Technical fields:

[0001] This invention belongs to the field of infrared remote sensing technology, and in particular relates to a method for high-speed small target extraction using infrared event streams. Background Technology

[0002] Because space-based detection targets are extremely small in scale and have long detection distances, the energy intensity captured during detection is very weak. Effective information is often drowned out by background information and noise from the detection devices themselves. Therefore, when detecting these "faint" targets, false alarms, missed detections, and mistracking are highly likely, making it difficult to guarantee the stability and reliability of the detection system. Traditional space-based detection methods utilize time-interval triggering to continuously acquire infrared images of the current moment and analyze these images in real time to achieve long-term, uninterrupted detection and tracking of "faint" targets. Traditional methods require processing large amounts of redundant background information from massive amounts of visual data over extended periods, resulting in significant computational resource and time consumption, placing a heavy load on computing and storage units, and generating useless energy and computing power consumption. In traditional methods, to effectively identify "weak" high-speed targets at ultra-long distances (above 2000 km), an extremely high detector pixel resolution (1K+) and high frame rate (100Hz+) sampling are required to acquire images. If deep neural networks are used to process the images, the on-chip processing unit needs to have a computing power of about 600 TFLOPS (using a ResNet50 network to process 100 1024×1024 grayscale images in 1 second). Existing technologies cannot achieve such processing power on a system-on-a-chip. Summary of the Invention:

[0003] The purpose of this invention is to overcome the above-mentioned shortcomings of the prior art and provide a high-speed small target extraction method using infrared event streams.

[0004] The above-mentioned objective of the present invention is achieved through the following technical solution:

[0005] A method for high-speed small target extraction using infrared event streams, characterized in that the method includes the following steps:

[0006] 1) By using a high frame rate infrared detector, continuous frame images are converted into infrared event frames to achieve differential detection in the time dimension;

[0007] 2) Utilize the sparse nature of infrared event stream information to deploy a convolutional neural network;

[0008] 3) Perform frame extraction on high frame rate infrared image frames proportionally and preprocess the original image.

[0009] 4) Based on the target area obtained after analyzing infrared event information, local slice extraction is performed in the preprocessed image;

[0010] 5) Perform target identification and feature extraction on the regions extracted from the local slices;

[0011] 6) Identify target type based on the continuously acquired target time-series features.

[0012] Furthermore, in step 1), the high frame rate infrared detector has a frame rate range of 10Hz to 1000Hz; the conversion of continuous frames into infrared event frames refers to subtracting frames at fixed intervals, with the number of interval frames being a preset value.

[0013] Furthermore, the convolutional neural network in step 2) refers to a neural network constructed through pre-training that outputs the center position, length, width, and confidence level of the target.

[0014] Furthermore, in step 3), proportional frame extraction refers to extracting one frame of image at fixed intervals, with the interval parameter being a preset value; the preprocessing includes non-uniform correction and blind pixel compensation.

[0015] Furthermore, in step 4), the slice extraction has a slice size of N*M, where N and M are preset values.

[0016] Furthermore, the extraction in step 5) refers to recalculating the target confidence and location center using methods such as local contrast and local template matching. If the confidence is less than the threshold, it is considered an incorrect target and is discarded. The acquired features include target grayscale, target peak value, target size, and target image velocity.

[0017] Furthermore, the target type identification in step 6) refers to classifying the target type using the continuously acquired target feature sequence, and the methods used include neural networks, support vector machines, and linear classifiers. This invention has the following advantages compared to existing technologies:

[0018] (1) This invention utilizes the event stream mechanism to perform time-domain differential processing on directly acquired infrared images, thereby reducing system processing load and improving system processing performance. By deploying infrared detectors and utilizing the time dimension information of continuous frames, the traditional infrared images are converted into infrared event stream information. The sparsity characteristics of the event stream are used for effective processing, and the events are monitored in real time.

[0019] (2) Compared with traditional infrared image processing methods, the present invention has higher sensitivity to high-speed targets, less data processing and lower energy consumption; when the event stream information conversion is completed, a large amount of redundant information can be filtered out, and its data structure only contains the pixel coordinates of the event trigger and the trigger polarity (x, y, p), where x and y represent the row and column coordinates of the pixel triggering the event, and p can take one of (-1, 0, +1) to represent the triggering direction of the event, which will greatly reduce the amount of data processed.

[0020] (3) The present invention can take into account both the processing power and detection frame rate of the ultra-high resolution camera system; by using the infrared event stream information obtained after conversion and analyzing it, after finding the sensitive and suspicious target area, it can guide the infrared camera to perform effective frame selection to acquire part of the data readout and transmission in real time, thereby achieving effective complementarity between processing power and detection frequency. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the overall process of the present invention.

[0022] Figure 2 A comparison chart of the effects of event images and traditional infrared images.

[0023] Figure 3 Architecture diagram of a custom single-bit convolutional neural network Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments:

[0025] Figure 1 This is a flowchart illustrating the overall process for extracting small moving targets using high-speed infrared technology. The entire process is divided into three stages:

[0026] Phase 1:

[0027] An infrared camera is used to continuously detect information within space, and the acquired output frame rate sequences are grouped into N sets, where N is an adjustable parameter. Data transmission and processing are performed simultaneously in two modes:

[0028] The first mode is frame-by-frame acquisition, which extracts the (N / 2)th image from a sequence of (1,2,3,...,N) consecutive images and transmits it into the information processing unit for preprocessing. If N / 2 is not an integer, it is rounded up.

[0029] The second mode is event frame acquisition. Each time the data is acquired, the data sequences of the T+1 frame and the T frame are sent to the digital comparator logic, and the outputs +1, 0, -1, single-byte information are stored.

[0030] Figure 2This is a comparison chart of the effects of event images and traditional infrared images.

[0031] Phase Two:

[0032] The acquired single-byte event frame information is processed by a custom single-bit convolutional neural network, and finally the bounding box information of the target position is output in the form of (X,Y,W,H), where (X,Y) represents the pixel position of the upper left corner of the bounding box, and W and H represent the width and height of the bounding box, respectively. Figure 3 This is a diagram of the structure of a custom single-bit convolutional neural network.

[0033] Phase Three:

[0034] Based on the bounding box information (X, Y, W, H) obtained in the previous stage, spatial target confirmation is performed on the preprocessed frame-sampling data. The original image is then matted to obtain only the image information within the bounding box. Neural networks, support vector machines, and linear classifiers are used to confirm, identify, and classify the target information within the bounding box, and the target confirmation and recognition results are output. If a target is confirmed, the radiation information within the target area is also output. Finally, the target radiation information is analyzed and fused with the target morphology information to quickly and effectively obtain multi-dimensional fused information after the target appears.

[0035] Phase Four:

[0036] The acquired target spatial location information, radiation information, and morphological information are packaged and transmitted to the host computer for storage, enabling real-time and accurate detection, identification, and tracking of high spatial resolution targets without frequency attenuation.

[0037] The above description is only the best specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

[0038] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A method for high-speed small target extraction using infrared event streams, characterized in that, The method includes the following steps: 1) By using a high frame rate infrared detector, continuous frame images are converted into infrared event frames to achieve differential detection in the time dimension; 2) Utilize the sparse nature of infrared event stream information to deploy a convolutional neural network; 3) Perform frame extraction on high frame rate infrared image frames proportionally and preprocess the original image; 4) Based on the target area obtained after analyzing infrared event information, local slice extraction is performed in the preprocessed image; 5) Perform target identification and feature extraction on the regions extracted from the local slices; 6) Identify target type based on continuously acquired temporal features; The high frame rate infrared detector in step 1) has a frame rate range of 10Hz to 1000Hz; the conversion of continuous frames into infrared event frames refers to subtracting at fixed intervals, with the number of intervals being a preset value. The convolutional neural network in step 2) refers to a neural network constructed through pre-training to output the center position, length, width, and confidence score of the target. Step 3) involves proportional frame extraction, which means extracting one frame of the image at fixed intervals, with the interval parameter being a preset value; the preprocessing includes non-uniformity correction and blind pixel compensation. The extraction in step 5) refers to recalculating the target confidence and location center using local contrast or local template matching methods. If the confidence is less than the threshold, it is considered an incorrect target and is discarded. The acquired features include target grayscale, target peak value, target size, and target image velocity.

2. The high-speed small target extraction method using infrared event streams as described in claim 1, characterized in that, In step 4), the slice extraction has a slice size of N*M, where N and M are preset values.

3. The high-speed small target extraction method using infrared event streams as described in claim 1, characterized in that, Step 6) Target type identification refers to classifying the type of target using the continuously acquired target feature sequence. The methods used include neural networks, support vector machines, or linear classifiers.

Images