An adaptive detection method for small infrared targets based on FPGA and DSP

Abstract

The present invention discloses an infrared small target adaptive detection method based on FPGA and DSP, belonging to the field of infrared real-time detection technology. The method comprises the following steps: first, the FPGA caches an input original infrared grayscale image, and filters the image by using curvature filtering and top-hat transformation to obtain a filtered image; second, the FPGA performs adaptive threshold segmentation on the filtered image to generate a binary image, and compresses the image into a single-bit image data stream to transmit to the DSP; third, the DSP performs connected domain detection on the single-bit image data stream to obtain a connected domain detection result; finally, the DSP dynamically adjusts the adaptive threshold segmentation coefficient of the FPGA according to the connected domain detection result, and uses the coefficient for threshold segmentation of the next frame of image.

Description

Technical Field

[0001] The invention belongs to the technical field of infrared real-time detection, in particular to an infrared small target adaptive detection method based on FPGA and DSP. Background Art

[0002] With the development of national defense science and technology, infrared detection systems, characterized by passive operation, anti-interference capabilities, strong target recognition capabilities, and all-weather operation, have become widely used in infrared early warning, guidance, and surveillance. Infrared detection systems utilize infrared detectors to process and analyze acquired image data, extract target information, and coordinate with servo systems for stable tracking. Currently, there is extensive research both domestically and internationally on infrared detection algorithms, including traditional detection algorithms and those based on neural networks. Furthermore, there are also significant achievements in infrared detection systems, focusing primarily on the system's detection, tracking, and real-time processing capabilities.

[0003] The infrared detection system needs to accurately locate the target orientation and control the servo system to keep the target in the field of view at all times. This requires the system to be able to process image and target information in real time to ensure that the servo system can be adjusted to the specified angle in a timely manner.

[0004] Current infrared detection systems primarily utilize embedded architectures. Embedded circuits can meet military system requirements for ambient temperature and other parameters, while also offering small size, high integration, and strong processing capabilities. FPGAs, with their parallel computing capabilities, are generally used for image preprocessing, while DSPs employ serial processing for target detection. Existing FPGA and DSP technical solutions suffer from challenges such as difficulty implementing complex detection algorithms on FPGAs, DSP processing time consumption that fails to meet real-time system requirements, and significant variations in detection performance across different scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the fact that existing patents for image target detection primarily focus on optimizing image target detection algorithms, with little consideration given to the real-time performance of implementing detection algorithms on embedded platforms. Furthermore, existing embedded infrared target detection systems generally utilize an FPGA plus DSP implementation. If hardware resources are not properly allocated, either the FPGA or the DSP alone will handle a large number of operations, wasting system performance. This invention proposes an adaptive infrared small target detection method based on an FPGA and a DSP. This system rationally allocates and fully utilizes the processing resources of the FPGA and DSP, reducing the DSP's target detection time, meeting the system's real-time requirements, and effectively improving target detection efficiency.

[0006] The specific technical solution of the present invention is: an adaptive detection method for infrared small targets based on FPGA and DSP, characterized by comprising the following steps:

[0007] Step 1: FPGA caches the input original infrared grayscale image and filters the image using curvature filtering and top-hat transformation to obtain the filtered image, and then proceeds to step 2.

[0008] Step 2: FPGA performs adaptive threshold segmentation on the filtered image to generate a binary image, which is then compressed into a single-bit image data stream and transmitted to DSP, and then proceeds to step 3.

[0009] Step 3: DSP performs connected domain detection on the single-bit image data stream, obtains the connected domain detection result, and then proceeds to step 4.

[0010] Step 4: DSP dynamically adjusts the adaptive threshold segmentation coefficient of FPGA according to the connected domain detection result, which is used for the threshold segmentation of the next frame image.

[0011] The effects of the present invention are as follows: 1) compared with detecting a binary image, the method of performing connected domain detection on a single-bit image data stream by the DSP greatly reduces the amount of data transmitted from the FPGA to the DSP and reduces the time it takes for the DSP to perform connected domain detection; 2) compared with ordinary gradient features, the added curvature feature can improve the distinction between the target and the background and reduce the false alarm rate; 3) the adaptive threshold segmentation method can adapt to different detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The present invention discloses an infrared small target adaptive detection method based on FPGA and DSP, which includes the main processing flow of FPGA.

[0013] Figure 2 The present invention discloses an infrared small target adaptive detection method based on FPGA and DSP, and a main processing flow of DSP. DETAILED DESCRIPTION

[0014] The present invention is further described in detail below with reference to the accompanying drawings.

[0015] Combine Figure 1-2 The present invention proposes an adaptive detection method for infrared small targets based on FPGA and DSP, wherein the main processing flow of FPGA is as follows: Figure 1 As shown, it specifically includes the following steps:

[0016] Step 1: The FPGA caches the input original infrared grayscale image. Due to the large curvature variation between the infrared small target and the background, the curvature filter is used to extract the target. The top-hat transform is used to enhance the features of the bright target against the dark background, making it easier to extract the target. The process then proceeds to step 2.

[0017] Step 11: Cache the original infrared grayscale image information into RAM. Considering the real-time processing and the size of the small infrared target, a 5*5 template is selected for curvature filtering. The filtering calculation needs to be performed after caching every 5 lines of image data.

[0018] Step 12. Since the facet model in the curvature calculation model has the effect of suppressing the background and enhancing the target, the curvature of the pixel point of the image is calculated using the 5*5 template of the facet model. That is, the second-order derivatives of 0°, 45°, 90° and 135° are calculated using 5 lines of original infrared grayscale image data, respectively, as the curvature values ​​C0, C45, C90, C135 in the following four directions. The minimum value of these four values ​​Cmin=min{C0,C45,C90,C135} is taken as the curvature feature of the point to obtain the curvature image;

[0019] Step 13: Cache the original infrared grayscale image information into RAM. Also considering the real-time processing and the size of the small infrared target, the top-hat transformation uses a 3*3 template. It is necessary to complete the top-hat calculation after caching 3 lines of image data.

[0020] Step 14: perform an erosion operation on the three lines of cached original image data, and then perform an expansion operation on the three lines of eroded image data. Finally, subtract the expanded data from the original infrared grayscale image data to obtain a top-hat transformed image. The curvature image and the top-hat transformed image constitute the filtered image.

[0021] Step 2: FPGA performs adaptive threshold segmentation on the filtered image to generate a binary image, which is then compressed into a single-bit image data stream and transmitted to DSP, and then proceeds to step 3.

[0022] Step 21: Since the curvature filtering and top-hat transformation processing times of the FPGA are different, it is necessary to delay the obtained curvature image and top-hat transformation image to the same starting time, and then multiply each pixel information of the two to obtain the image information to be thresholded;

[0023] Step 22: Utilizing the parallel processing characteristics of FPGA, while multiplying the curvature image and the top-hat transformed image, the product of each pixel is accumulated. After the product of the entire image is processed, the cumulative sum of all pixels is obtained, and then the mean and standard deviation of the entire image are obtained.

[0024] Step 23: cache the image to be segmented by the threshold value, and then perform binary segmentation on the image according to the threshold value Th calculated by the threshold calculation formula. Values ​​greater than Th are set to 255, indicating that there is target information, and values ​​less than Th are set to 0, indicating that there is no target information, thereby obtaining a binary image;

[0025] The threshold calculation formula is: Th=μ+k*σ, where Th is the threshold, μ is the mean of the entire image, σ is the standard deviation of the entire image, and k is the adaptive threshold coefficient;

[0026] Step 24: The depth of a single pixel of the obtained binary image is 8 bits. In order to reduce the amount of data transmitted from FPGA to DSP, improve transmission efficiency and save processing time, the depth of a single pixel of the binary image is compressed to 1 bit, that is, 1 represents target information and 0 represents no target information. The memory that originally stored 1 pixel information can now store 8 pixels of information. This compressed binary single-bit image data stream is sent to DSP for processing.

[0027] The main processing flow of DSP is as follows Figure 2 As shown, it specifically includes the following steps:

[0028] Step 3: DSP performs connected domain detection on the binary single-bit data stream;

[0029] Step 31: The data depth of a single DSP processing is 32 bits. After receiving the binary single-bit image data stream of the image, a judgment is made every 32 pixels. If the 32 bits of data are all 0, it is considered that there is no target and the process is skipped. If the 32 bits of data are not 0, it is considered that there is target information. Then find the point where the data is 1 and deduce the position information of the point.

[0030] Step 32: After finding the point location information with data of 1, DSP uses the region growing method to determine whether there are points with data of 1 in the 8-neighborhood. All points in the same region are recorded as the same count value. Every time a new connected region is formed, the count value increases by 1 until the binary single-bit image data stream is traversed. The count value is the number of connected domains.

[0031] Step 4: The DSP dynamically adjusts the FPGA's adaptive threshold segmentation coefficient based on the connected domain detection results. A large segmentation coefficient can eliminate darker targets, leading to missed alarms. A small segmentation coefficient can detect brighter backgrounds, leading to false alarms. The threshold segmentation coefficient varies for different scenarios: simple scenarios use a small segmentation coefficient, while complex scenarios require a larger one. The DSP counts the number of connected domains. If the number is outside the set range, it adjusts the adaptive threshold coefficient k. For a large number of connected domains, the k value increases; for a small number, the k value decreases.

[0032] The present invention will be described in further detail below with reference to examples.

[0033] Examples:

[0034] The following uses the "An Infrared Small Target Adaptive Detection Method Based on FPGA and DSP" proposed in this patent to extract target information. The specific processing steps are as follows:

[0035] Infrared image information is input into the detection system according to the image clock. The FPGA stores the image grayscale values ​​of 5 consecutive rows through 5 RAM buffers, and uses 25 registers to cache the grayscale values ​​of 5 consecutive pixels corresponding to each row to obtain a 5×5 image grayscale data matrix. It is then multiplied with the curvature calculation matrix of the second-order derivative in 4 directions, and the minimum value of the four product results is used as the curvature value of the pixel point at the last position of the 5×5 matrix.

[0036] Similarly, during the top-hat operation, the FPGA uses three RAM buffers to store the grayscale values ​​of three consecutive rows of images, and uses nine registers to cache the grayscale values ​​of three consecutive pixels corresponding to each row to obtain a 3×3 image grayscale matrix. The minimum grayscale value in the matrix is ​​assigned to the last pixel of the 3×3 matrix. According to this operation, an eroded image is formed; the eroded image is passed through three RAM buffers, and nine registers are used to cache the erosion results of three consecutive pixels corresponding to each row to obtain a 3×3 erosion result matrix. The maximum erosion result in the matrix is ​​assigned to the last pixel of the 3×3 matrix. According to this operation, the open operation is completed; then the original image is delayed, kept at a starting time with the open operation image, and the open operation is subtracted to obtain the result of the top-hat operation.

[0037] The curvature filtering and top-hat operation results are delayed to the same starting time, and the dot product is performed to obtain the image to be thresholded. Once the entire image is processed, the mean and variance of the image to be thresholded are immediately obtained. Using the threshold calculation formula Th = μ + k * σ, the grayscale value of pixels exceeding Th is set to 255, and the grayscale value of pixels below Th is set to 0, forming a binary image. All of these operations are performed in the FPGA. From the time the image enters the system to the generation of the binary image, a total of two image frames are consumed, ensuring real-time performance.

[0038] After obtaining the binary image, the FPGA compresses the 8-bit data into 1 bit, representing the binarization result of each pixel with 1 bit, forming a single-bit binary image. The FPGA transmits this single-bit binary image data stream to the DSP via SRIO. The DSP performs connected domain detection on the binary single-bit image data stream, determining whether there are points with a binary result of 1 for every 32 bits (i.e., 32 pixels). Then, using an 8-neighborhood growth method, it connects adjacent points with a binary result of 1 to obtain the connected domain result. Compared to the traditional DSP connected domain detection of the original binary image, this method reduces the time required to perform connected domain detection on 32 pixels at a time by 1 / 32, significantly saving DSP processing time.

[0039] DSP then dynamically adjusts the adaptive threshold coefficient k in the threshold calculation formula according to the number of connected domains detected. It can adaptively retain target points according to different scenarios, reduce background points, and improve target detection efficiency.

[0040] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An adaptive detection method for infrared small targets based on FPGA and DSP, characterized in that: The following steps are involved: Step 1: FPGA caches the input original infrared grayscale image and filters the image using curvature filtering and top-hat transformation. That is, the curvature image and the top-hat transformation image are delayed to the same starting time, and then the pixel information of each of the two is multiplied to obtain the filtered image, and then go to step 2. Step 2: FPGA performs adaptive threshold segmentation on the filtered image to generate a binary image, which is then compressed into a single-bit image data stream and transmitted to DSP, and then proceeds to step 3. Step 3: DSP performs connected domain detection on the single-bit image data stream, obtains the connected domain detection result, and proceeds to step 4; Step 4: DSP dynamically adjusts the adaptive threshold segmentation coefficient of FPGA according to the connected domain detection result, which is used for the threshold segmentation of the next frame image; Among them, DSP dynamically adjusts the adaptive threshold segmentation coefficient of FPGA according to the results of connected domain detection, including DSP counting the number of all connected domains. If the number is not within the set range, the adaptive threshold coefficient k is adjusted. When the number of connected domains is large, the k value increases, and when the number of connected domains is small, the k value decreases.

2. The FPGA and DSP-based adaptive infrared small target detection method according to claim 1 is characterized in that: In step 1, the FPGA performs curvature filtering and top-hat transformation on the input original infrared grayscale image, including the following steps: Step 11: Cache the original infrared grayscale image information into RAM. After caching 5 lines of image data, start the curvature filter calculation. Step 12: Use the 5*5 template of the facet model to calculate the curvature of the pixel point of the image. That is, use 5 lines of original infrared grayscale image data to calculate the second-order derivatives of 0°, 45°, 90° and 135° respectively, as the curvature values ​​of the following four directions C0, C45, C90, C135, and take the minimum value of these four values ​​Cmin=min{C0,C45,C90,C135} as the curvature feature of the point to obtain the curvature image; Step 13: Cache the original infrared grayscale image information into RAM. After caching 3 lines of image data, start the top hat transform calculation. Step 14: perform an erosion operation on the three lines of cached original image data, and then perform a dilation operation on the three lines of eroded image data. Finally, subtract the dilated data from the original infrared grayscale image data to obtain a top-hat transformed image. The curvature image and the top-hat transformed image constitute the filtered image.

3. The FPGA and DSP-based adaptive infrared small target detection method according to claim 2 is characterized in that: In step 2, the FPGA performs adaptive threshold segmentation on the filtered image to generate a binary image, and compresses it into a single-bit image data stream and transmits it to the DSP, including the following steps: Step 21: Delay the curvature image and the top-hat transformed image to the same starting time, and then multiply each pixel information of the two to obtain the image information to be threshold segmented; Step 22: While multiplying the curvature image and the top-hat transformed image, accumulate the product of each pixel. After the product of the entire image is processed, the cumulative sum of all pixels is obtained, and then the mean and standard deviation of the entire image are obtained. Step 23: cache the image to be segmented by the threshold value, and then perform binary segmentation on the image according to the threshold value Th calculated by the threshold calculation formula. Values ​​greater than Th are set to 255, indicating that there is target information, and values ​​less than Th are set to 0, indicating that there is no target information, to obtain a binary image; The threshold calculation formula is: Th=μ+k*σ, where Th is the threshold, μ is the mean of the entire image, σ is the standard deviation of the entire image, and k is the adaptive threshold coefficient; Step 24: The obtained binary image has a single pixel depth of 8 bits, which is compressed to 1 bit, that is, 1 represents target information and 0 represents no target information. The memory that originally stored 1 pixel information can now store 8 pixels of information. This compressed binary single-bit image data stream is sent to the DSP for processing.

4. The FPGA and DSP-based adaptive infrared small target detection method according to claim 3 is characterized in that: In step 3, the DSP performs connected domain detection on the binary single-bit image data stream, including the following steps: Step 31: The data depth of a single DSP processing is 32 bits. After receiving the binary single-bit image data stream of the image, a judgment is made every 32 pixels. If the 32 bits of data are all 0, it is considered that there is no target and the process is skipped. If the 32 bits of data are not 0, it is considered that there is target information. Then find the point where the data is 1 and infer the location information of the point. Step 32: After finding the point location information with data of 1, DSP uses the region growing method to determine whether there are points with data of 1 in the 8-neighborhood. All points in the same region are recorded as the same count value. Every time a new connected region is formed, the count value increases by 1 until the single-bit image data stream is traversed. The count value is the number of connected domains.

Images