Method for identifying mass center of infrared image in real time based on FPGA (Field Programmable Gate Array)

Abstract

The invention relates to an infrared image processing method, in particular to a method for recognizing the mass center of an infrared image in real time based on an FPGA, and the method comprises the steps: S1, collecting an original image of a target; s2, removing salt and pepper noise in the original image through median filtering; s3, performing secondary filtering on the image through Gaussian filtering; s4, performing Sobel edge detection on the target image to obtain a target contour of the target; s5, obtaining coordinates of target contour pixel points on the leftmost side, the rightmost side, the lowermost side and the uppermost side of the target in combination with the target contour obtained in the step S4 and the target in the target image; and S6, according to the coordinates of the target contour pixel points on the leftmost side, the rightmost side, the lowermost side and the uppermost side, calculating and outputting a centroid coordinate. According to the invention, the mass center of the target in the image can be obtained with a small calculation amount, and the method is not affected by the image quality.

Description

Technical Field

[0001] The invention relates to an infrared image processing method, and in particular to a method for real-time identification of infrared image centroids based on FPGA. Background Art

[0002] As a high-performance infrared imaging device, infrared focal plane detectors have demonstrated their unique advantages and application value in many fields. In military equipment and aerospace equipment (such as infrared warning equipment, infrared night vision equipment, laser warning equipment, etc.), infrared focal plane detectors play an irreplaceable role and greatly improve combat effectiveness and safety. At the same time, in the fields of industrial defect detection and medical infrared inspection, infrared focal plane detectors also provide strong support for industrial production and medical diagnosis with their non-contact and high sensitivity.

[0003] When a target appears in the field of view of an infrared device, how to accurately and quickly identify key information such as the target's position, size, and moving speed becomes an important challenge for infrared detection technology. To this end, infrared image processing technology came into being, which mainly includes three key steps: preprocessing, feature extraction, and target classification.

[0004] Hough transform is a commonly used infrared image processing technology. However, when processing images with higher resolution, its calculation complexity is higher, and its processing effect depends largely on the quality of the image. If the image has more noise, the Hough transform cannot accurately monitor the shape and center of mass of the target. Summary of the invention

[0005] The purpose of the present invention is to solve the technical problems that the existing Hough transform image processing technology has high computational complexity when processing images and cannot accurately monitor the shape and center of mass of the target when there is a lot of image noise, and to provide a method for real-time identification of the center of mass of infrared images based on FPGA.

[0006] To solve the above technical problems, the technical solutions provided by the present invention are as follows: A method for real-time identification of infrared image centroid based on FPGA, comprising the following steps: S1, turn on the infrared focal plane detector, move the target into the field of view and collect the original image of the target; S2, removing salt and pepper noise in the original image by median filtering to obtain a preliminary filtered image; S3, performing secondary filtering on the image after preliminary filtering in step S2 by Gaussian filtering to obtain a final target image; S4, performing Sobel edge detection on the target image obtained in step S3 to obtain the target contour of the target; S5. Combining the target contour obtained in step S4 and the target in the target image, obtain the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target, and record them as ; S6, calculating the centroid coordinates Z (R, C) according to the coordinates of the leftmost, rightmost, bottommost and topmost target contour pixel points in step S5 and outputting them; in, .

[0007] Furthermore, step S2 is specifically as follows: S2.1, open a 3×3 grid data window in the field of view through FPGA; S2.2, arranging pixel values ​​in each window of the 3×3 grid data window; S2.3, taking the median value in the arrangement to replace the pixel value in the middle window of the 3×3 grid data window; S2.4, move the 3×3 grid data window, traverse the original image and repeat steps S2.2 to S2.3 to complete the removal of salt and pepper noise in the original image and obtain the image after preliminary filtering.

[0008] Furthermore, step S3 is specifically as follows: S3.1, set the size of the Gaussian kernel to 3×3, and open a 3×3 Gaussian kernel data window in the field of view through FPGA; S3.2, setting corresponding Gaussian kernel coefficients for each pixel value in the Gaussian kernel data window; S3.3, performing convolution on the pixel values ​​in the Gaussian kernel data window and the corresponding Gaussian kernel coefficients, and recording the sum as E; S3.4, sum the Gaussian kernel coefficients, denoted as S; S3.5, take E / S as the pixel value of the middle window of the Gaussian kernel data window; S3.6. Move the 3×3 Gaussian kernel data window, traverse the image and repeat steps S3.2 to S3.5, perform secondary filtering on the image after preliminary filtering in step S2 to obtain the final target image.

[0009] Furthermore, step S4 is specifically as follows: S4.1, set the size of the convolution kernel to 3×3, and open a 3×3 convolution kernel data window in the field of view through FPGA; S4.2, respectively setting a corresponding row direction convolution factor and a column direction convolution factor for each pixel value in the convolution kernel data window; S4.3, convolve and sum the pixel values ​​in the convolution kernel data window with the corresponding row direction convolution factors and column direction convolution factors, respectively, to obtain row direction gradient data Gx and column direction gradient data Gy; S4.4. Calculate the gradient change rate S based on the row direction gradient data Gx and the column direction gradient data Gy: ; When the gradient change rate S of a pixel point in the target image is greater than the threshold SetV, the pixel point is recorded as a target contour pixel point and marked as 1; otherwise, the pixel point is marked as 0; S4.5. Move the 3×3 convolution kernel data window, traverse the image and repeat steps S4.2 to S4.4 to complete the Sobel edge detection of the target image; at the same time, remove the pixels marked as 0 to obtain the target contour composed of the target contour pixels.

[0010] Furthermore, in step S4.4, the threshold , select a value from the set as the threshold, where X represents the data bit width of the pixel.

[0011] Furthermore, step S5 is specifically as follows: S5.1. Extract the pixel coordinates of the leftmost, rightmost, bottommost and topmost pixels of the target in the target image and record them as ; S5.2. Extract the coordinates of the leftmost, rightmost, bottommost and topmost target contour pixel points of the target contour and record them as ; S5.3 If ,Will As the leftmost pixel coordinate of the target; otherwise, As the leftmost pixel coordinate of the target; like ,Will As the rightmost pixel coordinate of the target; otherwise, As the rightmost pixel coordinate of the target; like ,Will As the coordinates of the pixel point at the bottom of the target; otherwise, As the coordinates of the pixel point at the bottom of the target; like ,Will As the coordinates of the top pixel of the target; otherwise, As the coordinates of the top pixel of the target; Update the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target, and record them as .

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The method for real-time identification of infrared image centroid based on FPGA provided by the present invention removes salt and pepper noise in the image by performing median filtering on the original image collected by the infrared focal plane detector, so that the image is more uniform and has no prominent noise, thereby avoiding the problem of large deviation of the centroid position; the image is then filtered twice by Gaussian filtering to make the values ​​of the image pixels more reliable and prevent the deviation of the centroid detection; Sobel edge detection is performed on the target image after filtering to obtain the target contour of the target, and the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target are obtained to accurately calculate and output the centroid coordinates; moreover, the present invention mainly obtains the centroid position of the target through the geometric characteristics of the target. Compared with the Hough transform processing method, the method for identifying the centroid of the infrared image of the present invention has a small amount of calculation, can be implemented by relying on an FPGA chip, and can still calculate the centroid position well for irregular images.

[0013] 2. The method for real-time identification of infrared image centroid based on FPGA provided by the present invention can calculate the target centroid in the image in real time, and can be applied to occasions with high real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a target image in an embodiment of the present invention; Figure 2 It is an edge recognition image after image processing in an embodiment of the present invention; Figure 3 The centroid logic diagram is output for FPGA processing in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The technical solutions in the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] The method for real-time identification of infrared image centroid based on FPGA provided by the present invention is specifically as follows: (1) Turn on the infrared focal plane detector, move the target into the field of view and collect the original image of the target. The original image resolution is 512×512 pixels; (2) Open a 3×3 grid data window in the field of view through FPGA. The grid data window is shown in Table 1: Table 1: Raster data window and pixel values

[0017] In Table 1, A11, A12, A13, A21, A22, A23, A31, A32, and A33 are the pixel values ​​of each grid in the grid data window; (3) Arrange the pixel values ​​in each window of the 3×3 grid data window; (4) Take the median value in the arrangement to replace the pixel value in the middle window of the 3×3 grid data window; that is: ; (5) Move the 3×3 grid data window, traverse the original image and repeat steps (3) to (4) to remove the salt and pepper noise in the original image and obtain the image after preliminary filtering; (6) Set the size of the Gaussian kernel to 3×3, and use FPGA to open a 3×3 Gaussian kernel data window in the field of view; the Gaussian kernel data window is shown in Table 2.

[0018] Table 2: Gaussian kernel data window and pixel values

[0019] In Table 2, B11, B12, B13, B21, B22, B23, B31, B32, and B33 are the pixel values ​​of each window within the Gaussian kernel data window; (7) Set the corresponding Gaussian kernel coefficient for each pixel value in the Gaussian kernel data window; the Gaussian kernel coefficient is shown in Table 3: Table 3: Gaussian kernel coefficients

[0020] (8) Convolve the pixel value in the Gaussian kernel data window and the corresponding Gaussian kernel coefficient and sum it, and record it as E; E=(B11×1 + B12×3 + B13×1 + B21×3 + B23×3 + B31×1 + B32×3 + B33×1); (9) Sum the Gaussian kernel coefficients, denoted by S; S=1 + 3 + 1 + 3 + 3 + 1 + 3 + 1; (10) Take E / S as the pixel value of the middle window of the Gaussian kernel data window; That is, B22=E / S=(B11×1 + B12×3 + B13×1 + B21×3 + B23×3 + B31×1 + B32×3 + B33×1) / S; (11) Move the 3×3 Gaussian kernel data window, traverse the image and repeat steps (7) to (10) to complete the secondary filtering of the image after the initial filtering and obtain the final target image; (12) Set the size of the convolution kernel to 3×3, and open a 3×3 convolution kernel data window in the field of view through FPGA; the convolution kernel data window is shown in Table 4: Table 4: Convolution kernel data window and pixel values

[0021] In Table 4, C11, C12, C13, C21, C22, C23, C31, C32, and C33 are the pixel values ​​of each window in the convolution kernel data window; (13) For each pixel value in the convolution kernel data window, the corresponding row direction convolution factor and column direction convolution factor are set respectively; the row direction convolution factor and column direction convolution factor are shown in Table 5 and Table 6 respectively: Table 5: Row-wise convolution factors

[0022] Table 6: Column-wise convolution factors

[0023] (14) Convolve and sum the pixel values ​​in the convolution kernel data window with the corresponding row direction convolution factor and column direction convolution factor to obtain the row direction gradient data Gx and the column direction gradient data Gy; That is: the row direction gradient data Gx = (C11×-1) + (C12×-2) + (C13×-1) + (C21×0) + (C22×0) + (C23×0) + (C11×1) + (C12×2) + (C13×1); Column direction gradient data Gy=(C11×-1)+(C21×-2)+(C31×-1)+(C12×0)+(C22×0)+(C32×0)+(C13×1)+(C23×2)+(C33×1); (15) Calculate the gradient change rate S based on the row direction gradient data Gx and the column direction gradient data Gy: ; When the gradient change rate S of the pixel point in the target image is greater than the threshold SetV, the threshold , select a value from the set as the threshold, where X represents the data bit width of the pixel; record the pixel as the target contour pixel and mark the pixel as 1; otherwise, mark the pixel as 0; (16) Move the 3×3 convolution kernel data window, traverse the image and repeat steps (13) to (15) to complete the Sobel edge detection of the target image; at the same time, remove the pixels marked as 0 to obtain the target contour composed of the target contour pixels; (17) Extract the coordinates of the leftmost, rightmost, bottommost and topmost pixels of the target in the target image and record them as ; (18) Extract the coordinates of the leftmost, rightmost, bottommost and topmost target contour pixel points in the target contour and record them as ; (19) If ,Will As the leftmost pixel coordinate of the target; otherwise, As the leftmost pixel coordinate of the target; like ,Will As the rightmost pixel coordinate of the target; otherwise, As the rightmost pixel coordinate of the target; like ,Will As the coordinates of the pixel point at the bottom of the target; otherwise, As the coordinates of the pixel point at the bottom of the target; like ,Will As the coordinates of the top pixel of the target; otherwise, As the coordinates of the top pixel of the target; Update the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target, and record them as ; (20) Calculate the centroid coordinates Z(R, C) based on the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target and output them; in, .

[0024] Specific examples are given below.

[0025] Example 1

[0026] Move the target into the center of the field of view, first perform median filtering and Gaussian filtering on the original image to obtain the target image. Figure 1 As shown; secondly, Sobel edge detection is performed on the target image to obtain the target contour. The target contour is as follows Figure 2 Then, the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target are obtained and recorded as ; Finally, the FPGA calculates and outputs the centroid coordinates Z (R, C), such as Figure 3 As shown, the row coordinate R (C_ROW) = 256, and the column coordinate C (C_COL) = 256.

[0027] The centroid of the original image monitored by Hough transform is (256, 256), which is the same as the centroid output by the scheme of this embodiment, indicating that the method of the present invention based on FPGA to identify the centroid of infrared images in real time can realize real-time image processing and output the centroid. Compared with the current Hough transform detection of the centroid, it saves more chip resources and can identify the centroid position more quickly. It can be suitable for occasions with few chip resources, high requirements for centroid detection speed, and high real-time requirements.

[0028] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for real-time identification of infrared image centroid based on FPGA, characterized in that: The steps include: S1, turn on the infrared focal plane detector, move the target into the field of view and collect the original image of the target; S2, removing salt and pepper noise in the original image by median filtering to obtain a preliminary filtered image; S3, performing secondary filtering on the image after preliminary filtering in step S2 by Gaussian filtering to obtain a final target image; S4, performing Sobel edge detection on the target image obtained in step S3 to obtain the target contour of the target; S5. Combining the target contour obtained in step S4 and the target in the target image, obtain the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target, and record them as ; S6, calculating the centroid coordinates Z (R, C) according to the coordinates of the leftmost, rightmost, bottommost and topmost target contour pixel points in step S5 and outputting them; in, .

2. The method for real-time identification of infrared image centroid based on FPGA according to claim 1 is characterized in that: Step S2 is specifically as follows: S2.1, open a 3×3 grid data window in the field of view through FPGA; S2.2, arranging pixel values ​​in each window of the 3×3 grid data window; S2.3, taking the median value in the arrangement to replace the pixel value in the middle window of the 3×3 grid data window; S2.4, move the 3×3 grid data window, traverse the original image and repeat steps S2.2 to S2.3 to complete the removal of salt and pepper noise in the original image and obtain the image after preliminary filtering.

3. The method for real-time identification of infrared image centroid based on FPGA according to claim 1 is characterized in that: Step S3 is specifically as follows: S3.1, set the size of the Gaussian kernel to 3×3, and open a 3×3 Gaussian kernel data window in the field of view through FPGA; S3.2, setting corresponding Gaussian kernel coefficients for each pixel value in the Gaussian kernel data window; S3.3, performing convolution on the pixel values ​​in the Gaussian kernel data window and the corresponding Gaussian kernel coefficients, and recording the sum as E; S3.4, sum the Gaussian kernel coefficients, denoted as S; S3.5, take E / S as the pixel value of the middle window of the Gaussian kernel data window; S3.

6. Move the 3×3 Gaussian kernel data window, traverse the image and repeat steps S3.2 to S3.5, perform secondary filtering on the image after preliminary filtering in step S2 to obtain the final target image.

4. The method for real-time identification of infrared image centroid based on FPGA according to claim 1 is characterized in that: Step S4 is specifically as follows: S4.1, set the size of the convolution kernel to 3×3, and open a 3×3 convolution kernel data window in the field of view through FPGA; S4.2, respectively setting a corresponding row direction convolution factor and a column direction convolution factor for each pixel value in the convolution kernel data window; S4.3, convolve and sum the pixel values ​​in the convolution kernel data window with the corresponding row direction convolution factors and column direction convolution factors, respectively, to obtain row direction gradient data Gx and column direction gradient data Gy; S4.

4. Calculate the gradient change rate S based on the row direction gradient data Gx and the column direction gradient data Gy: ; When the gradient change rate S of a pixel point in the target image is greater than the threshold SetV, the pixel point is recorded as a target contour pixel point and marked as 1; otherwise, the pixel point is marked as 0; S4.

5. Move the 3×3 convolution kernel data window, traverse the image and repeat steps S4.2 to S4.4 to complete the Sobel edge detection of the target image; at the same time, remove the pixels marked as 0 to obtain the target contour composed of the target contour pixels.

5. The method for real-time identification of infrared image centroid based on FPGA according to claim 4 is characterized in that: In step S4.4, the threshold , select a value from the set as the threshold, where X represents the data bit width of the pixel.

6. The method for real-time identification of infrared image centroid based on FPGA according to claim 1 is characterized in that: Step S5 is specifically as follows: S5.

1. Extract the pixel coordinates of the leftmost, rightmost, bottommost and topmost pixels of the target in the target image and record them as ; S5.

2. Extract the coordinates of the leftmost, rightmost, bottommost and topmost target contour pixel points of the target contour and record them as ; S5.3 If ,Will As the leftmost pixel coordinate of the target; otherwise, As the leftmost pixel coordinate of the target; like ,Will As the rightmost pixel coordinate of the target; otherwise, As the rightmost pixel coordinate of the target; like ,Will As the coordinates of the pixel point at the bottom of the target; otherwise, As the coordinates of the pixel point at the bottom of the target; like ,Will As the coordinates of the top pixel of the target; otherwise, As the coordinates of the top pixel of the target; Update the coordinates of the target contour pixel points on the leftmost, rightmost, bottommost and topmost sides of the target, and record them as .

Images