Intelligent infrared image processing system and application thereof

Abstract

This invention relates to the field of intelligent infrared image processing system algorithm implementation technology, specifically to an intelligent infrared image processing system and its application. The processing system's image processing procedure includes the following steps: performing non-uniform correction on the acquired infrared image data, i.e., determining the non-uniform correction interval where each pixel value in the image belongs, and correcting the pixel according to the corresponding gain coefficient and correction coefficient based on its interval; performing blind pixel detection on the corrected image, and replacing defective pixels using neighborhood median filtering; after replacement, histogram equalization, arbitrary windowing, and image flipping / mirroring functions can be performed. This system has advantages such as low cost, simple operation, fast data acquisition, and real-time image processing.

Description

Technical Field

[0001] This invention relates to the field of algorithm implementation technology for intelligent infrared image processing systems, and specifically to an intelligent infrared image processing system and its applications. Background Technology

[0002] Intelligent infrared image processing systems use infrared sensors to replace human eyes and dedicated high-speed microprocessors to simulate human logical functions such as analysis, reasoning, judgment, and decision-making, replicating human thought processes through algorithms. While "seeing," the intelligent infrared image processing system also makes judgments and provides feedback on the information it sees, thus replacing the high-speed microcomputer in completing the thought process. Based on image processing algorithms, the intelligent infrared image processing system proposes image processing algorithms such as image correction, blind pixel detection, pixel substitution, histogram equalization, image windowing, and image flipping / mirroring, achieving multi-functional image processing capabilities and initially reaching a level of intelligent infrared image processing.

[0003] Significant breakthroughs have been achieved in infrared image processing algorithms at present. However, integrating multiple algorithms while simultaneously addressing the increased computational load and storage space requirements presents a challenge for intelligent infrared image processing systems. The desired functionalities include: detector control and image data acquisition; on-chip implementation of algorithms such as image windowing, image flipping, and image mirroring; collaborative processing between the on-chip system and high-speed SDRAM system, with off-chip storage space used for multi-functional image processing, including real-time preprocessing of images such as blind pixel detection, pixel substitution, and histogram equalization; and the provision of non-uniformity correction coefficients by the high-speed SDRAM system, which can significantly reduce on-chip data throughput, thereby enhancing target image processing capabilities and increasing both computational load and signal storage capacity. Summary of the Invention

[0004] To enhance target image processing capabilities while simultaneously increasing algorithm computation and signal storage capacity, this invention proposes an intelligent infrared image processing system. The image processing procedure includes the following steps:

[0005] Non-uniform correction is performed on the acquired infrared image data, that is, the non-uniform correction interval of each pixel value in the image is determined, and the corresponding gain coefficient and correction coefficient are called according to the interval to correct the pixel.

[0006] Blind pixel detection is performed on the corrected image, and defective pixels are replaced using median filtering.

[0007] After replacement, the functions of histogram equalization, arbitrary windowing, and image flipping / mirroring can be performed.

[0008] Compared with the prior art, the present invention has the following beneficial effects:

[0009] 1. The intelligent infrared image processing system of the present invention has the advantages of low cost, simple operation, fast data acquisition, and real-time image processing.

[0010] 2. The real-time algorithm of the intelligent infrared image processing system of this invention includes correction, blind pixel detection, pixel substitution, histogram equalization, image windowing, and image flipping / mirroring. It offers diverse image processing functions.

[0011] 3. The intelligent infrared image processing system of the present invention has the capabilities of online acquisition, bias control, timing control, real-time imaging, and online evaluation, which can improve the evaluation and verification efficiency of the intelligent infrared image processing system.

[0012] 4. This invention enables human-machine delivery. The host computer evaluation system includes an image testing system and a program control terminal. The infrared image processing system can perform real-time imaging, enable and disable online functions, configure different function modes, and achieve intelligent infrared image processing according to personalized needs.

[0013] 5. This invention uses a dedicated “FPGA + high-speed module SDRAM” model to realize real-time processing of intelligent algorithms. The high-speed module SDRAM has a data space of more than 600Mbits, which can perform real-time multi-functional image processing on an array of more than 640*512 pixels.

[0014] 6. The correction algorithm of this invention can achieve 6-point non-uniformity correction. Compared with the two-point correction method, it has more advantages in complex nonlinear non-uniformity correction cases. It is superior to the two-point non-uniformity correction in terms of accuracy, adaptability and dynamic adaptability. Blind pixel detection can identify and identify overheated pixels, dead pixels and noisy pixels in real time. Pixel replacement adopts the median value filtering algorithm for replacement, automatic identification and replacement, real-time output, and blind pixel rate is less than 1%.

[0015] 7. This invention's histogram equalization automatically redistributes pixel brightness values, improving image contrast, revealing details, and resulting in a clearer and richer visual effect. It also enhances image details and texture, optimizes dynamic range, and makes images more striking.

[0016] 8. The image windowing of this invention can achieve arbitrary windowing of the entire array; image flipping can "correct" the image captured by the inverted infrared thermal imager, so that the operator can see an upright image that conforms to visual habits as needed; image mirroring can seamlessly switch between two images, or can perform image fusion and precise superposition to form "picture-in-picture" or "thermal superposition", making the image richer.

[0017] 9. The real-time algorithms of the intelligent infrared image processing system in this invention include: correction, blind pixel detection, pixel replacement, histogram equalization, image windowing, and image flipping and mirroring. The host computer evaluation system can activate individual algorithms, combined algorithms, or fully automatic algorithms as needed to meet personalized infrared image processing requirements. Attached Figure Description

[0018] Figure 1 This is a block diagram illustrating the algorithm implementation of an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0019] Figure 2 This is a block diagram illustrating the implementation of a correction algorithm for an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0020] Figure 3 This is a block diagram illustrating the implementation of a blind pixel detection algorithm in an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0021] Figure 4 This is a schematic diagram of blind pixel identification in an intelligent infrared image processing system according to Embodiment 1 of the present invention;

[0022] Figure 5 This is a block diagram illustrating the implementation of a pixel substitution algorithm in an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0023] Figure 6 This is a schematic diagram illustrating the identification and judgment of the A / B value of a counter in an intelligent infrared image processing system provided in Embodiment 1 of the present invention;

[0024] Figure 7 This is a schematic diagram of pixel replacement in an intelligent infrared image processing system according to Embodiment 1 of the present invention;

[0025] Figure 8 This is a block diagram illustrating the implementation of a histogram equalization algorithm in an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0026] Figure 9 This invention provides a data balancing process for an intelligent infrared image processing system according to Embodiment 1.

[0027] Figure 10 This is a block diagram illustrating the windowing algorithm implementation of an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0028] Figure 11 This is a schematic diagram of image windowing in an intelligent infrared image processing system provided in Embodiment 1 of the present invention;

[0029] Figure 12 This is a block diagram illustrating the implementation of a mirror flipping algorithm in an intelligent infrared image processing system according to Embodiment 1 of the present invention.

[0030] Figure 13 A reference comparison diagram for evaluating the correction algorithm of an intelligent infrared image processing system provided in Embodiment 2 of the present invention;

[0031] Figure 14 A reference comparison diagram for evaluating the blind pixel detection and pixel substitution algorithms of an intelligent infrared image processing system provided in Embodiments 3 and 4 of the present invention;

[0032] Figure 15 This is a reference comparison diagram for evaluating the histogram equalization algorithm of an intelligent infrared image processing system provided in Embodiment 5 of the present invention;

[0033] Figure 16 This is a reference comparison diagram for evaluating the image windowing algorithm of an intelligent infrared image processing system provided in Embodiment 6 of the present invention;

[0034] Figure 17 This is a reference comparison diagram for evaluating the image flipping and mirroring algorithm of an intelligent infrared image processing system provided in Embodiment 7 of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] This invention provides an intelligent infrared image processing system, the image processing procedure of which includes the following steps:

[0037] Non-uniform correction is performed on the acquired infrared image data, that is, the non-uniform correction interval of each pixel value in the image is determined, and the corresponding gain coefficient and correction coefficient are called according to the interval to correct the pixel.

[0038] Blind pixel detection is performed on the corrected image, and defective pixels are replaced using median filtering.

[0039] After replacement, the functions of histogram equalization, arbitrary windowing, and image flipping / mirroring can be performed.

[0040] Example 1

[0041] This embodiment provides an intelligent infrared image processing system. In addition to image processing, the system integrates image acquisition functionality. It includes an evaluation system, a control board, and an infrared detector. The evaluation system further includes a program control terminal and an image testing system. The control board includes an FPGA, a high-speed SDRAM module, a low-speed module, a data acquisition unit, an SPI interface, and Flash memory.

[0042] The program control terminal is used by the host computer to control the FPGA to ensure the algorithm implementation; at the same time, it inputs the non-uniformity correction coefficient to the high-speed module (SDRAM) to ensure high-speed data interaction between the FPGA and SDRAM.

[0043] An image testing system is used to verify and evaluate the algorithms of intelligent infrared image processing systems.

[0044] FPGA is used to acquire image data from infrared detectors and implement algorithms. Image algorithms include real-time image processing such as correction, blind pixel detection, pixel replacement, histogram equalization, image windowing, and image flipping / mirroring.

[0045] High-speed SDRAM modules are used to provide corresponding control data and calculation parameters to the FPGA, including the SDRAM controller and SDRAM memory;

[0046] The SDRAM controller provides non-uniform correction coefficients for the correction function; compares and accumulates for the blind pixel detection function; provides accumulated values ​​for pixel substitution; and performs operations such as current frame probability density statistics and previous frame probability density mapping for histogram equalization. The FPGA also has high-speed and low-speed modules mounted on it. The high-speed module is mounted on general-purpose SDRAM, forming a programmable control system with the FPGA as the core. The image algorithm is mounted on the high-speed module, which is externally connected to the SDRAM controller and SDRAM memory, which can expand the image data storage space. The FPGA directly addresses and accesses the parameters of each functional block, and configures the high-speed processing channel at the same time.

[0047] SDRAM memory enables image data reception / storage, image data reading and processing, and high-speed image data transmission. Specifically, this embodiment uses SDRAM as the data memory, which contains an SRAM hard core module. Since the SRAM controller of this system needs to support 1-byte read and write access, the data bit width of a single SRAM is 8-bit, and the granularity of the SRAM is 1K*8bits. If a total of N SRAMs are needed, they are arranged in an array of 4 per row, for a total of N / 4 rows. The main functions of this SRAM memory are: it can be accessed in bytes, half words (16 bits), or full words (32 bits). The starting address of the SRAM is 0x20000000. It has a bus interface and supports little-endian storage.

[0048] The low-speed module is used to provide the corresponding timing logic and bias voltage level to the infrared detector, including timing control and bias voltage control.

[0049] Timing control is used to generate control timing outside the infrared image detector signal acquisition circuit, including register write operations, to control the normal operation of the external infrared image detector.

[0050] Bias control, whose main function is for the FPGA to output serial control data to the DAC, generates a signal that conforms to the DAC's working timing to control the DAC and achieve bias output;

[0051] Data acquisition is used to receive serial image data, which is acquired via an ADC. The FPGA implements functions such as serial data reception, ADC register writing and reading, and ADC data processing.

[0052] The SPI interface is used to receive parameter configuration information, convert it, and send it to the FPGA. The FPGA then redistributes these parameters to various modules.

[0053] Flash memory is used for FPGA program startup and power-on initialization configuration.

[0054] Infrared detectors are used to capture targets and focus infrared radiation energy. Short-wave, medium-wave, and long-wave infrared lenses can be installed as needed to enable signal readout and communication with the main control board.

[0055] Example 2

[0056] This invention employs a non-uniform correction, multi-point correction method, which requires writing non-uniform correction coefficients through the serial communication interface of the program control terminal in the evaluation system and storing them in the high-speed module SDRAM memory.

[0057] The flowchart of the non-uniform correction algorithm is as follows: Figure 2 As shown, when the FPGA performs non-uniform correction on image data, it receives image data in real time through the LVDS interface, determines the grayscale value range of the image data, sends a range segment Xi read control operation to the SDRAM controller based on the grayscale value range, and simultaneously reads the corresponding correction coefficients from the SDRAM memory. The FPGA then performs correction processing on the original image using the correction coefficients and outputs the corrected image data. The high-speed module SDRAM storage space requirements are as follows: Each pixel requires 6 points of 16-bit gain and bias correction coefficients, totaling 6*(16+16)=192 bits. Based on 640*512 pixels, the storage space for non-uniform correction is 192*640*512=62914560=60Mbits. Considering reserving 10 integration times, 600Mbits of data space is required.

[0058] like Figure 2 As shown, non-uniformity correction consists of two processes: non-uniformity correction coefficient storage and non-uniformity correction processing. In the initial state, the host computer stores the pre-calibrated six segments of non-uniformity correction coefficients corresponding to each pixel in the high-speed module SDRAM memory via the FPGA. In the non-uniformity correction processing, the input pixel image data value is first compared to determine the position within the six non-uniformity correction segments. Then, the correction coefficients for the corresponding segments are retrieved from the SDRAM for correction processing. Finally, the corrected image data is output.

[0059] Each pixel corresponds to 6 non-uniform correction intervals, and each interval corresponds to two coefficients: a 16-bit gain coefficient and a 16-bit offset coefficient.

[0060] The specific implementation steps are as follows:

[0061] a) The pre-calibrated non-uniformity correction coefficients (each pixel corresponds to a certain segment, and each segment corresponds to a gain coefficient a and a correction coefficient b, for a total of 6 segments) are stored in SDRAM. The 16 bits of a and 16 bits of b are combined into 32 bits of data, which are stored in SDRAM and read out from SDRAM at the same time.

[0062] b) During image data transmission, the pixel grayscale value determines the current interval: the interval position is set to [0, x1), [x1, x2), [x2, x3), [x3, x4), [x4, x5), [x5, x6). The correction interval values ​​x1~x6 must be configurable by registers. The default interval interval is 65536 / 6. Once the interval for each pixel is determined, the coefficients a and b for that interval are read from this SDRAM.

[0063] c) Correct the correction coefficients of the determined interval segments: pixel(i,j)=a* pixel(i,j)>>16+b, where pixel(i,j) represents each pixel of the image, that is, the pixel data and gain data are multiplied, the lower 16 bits of data are discarded, and then added to the offset coefficient. After the addition, the upper 16 bits are truncated and the pixel data is finally output.

[0064] Example 3

[0065] like Figure 3As shown, an intelligent infrared image processing system implements a blind pixel detection algorithm. Blind pixel compensation in the intelligent infrared image processing system is divided into blind pixel detection and pixel replacement. In this invention, blind pixel detection is performed on the corrected image input data. In blind pixel detection of multiple consecutive frames of image data, the detection results need to be read from the SDRAM memory in the high-speed module for blind pixel detection. After detecting the current pixel, the results are written back to the SDRAM memory. In the pixel replacement algorithm, the accumulated value of the detection results is read from the SDRAM memory to determine the type of blind pixel, and then blind pixel compensation processing is performed. SDRAM storage space requirements: Each pixel needs to store 4 bits of accumulated values ​​A and B, totaling 8 bits. Calculated based on 640*512 pixels, the storage space for blind pixel identification and pixel replacement is the same, both being 8*640*512=2621440=2.5Mbits.

[0066] Blind pixel detection and processing flow, such as Figure 3 As shown, during blind pixel detection, the received 3×3 adjacent pixel data undergoes pixel (top, bottom, left, and right) grayscale comparison. A is used to record the number of times the pixel's maximum value is reached, and B is used to record the number of times the pixel's minimum value is reached. When accumulating the comparison results for each pixel in each frame, the accumulated value (A / B) of the pixel at that position in the previous frame needs to be read from the high-speed module's SDRAM memory. After comparison and accumulation processing, the result is stored in the SDRAM memory. The blind pixel detection process ends when the set number of image frames N is reached. The specific implementation steps are as follows:

[0067] Algorithm implementation for a single frame of an image: such as Figure 4 As shown, the grayscale difference between a single pixel and the pixels above, below, to the left, and to the right is calculated. Specifically, the 16-bit data value read from pixel (i,j) is compared with the 16-bit data values ​​of the pixels above, below, to the left, and to the right. The result is represented by two signals, A and B. If the value of pixel (i,j) is greater than the values ​​of all four pixels, signal A outputs "1", indicating that it is greater than all four pixel values; in this case, signal B is "0". If the value of pixel (i,j) is less than the values ​​of all four pixels, signal B outputs "1", indicating that it is less than all four pixel values; in this case, signal A is "0". Otherwise, both A and B are "0".

[0068] Example 4

[0069] like Figure 5As shown, a pixel substitution algorithm for an intelligent infrared image processing system is implemented. In Example 3, pixel substitution processing is performed after blind pixel detection. During pixel substitution processing, while receiving image data, the A / B accumulated value of the corresponding pixel is read from the high-speed module SDRAM memory. Then, blind pixel type determination is performed. Based on the blind pixel type determination result, pixel substitution processing is performed on the blind pixels, and finally, the processed image data is output. The specific implementation steps are as follows:

[0070] A / B value determination: such as Figure 5 As shown, the A / B value of a frame of image is judged, and the A / B values ​​generated at the same position in each frame of image are accumulated separately. The accumulated result of each frame of image is stored separately. In this embodiment, only 15 frames of images are accumulated and stored. Taking the pixel position (4,100) in a frame of image as an example, the accumulation of the 15 frames of images is based on the accumulation of the A and B values ​​of the pixel at position (4,100). The maximum accumulation result is 15, so it is stored in 4 bits. Therefore, the entire 640×512 area array requires 640×512×4 bits×2 memory to complete the storage of the accumulated A and B values ​​(from frame 1 to frame 15). After the 15 frames of image data are processed, the accumulated A and B values ​​remain unchanged. Subsequent image data will only undergo serial shift buffer updates, but will not perform the accumulation calculation of the judgment values ​​A and B of blind flash elements. After the system issues a recalculation command, A and B are cleared, and the accumulation of blind flash pixel judgment values ​​A and B is restarted for the cached image data. For pixels at the edge, only the values ​​of pixels above, below, to the left, and to the right of that position are compared. The accumulated values ​​of 15 frames of data pixels (two 4-bit counters A and B represent constant high and constant low values, respectively) are statistically analyzed: a constant high counter A ≥ 13 indicates an overheated pixel; a constant low counter B ≥ 13 indicates a dead pixel; the sum of the two counters A and B ≥ 8 indicates noise; other cases are non-defect pixels. The recognition information is represented by a 2-bit output, with four recognition categories: 00 non-defect pixel, 01 overheated pixel, 10 dead pixel, and 11 noise. Blind flash pixels include: overheated pixels, dead pixels, and noise.

[0071] Determine the type of blind cell: The type of blind cell is determined based on two 4-bit registers, A and B. For example... Figure 6 As shown, Figure 6 (a) is a schematic diagram of A / B value labeling. Figure 6 (b) is a schematic diagram of A / B value judgment. The A / B value is recorded by the counter and the type is judged by the A / B value: if the counter A≥13, the counter B≥13, and A+B≥8, the A / B value judgment diagram is marked as "1", otherwise it is marked as "0". After the blind flashing cell type judgment is performed on all cell positions, the cell replacement is performed.

[0072] Pixel substitution: The A / B value comparison diagram uses the median value to replace the pixel marked "1". The values ​​of pixel (i,j) and the values ​​of the pixels above, below, to the left, and right are sorted, and the median value is used to replace the value of pixel (i,j). Figure 7 The diagram illustrates the pixel replacement algorithm. Taking pixel position (2,2) as an example, its top, bottom, left, and right positions are (1,2), (2,1), (2,3), and (3,2), respectively. The gray values ​​of these five coordinates are then sorted. Since the gray value of position (2,3) is the middle value, the gray value of pixel position (2,2) is replaced with the gray value of position (2,3).

[0073] Pixel Output: Pixels are classified into missing pixels (2 bits) and normal pixels. Based on the replacement steps described above, the missing pixels (2 bits) are replaced using an intermediate value filtering algorithm, while normal pixels are output directly. All output recognition categories are calculated in real time, and the results, along with the corresponding pixel data, are output in real time. This achieves automatic recognition and replacement of blind pixels, resulting in real-time image output.

[0074] Example 5

[0075] like Figure 8 As shown, an intelligent infrared image processing system implements a histogram equalization algorithm. This invention performs histogram equalization on images. The core benefit of histogram equalization is its significant improvement in image contrast. It automatically redistributes pixel brightness values ​​based on statistical methods, increasing the global contrast of the image, thereby revealing hidden details and making the overall visual effect of the image clearer, richer, and more information-rich. Simultaneously, histogram equalization can also enhance image details and texture, and optimize the image's dynamic range.

[0076] Histogram equalization uses grayscale value statistical processing, requiring the reading of grayscale value statistical results from the high-speed SDRAM module. After statistical processing, the grayscale values ​​are sequentially accumulated, and the accumulated results are stored in the SDRAM. During mapping, the accumulated grayscale value is read from the SDRAM based on the input pixel grayscale value, and then the mapping process is performed before outputting the image data. SDRAM storage space requirements are summarized as follows: 16 bits of pixel data, with a grayscale value range of 0~65535, requires 19 bits to cover the number of pixels (assuming 640*512 pixels). Therefore, each grayscale value requires 19 bits for statistical processing, resulting in a total storage space of 65536*19 = 1.1875 Mbits for the statistical results. Since the storage space for the accumulated results after grayscale value statistics can be the same as the storage space for the statistical values, the total storage space required for image equalization is 65536*19*2 = 2.375 Mbits.

[0077] The histogram equalization algorithm is divided into two stages: a preparation stage and a real-time compensation stage. The preparation stage mainly involves acquiring and storing the mapping vector, i.e., grayscale value statistics and sequential accumulation; the real-time compensation stage performs real-time equalization of the image using the mapping vector, such as... Figure 9 As shown. The specific implementation steps of image equalization are as follows:

[0078] a) Allocate SDRAM storage space: Since the image data is 16 bits, the range of the image grayscale value is 0~65535 (2 16 -1), so 65536 (2) needs to be allocated. 16 There are 19 storage spaces, and each space needs to be allocated 19 bits (640 * 512 < 2). 19 Each storage space corresponds to a grayscale value. The starting address of this storage space is defined as addr0, and the storage space corresponding to the grayscale value i (i is located in [0~65535]) is addr0+i. Initially, all grayscale value statistics are reset to 0.

[0079] b) For each input pixel of image data, based on the gray value i of this pixel, read the gray value statistics h(i) corresponding to the gray value from SDRAM, then h(i) = h(i) + 1, and then store the accumulated result into the gray value statistics storage space corresponding to SDRAM, similar to generating a gray value statistics lookup table.

[0080] c) Repeat the above steps until the grayscale values ​​of all pixels in a frame of the image have been counted. The frame interval for grayscale value counting is controlled by a register (i.e., the update frequency is controlled by a register).

[0081] d) Read the statistical results of grayscale values ​​from SDRAM sequentially, then accumulate them sequentially, and store the accumulated result S(i) back into SDRAM. Here, it is necessary to reallocate the storage space for the accumulated value in SDRAM and store S(65535) in a register, as shown in the following formula:

[0082]

[0083] in, This represents the cumulative sum of the storage space corresponding to pixels 0 to 1 of the previous frame image; This invention provides a statistical storage space for the grayscale values ​​of pixel value j in the previous frame image. Considering the limitations of FPGA computing power and computation time, this invention utilizes the statistical distribution probability of pixel values ​​from the previous frame image as a mapping output for the next frame image. Because the image algorithm outputs data in real time and the images are refreshed very quickly, the two images are essentially the same, so visually there is no difference between the two frames.

[0084] e) After the above operations are completed, the next frame of the image arrives. For each input pixel of image data, the accumulated value S(i) corresponding to the gray value i of the pixel is read from the SDRAM, and then the mapping process shown in the following formula is performed to obtain the image data of the equalization process.

[0085]

[0086] in, This represents the grayscale value i of the current frame image after equalization; g2 represents the cumulative sum of the storage space corresponding to all pixels of the previous frame image; 16 Indicates will Mapped to 0~2 16 Output within the range of -1, each For a given grayscale value, let adjacent... The grayscale values ​​obtained by mapping between them have the same interval. For example, three adjacent ratios a, b, and c are mapped to 0~2. 16 If the values ​​in the range of -1 are a', b', and c', then b' - a' = c' - b'.

[0087] Example 6

[0088] like Figure 10 As shown, to achieve the above objectives, this invention provides an image windowing algorithm implementation for an intelligent infrared image processing system. Image windowing is used to display local feature data images. Windowing technology "stretches" the image information of interest to the entire display area, making details that are difficult to perceive with the naked eye clearly discernible. First, the image windowing algorithm is implemented on the image data after histogram equalization. The data transmission timing is set as follows:

[0089] During data reading, a windowing function is implemented, meaning data is read starting from a specified starting cell. This is achieved via SPI control from the host computer program (evaluation system), obtaining the initial cell coordinates (x0, y0) and window size (column width and row depth) for the image data to be windowed. Specifically, the first row of cells is read from (x0, y0) to (x0, y0 + width - 1), and the last cell is read from (x0 + depth - 1, y0 + width - 1).

[0090] Window opening diagram as shown Figure 11 As shown, this involves extracting a region of interest from the original image. The initial pixel coordinates (x0, y0) are (2, 3), and the window size (column width, row depth) is (m-3, n-2).

[0091] Example 7

[0092] like Figure 12 As shown, to achieve the above objectives, this invention provides an image flipping and mirroring algorithm implementation for an intelligent infrared image processing system. Image flipping and mirroring include two functions: image flipping and image mirroring. Image flipping refers to vertically flipping the image, swapping its top and bottom directions; image mirroring refers to horizontally flipping the image, like looking in a mirror, swapping its left and right directions. Infrared image flipping can "correct" the image captured by an inverted infrared thermal imager, allowing the operator to see an upright image that conforms to visual habits; infrared image mirroring can seamlessly switch between the two images, or perform image fusion and precise overlay to form "picture-in-picture" or "thermal overlay." The image flipping and mirroring algorithm is implemented using image data after windowing, and the data transmission timing is as follows:

[0093] During data transmission, a mirroring function is implemented, meaning data is read according to a configured reading order, such as top-to-bottom, left-to-right, bottom-to-top, or right-to-left. Flipping and mirroring are controlled by configuring flip and mirroring coefficients.

[0094] The flip factor control is set to 0: top to bottom (default); 1: bottom to top. That is, when it is 0, the row address counter increments; when it is 1, the row address counter decrements.

[0095] The mirroring factor is set to 0: left to right (default); 1: right to left. That is, when it is 0, the column address counter increments; when it is 1, the column address counter decrements.

[0096] Example 8

[0097] like Figures 13-17 As shown, to achieve the above objectives, this invention provides an evaluation method for an intelligent infrared image processing system algorithm. The evaluation method is as follows:

[0098] ① Perform image testing and verification of the correction algorithm. Configure test cases on the image processing system, enter the non-uniformity correction mode, and then input a non-uniform image (e.g., applying two horizontal lines) to the FPGA. Figure 13 As shown in (a), the non-uniformity correction coefficient is input through the program control terminal to enable the non-uniformity correction function. The acquired image is used to determine whether the two applied horizontal bars are corrected and whether the remaining image portion is not corrected. The corrected image is shown below. Figure 13 As shown in (b), after non-uniform correction, the two horizontal lines in the image disappear, indicating that the non-uniform correction is effective.

[0099] ② Image testing and verification of the blind pixel detection and pixel substitution algorithms. By configuring test cases on the image processing system, entering the blind pixel detection and pixel substitution mode, and then inputting an image with blind pixels (e.g., an image with a blind pixel rate > 7%) into the FPGA, such as... Figure 14 As shown in (a), by enabling blind pixel detection and pixel substitution, blind pixels in specific regions are compensated (blind pixel rate of the image < 1%), it can be seen that... Figure 14 In (b), all blind flash elements were identified and replaced.

[0100] ③ Image testing and verification of the histogram equalization algorithm. By configuring test cases for the image processing system and entering histogram equalization mode, the image after blind pixel compensation is input to the FPGA, such as... Figure 15 (a) As shown in the figure, the image testing system is used to acquire and display the output video, and the histogram before equalization is recorded, as follows. Figure 15 (c) As shown in Figure 1. The image equalization function is enabled via the "Programmable Controller" of the host computer evaluation system. The image is then captured and displayed using an image testing system. The image after enabling the histogram equalization function is as follows: Figure 15 As shown in (b), record the histogram after equalization as follows. Figure 15 As shown in (d), image histogram equalization aims to improve image contrast, that is, to increase the overall contrast of the image, making the image details richer and easier to observe, thereby improving the overall image quality. Image equalization can achieve image enhancement; it is visually apparent that the equalized image is more striking. Comparing the output image and histogram before and after image equalization to see if they meet expectations determines that the image equalization function is qualified.

[0101] ④ Perform image testing and verification on the image windowing algorithm. Configure test cases on the image processing system, enter image windowing mode, then input a 640*512 image array into the FPGA, use the image testing system to acquire and output video, obtain and display the images, such as... Figure 16 As shown in (a). The image windowing function is enabled via the "Programmable Controller" of the host computer evaluation system. The image is then acquired and displayed using the image testing system, as shown below. Figure 16 As shown in (b), a 192*256 image array is displayed. If the image testing system shows that the windowing function is normal, then the image windowing function is deemed qualified.

[0102] ⑤ Perform image testing and verification on the image flipping and mirroring algorithm. Configure test cases on the image processing system, enter the image flipping and mirroring mode, then input an image with diagonal stripes into the FPGA, use the image testing system to capture and output video, obtain the image, and display it, as shown below. Figure 17 As shown in (a). The image flipping function is enabled through the "program controller" of the host computer evaluation system, and the image is acquired and displayed using the image testing system, as shown. Figure 17 As shown in (b); enable the image mirroring function, use the image testing system to acquire and display the image, as shown. Figure 17 As shown in (c); enable the image flipping and mirroring function, acquire and display the image using the image testing system, as shown. Figure 17 As shown in (d). If the image testing system shows that the flipping and mirroring functions are normal, then the function is deemed qualified.

[0103] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent infrared image processing system, characterized by, The image processing procedure includes the following steps: Non-uniform correction is performed on the acquired infrared image data, that is, the non-uniform correction interval of each pixel value in the image is determined, and the corresponding gain coefficient and correction coefficient are called according to the interval to correct the pixel. Blind pixel detection is performed on the corrected image, and defective pixels are replaced using neighborhood median filtering. After replacement, the functions of histogram equalization, arbitrary windowing, and image flipping / mirroring can be performed.

2. The intelligent infrared image processing system according to claim 1, characterized in that, The gain and offset coefficients are used to correct the pixel by multiplying the 16-bit gain coefficient by the 16-bit pixel value, discarding the lowest 16 bits, adding the result to the 16-bit offset coefficient, and then truncating and retaining the highest 16 bits as the corrected pixel data.

3. The intelligent infrared image processing system according to claim 2, characterized in that, The 16-bit grayscale space is evenly divided into 6 segments, each corresponding to a gain coefficient and a correction coefficient. If the input image pixel size is H×W and 10 integration times are reserved, then the data space required to perform non-uniform correction is at least H×W×6×(16+16)×10 bits.

4. The intelligent infrared image processing system according to claim 1 or 2, characterized in that, The process of performing blind pixel detection on the corrected image includes: In N consecutive image data frames, calculate the size relationship between the current pixel and its upper, lower, left, and right adjacent pixels in each frame; For a frame of an image, if the current pixel value is the maximum value compared with its neighboring pixels, then set the first comparison value to 1 and the second comparison value to 0; if the current pixel value is the minimum value compared with its neighboring pixels, then set the first comparison value to 0 and the second comparison value to 1. The first accumulated value is obtained by accumulating the first comparison value of the current pixel for N consecutive frames, and the second accumulated value is obtained by accumulating the second comparison value of the current pixel for N consecutive frames. If the sum of the first accumulated value and the second accumulated value is less than If the value is positive, then the pixel is a non-blind pixel; otherwise, the pixel is a blind pixel.

5. The intelligent infrared image processing system according to claim 4, characterized in that, For the first and second accumulated values ​​of each pixel, reserve bit space.

6. The intelligent infrared image processing system according to claim 4, characterized in that, The image after blind pixel compensation undergoes further equalization processing, including: For the first frame of the input image, allocate a storage space for each pixel value within each pixel range to record the number of times the corresponding pixel value appears in the image. The size of each storage space is [size missing]. , where H×W is the pixel size of the input image, and all values ​​stored in the storage space are initialized to 0; To count all pixels in an image, for the nth pixel in the input image, if its pixel value is i, then let the value of the i-th storage space be h(i) = h(i) + 1; When equalizing the i-th grayscale value of the current frame image, the cumulative sum of the storage space corresponding to pixels 0 to i of the previous frame image is calculated, and the proportion of this cumulative sum to the total image size is calculated. The image is then equalized based on this proportion. The equalization process is represented as follows: ； in, This represents the grayscale value i of the current frame image after equalization. This represents the cumulative sum of the storage space corresponding to pixels 0 to 1 of the previous frame image; g2 represents the cumulative sum of the storage space corresponding to all pixels of the previous frame image; 16 Indicates will Mapping to 2 16 Output within the range, each It corresponds to a grayscale value.

7. The intelligent infrared image processing system according to claim 1, characterized in that, The system includes an FPGA, a high-speed SDRAM module, and a low-speed module, wherein: FPGA is used to acquire image data from infrared detectors and implement algorithms, including correction, blind pixel detection, pixel replacement, histogram equalization, image windowing, and image flipping and mirroring, for real-time image processing. The high-speed module SDRAM, SDRAM controller, and SDRAM memory are used for providing non-uniform correction coefficients for correction algorithms, comparing and accumulating data for blind pixel detection algorithms, providing accumulated values ​​for pixel replacement algorithms, and performing current frame probability density statistics and previous frame probability density mapping for histogram equalization algorithms. The SDRAM memory is used to implement image data reception / storage, image data reading and processing, and high-speed image data transmission. The low-speed module is used to provide the corresponding timing logic and bias voltage level to the infrared detector, including timing control and bias voltage control.

8. An application of an intelligent infrared image processing system, characterized in that, Including the intelligent infrared image processing system of claim 4, the type of blind cell is determined based on a first accumulated value and a second accumulated value, wherein: When the first accumulated value is ≥13, the current pixel is an overheated element; When the second accumulated value is ≥13, the current pixel is a dead pixel; When the sum of the first accumulated value and the second accumulated value is ≥8, the current pixel is noise.

9. An application of an intelligent infrared image processing system, characterized in that, The intelligent infrared image processing system according to claim 6 performs image windowing on the equalized image, specifically including: The initial cell coordinates (x0, y0) and window row and column dimensions are required to start reading data from the specified starting pixel. The window row and column dimensions include the column length (width) and the row depth (depth). Then read the first line of image data, starting from (x0, y0) and ending at (x0, y0+width-1). Read the depth line, with the last pixel being (x0+depth-1, y0+width-1).

10. An application of an intelligent infrared image processing system, characterized in that, The intelligent infrared image processing system according to claim 6, which re-reads the equalized image according to a set control coefficient to achieve image flipping, specifically includes: When the flip factor control is set to 0, the row data of the image is read sequentially from top to bottom; when the flip factor control is set to 1, the row data of the image is read sequentially from bottom to top. When the mirror factor control is set to 0, one column of image data is read sequentially from left to right; when the mirror factor control is set to 1, one column of image data is read sequentially from right to left.

Images