Image dead pixel compensation method and system based on adaptive gradient descent optimization

Abstract

The invention discloses an adaptive gradient descent optimization image dead pixel compensation method and system, and relates to the technical field of image processing, and the method comprises the steps: intercepting image frames of a received image data stream collected by a short wave infrared sensor according to a preset interval; calculating a gradient descent iteration value of the image group by adopting a self-adaptive gradient descent iteration algorithm; traversing each pixel point of each image frame in the image group, pre-estimating a pre-estimated pixel value corresponding to each pixel point according to the gradient descent iteration value, and calculating a decision factor according to a difference value between the pixel value of each pixel point and the pre-estimated pixel value corresponding to the pixel value; and judging whether each pixel point is a defective pixel or not according to a comparison result of the judgment factor of each pixel point and a preset threshold value, and if the pixel point is the defective pixel, replacing the pixel value of the defective pixel with the estimated pixel value. The self-adaptive gradient descent decision factor of each pixel is calculated in real time, and the dead pixel is identified by judging the difference between the estimated value and the actual value, so that real-time, self-adaptive and high-precision identification and compensation of the dead pixel are realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an adaptive gradient descent optimized method and system for image defect compensation. Background Technology

[0002] InGaAs (Indium Gallium Arsenide) short-wave infrared cameras are commonly used on star-sensitive spacecraft for astronomical observation. During long-term operation in near-Earth orbit, InGaAs short-wave infrared cameras suffer from sensor manufacturing defects, aging of components over time, and external radiation. This can lead to insufficient light sensitivity or damage to the pixels, resulting in abnormally large or small pixel values, severely impacting image quality. Traditional identification and compensation techniques rely on two approaches: First, a lookup table method identifies points at fixed locations and compensates using the average of neighboring points. However, as the camera ages, new bad pixels appear, and this method lacks online adaptability and has low accuracy. Second, traditional software on a CPU or DSP calculates the mean and variance of the image between frames in real-time, setting a fixed bad pixel threshold. This method is computationally intensive and cannot achieve real-time processing for high-frame-rate, high-resolution short-wave infrared cameras. Field-Programmable Gate Arrays (FPGAs) offer real-time parallel processing, reconfigurability, and scalability. Implementing a real-time, adaptive, and high-accuracy identification and compensation method on an FPGA has become a key technology. Summary of the Invention

[0003] The purpose of this invention is to provide an image defect compensation method and system optimized by adaptive gradient descent. By calculating the adaptive gradient descent decision factor for each pixel in real time, and identifying defective pixels by judging the difference between the predicted value and the actual value, defective pixels can be identified in real time, adaptively, and with high accuracy.

[0004] To achieve the above objectives, this application provides the following solution: On the one hand, the present invention provides an image defect compensation method optimized by adaptive gradient descent, specifically including the following steps: S1. Receive the image data stream collected by the shortwave infrared sensor, and extract several image frames from the image data stream at preset intervals to store them as an image group; S2. Calculate the gradient descent iteration value of the image group using the adaptive gradient descent iterative algorithm; S3. Traverse each pixel of each image frame in the image group, estimate the estimated pixel value of each pixel based on the gradient descent iteration value, and calculate the decision factor based on the difference between the pixel value of each pixel and its corresponding estimated pixel value. S4. Determine whether each pixel is a bad pixel based on the comparison result between the decision factor of each pixel and the preset threshold. If it is a bad pixel, replace the bad pixel value with the estimated pixel value.

[0005] In some specific implementation schemes, the specific process of step S2 is as follows: S21. Select the first frame image from the image group, calculate the gradient descent iteration value, and store the first frame image into the FIFO cache module row by row and pixel by pixel. S22. Set up a sliding window, traverse each pixel to be identified in the first frame image, and use the sliding window to read the window data centered on the pixel to be identified from the FIFO buffer module in turn. S23. Calculate the neighborhood gradient between the pixel to be identified and its neighboring pixels within the window data to obtain the neighborhood gradient value matrix corresponding to the window data. S24. Adaptively calculate the corresponding weight value for each neighborhood gradient value in the neighborhood gradient value matrix to obtain the weight matrix. Perform iterative calculation based on the window data and the weight matrix to obtain the gradient descent iteration value corresponding to the window data of each pixel to be identified. S25. Repeat steps S22-S24 until every pixel to be identified in the first frame image has been traversed. Use the gradient descent iteration value corresponding to the last pixel to be identified as the gradient descent iteration value of the image group.

[0006] In some specific implementations, when setting up a sliding window to traverse each pixel to be identified in the first frame image, the following steps are also included: When traversing each pixel to be identified, the window data containing each pixel is processed by median filtering, and the pixel to be identified is marked according to the result of median filtering, thus obtaining the marked window data; The neighborhood gradient is calculated by selecting the window data containing the pixels to be identified with normal labels, and obtaining the neighborhood gradient matrix and the corresponding gradient descent iteration value for each pixel to be identified with normal labels.

[0007] In some specific implementation schemes, the process of applying median filtering to window data is as follows: The median value of each pixel is calculated by performing median calculation on the data of all pixels in the window data except the pixel to be identified. Calculate the difference between the data of the pixel to be identified and the median value of the pixel; The difference is compared with a preset pixel threshold, and the pixel to be identified is marked according to the comparison result.

[0008] In some specific implementation schemes, the process of marking each pixel to be identified based on the comparison results is as follows: When the difference is greater than the preset pixel threshold, the mask of the pixel to be identified is marked as abnormal; when the difference is less than or equal to the preset pixel threshold, the mask of the pixel to be identified is marked as normal.

[0009] In some specific implementations, the process of adaptively calculating the corresponding weight value for each neighborhood gradient value in the neighborhood gradient value matrix in step S24 is as follows: For the j-th neighborhood gradient value DIFF_P(j), its corresponding weight value W_P(j) is: W_P(j)=1 / (1+DIFF_P(j)*DIFF_P(j)).

[0010] In some specific implementation schemes, the iterative calculation process is as follows: Get the gradient descent iteration value calculated in the previous iteration corresponding to the current iteration number; Based on the window data and weight matrix of the pixel to be identified in the current iteration number, as well as the gradient descent iteration value calculated in the previous iteration, calculate the total gradient value within the sliding window in the current iteration number. Calculate and update the gradient descent iteration value for the current iteration number based on the iteration rate and the total gradient value.

[0011] In some specific implementation schemes, the total gradient value D is calculated as follows: D=Σ(W_P*(Z i -P) 2 ) Where W_P represents the weight matrix, P represents the window data matrix containing the pixel to be identified in the current iteration, and Z... i This represents the gradient descent iteration value calculated in the previous iteration.

[0012] In some specific implementations, the gradient descent iteration value Z for the current iteration number is calculated. i+1 : Z i+1 =Z i -mu*D; Where mu represents the iteration rate.

[0013] Secondly, this application provides an adaptive gradient descent optimized image bad pixel compensation system, comprising: The image receiving module is used to receive image data streams acquired by the shortwave infrared sensor; The adaptive gradient descent iterative value calculation module is used to extract a number of image frames from the image data stream at preset intervals and store them as an image group, and to calculate the gradient descent iterative value of the image group using the adaptive gradient descent iterative algorithm. The adaptive bad pixel identification and compensation module is used to traverse each pixel of each image frame in the image group, estimate the estimated pixel value corresponding to each pixel based on the gradient descent iteration value, and calculate the decision factor based on the difference between the pixel value of each pixel and its corresponding estimated pixel value. The system determines whether each pixel is a bad pixel by comparing its decision factor with a preset threshold. If a pixel is a bad pixel, the estimated pixel value is used to replace the bad pixel value.

[0014] The beneficial effects of this invention are as follows: This invention optimizes the algorithm to learn the spatial correlation features of images and leverages the parallel pipeline processing capability of FPGA to decompose the recognition and compensation steps into parallel logical operations. The processing time of each pixel is completed within a few clock cycles, matching the data throughput of camera pixels, enabling real-time bad pixel processing frame by frame and line by line with low latency, thus achieving real-time image processing. Median filtering is used to identify obvious bad pixels, and gradient descent iterative calculations are performed only on normal pixel data. An adaptive gradient descent algorithm is employed to calculate adaptive gradient descent iterative values, which are then applied to neighboring pixels to calculate a predicted value. Bad pixels are identified by comparing the predicted value with the actual value. An adaptive decision factor based on gradient descent dynamically adjusts the decision sensitivity according to the characteristics of local image data, reducing the false detection rate in complex scenes and significantly improving the accuracy of bad pixel identification. The identification process exhibits high adaptability and accuracy. This invention can also identify new dead pixels caused by factors such as component aging and external interference after the camera leaves the factory. It can dynamically identify dead pixels, improve the long-term reliability and environmental adaptability of the camera, and thus achieve real-time, adaptive, and high-precision identification and compensation of dead pixels. Attached Figure Description

[0015] Figure 1 A flowchart of the adaptive gradient descent optimized image defect compensation method provided in this embodiment of the invention; Figure 2 This is a block diagram of an image defect compensation system optimized by adaptive gradient descent, provided in an embodiment of the present invention. Figure 3 This is a pixel distribution diagram in the window data provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0018] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0019] Furthermore, for clarity and brevity, descriptions of well-known structures, functions, and configurations may have been omitted. Those skilled in the art will recognize that various changes and modifications can be made to the examples described herein without departing from the spirit and scope of this disclosure.

[0020] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0021] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0022] Example 1 like Figure 1 As shown, this embodiment provides an adaptive gradient descent optimized image defect compensation method, which specifically includes the following steps: S1. Receive the image data stream collected by the shortwave infrared sensor, and extract several image frames from the image data stream at preset intervals to store them as an image group; S2. Calculate the gradient descent iteration value of the image group using the adaptive gradient descent iterative algorithm; The specific process of step S2 is as follows: S21. Select the first frame image from the image group, calculate the gradient descent iteration value, and store the first frame image into the FIFO cache module row by row and pixel by pixel. S22. Set up a sliding window, traverse each pixel to be identified in the first frame image, and use the sliding window to read the window data centered on the pixel to be identified from the FIFO buffer module in turn. To ensure accuracy in subsequent iterative calculations, the pixels to be identified are pre-identified to remove obviously defective pixels and avoid their participation in subsequent calculations. Therefore, when setting up a sliding window to traverse each pixel to be identified in the first frame image, the following steps are also included: When traversing each pixel to be identified, the window data containing each pixel is processed by median filtering, and the pixel to be identified is marked according to the result of median filtering, thus obtaining the marked window data; The specific process of median filtering is as follows: The median value of each pixel is calculated by performing median calculation on the data of all pixels in the window data except the pixel to be identified. Calculate the difference between the data of the pixel to be identified and the median value of the pixel; The difference is compared with a preset pixel threshold, and the pixel to be identified is marked according to the comparison result.

[0023] When the difference is greater than the preset pixel threshold, the mask of the pixel to be identified is marked as abnormal; when the difference is less than or equal to the preset pixel threshold, the mask of the pixel to be identified is marked as normal.

[0024] The following calculation of neighborhood gradients will only select the window data containing the pixels to be identified with normal labels for neighborhood gradient calculation, so as to obtain the neighborhood gradient matrix of each pixel to be identified with normal labels and the corresponding gradient descent iteration value.

[0025] S23. Traverse the window data of each pixel to be identified that has a normal label: Calculate the neighborhood gradient between the pixel to be identified and its neighboring pixels within the window data to obtain the neighborhood gradient value matrix corresponding to the window data; S24. Adaptively calculate the corresponding weight value for each neighborhood gradient value in the neighborhood gradient value matrix to obtain the weight matrix. Perform iterative calculation based on the window data and the weight matrix to obtain the gradient descent iteration value corresponding to the window data of each pixel to be identified. The specific process of adaptively calculating the corresponding weight value for each neighborhood gradient value in the neighborhood gradient value matrix in step S24 is as follows: For the j-th neighborhood gradient value DIFF_P(j), its corresponding weight value W_P(j) is: W_P(j)=1 / (1+DIFF_P(j)*DIFF_P(j)).

[0026] The specific process of iterative calculation is as follows: Get the gradient descent iteration value calculated in the previous iteration corresponding to the current iteration number; Based on the window data and weight matrix of the pixel to be identified in the current iteration, and the gradient descent iteration value calculated in the previous iteration, calculate the total gradient value D within the sliding window in the current iteration: D=Σ(W_P*(Z i -P) 2 ) Where W_P represents the weight matrix, P represents the window data matrix containing the pixel to be identified in the current iteration, and Z... i This represents the gradient descent iteration value calculated in the previous iteration.

[0027] Calculate and update the gradient descent iteration value Z for the current iteration number based on the iteration rate and the total gradient value. i+1 : Z i+1 =Z i -mu*D; Where mu represents the iteration rate.

[0028] S25. Repeat steps S22-S24 until all the window data of each pixel to be identified with a normal label in the first frame image has been traversed. The gradient descent iteration value corresponding to the last pixel to be identified with a normal label is taken as the gradient descent iteration value of the image group.

[0029] S3. Traverse each pixel of each image frame in the image group, estimate the estimated pixel value of each pixel based on the gradient descent iteration value, and calculate the decision factor based on the difference between the pixel value of each pixel and its corresponding estimated pixel value. S4. Determine whether each pixel is a bad pixel based on the comparison result between the decision factor of each pixel and the preset threshold. If it is a bad pixel, replace the bad pixel value with the estimated pixel value.

[0030] It is understood that the image data in this embodiment originates from a short-wave infrared sensor, and the method is implemented on FPGA-based short-wave infrared camera products. The image data from the InGaAs short-wave infrared sensor is output to the FPGA in a row-by-row, pixel-by-pixel image data stream manner. After receiving the image data stream, the FPGA uses a FIFO to buffer the data, caching the data of all pixels in the current row and neighboring rows in the FIFO buffer module. Based on the cached pixel data of the current row and neighboring rows, pipelined neighboring pixel data is generated to facilitate the extraction of neighboring pixel data. To identify each pixel in each frame of an image group, since the states of adjacent frames are generally relatively stable and the resulting gradient descent iteration value Z is also relatively fixed, several frames within a set interval (e.g., ten frames) can be stored as a group of images to be identified. For each pixel in each frame of the image group, in order to obtain a gradient descent iteration value, the first frame of the image group is selected to calculate the gradient descent iteration value, and the gradient descent iteration value of the first frame is used as the gradient descent iteration value of the entire image group.

[0031] Calculating the gradient descent iterative values ​​of the first frame image requires calculating the values ​​of each pixel in the first frame image. For each pixel in the first frame image, identification and iterative calculation are performed. Taking a 3x3 sliding window as an example, the pixel data read by each sliding window forms a 3x3 matrix centered on the pixel to be identified, such as... Figure 3 As shown, in this 3x3 matrix, C(m) is the m-th column, R(n) is the n-th row, P(n,m) is the pixel in the m-th column of the n-th row, and P(n,m) is the pixel to be identified. To improve the accuracy of subsequent defective pixel identification and ensure that only normal pixel data is used for gradient descent iteration calculation, median filtering is applied to the 3x3 window data. Median filtering is used to filter out obvious defects and record their masks. Obvious defects are not included in subsequent gradient descent iteration calculations. This ensures that only normal pixel data is used for gradient descent iteration calculations: The median of the data in the 3x3 window excluding the current pixel to be identified, P(n,m), is calculated to obtain the median of the pixels surrounding P(n,m). The difference Dm between the calculated median and the current pixel P(n,m) is compared with a threshold Tm. If the absolute value of Dm is greater than Tm, the current pixel mask is kept to 0; if the absolute value of Dm is not greater than Tm, the current pixel mask is kept to 1. The difference between the median and the current pixel P(n,m) is used to filter whether P(n,m) is an obvious defective pixel. Obvious defects are not included in subsequent gradient descent iteration calculations. For pixel windows with a mask of 0, subsequent gradient descent iterations are not performed. Subsequent gradient descent iterations are only performed in the next 3x3 sliding window with a mask of 1.

[0032] The window data containing the normal pixel to be identified, whose mask keep is 1, is selected for neighborhood gradient calculation. The gradient descent iteration value Z is obtained through adaptive learning using the normal pixel to be identified. For the current normal pixel to be identified P(n,m), an adaptive decision factor G(n,m) is calculated in parallel using the following steps: 1. Calculate the neighborhood gradient: Calculate the gradient value DIFF_P between the current pixel and its neighboring pixels. DIFF_P=([DIFF_P(1),DIFF_P(2),DIFF_P(3)], [DIFF_P(4),0,DIFF_P(5)], [DIFF_P(6),DIFF_P(7),DIFF_P(8)]); Right now: DIFF_P(1)=P(n,m)-P(n-1,m-1) DIFF_P(2)=P(n,m)-P(n-1,m) DIFF_P(3)=P(n,m)-P(n-1,m+1) DIFF_P(4)=P(n,m)-P(n,m-1) DIFF_P(5)=P(n,m)-P(n,m-1) DIFF_P(6)=P(n,m)-P(n+1,m-1) DIFF_P(7)=P(n,m)-P(n+1,m) DIFF_P(8)=P(n,m)-P(n+1,m+1) 2. Calculate the weights: Calculate the adaptive weights W_P(1), W_P(2), W_P(3), W_P(4), W_P(5), W_P(6), W_P(7), W_P(8) for each gradient value.

[0033] W_P=([W_P(1),W_P(2),W_P(3)], [W_P(4),0,W_P(5)], [W_P(6),W_P(7),W_P(8)]); The weights are inversely proportional to the absolute value of the gradient. The relationship between weights and gradient values ​​is as follows: W_P(j) = 1 / (1 + DIFF_P(j) * DIFF_P(j)); 3. Iterative calculation of gradient descent: a. Initialization: Z0=([0,0.25,0], [0.25,0,0.25], [0,0.25,0]); b. Calculate the gradient value of the window data: D=Σ(W_P*(Z) i -P) 2Zi is the i-th iteration; P represents the pixel matrix composed of all pixel values ​​within the 3x3 window where the currently normal pixel to be identified P(n,m) is located; the number of iterations of the first frame image is equal to the number of 3x3 windows with the center pixel mask keep set to 1, and the Z iteration value will eventually converge to a relatively stable iteration value.

[0034] Understandably, for a group of images, after iteratively calculating the gradient descent iteration value Z for all pixels in the first frame with a mask keep of 1, we obtain Z. Here, Z is a two-dimensional array with dimensions the same as the window size. For example, in a 3x3 window, Z is a 3x3 array. Only one iteration value of Z exists and is continuously updated with each iteration. This applies to all image frames in the image group and all pixels within those frames.

[0035] c. Update iteration value: Z i+1 =Zi-mu*D; mu is the iteration rate, with a default value of 0.01. The iteration rate can be modified through register configuration. 4. Calculate the decision factor: Multiply and add the window matrix data of the pixel P(n,m) to be detected in the data set with the gradient descent iteration value Z corresponding to the data set to obtain the estimated value of pixel P(n,m) P0(n,m)=Σ(P*Z). Therefore, the decision factor is the absolute value of the difference between the estimated value and the actual pixel value: G(n,m)=|P(n,m)-P0(n,m)|. The decision factor G(n,m) is compared with the threshold T, which can be dynamically adjusted. When the decision factor G(n,m) is greater than the threshold T, it indicates that the current pixel P(n,m) is a bad pixel and needs compensation; when the decision factor G(n,m) is not greater than the threshold T, it indicates that the current pixel P(n,m) is a normal pixel; when the decision factor G(n,m) in S06 is greater than the threshold T, the estimated value P0(n,m) of P(n,m) is used instead.

[0036] The key point of this embodiment is that it first eliminates obvious bad pixels in the image by median filtering, calculates the adaptive gradient descent iteration value, applies the adaptive gradient descent iteration value to the neighboring pixels, calculates the estimated value of the pixel, and identifies bad pixels by judging the difference between the estimated value and the actual value, thereby realizing real-time, adaptive, and high-precision identification and compensation of bad pixels.

[0037] Example 2 like Figure 2 As shown, this embodiment provides an adaptive gradient descent optimized image defect compensation system, including: The image receiving module is used to receive the image data stream acquired by the shortwave infrared sensor. The specific implementation process is as follows: S01, Image Reception: The image data of the InGaAs short-wave infrared sensor is output to the FPGA line by line and pixel by pixel. The image receiving module transmits the received image data to the adaptive gradient descent iterative value calculation module and the adaptive bad pixel identification and compensation module respectively. The adaptive gradient descent iterative value calculation module is used to extract a number of image frames from the image data stream at preset intervals and store them as an image group, and to calculate the gradient descent iterative value of the image group using the adaptive gradient descent iterative algorithm. The adaptive gradient descent iterative value calculation module includes a cascaded adaptive learning image frame selection module, BRAM memory, row FIFO cache module, neighborhood data cache module, median filtering preprocessing module, neighborhood gradient calculation module, weight calculation module, gradient descent iterative value calculation, and gradient descent iterative value update module. S02. The adaptive learning image frame selection module has an image selection interval parameter, which is set to 10 here. That is, every 10 frames of images are output and stored in BRAM. The adaptive gradient descent iteration value does not need to be trained for each frame. S03. Read the image data from the BRAM and store it in the row FIFO buffer module; S04, The neighborhood data caching module generates a 3x3 window of the current pixel based on the image data cached in the row FIFO; S05, Median filtering preprocessing module, calculates the median of the data in the 3x3 window excluding the current pixel, compares the difference Dm between the calculated median and the current pixel data with the threshold Tm, and sets the current pixel mask keep to 0 when the absolute value of Dm is greater than Tm, and sets the current pixel mask keep to 1 when the absolute value of Dm is not greater than Tm. S06, Neighborhood Gradient Calculation Module: Selects the pixel with a mask keep of 1 for the current pixel and calculates its neighborhood gradient. A subtractor is used to calculate the gradient value DIFF_P between the current pixel and its neighboring pixels. DIFF_P=([DIFF_P(1),DIFF_P(2),DIFF_P(3)], [DIFF_P(4),0,DIFF_P(5)], [DIFF_P(6),DIFF_P(7),DIFF_P(8)]); Right now: DIFF_P(1)=P(n,m)-P(n-1,m-1) DIFF_P(2)=P(n,m)-P(n-1,m) DIFF_P(3)=P(n,m)-P(n-1,m+1) DIFF_P(4)=P(n,m)-P(n,m-1) DIFF_P(5)=P(n,m)-P(n,m-1) DIFF_P(6)=P(n,m)-P(n+1,m-1) DIFF_P(7)=P(n,m)-P(n+1,m) DIFF_P(8)=P(n,m)-P(n+1,m+1) S07, Calculate Weights Module: Use multipliers, adders, and dividers to calculate the adaptive weights W_P(1), W_P(2), W_P(3), W_P(4), W_P(5), W_P(6), W_P(7), W_P(8) for each gradient value. Weight W_P(j) = 1 / (1 + DIFF_P(j) * DIFF_P(j)) S08. This step initializes iterative values ​​upon camera startup; it is not executed at other times. Z0=([0,0.25,0], [0.25,0,0.25], [0,0.25,0]); S09. Use multipliers, subtractors, and adders to calculate the gradient value: Gradient value D = Σ(W_P*(Z)). i -P) 2 Z i For the i-th iteration; S10. Calculate iterative values ​​using multipliers and subtractors: Z i+1 =Z i -mu*D;mu represents the iteration rate, with a default value of 0.01. The iteration rate can be modified through register configuration. S11. Repeat S05~S10 until the gradient descent iteration value corresponding to the last pixel of the frame image is calculated, then update the gradient descent iteration value Z. S12. After a frame of image is output and stored in BRAM in S02, S03~S11 are executed repeatedly. The adaptive bad pixel identification and compensation module is used to traverse each pixel of each image frame in the image group, estimate the estimated pixel value corresponding to each pixel based on the gradient descent iteration value, and calculate the decision factor based on the difference between the pixel value of each pixel and its corresponding estimated pixel value. The system determines whether each pixel is a bad pixel by comparing its decision factor with a preset threshold. If a pixel is a bad pixel, the estimated pixel value is used to replace the bad pixel value.

[0038] The adaptive bad pixel identification and compensation module includes a cascaded row FIFO buffer module, a neighborhood data buffer module, an adaptive judgment factor module, a bad pixel identification module, and an image compensation output module. The specific implementation process is as follows: S13. The image data of the adaptive bad pixel identification and compensation module is stored in the row FIFO cache module; S14. The neighborhood data caching module generates a 3x3 window of the current pixel based on the image data cached in the row FIFO. S15, the adaptive decision factor module, based on the pixel point P(n,m) from S14 and the gradient descent iteration value Z from S11, uses multipliers and adders to estimate the value and calculate P0(n,m)=Σ(P*Z). It then uses a subtractor to calculate the decision factor G(n,m)=|P(n,m)-P0(n,m)|.

[0039] S16, Defect Pixel Detection Module: Compares the decision factor G(n,m) with the threshold T, which can be dynamically adjusted. When the decision factor G(n,m) is greater than the threshold T, it indicates that the current pixel P(n,m) is a defective pixel and needs compensation; when the decision factor G(n,m) is not greater than the threshold T, it indicates that the current pixel P(n,m) is a normal pixel. S17, Image Compensation Output Module: When the decision factor G(n,m) in S16 is greater than the threshold T, the estimated value P0(n,m) of P(n,m) is used instead.

[0040] Example 3 This embodiment provides a computer-readable storage medium, including: One or more processors; A storage unit is used to store one or more programs that, when executed by one or more processors, enable the one or more processors to implement the adaptive gradient descent optimized image defect compensation method described in the first aspect.

[0041] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An adaptive gradient descent optimized image bad pixel compensation method, characterized in that, Specifically comprising the following steps: S1, receiving image data stream collected by a short-wave infrared sensor, and storing a plurality of image frames in the image data stream as an image group at a preset interval; S2, calculating a gradient descent iteration value of the image group by using an adaptive gradient descent iteration algorithm; S3, traversing each pixel point of each image frame in the image group, estimating a predicted pixel value corresponding to each pixel point according to the gradient descent iteration value, and calculating a decision factor according to a difference between the pixel value of each pixel point and the predicted pixel value corresponding thereto; S4, judging whether each pixel point is a bad pixel according to a comparison result of the decision factor of each pixel point and a preset threshold, and replacing the bad pixel value with the predicted pixel value if the pixel point is a bad pixel.

2. The method of claim 1, wherein, The specific process of step S2 is as follows: S21, selecting a first image from the image group to calculate the gradient descent iteration value, and storing the first image in the FIFO cache module in a row-by-pixel point manner; S22, setting a sliding window, traversing each to-be-identified pixel point in the first image, and reading out window data centered on the to-be-identified pixel point from the FIFO cache module in turn by using the sliding window; S23, calculating a neighborhood gradient between the to-be-identified pixel point and the neighborhood pixel points in the window data, and obtaining a neighborhood gradient value matrix corresponding to the window data; S24, adaptively calculating a weight value corresponding to each neighborhood gradient value in the neighborhood gradient value matrix to obtain a weight matrix, and iteratively calculating according to the window data and the weight matrix to obtain a gradient descent iteration value corresponding to the window data in which each to-be-identified pixel point is located; S25, repeating steps S22-S24 until each to-be-identified pixel point in the first image is traversed, and taking the gradient descent iteration value corresponding to the last to-be-identified pixel point as the gradient descent iteration value of the image group.

3. The method of claim 2, wherein, When the sliding window is set to traverse each to-be-identified pixel point in the first image, the method further comprises the following steps: When each to-be-identified pixel point is traversed, the window data in which the to-be-identified pixel point is located is processed by using median filtering, and the to-be-identified pixel point is marked according to the median filtering result to obtain marked window data; The window data in which the to-be-identified pixel point with a normal mark is located is selected to calculate the neighborhood gradient, and a neighborhood gradient matrix of each to-be-identified pixel point with a normal mark and a corresponding gradient descent iteration value are obtained.

4. The image bad pixel compensation method of claim 3, wherein, The specific process of processing the window data by using median filtering is as follows: Median values of all pixel points except the to-be-identified pixel point in the window data are calculated to obtain a pixel median value; A difference between the data of the to-be-identified pixel point and the pixel median value is calculated; The difference is compared with a preset pixel threshold, and the to-be-identified pixel point is marked according to the comparison result.

5. The method of claim 4, wherein, The specific process of marking each to-be-identified pixel point according to the comparison result is as follows: When the difference is greater than the preset pixel threshold, the mask of the to-be-identified pixel point is marked as abnormal; and when the difference is less than or equal to the preset pixel threshold, the mask of the to-be-identified pixel point is marked as normal.

6. The method of claim 2, wherein, The specific process of adaptively calculating a weight value corresponding to each neighborhood gradient value in the neighborhood gradient value matrix in step S24 is as follows: For the jth neighborhood gradient value DIFF_P(j), its corresponding weight value W_P(j) is: W_P(j)=1 / (1+DIFF_P(j)*DIFF_P(j)).

7. The method of claim 2, wherein the method further comprises: The specific process of iterative calculation is: obtaining a gradient descent iteration value calculated after last iteration corresponding to the current iteration number; calculating a total gradient value in the sliding window according to the window data and the weight matrix of the pixel point to be identified in the current iteration number and the gradient descent iteration value calculated after the last iteration; calculating and updating the gradient descent iteration value of the current iteration number according to the iteration rate and the total gradient value.

8. The method of claim 7, wherein, The calculation method of the total gradient value D is: D =∑(W_P*(Z i -P) 2 ) Wherein, W P represents a weight matrix, P represents a window data matrix in which a pixel point to be identified is located in the current iteration number, Z i represents the gradient descent iteration value after the last iteration calculation.

9. The method of claim 8, wherein, Computing gradient descent iteration value Z for current iteration number i+1 : Z i+1 =Z i -mu*D; Wherein, mu represents the iteration rate.

10. An adaptive gradient descent optimized image bad pixel compensation system, comprising: It comprises: An image receiving module configured to receive image data streams collected by a short-wave infrared sensor; An adaptive gradient descent iteration value calculation module configured to extract a plurality of image frames from the image data streams as an image group at a preset interval, and calculate a gradient descent iteration value of the image group by using an adaptive gradient descent iteration algorithm; An adaptive bad pixel identification and compensation module configured to traverse each pixel point of each image frame in the image group, estimate an estimated pixel value corresponding to each pixel point according to the gradient descent iteration value, and calculate a decision factor according to the difference between the pixel value of each pixel point and the estimated pixel value corresponding thereto; Judge whether each pixel point is a bad pixel according to the comparison result of the decision factor of each pixel point and a preset threshold, and replace the bad pixel value with the estimated pixel value if it is a bad pixel.

Images