Image correction method and device and electronic equipment
By combining a decision tree model and dimensionality reduction information with an image correction method based on pixel feature values and noise matrix in infrared imaging devices, the problem of non-uniform response in infrared imaging devices is solved, and high-precision fixed noise correction, especially effective suppression of low-frequency noise, is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-03-10
AI Technical Summary
Existing infrared imaging equipment suffers from low correction accuracy in addressing non-uniform response issues, particularly in its insufficient ability to correct low-frequency noise, making it difficult to track the slowly changing noise drift phenomenon between frames caused by detector temperature drift.
By using the pixel feature values and noise matrix of the current image frame and the previous image frame, the target pixel that needs to be corrected is locked. The decision tree model is used to determine the motion state of the device. Combined with the dimensionality reduction information, the image frame that needs to be corrected is selected. The noise value is determined and updated by multi-frame iteration, so as to achieve integrated suppression of high and low frequency noise.
It significantly improves the correction accuracy of non-uniform response, overcomes the difficulties of traditional baffled devices in correcting low-frequency noise, and achieves high-precision positioning and correction of fixed noise sources.
Smart Images
Figure CN121639504A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to image correction methods, apparatus and electronic devices. Background Technology
[0002] In real-world scenarios, common image acquisition devices, such as infrared thermal imaging devices, often suffer from non-uniform response problems. This is mainly caused by the manufacturing process of the detector in the image acquisition device. Non-uniform response can significantly affect image quality, manifesting as severe fixed pattern noise.
[0003] Currently, some infrared imaging devices that include baffles can mitigate the impact of fixed pattern noise by periodically applying baffles. However, the baffle-based approach is difficult to track the slow-changing noise drift caused by detector temperature drift between frames. In particular, it lacks the ability to correct low-frequency noise such as vignetting and wide vertical stripes, resulting in low correction accuracy for non-uniform responses. Summary of the Invention
[0004] In view of this, this application provides an image correction method, apparatus, and electronic device to improve the correction accuracy for non-uniform responses.
[0005] The technical solution provided in this application is as follows: According to an embodiment of the first aspect of this application, an image correction method is provided, the method comprising: For the current image frame that needs correction, based on the pixel feature values of each pixel in the current image frame, the pixel feature values of each pixel in the previous image frame, and the current noise matrix, target pixels to be corrected are selected from the current image frame. Specifically, the difference between the pixel feature value of the target pixel and the pixel feature value of the reference pixel in the previous image frame is within a set difference range; the position of the target pixel in the current image frame is the same as the position of the reference pixel in the previous image frame; the current noise matrix is determined based on the noise values of pixels in the most recently corrected image frame; and the current noise matrix contains noise values that match the target pixels. For each target pixel, the noise value of the target pixel is determined based on the pixel feature value of the target pixel, the pixel feature value of the reference pixel of the target pixel in the previous image frame, the global pixel feature value used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel; the global pixel feature value of the previous image frame is determined based on the pixel feature values of each pixel in the previous image frame. The target pixel is corrected based on its noise value.
[0006] According to an embodiment of a second aspect of this application, an image correction apparatus is provided, the apparatus comprising: The selection unit is used to select target pixels to be corrected from the current image frame based on the pixel feature values of each pixel in the current image frame, the pixel feature values of each pixel in the previous image frame, and the current noise matrix. Specifically, the difference between the pixel feature value of the target pixel and the pixel feature value of a reference pixel in the previous image frame is within a set difference range; the position of the target pixel in the current image frame is the same as the position of the reference pixel in the previous image frame; the current noise matrix is determined based on the noise values of pixels in the most recently corrected image frame; and the current noise matrix contains noise values that match the target pixel. The determining unit is configured to, for each target pixel, determine the noise value of the target pixel based on the pixel feature value of the target pixel, the pixel feature value of the reference pixel of the target pixel in the previous image frame, the global pixel feature value used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel; wherein the global pixel feature value of the previous image frame is determined based on the pixel feature values of each pixel in the previous image frame; The correction unit is used to correct the target pixel based on the noise value of the target pixel.
[0007] According to an embodiment of a third aspect of this application, an electronic device is provided, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute the machine-executable instructions to implement the method described in the first aspect.
[0008] As can be seen from the above technical solutions, this application locks down the target pixel that needs to be corrected by using the pixel feature values of each pixel in the current image frame, the pixel feature values of each pixel in the previous image frame, and the noise value recorded in the current noise matrix. This significantly improves the positioning accuracy of fixed noise sources that cause non-uniform responses. Furthermore, for each target pixel, the noise value of the target pixel is determined based on its pixel feature values, the pixel feature values of the reference pixel in the previous image frame, the global pixel feature values used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel. This effectively suppresses both high and low frequency noise in an integrated manner, overcoming the limitation of traditional baffled devices that have difficulty correcting low frequency noise, and improving the correction accuracy for non-uniform responses. Attached Figure Description
[0009] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.
[0010] Figure 1This is a schematic flowchart of the image correction method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the current image frame acquisition method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the overall image correction process provided in the embodiments of this application; Figure 4 This is a schematic diagram showing the comparison of correction results provided in the embodiments of this application; Figure 5 This is a structural diagram of the image correction device provided in an embodiment of this application; Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation
[0011] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.
[0012] In real-world scenarios, common image acquisition devices, such as infrared thermal imaging devices, often suffer from non-uniform response problems. Taking infrared thermal imaging devices as an example, the non-uniform response problem is mainly caused by the detector manufacturing process. Non-uniform response will significantly affect image quality, manifesting as severe fixed pattern noise.
[0013] In related technologies, some infrared thermal imaging devices with baffles can mitigate this phenomenon by periodically applying baffles (i.e., periodically using baffles to block the lens), but it is difficult to eliminate it completely. Devices without baffles cannot update the background in real time, so the fixed pattern noise is more serious, especially the low-frequency noise. However, the solutions in related technologies generally have insufficient ability to handle large-area low-frequency noise, resulting in low correction accuracy for non-uniform responses.
[0014] Based on this, this application proposes an image correction method to improve the correction accuracy for non-uniform responses.
[0015] Please refer to Figure 1 , Figure 1 This is a schematic flowchart of the image correction method provided in the embodiments of this application.
[0016] In this embodiment, the image to be corrected can be an infrared thermal imaging image, an industrial linear scan image, etc., and this application does not impose any limitations.
[0017] Because the non-uniform response problem of infrared thermal imaging equipment is particularly serious and difficult to correct, this application will describe the correction of infrared thermal imaging images as an example.
[0018] like Figure 1 As shown, the method includes the following steps: Step 101: For the current image frame that needs to be corrected, select the target pixel that needs to be corrected from the current image frame based on the pixel feature values of each pixel in the current image frame, the pixel feature values of each pixel in the previous image frame, and the current noise matrix.
[0019] The target pixel here is the pixel in the current image frame that needs to be corrected. There can be one or more target pixels. The pixel corresponding to the target pixel in the previous image frame can be called the reference pixel. That is, the position of the target pixel in the current image frame is the same as the position of the reference pixel in the previous image frame.
[0020] In this embodiment, the difference between the pixel feature value of the target pixel and the pixel feature value of the reference pixel in the previous image frame is within a set difference range. The specific method for determining the target feature point will be described in detail below, and will not be repeated here.
[0021] As an example, taking an infrared thermal imaging image frame as the current image frame, the pixel feature value of a pixel in the current image frame can be the grayscale value of that pixel. When the current image frame is another type of image frame, the pixel feature value of a pixel can be the original output value of each pixel of the sensor, and this application does not impose any limitations on this.
[0022] It should be noted that the current noise matrix here is determined based on the noise values of pixels in the most recent image frame that has been corrected. That is, the noise matrix is only updated in image frames that need correction; in image frames that do not need correction, the latest noise matrix can be applied directly. The current noise matrix contains the noise values that match each target pixel.
[0023] As an example, the size of the noise matrix can be the same as the size of the pixel matrix of the current image frame. Each element in the noise matrix corresponds to a pixel in the pixel matrix, and the elements in the noise matrix are the noise values matched by the corresponding pixels in the pixel matrix.
[0024] This concludes the description of step 101. We will now proceed to step 102.
[0025] Step 102: For each target pixel, determine the noise value of the target pixel based on the pixel feature value of the target pixel, the pixel feature value of the reference pixel of the target pixel in the previous image frame, the global pixel feature value used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel.
[0026] In this embodiment, after determining the target pixel that needs to be corrected, the noise value of each target pixel can be further determined. Here, the global pixel feature value of the previous image frame is determined based on the pixel feature value of each pixel in the previous image frame.
[0027] As an example, the global pixel feature value can be the average value of the pixel feature values of each pixel in the previous image frame (such as the average value of grayscale), or the median value of the pixel feature values of each pixel in the previous image frame (such as the median value of grayscale). This application does not limit this. The following description takes the example that the global pixel feature value is the average value of the pixel feature values of each pixel in the previous image frame.
[0028] This concludes the description of step 102. We will now proceed to step 103.
[0029] Step 103: Correct the target pixel based on the noise value of the target pixel.
[0030] In this embodiment, after determining the noise value of each target pixel, the pixel feature value of the target pixel can be corrected based on the noise value of the target pixel.
[0031] The process of correcting the target pixels will be described in detail below, and will not be repeated here.
[0032] This concludes the discussion on... Figure 1 The description.
[0033] This application identifies the target pixel to be corrected by using the pixel feature values of each pixel in the current image frame, the pixel feature values of each pixel in the previous image frame, and the noise values recorded in the current noise matrix. This significantly improves the localization accuracy of fixed noise sources that cause non-uniform responses. Furthermore, for each target pixel, the application determines its noise value based on its pixel feature values, the pixel feature values of a reference pixel in the previous image frame, the global pixel feature values representing the previous image frame, and the noise values in the current noise matrix that match the target pixel's position. This integrated and effective suppression of high and low frequency noise overcomes the limitation of traditional baffled devices, which struggle to correct low-frequency noise, and improves the correction accuracy for non-uniform responses.
[0034] The following section first introduces the method for obtaining the current image frame that needs to be corrected in step 101: In this embodiment, the current frame image contains data acquired by the image acquisition device under the current dimensional direction information. For example, when the image acquisition device is an infrared thermal imaging device, the infrared thermal imaging device also includes a gyroscope. The dimensional direction information here includes at least pitch angle and yaw angle, and may also include roll angle.
[0035] The dimensionality information used by the image acquisition device when acquiring data is reduced to obtain dimensionality information; the dimensionality information used by the image acquisition device when acquiring data includes at least the pitch angle and yaw angle of the image acquisition device when acquiring data.
[0036] The dimensionality reduction information is input into the trained decision tree model to obtain the state of the image acquisition device during the data acquisition process. The trained decision tree model is obtained by training the decision tree model after dimensionality reduction processing of the sample dimensionality and orientation information of the acquired motion state, vibration state and static state. If the image acquisition device is determined to be in motion during the data acquisition process, then if the dimensionality reduction information meets the requirements for low-frequency noise correction of the image, the current image frame is determined to be the current image frame that needs to be corrected.
[0037] In this embodiment, considering that the fixed pattern noise generated by the non-uniform response is a spatially fixed and temporally stable noise pattern, and that the fixed pattern noise does not change with the scene, it is usually possible to distinguish the fixed pattern noise from the current scene when the device is in motion. Therefore, when determining whether the current image frame is the current image frame that needs to be corrected, it is first necessary to determine the motion state and filter out the image frames acquired by the image acquisition device that is in motion when acquiring image data for correction.
[0038] As an example, motion state determination can be made using dimensional orientation information acquired by an image acquisition device. Taking an infrared thermal imaging device as an example, the gyroscope in the infrared thermal imaging device can acquire data in three directions in one frame, acquiring 16 sets of data (such as raw angular velocity data) in each direction, for a total of 48 data. Since the amount of data acquired in one frame is too large, the acquired dimensional orientation information can be reduced in dimensionality, for example, by reducing the dimensional orientation information to obtain four dimensional data.
[0039] Assuming the dimension information collected in the pitch direction is L1, the dimension information collected in the yaw direction is L2, and the dimension information collected in the roll direction is L3, the dimension-reduced data can be obtained using the following formula: in, This is pitch angle dimensionality reduction information, used to characterize the angular displacement in the pitch direction within one frame period using quantitative estimation. For the summation operator, This is the absolute value operator. L 1 This refers to the dimensional direction information collected in the pitch direction; The yaw angle is a dimensionality-reduced information used to represent a quantitative estimate of the angular displacement in the yaw direction within one frame period. L 2 The dimensional direction information collected in the yaw angle direction; This is a dimensionality reduction information of the roll angle, used to represent a quantized estimate of the angular displacement in the roll direction within one frame period. L 3 This refers to the dimensional direction information acquired in the roll angle direction; This is a dimensionality-reduced information representation of the fluctuation amplitude, used to characterize the sum of the maximum fluctuation amplitudes of the angular velocities in the three directions mentioned above. This is the range operator.
[0040] After obtaining the dimensionality reduction information, the information can be input into the trained decision tree model to obtain the state of the image acquisition device during the data acquisition process.
[0041] In this embodiment, a large amount of dimensionality information of motion, vibration and static states can be manually collected. This dimensionality information can be reduced to obtain sample dimensionality information. The obtained sample dimensionality information can then be used to train a decision tree to obtain the trained decision tree model.
[0042] Given that the image acquisition device is in motion during the data acquisition process, based on the decision tree model, and considering that low-frequency noise correction is usually effective only when there is motion in both the pitch and yaw directions, it is possible to further determine whether the dimensionality reduction information meets the requirements for low-frequency noise correction of the image.
[0043] As an example, the following formula can be used to determine whether the dimensionality reduction information meets the requirements for low-frequency noise correction of an image: in, For pitch angle dimensionality reduction information, For yaw angle dimensionality reduction information, The first pitch angle threshold, The first yaw angle threshold, The second pitch angle threshold, This is the second yaw angle threshold; here Much larger , Greater than , Less than , Greater than .
[0044] In this embodiment, if the pitch angle dimensionality reduction information is determined... and yaw angle dimensionality reduction information If the above conditions are met, then the dimensionality reduction information is determined to be consistent with the dimensionality reduction information required for low-frequency noise correction of the image.
[0045] If the current image frame is determined to be in motion state during the acquisition process of the image acquisition device, and the dimensionality reduction information meets the dimensionality reduction information required for low-frequency noise correction of the image, then the current image frame is determined to be the current image frame that needs to be corrected.
[0046] The following is based on Figure 2 The method for obtaining the current image frame that needs to be corrected is described in general.
[0047] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the current image frame acquisition method provided in an embodiment of this application.
[0048] like Figure 2 As shown, you can first obtain the dimensional orientation information used by the image acquisition device during the acquisition of the current image frame, such as the dimensional orientation information acquired by the gyroscope.
[0049] Since the amount of dimensional information data collected is too large, we can first perform dimensionality reduction processing on the information in each dimension to obtain dimensionality-reduced information.
[0050] Furthermore, the dimensionality reduction information obtained after dimensionality reduction is input into the trained decision tree model to determine the state of the acquisition device during the acquisition of the current image frame.
[0051] If it is determined that the image acquisition device is in motion during the acquisition of the current image frame, it can be further determined whether the dimensionality reduction information meets the dimensionality reduction information required for low-frequency noise correction of the image. If it is determined that the dimensionality reduction information meets the dimensionality reduction information required for low-frequency noise correction of the image, the current image frame is determined to be the current image frame that needs to be corrected.
[0052] This concludes the description of the method for obtaining the current image frame that needs correction.
[0053] The specific method for selecting the target pixels to be corrected from the current image frame is described below.
[0054] In this embodiment, for each pixel in the current image frame, the overall deviation between the joint pixel feature value of the pixel feature value of the current pixel and the pixel feature value of the corresponding pixel in the previous image frame and the global pixel feature value can be determined based on the pixel feature value of the current pixel, the pixel feature value of the corresponding pixel in the previous image frame, and the global pixel feature value. Furthermore, based on the pixel feature value of the pixel and the noise value of the corresponding pixel recorded in the noise matrix, a reference feature value of the pixel is determined; based on the reference feature value of the pixel and the global pixel feature value, a first degree of deviation between the reference feature value of the pixel and the global pixel feature value is determined. Furthermore, based on the pixel feature value of the corresponding pixel in the previous image frame and the noise value of the corresponding pixel recorded in the noise matrix, a reference feature value of the corresponding pixel in the previous image frame is determined. Based on the reference feature value of the corresponding pixel in the previous image frame and the global pixel feature value, a second degree of deviation between the reference feature value and the global pixel feature value of the corresponding pixel in the previous image frame is determined. If the overall deviation is less than the first threshold, the first deviation is less than the second threshold, and the second deviation is less than the second threshold, then the pixel is determined to be the target pixel.
[0055] Specifically, the current image frame data is denoted as Y, and the previous image frame data is denoted as Y'. Both the current and previous image frame data are the original images without noise correction. Here, we take the global pixel feature value as the average of the pixel feature values (such as grayscale values) of each pixel in the previous image frame as an example. The following judgment conditions can be used to judge each pixel in the current image frame: in, This represents the pixel feature value of the pixel located in the i-th row and j-th column of the current image frame. This represents the pixel feature value of the pixel located in the i-th row and j-th column of the previous image frame, and mean represents the global pixel feature value, which can be the average of the pixel feature values of all pixels in the previous image frame. This represents the noise value matched by the pixel located in the i-th row and j-th column of the current noise matrix. The noise matrix can have the same size as the pixel matrix of the current image frame, and each element in the noise matrix corresponds to a pixel in the pixel matrix. The first threshold, The second threshold is the threshold value, and the first threshold value can be greater than the second threshold value.
[0056] It should be noted that, It can be considered as the overall deviation between the joint pixel feature value that determines the pixel feature value of this pixel and the pixel feature value of the corresponding pixel in the previous image frame, and the global pixel feature value. This can be considered as the first degree of deviation between the reference feature value of this pixel and the global pixel feature value. It can be considered as the second degree of deviation between the reference feature value of the pixel at the corresponding position in the previous image frame and the global pixel feature value.
[0057] In this embodiment, if any pixel in the current image frame satisfies the above three conditions at the same time, it indicates that the pixel feature value of the pixel in the current frame and the previous frame does not deviate too much from the global pixel feature value, and the pixel feature value of the pixel does not change significantly between the two frames. At this time, the pixel can be considered as the target pixel that needs to be corrected.
[0058] This concludes the description of the method for identifying target pixels.
[0059] The process of determining the noise value of the target pixel is described below.
[0060] In this embodiment, the specific method for determining the noise value of the target pixel based on the pixel feature value of the target pixel, the pixel feature value of the reference pixel of the target pixel in the previous image frame, the global pixel feature value used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel may include: The first noise perturbation component is determined based on the pixel feature value of the target pixel and the global pixel feature value; the first noise perturbation component refers to the noise perturbation component of the target pixel in the current image frame. The second noise perturbation component is determined based on the pixel feature value of the reference pixel of the target pixel in the previous image frame and the global pixel feature value; the second noise perturbation component refers to the noise perturbation component of the reference pixel of the target pixel in the previous image frame. The noise value of the target pixel is determined based on the first noise perturbation component, the second noise perturbation component, and the noise value in the current noise matrix that matches the position of the target pixel.
[0061] In this embodiment, the noise value of the target pixel can be composed of three parts: the noise perturbation component of the target pixel in the current image frame (first noise perturbation component), which is used to characterize the direct observation contribution of the current image frame to the noise estimation of the pixel; the noise perturbation component of the reference pixel of the target pixel in the previous image frame (second noise perturbation component), which is used to characterize the auxiliary reference information of the previous image for noise estimation; and the noise value in the current noise matrix that matches the position of the target pixel, which is used to characterize the historical cumulative result of noise estimation.
[0062] Specifically, the noise value of the target pixel can be calculated using the following formula: Where, n The noise value of the target pixel located in the i-th row and j-th column. This represents the pixel feature value of the pixel located in the i-th row and j-th column of the current image frame. The mean represents the pixel feature value located at the i-th row and j-th column in the previous image frame, and the mean represents the global pixel feature value. This represents the noise value matched by the pixel located in the i-th row and j-th column of the current noise matrix. a is the first noise perturbation component parameter, b is the second noise perturbation component parameter, and c is the noise matrix perturbation component parameter.
[0063] Preferably, a can be 0.0062, b can be 0.0062, and c can be 0.9875.
[0064] The noise estimate is iteratively updated in small increments using parameters a, b, and c. a and b are set to smaller values to incorporate new observational information; c is close to 1 to maintain the stability of historical noise estimates. This avoids amplification of noise estimation errors caused by anomalies in single-frame images or sudden scene changes, achieving stable and adaptive non-uniform correction under unshielded conditions.
[0065] This concludes the description of the method for determining the noise value of the target pixel.
[0066] The following describes the specific method for correcting target pixels based on their noise values.
[0067] In this embodiment, the noise matrix can be updated based on the noise value of the target pixel to obtain the updated noise matrix; the updated noise matrix records the noise value corresponding to each pixel in the current image frame; the pixel feature value of each pixel in the current image frame is corrected based on the updated noise matrix.
[0068] As an example, the noise value matched by the target pixel in the noise matrix can be directly updated to the noise value of the target pixel obtained in step 102. The noise value of the target pixel can be directly subtracted from the pixel feature value of the target pixel to obtain the corrected pixel feature value of the target pixel. For other non-target pixels, the noise value matched by the non-target noise point in the current noise matrix can be directly subtracted to obtain the corrected pixel feature value of the non-target pixel, so as to complete the correction of the current image frame.
[0069] Considering that in real-world scenarios, there may be amplification of noise estimation errors due to single-frame image anomalies or sudden scene changes, this application can control the update step size based on the change in noise values when updating the noise matrix, thus avoiding sudden changes in noise values.
[0070] As an example, the noise step size corresponding to the target pixel can be determined based on the reference noise value of the target pixel and the noise value of the corresponding pixel in the previous frame of infrared image recorded in the noise matrix. For each target pixel, the noise value corresponding to the target pixel recorded in the noise matrix is updated according to the noise step size corresponding to the target pixel. For each non-target pixel, the noise value corresponding to the non-target pixel recorded in the noise matrix remains unchanged.
[0071] Specifically, for each target pixel, the update method for the matched noise value in the noise matrix can be determined according to the following formula: Where d is the change in noise value of the target pixel (denoted as noise step size), and n The noise value of the target pixel located in the i-th row and j-th column. This represents the noise value matched by the pixel located in the i-th row and j-th column of the current noise matrix. The first noise threshold, The first noise threshold, The first noise threshold, The first noise threshold, .
[0072] In this embodiment, the method for updating the noise value matching in the noise matrix can be determined according to the degree of change of the noise value of the target pixel. The larger the change d of the noise value of the target pixel, the more likely it is to be a sudden noise change. At this time, d can be controlled within a small range to avoid amplifying the correction error.
[0073] It should be noted that the above formula is merely exemplary and this application does not impose any limitations on it.
[0074] For non-target pixels, the noise value matched in the noise matrix does not need to be updated.
[0075] After updating the noise matrix, it can be seen that the noise matrix is continuously updated by time-domain calculation using the current frame's uncorrected image and the previous frame's uncorrected image as the image acquisition device moves. The noise value of the target pixel is also determined in real time, which is a multi-frame iterative process.
[0076] The corrected image Z is obtained by subtracting the updated noise matrix from Y, and can be displayed as a clean image.
[0077] Specifically: in, The current image frame after correction. This represents the current image frame before correction. This is the updated noise matrix.
[0078] This concludes the description of the specific method for correcting target pixels.
[0079] In this embodiment, if it is determined that the image acquisition device is not in motion during the data acquisition process, or if the dimensional direction information does not meet the dimensional direction information required for low-frequency noise correction of the image, then the pixel feature values of each pixel in the current image frame are corrected according to the current noise matrix.
[0080] In this embodiment, if the current image frame is not the current image frame that needs to be corrected, there is no need to update the current noise matrix. In this case, the latest noise matrix can be used directly for image correction.
[0081] The following is through Figure 3 The overall process of image correction proposed in this application is described.
[0082] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the overall image correction process provided in the embodiments of this application.
[0083] like Figure 3As shown, the target pixels that need to be corrected are first determined from the current image frame that needs to be corrected.
[0084] Furthermore, the noise value of the target pixel is determined based on the pixel feature value of the target pixel, the pixel feature value of the reference pixel of the target pixel in the previous image frame, the global pixel feature value used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel.
[0085] After determining the noise value of the target pixel, the noise matrix can be updated based on the noise value of the target pixel. The change in the noise value of the target pixel (denoted as the noise step size) can be determined, and different methods for updating the noise matrix can be selected based on different noise step sizes.
[0086] After updating the noise matrix, the current image frame can be corrected based on the noise matrix. That is, the noise value matched by each pixel in the noise matrix is subtracted from the pixel feature value of each pixel in the current image frame to obtain the corrected current image frame.
[0087] This concludes the discussion. Figure 3 The description.
[0088] Please refer to Figure 4 , Figure 4 This is a schematic diagram showing the comparison of correction results provided in the embodiments of this application.
[0089] like Figure 4 As shown, the left column of images consists of multiple infrared thermal imaging image frames acquired by an infrared thermal imaging device before correction. It can be seen that there is obvious low-frequency noise in the form of wide vertical stripes in the area outlined in the image before correction, which has a significant impact on the image imaging effect.
[0090] The right column shows the image obtained after correcting the left infrared thermal imaging image frame using the image correction method proposed in this application. As can be seen, in the area bounded by the box in the corresponding left image, the low-frequency noise in the form of wide vertical stripes has obviously disappeared, and the image imaging effect is better.
[0091] It should be noted that since the image correction method proposed in this application only needs to process the acquired image frame data to achieve joint correction of high and low frequency noise, it does not require a baffle to block the lens. Therefore, it can be used on image acquisition devices with or without baffles to solve the corresponding non-uniform correction problem.
[0092] Please refer to Figure 5 , Figure 5 This is a structural diagram of an image correction device proposed in an embodiment of this application. Figure 5As shown, the device may include a pixel selection unit 501, a determination unit 502, and a correction unit 503. Specifically, the device includes: Selection unit 501 is used to select target pixels to be corrected from the current image frame based on the pixel feature values of each pixel in the current image frame, the pixel feature values of each pixel in the previous image frame, and the current noise matrix. Specifically, the difference between the pixel feature value of the target pixel and the pixel feature value of the reference pixel in the previous image frame is within a set difference range; the position of the target pixel in the current image frame is the same as the position of the reference pixel in the previous image frame; the current noise matrix is determined based on the noise values of pixels in the most recently corrected image frame; and the current noise matrix contains noise values that match the target pixel. The determining unit 502 is configured to, for each target pixel, determine the noise value of the target pixel based on the pixel feature value of the target pixel, the pixel feature value of the reference pixel of the target pixel in the previous image frame, the global pixel feature value used to characterize the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel; the global pixel feature value of the previous image frame is determined based on the pixel feature values of each pixel in the previous image frame; The correction unit 503 is used to correct the target pixel based on the noise value of the target pixel.
[0093] Optionally, the current frame image contains data acquired by the image acquisition device under the current dimensional orientation information; the current image frame that needs to be corrected is determined according to the following method: The dimensionality and orientation information used by the image acquisition device when acquiring data is reduced to obtain dimensionality reduction information; the dimensionality and orientation information used by the image acquisition device when acquiring data includes at least the pitch angle and yaw angle of the image acquisition device when acquiring data; The dimensionality reduction information is input into the trained decision tree model to obtain the state of the image acquisition device during the data acquisition process. The trained decision tree model is obtained by training the decision tree model after dimensionality reduction processing of the sample dimensionality and orientation information of the acquired motion state, vibration state and static state. If the image acquisition device is determined to be in motion during the data acquisition process, then if the dimensionality reduction information meets the dimensionality reduction information required for low-frequency noise correction of the image, the current image frame is determined to be the current image frame that needs to be corrected. And / or, the selection unit 501 is specifically used for: For each pixel in the current image frame, based on the pixel feature value of the pixel, the pixel feature value of the corresponding pixel in the previous image frame, and the global pixel feature value, determine the joint pixel feature value between the pixel feature value of the current pixel and the pixel feature value of the corresponding pixel in the previous image frame, and the overall deviation between the joint pixel feature value of the current pixel and the global pixel feature value. Furthermore, based on the pixel feature value of the pixel and the noise value of the corresponding pixel recorded in the noise matrix, a reference feature value of the pixel is determined; based on the reference feature value of the pixel and the global pixel feature value, a first degree of deviation between the reference feature value of the pixel and the global pixel feature value is determined. Furthermore, based on the pixel feature value of the corresponding pixel in the previous image frame and the noise value of the corresponding pixel recorded in the noise matrix, a reference feature value of the corresponding pixel in the previous image frame is determined. Based on the reference feature value of the corresponding pixel in the previous image frame and the global pixel feature value, a second degree of deviation between the reference feature value and the global pixel feature value of the corresponding pixel in the previous image frame is determined. If the overall deviation is less than the first threshold, the first deviation is less than the second threshold, and the second deviation is less than the second threshold, then the pixel is determined to be the target pixel. And / or, the determination unit 502 is specifically used for: The first noise perturbation component is determined based on the pixel feature value of the target pixel and the global pixel feature value; the first noise perturbation component refers to the noise perturbation component of the target pixel in the current image frame. The second noise perturbation component is determined based on the pixel feature value of the reference pixel of the target pixel in the previous image frame and the global pixel feature value; the second noise perturbation component refers to the noise perturbation component of the reference pixel of the target pixel in the previous image frame. The noise value of the target pixel is determined based on the first noise perturbation component, the second noise perturbation component, and the noise value in the current noise matrix that matches the position of the target pixel. And / or, the correction unit 503 is specifically used for: The noise matrix is updated based on the noise value of the target pixel to obtain the updated noise matrix; the updated noise matrix records the noise value corresponding to each pixel in the current image frame; The pixel feature values of each pixel in the current image frame are corrected based on the updated noise matrix; And / or, the correction unit 503 is specifically used for: The noise step size corresponding to the target pixel is determined based on the reference noise value of the target pixel and the noise value of the corresponding pixel in the previous frame of infrared image recorded in the noise matrix. For each target pixel, the noise value corresponding to the target pixel recorded in the noise matrix is updated according to the noise step size corresponding to the target pixel; wherein, for each non-target pixel, the noise value corresponding to the non-target pixel recorded in the noise matrix remains unchanged. And / or, after inputting the dimensionality reduction information into the trained decision tree model to obtain the state of the image acquisition device during the data acquisition process, the correction unit 503 is also used to: If it is determined that the image acquisition device is not in motion during the data acquisition process, or if the dimensional orientation information does not meet the dimensional orientation information required for low-frequency noise correction of the image, then the pixel feature values of each pixel in the current image frame are corrected according to the current noise matrix.
[0094] This concludes the process. Figure 5 Description of the image correction device.
[0095] This application also provides embodiments that... Figure 5 Hardware structure description of the illustrated device. This hardware structure is... Figure 6 The structure in the illustrated electronic device. Please refer to [link / reference]. Figure 6 , Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 6 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.
[0096] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.
[0097] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.
[0098] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application. The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. An image correction method characterized by, The method comprises: For a current image frame needing correction, target pixel points needing correction are selected from the current image frame based on pixel feature values of pixel points in the current image frame, pixel feature values of pixel points in a previous image frame, and a current noise matrix; wherein a difference between the pixel feature value of the target pixel point and a pixel feature value of a reference pixel point in the previous image frame is within a set difference range; a position of the target pixel point in the current image frame is the same as a position of the reference pixel point in the previous image frame; the current noise matrix is determined based on noise values of pixel points in a most recent image frame that has been corrected; noise values matched with the target pixel points exist in the current noise matrix; For each target pixel point, a noise value of the target pixel point is determined based on the pixel feature value of the target pixel point, a pixel feature value of a reference pixel point of the target pixel point in the previous image frame, a global pixel feature value of the previous image frame, and a noise value matched with a position of the target pixel point in the current noise matrix; the global pixel feature value of the previous image frame is determined based on pixel feature values of pixel points in the previous image frame; The target pixel points are corrected based on the noise values of the target pixel points.
2. The method of claim 1, wherein, The current frame image contains data collected by an image collection device under current dimension direction information; the current image frame needing correction is determined according to the following method: Dimension reduction is performed on dimension direction information used by the image collection device when collecting data to obtain reduced dimension information; the dimension direction information used by the image collection device when collecting data at least includes a pitch angle and a yaw angle of the image collection device when collecting data; The reduced dimension information is input into a trained decision tree model to obtain a state of the image collection device in a data collection process; the trained decision tree model is obtained by training a decision tree model based on reduced dimension information of samples in motion states, vibration states, and static states; If it is determined that the state of the image collection device in the data collection process is a motion state, and the reduced dimension information meets the requirement of reduced dimension information for low-frequency noise correction of an image, it is determined that the current image frame is the current image frame needing correction.
3. The method of claim 1, wherein, The method comprises: For each pixel point in the current image frame, a joint pixel feature value between the pixel feature value of the pixel point and the pixel feature value of a pixel point at a corresponding position in the previous image frame and an overall deviation degree between the joint pixel feature value and the global pixel feature value are determined based on the pixel feature value of the pixel point, the pixel feature value of the pixel point at the corresponding position in the previous image frame, and the global pixel feature value. and, according to the pixel feature value of the pixel point and the noise value of the pixel corresponding to the pixel recorded in the noise matrix, determine a reference feature value of the pixel point; and according to the reference feature value of the pixel point and the global pixel feature value, determine a first deviation degree between the reference feature value of the pixel point and the global pixel feature value; and, according to the pixel feature value of the pixel point in the corresponding position in the previous image frame and the noise value of the pixel corresponding to the pixel recorded in the noise matrix, determine a reference feature value of the pixel point in the corresponding position in the previous image frame, and according to the reference feature value of the pixel point in the corresponding position in the previous image frame and the global pixel feature value, determine a second deviation degree between the reference feature value of the pixel point in the corresponding position in the previous image frame and the global pixel feature value; if the overall deviation degree is less than a first threshold value, the first deviation degree is less than a second threshold value, and the second deviation degree is less than the second threshold value, the pixel point is determined as the target pixel point.
4. The method of claim 1, wherein, The determination of the noise value of the target pixel point based on the pixel feature value of the target pixel point, the pixel feature value of the reference pixel point of the target pixel point in the previous image frame, the global pixel feature value of the previous image frame, and the noise value in the current noise matrix matching the position of the target pixel point comprises: determining a first noise disturbance component according to the pixel feature value of the target pixel point and the global pixel feature value; the first noise disturbance component refers to a noise disturbance component of the target pixel point in the current image frame; determining a second noise disturbance component according to the pixel feature value of the reference pixel point of the target pixel point in the previous image frame and the global pixel feature value; the second noise disturbance component refers to a noise disturbance component of the reference pixel point of the target pixel point in the previous image frame; determining the noise value of the target pixel point according to the first noise disturbance component, the second noise disturbance component, and the noise value in the current noise matrix matching the position of the target pixel point.
5. The method of claim 1, wherein, The correction of the target pixel point based on the noise value of the target pixel point comprises: updating the noise matrix based on the noise value of the target pixel point to obtain an updated noise matrix; the updated noise matrix records the noise values corresponding to the pixel points in the current image frame; correcting the pixel feature values of the pixel points in the current image frame based on the updated noise matrix.
6. The method of claim 5, wherein, The updating of the noise matrix based on the noise value of the target pixel point comprises: determining a noise step corresponding to the target pixel point according to the reference noise value of the target pixel point and the noise value of the pixel corresponding to the target pixel point in the previous infrared image recorded in the noise matrix; for each target pixel point, updating the noise value corresponding to the target pixel point recorded in the noise matrix according to the noise step corresponding to the target pixel point; wherein for each non-target pixel point, the noise value corresponding to the non-target pixel point recorded in the noise matrix is kept unchanged.
7. The method of claim 2, wherein, After the dimension reduction information is input into the trained decision tree model to obtain the state of the image acquisition device in the data acquisition process, the method further includes: If it is determined that the state of the image acquisition device in the data acquisition process is not the motion state, or the dimension direction information does not meet the required dimension direction information when the image is corrected for low-frequency noise, then the pixel feature values of each pixel point in the current image frame are corrected according to the current noise matrix.
8. An image correction apparatus characterized by comprising: The device includes: The selection unit is configured to select target pixel points that need to be corrected from the current image frame based on the pixel feature values of each pixel point in the current image frame, the pixel feature values of each pixel point in the previous image frame, and the current noise matrix, wherein the difference between the pixel feature value of the target pixel point and the pixel feature value of the reference pixel point in the previous image frame is within a set difference range, the position of the target pixel point in the current image frame is the same as the position of the reference pixel point in the previous image frame, the current noise matrix is determined based on the noise values of the pixel points in the most recent image frame that has been corrected, and the noise values of each target pixel point are matched in the current noise matrix; The determination unit is configured to determine the noise value of each target pixel point based on the pixel feature value of the target pixel point, the pixel feature value of the reference pixel point of the target pixel point in the previous image frame, the global pixel feature value of the previous image frame, and the noise value in the current noise matrix that matches the position of the target pixel point, wherein the global pixel feature value of the previous image frame is determined based on the pixel feature values of each pixel point in the previous image frame; The correction unit is configured to correct the target pixel point based on the noise value of the target pixel point.
9. The apparatus of claim 8, wherein, The current frame image includes data collected by the image acquisition device under the current dimension direction information; and the current image frame that needs to be corrected is determined according to the following method: The dimension direction information used by the image acquisition device when collecting data is dimensionally reduced to obtain dimension reduction information; the dimension direction information used by the image acquisition device when collecting data includes at least the pitch angle and the yaw angle of the image acquisition device when collecting data; The dimension reduction information is input into a trained decision tree model to obtain the state of the image acquisition device in the data acquisition process; the trained decision tree model is obtained by training the decision tree model based on the dimension reduction information of the samples in the motion state, the vibration state, and the static state after dimension reduction processing; If it is determined that the state of the image acquisition device in the data acquisition process is the motion state, and the dimension reduction information meets the required dimension reduction information when the image is corrected for low-frequency noise, then the current image frame is determined to be the current image frame that needs to be corrected; And / or, the selection unit is specifically configured to: For each pixel point in the current image frame, according to the pixel feature value of the pixel point, the pixel feature value of the pixel point at the corresponding position in the previous image frame, and the global pixel feature value, determine the overall deviation between the pixel feature value of the pixel point and the pixel feature value of the pixel point at the corresponding position in the previous image frame, and the global pixel feature value; And, according to the pixel feature value of the pixel point and the noise value of the pixel corresponding to the pixel recorded in the noise matrix, determine the reference feature value of the pixel point; According to the reference feature value of the pixel point and the global pixel feature value, determine the first deviation between the reference feature value of the pixel point and the global pixel feature value; And, according to the pixel feature value of the pixel point at the corresponding position in the previous image frame and the noise value of the pixel corresponding to the pixel recorded in the noise matrix, determine the reference feature value of the pixel point at the corresponding position in the previous image frame, and according to the reference feature value of the pixel point at the corresponding position in the previous image frame and the global pixel feature value, determine the second deviation between the reference feature value of the pixel point at the corresponding position in the previous image frame and the global pixel feature value; If the overall deviation is less than a first threshold, the first deviation is less than a second threshold, and the second deviation is less than a second threshold, the pixel point is determined to be the target pixel point; And / or, the determination unit is specifically used for: According to the pixel feature value of the target pixel point and the global pixel feature value, determine a first noise disturbance component; the first noise disturbance component refers to the noise disturbance component of the target pixel point in the current image frame; According to the pixel feature value of the reference pixel point of the target pixel point in the previous image frame and the global pixel feature value, determine a second noise disturbance component; The second noise disturbance component refers to the noise disturbance component of the reference pixel point of the target pixel point in the previous image frame; According to the first noise disturbance component, the second noise disturbance component, and the noise value matching the position of the target pixel point in the current noise matrix, determine the noise value of the target pixel point; And / or, the correction unit is specifically used for: Update the noise matrix based on the noise value of the target pixel point to obtain an updated noise matrix; the updated noise matrix records the noise values corresponding to each pixel point in the current image frame; Correct the pixel feature values of each pixel point in the current image frame based on the updated noise matrix; And / or, the correction unit is specifically used for: According to the reference noise value of the target pixel point and the noise value of the pixel corresponding to the target pixel point in the previous infrared image recorded in the noise matrix, determine the noise step corresponding to the target pixel point; For each target pixel point, update the noise value corresponding to the target pixel point recorded in the noise matrix according to the noise step corresponding to the target pixel point; wherein for each non-target pixel point, keep the noise value corresponding to the non-target pixel point recorded in the noise matrix unchanged; And / or, after inputting the dimension reduction information into the trained decision tree model to obtain the state of the image acquisition device in the data acquisition process, the correction unit is further configured to: If it is determined that the state of the image acquisition device in the data acquisition process is not the motion state, or the dimension direction information does not meet the required dimension direction information when the image is corrected for low-frequency noise, then the pixel feature values of each pixel point in the current image frame are corrected according to the current noise matrix.
10. An electronic device, comprising: Comprise: A processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions capable of being executed by the processor; The processor is configured to execute the machine executable instructions to implement the method of any one of claims 1 to 7.