Camera image restoration correction method, device and medium in strong radiation environment

Abstract

The application discloses a camera image restoration correction method and device in a strong radiation environment and a medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring an original camera image and image acquisition parameters. The image acquisition parameters are corresponding dose, dose rate, sensor temperature and exposure time of the camera when the original camera image is shot in a strong radiation environment. Then, based on a degradation parameter library, a degradation parameter corresponding to the image acquisition parameters is determined. The degradation parameter comprises a bad point mask, a salt and pepper noise probability graph, a dark current graph, a gain correction graph, a bias correction graph, camera internal parameters and distortion parameters. Finally, the original camera image is subjected to image restoration based on the bad point mask, the salt and pepper noise probability graph, the dark current graph, the gain correction graph and the bias correction graph, so as to obtain a restored image. The restored image is subjected to distortion correction based on the camera internal parameters and the distortion parameters, so as to obtain a restored and corrected image. The application can improve the processing effect.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device and medium for restoring and correcting camera images under strong radiation conditions. Background Technology

[0002] In nuclear power plants (such as around nuclear power plant reactors), nuclear waste disposal sites, high-energy physics experimental devices, space detectors, and other high-radiation environments, cameras are often used for remote monitoring, target identification, displacement measurement, equipment inspection, and anomaly detection. Since such high-radiation environments are usually not suitable for long-term personnel entry, the reliability and image quality of the cameras directly affect the accuracy of subsequent monitoring, control, and decision-making.

[0003] However, strong radiation environments can affect camera sensors, readout circuits, and image acquisition links. Typical manifestations include random bright spots, dark spots, flickering spots, an increase in the number of bad pixels with cumulative dose, pulse interference similar to salt-and-pepper noise, inconsistent pixel response, increased dark current, decreased image contrast, and camera calibration parameter drift in raw camera images. Currently, median filtering, bilateral filtering, nonlocal averaging, BM3D (Block-Matching 3-D filtering), or common deep learning denoising methods are commonly used to process raw camera images. However, these methods typically only consider the statistical characteristics of the raw camera image and lack the physical constraints between dose, sensor pixel damage, dark current variations, and bad pixel generation. They are ill-suited to the complex degradation under different dose and dose rate conditions in strong radiation environments, resulting in poor processing performance. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, and medium for restoring and correcting camera images under strong radiation environments, which can improve the processing effect.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] In a first aspect, this application provides a method for restoring and correcting camera images under strong radiation conditions, including: Acquire raw camera images and image acquisition parameters; the image acquisition parameters are the dose, dose rate, sensor temperature, and exposure time corresponding to the camera capturing raw camera images in a strong radiation environment. Based on the degradation parameter library, the degradation parameters corresponding to the image acquisition parameters are determined; the degradation parameter library includes degradation parameter sample values ​​corresponding to each image acquisition parameter sample value among multiple image acquisition parameter sample values, and the degradation parameters include bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters and distortion parameters; Based on the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, and the bias correction map, the original camera image is restored to obtain a restored image. Based on the camera intrinsic parameters and the distortion parameters, the restored image is distorted to obtain a restored and corrected image.

[0007] Optionally, methods for establishing a degradation parameter library include: Acquire the dark field image, flat field image, calibration plate image, and target scene image corresponding to each of the multiple image acquisition parameter sample values; For each of the image acquisition parameter sample values, a bad pixel mask and dark current map corresponding to the image acquisition parameter sample value are determined based on the dark field image; a gain correction map and bias correction map corresponding to the image acquisition parameter sample value are determined based on the flat field image; camera intrinsic parameters and distortion parameters corresponding to the image acquisition parameter sample value are determined based on the calibration board image; and a salt-and-pepper noise probability map corresponding to the image acquisition parameter sample value is determined based on the target scene image, thereby determining the degradation parameter sample value corresponding to the image acquisition parameter sample value. The degradation parameter sample value corresponding to each of the multiple image acquisition parameter sample values ​​is stored to obtain the degradation parameter library.

[0008] Optionally, based on a degradation parameter library, the degradation parameters corresponding to the image acquisition parameters are determined, specifically including: Determine whether the image acquisition parameters are the same as the sample values ​​of image acquisition parameters in the degradation parameter library; If so, the degradation parameter sample value corresponding to the same image acquisition parameter sample value as the image acquisition parameter is taken as the degradation parameter corresponding to the image acquisition parameter; If not, when the image acquisition parameter is located between two image acquisition parameter sample values ​​in the degradation parameter library, interpolation is performed on the degradation parameter sample values ​​corresponding to the two image acquisition parameter sample values ​​to obtain the degradation parameter corresponding to the image acquisition parameter.

[0009] Optionally, image restoration is performed on the original camera image based on the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, and the bias correction map to obtain a restored image, specifically including: Based on the bad pixel mask, bad pixels in the original camera image are determined. The pixel values ​​of the pixels in the neighborhood corresponding to the bad pixels are weighted and summed to obtain the repaired pixel values. The pixel values ​​of the bad pixels in the original camera image are replaced with the repaired pixel values ​​to obtain the image after bad pixel repair. Based on the salt-and-pepper noise probability map, salt-and-pepper noise points in the image after bad pixel repair are determined. The pixel values ​​of the pixels in the neighborhood corresponding to the salt-and-pepper noise points are weighted and summed to obtain the suppressed pixel values. The pixel values ​​of the salt-and-pepper noise points in the image after bad pixel repair are replaced with the suppressed pixel values ​​to obtain the image after salt-and-pepper noise suppression. The difference between the image after salt-and-pepper noise suppression and the dark current map is calculated to obtain the image after dark current subtraction. The dark current subtraction image is processed based on the gain correction map and the bias correction map to obtain the response non-uniformity correction image; The image after response nonuniformity correction is restored by constraining data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiative degradation physical consistency, to obtain the restored image.

[0010] Optionally, determining salt-and-pepper noise points in the image after bad pixel repair based on the salt-and-pepper noise probability map specifically includes: for each pixel in the image after bad pixel repair, determining the median deviation threshold of the pixel based on the salt-and-pepper noise probability map, and using the salt-and-pepper noise point detection formula based on the median deviation threshold of the pixel to determine whether the pixel is a salt-and-pepper noise point, so as to determine the salt-and-pepper noise points in the image after bad pixel repair. The formula for detecting salt and pepper noise points is as follows: ; in, For pixels The salt-and-pepper noise point detection result, if the value is 1, is a salt-and-pepper noise point; Pixels in the image after bad pixel repair Pixel values; This is the saturation discrimination margin; This represents the pixel saturation value. Pixels in the image after bad pixel repair Corresponding neighborhood The median pixel value of an inner pixel; Pixels in the image after bad pixel repair With dose The changing median deviation threshold.

[0011] Optionally, the dark current subtraction image is processed based on the gain correction map and the bias correction map to obtain a response non-uniformity corrected image. Specifically, this includes: processing the dark current subtraction image using a response non-uniformity correction formula based on the gain correction map and the bias correction map to obtain a response non-uniformity corrected image. The formula for correcting non-uniformity of response is: ; in, In response to pixels in the image after non-uniformity correction Pixel values; For pixels in the image after dark current subtraction Pixel values; For dosage Pixels in the offset correction image below Pixel values; For dosage Pixels in the gain correction image below The pixel value.

[0012] Optionally, image restoration is performed on the image after response non-uniformity correction, constrained by data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiative degradation physical consistency, to obtain the restored image, specifically including: Based on data consistency, edge preservation, residual salt-and-pepper noise sparsity, and physical consistency of radiative degradation, a restoration objective function is constructed. Using the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, the bias correction map, and the image after response non-uniformity correction as input, the restoration objective function is optimized and solved to obtain the restored image corresponding to the minimum restoration objective function.

[0013] Optionally, distortion correction is performed on the restored image based on the camera intrinsic parameters and the distortion parameters to obtain a restored and corrected image, specifically including: For each pixel in the restored image, based on the camera intrinsic parameters, the pixel coordinates of the pixel are converted into the distorted coordinates of the pixel, and based on the distortion parameters, the distorted coordinates of the pixel are converted into the undistorted coordinates of the pixel, thus obtaining the restored and corrected image.

[0014] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the above-described method for restoring and correcting camera images under strong radiation conditions.

[0015] Thirdly, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for restoring and correcting camera images under strong radiation conditions.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, device, and medium for restoring and correcting camera images under strong radiation conditions. Considering that the original camera image includes random bright spots, dark spots, flickering spots, and the number of bad spots increasing with cumulative dose (which are considered bad spots), the original camera image includes pulse interference similar to salt-and-pepper noise (which is salt-and-pepper noise), the original camera image includes inconsistent pixel response, increased dark current, and decreased image contrast (which are considered response degradation), and the original camera image includes camera calibration parameter drift (which is considered distortion), the degradation parameters are designed to include a bad spot mask, a salt-and-pepper noise probability map, a dark current map, a gain correction map, a bias correction map, camera intrinsic parameters, and distortion parameters. First, the original camera image and image acquisition parameters are acquired. The image acquisition parameters are the dose, dose rate, sensor temperature, and exposure time corresponding to the original camera image captured under strong radiation conditions. Then, based on... A degradation parameter library is used to determine the degradation parameters corresponding to the image acquisition parameters. Finally, based on the bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, and bias correction map in the degradation parameters, the original camera image is restored to obtain the restored image. Based on the camera intrinsic parameters and distortion parameters in the degradation parameters, the distortion of the restored image is corrected to obtain the corrected image. It can perform image restoration and distortion correction on the original camera image based on the degradation parameters under different image acquisition parameters. Instead of starting from the statistical features of the original camera image, it introduces a degradation parameter library to determine the degradation parameters based on the image acquisition parameters. This solves the problem of lack of physical constraints between dose, sensor pixel damage, dark current changes, and bad pixel generation. It can adapt to the composite degradation under different dose and dose rate conditions in strong radiation environments and improve the processing effect. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 An application environment diagram for a camera image restoration and correction method under strong radiation environment provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating a camera image restoration and correction method under strong radiation conditions, provided as an embodiment of this application; Figure 3 This is a schematic diagram of the overall structure of a high-radiation environment experimental data acquisition system provided in an embodiment of this application; Figure 4 This is a schematic diagram of the camera image degradation mechanism under strong radiation environment provided in an embodiment of this application; Figure 5A schematic diagram illustrating the construction process of a degradation parameter library provided in an embodiment of this application; Figure 6 A schematic diagram illustrating the process of image restoration and distortion correction combined with a physical model, provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] The camera image restoration and correction method provided in this application embodiment for strong radiation environments can be applied to, for example... Figure 1 The application environment shown illustrates this. The terminal communicates with the server via a network. A data storage system stores the data the server needs to process. This system can be set up independently, integrated into the server, or located in the cloud or on another server. The terminal can send an image processing request to the server. Upon receiving the request, the server acquires the original camera image and image acquisition parameters. These parameters include the dose, dose rate, sensor temperature, and exposure time corresponding to the original image captured under strong radiation conditions. Based on a degradation parameter library, the server determines the corresponding degradation parameters, including bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters. The server then performs image restoration on the original camera image based on the bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, and bias correction map, obtaining a restored image. Finally, it performs distortion correction on the restored image based on the camera intrinsic parameters and distortion parameters, obtaining a restored and corrected image. The server can then feed back this image processing result—the restored and corrected image—to the terminal.

[0022] In addition, in some embodiments, the camera image restoration and correction method under strong radiation environment can also be implemented by a server or a terminal alone. For example, the terminal can directly process the image processing request to be processed, or the server can obtain the image processing request to be processed from the data storage system and process it.

[0023] The terminals can be, but are not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Servers can be implemented using independent servers, server clusters composed of multiple servers, or cloud servers.

[0024] In one exemplary embodiment, such as Figure 2 As shown, a method for restoring and correcting camera images under strong radiation environments is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 The following steps, S1 to S3, are used as an example to illustrate the process of using a server in the example.

[0025] Step S1: Obtain the original camera image and image acquisition parameters; the image acquisition parameters are the dose, dose rate, sensor temperature and exposure time corresponding to the original camera image taken under strong radiation environment.

[0026] Step S2: Based on the degradation parameter library, determine the degradation parameters corresponding to the image acquisition parameters; the degradation parameter library includes degradation parameter sample values ​​corresponding to each image acquisition parameter sample value among multiple image acquisition parameter sample values, and the degradation parameters include bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters.

[0027] Step S3: Based on the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, and the bias correction map, the original camera image is restored to obtain a restored image. Based on the camera intrinsic parameters and the distortion parameters, the restored image is distorted to obtain a restored and corrected image.

[0028] By implementing steps S1 to S3 as described above, this embodiment can improve the processing effect of the original camera image.

[0029] Considering that the original camera image includes random bright spots, dark spots, flickering spots, and the number of bad spots increases with the cumulative dose, which are considered bad spots; the original camera image includes pulse interference similar to salt-and-pepper noise, which is considered salt-and-pepper noise; the original camera image includes inconsistent pixel response, increased dark current, and decreased image contrast, which are considered response degradation; and the original camera image includes camera calibration parameter drift, which is considered distortion, this embodiment provides a method for modeling, calibrating, and correcting bad spots, salt-and-pepper noise, response degradation, and distortion of camera images under strong radiation conditions, so as to realize image restoration and distortion correction of the original camera image.

[0030] To implement the aforementioned camera image restoration and correction method under strong radiation conditions, it is necessary to first establish a degradation parameter library, such as... Figure 3 , Figure 4 and Figure 5 As shown, based on multi-dose experimental calibration data, a physical model of camera imaging degradation under strong radiation environment is first established, and then degradation parameters are extracted to form a degradation parameter library. The process of establishing the degradation parameter library will be described in detail below.

[0031] (a) Acquisition of multi-dose experimental calibration data This embodiment establishes strong radiation experimental acquisition conditions (i.e., strong radiation environment), and acquires multi-dose experimental calibration data under strong radiation experimental acquisition conditions. The multi-dose experimental calibration data includes dark field image, flat field image, calibration plate image and target scene image corresponding to each image acquisition parameter sample value among multiple image acquisition parameter sample values. The image acquisition parameter sample values ​​are the image acquisition parameters used when establishing the degradation parameter library, which also include dose, dose rate, sensor temperature and exposure time.

[0032] This embodiment can build a high-radiation environment experimental data acquisition system to obtain multi-dose experimental calibration data, and further establish a degradation parameter library, such as... Figure 3As shown, the high-radiation environment experimental acquisition system includes a high-radiation environment or irradiation simulation device, a camera under test, a standard test target, a dosimeter, a thermometer, an image acquisition and synchronization control unit, and an image processing unit. The high-radiation environment or irradiation simulation device, standard test target, dosimeter, and thermometer constitute the experimental calibration and state perception part; the camera under test and the image acquisition and synchronization control unit constitute the image data acquisition part; and the image processing unit constitutes the degradation parameter extraction and storage part. The experimental calibration and state perception part and the image data acquisition part correlate dose, dose rate, sensor temperature, exposure time, and degradation images (i.e., dark field images, flat field images, calibration plate images, and target scene images) using timestamps to form multi-dose experimental calibration data. Further processing of the multi-dose experimental calibration data is performed based on the degradation parameter extraction and storage part to form a degradation parameter library. The image processing unit can subsequently use the degradation parameter library to call up and restore degradation parameters, thus forming a closed-loop structure of "irradiation conditions—image acquisition—parameter calibration—parameter calling—restoration correction".

[0033] The strong radiation environment or irradiation simulation device can be an actual strong radiation environment such as a nuclear power plant, or a strong radiation environment simulated using existing irradiation simulation devices (such as cobalt-60 sources or X-ray irradiation devices). It forms a controllable strong radiation environment with preset dose, dose rate and irradiation time in the target area covering the camera under test and the standard test target, which makes it easy to obtain degradation images of different degrees of degradation under this strong radiation environment.

[0034] The camera being tested can be of any type.

[0035] The standard test targets include dark field shading conditions, flat field uniform light source, grayscale target, resolution target, checkerboard or dot array calibration board, and typical target scene target board. These can be manually selected to provide dark field images (captured by the camera under test under dark field shading conditions), flat field images (captured by the camera under test under flat field uniform light source), calibration board images (captured by the camera under test of the grayscale target, resolution target, or checkerboard or dot array calibration board), and target scene images (captured by the camera under test of the typical target scene target board). The above four types of images constitute the degradation image.

[0036] The dosimeter is located in the target area and is used to measure the dose (cumulative dose) and dose rate (instantaneous dose rate) at the camera under test in real time. It also synchronously associates the dose and dose rate information with the degraded image as input to the degradation parameter library.

[0037] The thermometer is located in the target area and is used to measure the temperature information of the camera sensor, lens assembly and target area (if it is a simulated strong radiation device, that is, the irradiation chamber of an existing irradiation simulation device) in real time. It obtains the sensor temperature, lens temperature and target area temperature, and synchronously associates the sensor temperature, lens temperature and target area temperature information with the degradation image as input to the degradation parameter library.

[0038] It should be noted that among the three temperature channels mentioned above, only the sensor temperature is used to extract degradation parameters. This is because dark current, gain drift, and bad pixel activation threshold are directly determined by the sensor junction temperature. Lens temperature and target area temperature are used as auxiliary recording quantities for data validity determination (such as determining whether thermal equilibrium has been established and removing samples from periods of drastic temperature changes) and subsequent expansion.

[0039] The image acquisition and synchronization control unit controls the camera under test to capture images of a standard test target under different dose conditions. Specifically, it sequentially exposes the target under dark field shading, flat field uniform light source, grayscale target, resolution target, checkerboard or dot array calibration plate, and typical target scene target conditions, acquiring dark field images, flat field images, calibration plate images, and target scene images to obtain degraded images. The unit also uniformly numbers, timestamps, and stores the dose, dose rate, sensor temperature, exposure time, and degraded images, resulting in a degraded image for each of multiple image acquisition parameter sample values. Exposure time, a camera acquisition parameter, is not dependent on additional sensors. It is directly read from the camera's configuration and recorded along with the degraded images when the image acquisition and synchronization control unit sends the capture command to the camera under test. It is synchronously associated with the degraded images and serves as input to the degraded parameter library. The image acquisition and synchronization control unit can also read camera acquisition parameters including gain and frame rate.

[0040] Based on this, the camera under test, standard test target, dosimeter, and thermometer are placed in a strong radiation environment to acquire dark-field images, flat-field images, calibration plate images, and target scene images under different image acquisition parameter sample values. These images serve as benchmark data for subsequent degradation modeling. Specifically, based on a preset dose sequence, multiple doses are determined. Using the aforementioned strong radiation environment experimental acquisition system, degraded images such as dark-field images, flat-field images, calibration plate images, and target scene images are acquired at each dose, while simultaneously recording the dose, dose rate, sensor temperature, and exposure time to obtain multi-dose experimental calibration data. Simultaneously, benchmark dark-field images, benchmark flat-field images, benchmark calibration plate images, and benchmark target scene images are acquired under a radiation-free environment to obtain a radiation-free benchmark image.

[0041] (II) Establishment of a physical model for camera imaging degradation under strong radiation environment This embodiment establishes a physical model of camera imaging degradation under strong radiation conditions. Based on radiation-free reference images (or low-radiation reference images, including reference dark field images, reference flat field images, reference calibration plate images, and reference target scene images when the dose is 0 or close to 0 (e.g., the difference from 0 is less than a preset difference) and degradation images under different doses (including dark field images, flat field images, calibration plate images, and target scene images), radiation degradation is described as a complex degradation process composed of bad pixel generation, salt-and-pepper impulse noise, dark current drift, response non-uniformity, and geometric distortion drift.

[0042] At this time, as Figure 4 As shown, with dose, dose rate, sensor temperature and exposure time as input variables, a bad pixel occurrence probability model (or bad pixel occurrence model), a salt and pepper noise probability model (or salt and pepper noise formation model), a dark current drift model, a response non-uniformity model and a geometric distortion drift model are established respectively, so that the subsequent degradation parameter extraction and image restoration and correction are physically constrained.

[0043] Degraded images acquired under strong radiation environments are represented as the result of multi-source noise superimposed on a non-radioactive baseline image after applying a radiation degradation operator: (1) In equation (1), These are degraded images from a high-radiation environment. Each is a pixel x coordinates and y coordinate, For dosage, Exposure time; For operators that degrade due to strong radiation environments, A radiation-free reference image. These are the sensor acquisition parameters, which are acquisition condition vectors that affect the imaging response, excluding dose and exposure time. Specifically, they include dose rate, sensor temperature, analog gain, digital gain, pixel binning method, and readout mode. , , , These are readout noise, salt-and-pepper pulse noise, dark current drift noise, and dead pixel noise, respectively.

[0044] The above model is the overall model, which is a unified framework for the composite degradation process. In the following, sub-models are further established to characterize the bad spots, salt-and-pepper noise, dark current, response non-uniformity and geometric distortion.

[0045] (1) Defect occurrence probability model, used to characterize defective pixels When radiation particles or rays act on sensor pixels and readout circuits, they can cause constant bright spots, constant dark spots, flickering bad spots, and response drift spots. The probability of bad spots occurring can be modeled as a function of dose, dose rate, sensor temperature, exposure time, and pixel sensitivity. (2) In equation (2), For pixels The probability of it being a bad pixel; This is a function, and the specific function form can be designed by the user. For dosage, For dose rate, For sensor temperature, For the exposure time, For pixels The pixel sensitivity factor is a pixel-level empirical coefficient obtained statistically from dark-field images with multiple doses, representing pixel sensitivity.

[0046] The aforementioned defective pixel probability model can determine the probability of a pixel being a defective pixel based on dose, dose rate, sensor temperature, exposure time, and pixel sensitivity. Therefore, the influencing factors for determining the defective pixel mask include dose, dose rate, sensor temperature, and exposure time.

[0047] (2) Salt and pepper noise probability model, used to characterize salt and pepper noise The transient impulse response induced by strong radiation particles is modeled as a dose-dependent random impulse noise process: (3) In equation (3), For pixels in the target scene image The noise pixel value, which can be a grayscale value; For probability; The probability of a black pulse. For dosage; This represents the pixel saturation value. The probability of a white pulse; Pixels in a radiation-free reference image The baseline pixel value.

[0048] This formula shows that pixels With probability Take 0 (black pulse), with probability Get pixel saturation value (White pulse), with probability Maintain baseline pixel value , and It was obtained from target scene images with multiple doses.

[0049] The aforementioned salt-and-pepper noise probability model models the transient impulse response caused by radiated particles as a dose-dependent random impulse noise process. Based on this, the influencing factors of the salt-and-pepper noise probability map include dose.

[0050] (3) Dark current drift model, used to characterize dark current (4) In equation (4), For pixels Dark current; This is a function, and the specific function form can be designed by the user. For dosage, For sensor temperature, This refers to the exposure time.

[0051] Based on this, the influencing factors of dark current maps were determined to be dose, sensor temperature, and exposure time.

[0052] (4) Response nonuniformity model, used to characterize response nonuniformity Strong radiation causes changes in gain and bias of different pixels, and the image response can be expressed as: (5) In equation (5), pixels in a flat field image The response to non-uniform pixel values; For pixels Gain, with dose Related; Pixels in a radiation-free reference image The baseline pixel value; For pixels Bias, and dosage Related; For pixels Random noise.

[0053] Based on this, the factors influencing the gain correction plot and the bias correction plot are determined to include dose.

[0054] (5) Geometric distortion drift model, used to characterize geometric distortion Under long-term radiation conditions, camera intrinsic parameters and distortion parameters may drift. A dose-dependent model for these intrinsic parameters and distortion parameters is established. (6) (7) In the above formula, , Dosage Camera intrinsic parameters and distortion parameters; , These are the non-radioactive reference values ​​for the camera's intrinsic parameters and distortion parameters, respectively; , Dosage The amount of drift in the camera's intrinsic parameters and distortion parameters.

[0055] Based on this, the factors affecting camera intrinsic parameters include dose, and the factors affecting distortion parameters also include dose.

[0056] (III) Degradation Parameter Extraction This embodiment extracts degradation parameters based on degraded images acquired at multiple doses. Based on multi-dose experimental calibration data and combined with the aforementioned physical model of camera imaging degradation under strong radiation conditions, the mean, variance, extreme values, anomaly frequency, and temporal stability of each pixel are statistically analyzed. Defect mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters are extracted to obtain the degradation parameter sample values ​​corresponding to the sample values ​​of each image acquisition parameter. Specifically, the defect mask and dark current map are obtained from multiple frames of dark-field images; the salt-and-pepper noise probability map is obtained from multiple frames of target scene images; the gain correction map and bias correction map are obtained from multiple frames of flat-field images; and the camera intrinsic parameters and distortion parameters are obtained from multiple frames of calibration board images.

[0057] (1) Defect mask Fixed bright spots, fixed dark spots, and response drift points (if the average value of a pixel in multiple frames monotonically drifts with the cumulative dose, and the drift exceeds a threshold but does not reach saturation or complete black, it is judged as a response drift point) are identified based on statistical thresholds of multiple frames of dark field images. Specifically, statistical analysis is performed on multiple frames of dark field images under a certain image acquisition parameter sample value to identify stable abnormal pixels and generate a bad spot mask. (8) In equation (8), For pixels The result is a binary judgment of the defective pixel, where 1 represents a defective pixel and 0 represents a normal pixel. pixels in a dark field image The dark current pixel value; Pixels in a multi-frame dark field image neighborhood Mean pixel value of an inner pixel, neighborhood The range can be designed manually; This is the threshold coefficient, typically taken as 3 to 5; Pixels in a multi-frame dark field image neighborhood Standard deviation of pixel values ​​within a pixel; This means otherwise.

[0058] Based on the above equation (8), for a certain image acquisition parameter sample value, it will continuously satisfy the following in multiple consecutive frames (such as more than 90% or more than 95% of the frames) in multiple dark field images. The pixel of the criterion (i.e. A pixel with a value of 1 is designated as a stable abnormal pixel, which is a fixed bad pixel. Constant bright pixels and constant dark pixels are distinguished by the deviation sign, which is different from the occasional transient abnormal pixels. The probability of a stable abnormal pixel being a bad pixel is set to 1, and the probability of other pixels being bad pixels is set to 0. This gives us the bad pixel mask under a certain image acquisition parameter sample value. Based on this, by inputting multiple frames of dark field images under a certain image acquisition parameter sample value, we can output the bad pixel mask under that image acquisition parameter sample value.

[0059] (2) Salt and pepper noise probability diagram Transient salt-and-pepper noise is identified based on saturation state, neighborhood median deviation, and anomaly occurrence frequency. Specifically, for transient anomalous pixels, the probability of salt-and-pepper noise occurrence is estimated using multiple frames of target scene images with sample values ​​of a certain image acquisition parameter. (9) In equation (9), For dosage Pixels in multiple frames of target scene images The probability of salt-and-pepper noise occurrence is calculated, and a salt-and-pepper noise probability map can be generated based on the salt-and-pepper noise occurrence probabilities of all pixels. For dosage Pixels in multiple frames of target scene images The number of frames that were identified as transient abnormal pixels; This represents the total number of frames in the multi-frame target scene image.

[0060] Based on the above formula (9), transient abnormal pixels are pixels that only take 0 or pixel saturation value in individual frames of multi-frame target scene images and deviate from the neighborhood median value by more than a threshold, but recover to normal in subsequent frames (abnormal occurrence frequency is between 0 and 1, and appears randomly in time sequence). They refer to abnormal pixels that only appear randomly in individual frames and are unstable in time sequence. They are mutually exclusive with stable abnormal pixels in the bad pixel mask. The occurrence frequency of transient abnormal pixels is statistically calculated as the salt and pepper noise occurrence probability, and the salt and pepper noise probability map under a certain image acquisition parameter sample value can be obtained. Based on this, by inputting multi-frame target scene images under a certain image acquisition parameter sample value, the salt and pepper noise probability map under the image acquisition parameter sample value can be output.

[0061] (3) Dark current diagram Dark current is estimated based on the statistical results of dark field images. Specifically, based on multiple dark field images with sample values ​​of a certain image acquisition parameter, the dark current map under the sample values ​​of that image acquisition parameter is determined. Based on this, by inputting multiple dark field images with sample values ​​of a certain image acquisition parameter, the dark current map under the sample values ​​of that image acquisition parameter can be output.

[0062] (4) Gain correction diagram and bias correction diagram The response nonuniformity is estimated based on the statistical results of the flat field images. Specifically, based on multiple frames of flat field images under a certain image acquisition parameter sample value, the gain correction map and offset correction map under the image acquisition parameter sample value are determined. Based on this, by inputting multiple frames of flat field images under a certain image acquisition parameter sample value, the gain correction map and offset correction map under the image acquisition parameter sample value can be output.

[0063] (5) Camera intrinsic parameters and distortion parameters For calibration plate images, camera intrinsic parameters and distortion parameters are solved under different doses to form dose-related geometric distortion drift data. Specifically, based on multiple frames of calibration plate images under a certain image acquisition parameter sample value, the camera intrinsic parameters and distortion parameters under that image acquisition parameter sample value are determined. Based on this, by inputting multiple frames of calibration plate images under a certain image acquisition parameter sample value, the camera intrinsic parameters and distortion parameters under that image acquisition parameter sample value can be output.

[0064] In summary, the bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters can be extracted for each image acquisition parameter sample value, thus obtaining the degradation parameter sample value for each image acquisition parameter sample value.

[0065] (iv) Establishment of a degradation parameter library This embodiment establishes a degradation parameter library. The extracted bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters are indexed according to dose, dose rate, sensor temperature, and exposure time, forming a queryable, interpolable, and online-updable degradation parameter library. At this time, the degradation parameter library is used to store bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters under different doses, dose rates, sensor temperatures, and exposure times, and supports querying, interpolation, and online updates.

[0066] This embodiment can further use the camera number as an index. An index is then established based on (camera number, dose, dose rate, sensor temperature, exposure time) to obtain a degradation parameter library. This library is partitioned by camera number, with each camera corresponding to an independent partition. Each camera establishes its own set of degradation parameter library entries, including bad pixel masks, salt-and-pepper noise probability maps, dark current maps, gain correction maps, bias correction maps, camera intrinsic parameters, and distortion parameters under dose, dose rate, sensor temperature, and exposure time. This is because bad pixel distribution, dark current steps, and gain non-uniformity are individual characteristics of each device and cannot be directly transferred between cameras of the same model. Cameras of the same model can use existing partitions in the library as prior initial values ​​to shorten calibration time, but each camera still needs to be calibrated individually before being written into its own partition.

[0067] During the query, a multidimensional index table is built with the quintuple (camera number, dose, dose rate, sensor temperature, exposure time) as the primary key. The query first locates the partition by camera number, and then performs exact matching or nearest neighbor location in the four-dimensional image acquisition parameter space.

[0068] During interpolation, multidimensional interpolation within a four-dimensional index space (i.e., dose, dose rate, sensor temperature, and exposure time) is employed. The dose dimension can use monotonically conformal interpolation to conform to the physical constraint that cumulative damage increases monotonically and irreversibly with dose. The other three dimensions can use linear interpolation. Specifically, when a query point does not match a sample value but falls within a four-dimensional hypercube formed by adjacent sample values, degradation parameters are obtained by interpolation using the aforementioned multidimensional interpolation method, and the interpolation distance is simultaneously output as a confidence indicator for this set of degradation parameters. Multidimensional interpolation is a mature existing technology and will not be elaborated further here.

[0069] During online updates, when long-term drift or new bad pixels are detected, the online statistical results are written back to the degradation parameter library for updates in subsequent frames. Specifically, during online camera operation, dark-field images are periodically acquired, the current bad pixel mask and dark current map are re-estimated, and the data is fused with the corresponding entries in the degradation parameter library using a forgetting factor and then written back. The version number, update time, and number of samples are recorded. When the deviation between the online estimate and the value in the degradation parameter library exceeds a preset threshold, a recalibration prompt is triggered.

[0070] The degradation parameter library can be represented as: (10) In equation (10), For entries in the degenerate parameter library: For bad pixel masking; This is a probability plot of salt and pepper noise. Dark current diagram; This is a gain correction diagram; This is the bias correction diagram; For camera internal parameters; This is the distortion parameter.

[0071] This completes the offline calibration process from experimental data acquisition and physical degradation modeling to the construction of a degradation parameter library, providing input for online restoration and correction.

[0072] In this embodiment, the method for establishing the degradation parameter library includes: (1) Obtain the dark field image, flat field image, calibration plate image and target scene image corresponding to each of the multiple image acquisition parameter sample values.

[0073] (2) For each image acquisition parameter sample value, the bad pixel mask and dark current map corresponding to the image acquisition parameter sample value are determined based on the dark field image, the gain correction map and bias correction map corresponding to the image acquisition parameter sample value are determined based on the flat field image, the camera intrinsic parameters and distortion parameters corresponding to the image acquisition parameter sample value are determined based on the calibration board image, and the salt-and-pepper noise probability map corresponding to the image acquisition parameter sample value is determined based on the target scene image, so as to determine the degradation parameter sample value corresponding to the image acquisition parameter sample value.

[0074] (3) Store the degradation parameter sample value corresponding to each of the multiple image acquisition parameter sample values ​​to obtain the degradation parameter library.

[0075] like Figure 6 As shown, based on the established degradation parameter library, the image processing unit determines degradation parameters according to the dose, dose rate, sensor temperature, and exposure time corresponding to the original camera image, and sequentially completes bad pixel repair, salt-and-pepper noise suppression, dark current subtraction, response non-uniformity correction, physical model-constrained image restoration, and dose-adaptive geometric distortion correction, and outputs the restored and corrected image. At the same time, it can also output correction parameters and restoration quality indicators. Among them, the correction parameters refer to the bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters that are actually applied to the current original camera image. The restoration quality indicators are calculated synchronously during the restoration and correction process, including statistical indicators (such as residual outlier ratio, noise standard deviation, and edge preservation) of the images before and after restoration and correction (i.e., the original camera image and the restored and corrected image), as well as the convergence residual of the restoration objective function and the uncertainty map and residual noise map determined based on the convergence residual. The uncertainty map and residual noise map are obtained by pixel-by-pixel statistics of the convergence residual.

[0076] Based on this, the image processing unit reads the current original camera image and the corresponding image acquisition parameters, and completes degradation parameter matching and image restoration and correction based on the degradation parameter library.

[0077] The following is a detailed description of the camera image restoration and correction method under strong radiation environment in this embodiment.

[0078] In actual radiation scenarios, image restoration and distortion correction are performed based on the current image acquisition parameters. The degradation parameter library is called according to the current dose, dose rate, sensor temperature and exposure time, and bad pixel repair, salt and pepper noise suppression, dark current subtraction, response non-uniformity correction, physical model constrained image restoration and dose adaptive geometric distortion correction are performed in sequence.

[0079] (a) Determination of degradation parameters In this embodiment, the original camera image to be corrected is acquired under actual strong radiation environment and the degradation parameters are matched. During the online application phase, the camera under test acquires the current original camera image and obtains the corresponding image acquisition parameters such as dose, dose rate, sensor temperature and exposure time. The degradation parameters of the current original camera image are obtained by calling from the degradation parameter library or by interpolation based on the image acquisition parameters.

[0080] In this embodiment, the original camera image and image acquisition parameters are obtained. The image acquisition parameters are the dose, dose rate, sensor temperature, and exposure time corresponding to the original camera image captured by the camera in a strong radiation environment. Based on the degradation parameter library, the degradation parameters corresponding to the image acquisition parameters are determined. The degradation parameter library includes degradation parameter sample values ​​corresponding to each image acquisition parameter sample value among multiple image acquisition parameter sample values. The degradation parameters include bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters.

[0081] Specifically, based on the degradation parameter library, the degradation parameters corresponding to the image acquisition parameters are determined, including: (1) Determine whether the image acquisition parameters are the same as the sample values ​​of the image acquisition parameters in the degradation parameter library; (2) If so, the degradation parameter sample value corresponding to the same image acquisition parameter sample value as the image acquisition parameter is used as the degradation parameter corresponding to the image acquisition parameter; (3) If not, when the image acquisition parameter is located between two image acquisition parameter sample values ​​in the degradation parameter library, the degradation parameter sample values ​​corresponding to the two image acquisition parameter sample values ​​are interpolated to obtain the degradation parameter corresponding to the image acquisition parameter.

[0082] Interpolation employs the aforementioned multidimensional interpolation. Since the influencing factors of the defect mask are known to include dose, dose rate, sensor temperature, and exposure time; the influencing factors of the salt-and-pepper noise probability map include dose; the influencing factors of the dark current map include dose, sensor temperature, and exposure time; the influencing factors of the gain correction map include dose; the influencing factors of the bias correction map include dose; the influencing factors of the camera intrinsic parameters include dose; and the influencing factors of the distortion parameters include dose, if interpolation is required, interpolation can be performed only on the influencing factors. For example, interpolating dose, dose rate, sensor temperature, and exposure time yields the defect mask; interpolating dose yields the salt-and-pepper noise probability map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters; and interpolating dose, sensor temperature, and exposure time yields the dark current map.

[0083] Taking dose interpolation as an example, when the current dose is between two calibrated doses (i.e., doses in the degradation parameter library), the degradation parameters are obtained by interpolation: (11) (12) In the above formula, For dosage The degradation parameters can be any one of the following: salt-and-pepper noise probability map, gain correction map, bias correction map, camera intrinsic parameters, and distortion parameters. For linear interpolation weights; For dosage The degradation parameters below, For the degradation parameter library and the current dose Adjacent, and smaller than the current dose The calibrated dose; For dosage The degradation parameters below, For the degradation parameter library and the current dose Adjacent and greater than the current dose The calibrated dose.

[0084] If multiple parameters need to be interpolated at the same time, the aforementioned multidimensional interpolation can be used.

[0085] As an optional implementation, this embodiment can determine the degradation parameters by fitting a functional relationship. Specifically, after obtaining the bad pixel mask under each image acquisition parameter sample value, the bad pixel mask under each image acquisition parameter sample value is input into equation (2) to obtain the functional relationship between the probability of a pixel being a bad pixel and the dose, dose rate, sensor temperature and exposure time in the image acquisition parameter sample value. Subsequently, the dose, dose rate, sensor temperature and exposure time in the image acquisition parameters are input to output the probability of each pixel being a bad pixel, thus obtaining the bad pixel mask.

[0086] After obtaining the salt-and-pepper noise probability map for each image acquisition parameter sample value, considering that the dose-dependent salt-and-pepper noise probability map can be represented by a linear model or an exponential model: (13) (14) In the above formula, For dosage The probability plot of salt and pepper noise below; , , Dosage Dose rate and sensor temperature The linear influence coefficient on the probability of salt-and-pepper noise occurrence. The background constant; is the dose response coefficient of the exponential model.

[0087] By inputting the salt-and-pepper noise probability map under each image acquisition parameter sample value into equation (13) or equation (14), the functional relationship between the salt-and-pepper noise occurrence probability of a pixel and the dose in the image acquisition parameters can be obtained. Subsequently, by inputting the dose in the image acquisition parameters, the salt-and-pepper noise occurrence probability of each pixel can be output, and the salt-and-pepper noise probability map can be obtained.

[0088] After obtaining the dark current map under each image acquisition parameter sample value, input the dark current map under each image acquisition parameter sample value into equation (4) to obtain the functional relationship between the dark current of the pixel and the dose, sensor temperature and exposure time in the image acquisition parameters. Then, input the dose, sensor temperature and exposure time in the image acquisition parameters to output the dark current of each pixel and obtain the dark current map.

[0089] By inputting the flat field image under the sample values ​​of each image acquisition parameter into equation (5), the functional relationship between the pixel gain and bias and the dose in the image acquisition parameters can be obtained. Specifically, it can be obtained by least squares estimation. Subsequently, by inputting the dose in the image acquisition parameters, the gain and bias of each pixel can be output, and the gain correction map and bias correction map can be obtained.

[0090] After obtaining the camera intrinsic parameters and distortion parameters under each image acquisition parameter sample value, input the camera intrinsic parameters and distortion parameters under each image acquisition parameter sample value into equation (6) and equation (7) respectively, and the functional relationship between the camera intrinsic parameters and distortion parameters and the dose in the image acquisition parameters can be obtained. Then, input the dose in the image acquisition parameters to output the camera intrinsic parameters and distortion parameters.

[0091] It should be further explained that, for the bad pixel mask, whether it is obtained by interpolation or by fitting a function relationship, it is necessary to determine the probability of a certain pixel being a bad pixel and then binarize it with a preset probability threshold. If the probability is greater than the preset probability threshold, it is 1; otherwise, it is 0. Only then can the bad pixel mask under the current image acquisition parameters be obtained.

[0092] (II) Image Restoration and Correction In this embodiment, the original camera image is restored based on the bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map and bias correction map to obtain the restored image. The restored image is then distorted based on the camera intrinsic parameters and distortion parameters to obtain the restored and corrected image.

[0093] (1) Image restoration This embodiment performs radiated noise and response correction based on the current degradation parameters. First, a bad pixel mask is used to perform neighborhood weighted interpolation repair on fixed bad pixels and response drift points to complete bad pixel repair. Then, transient impulse points are detected and suppressed using the salt-and-pepper noise probability map and neighborhood median deviation to complete salt-and-pepper noise suppression. Next, dark current subtraction and response non-uniformity correction are completed using the dark current map, gain correction map, and bias correction map. Finally, image restoration is performed using the image restoration method of the physical model.

[0094] In this embodiment, image restoration is performed on the original camera image based on the bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, and bias correction map to obtain the restored image. Specifically, this includes: 1) Determine the bad pixels in the original camera image based on the bad pixel mask, perform a weighted summation of the pixel values ​​of the pixels in the neighborhood corresponding to the bad pixels to obtain the repaired pixel values, replace the pixel values ​​of the bad pixels in the original camera image with the repaired pixel values, and obtain the image after bad pixel repair.

[0095] Defect pixel repair uses a neighborhood-weighted interpolation formula. In this case, the repaired pixel value can be weighted and interpolated based on the effective neighboring pixels, spatial distance, gray-level similarity, and edge direction. (15) In equation (15), For pixels The repaired pixel value, which is the pixel value in the image after bad pixel repair. Pixel values; The neighborhood corresponding to the bad pixel contains only valid pixels, which are pixels that are not bad pixels. For the pixels in the neighborhood corresponding to the bad pixel The corresponding normalized weights are determined by spatial distance, gray-level similarity, and edge direction; For pixels in the neighborhood of the bad pixel in the original camera image The corresponding pixel value.

[0096] At this point, the bad pixel mask and the original camera image are input, and the image after bad pixel repair is output, thus completing the bad pixel repair.

[0097] 2) Based on the salt-and-pepper noise probability map, determine the salt-and-pepper noise points in the image after bad pixel repair. Perform a weighted summation on the pixel values ​​of the pixels in the neighborhood corresponding to the salt-and-pepper noise points to obtain the suppressed pixel values. Replace the pixel values ​​of the salt-and-pepper noise points in the image after bad pixel repair with the suppressed pixel values ​​to obtain the image after salt-and-pepper noise suppression.

[0098] Salt-and-pepper noise suppression uses a salt-and-pepper noise point detection formula determined by the deviation between pixel saturation state and neighborhood median to identify salt-and-pepper noise points, outputting a salt-and-pepper noise point marker map, and replacing the detected salt-and-pepper noise points with the neighborhood median or weighted summation value.

[0099] Specifically, determining salt-and-pepper noise points in the image after bad pixel repair based on the salt-and-pepper noise probability map includes: for each pixel in the image after bad pixel repair, determining the median deviation threshold of the pixel based on the salt-and-pepper noise probability map, and using the salt-and-pepper noise point detection formula based on the median deviation threshold of the pixel to determine whether the pixel is a salt-and-pepper noise point, so as to determine the salt-and-pepper noise points in the image after bad pixel repair.

[0100] Salt-and-pepper noise detection can be performed based on pixel saturation state and neighborhood median deviation. The formula for detecting salt-and-pepper noise points is as follows: (16) In equation (16), For pixels The salt-and-pepper noise point detection results, if If the value is 1, then the pixel Salt and pepper noise point; Pixels in the image after bad pixel repair Pixel values; This is the saturation discrimination margin; This represents the pixel saturation value. Pixels in the image after bad pixel repair Corresponding neighborhood The median pixel value of an inner pixel; Pixels in the image after bad pixel repair With dose The varying median deviation threshold can be determined manually based on the salt-and-pepper noise probability map.

[0101] 3) Calculate the difference between the image after salt-and-pepper noise suppression and the dark current map to obtain the image after dark current subtraction.

[0102] Dark current subtraction uses the dark current subtraction formula and outputs the image after dark current subtraction.

[0103] The dark current deduction formula is: (17) In equation (17), Pixels in the image after dark current subtraction Pixel values; Pixels in the image after salt-and-pepper noise suppression Pixel values; For the degradation parameter library and the current dose Sensor temperature and exposure time Pixels in the matched dark current map The pixel value.

[0104] 4) The dark current subtraction image is processed based on the gain correction map and the bias correction map to obtain the response non-uniformity correction image.

[0105] Response nonuniformity correction (also known as response nonuniformity normalization) uses the response nonuniformity correction formula to output a normalized response nonuniformity corrected image.

[0106] Specifically, the dark current subtraction image is processed based on the gain correction map and the bias correction map to obtain the response non-uniformity corrected image. This process includes: processing the dark current subtraction image using the response non-uniformity correction formula based on the gain correction map and the bias correction map to obtain the response non-uniformity corrected image.

[0107] The formula for correcting response nonuniformity is: (18) In equation (18), In response to pixels in the image after non-uniformity correction Pixel values; Pixels in the image after dark current subtraction Pixel values; For dosage Pixels in the offset correction image below Pixel values; For dosage Pixels in the gain correction image below The pixel value.

[0108] The above formula can eliminate the difference in response between pixels.

[0109] 5) Using data consistency, edge preservation, residual salt-and-pepper noise sparsity (also known as residual sparsity) and radiative degradation physical consistency as constraints, image restoration is performed on the image after response nonuniformity correction to obtain the restored image.

[0110] This embodiment performs physical model-constrained image restoration based on the image after response non-uniformity correction. It solves for the restored image by constraining data consistency, edge preservation, residual salt-and-pepper noise sparsity, and physical consistency of radiation degradation, and outputs restoration quality index to determine whether the current restored image meets the requirements of visual monitoring or measurement.

[0111] The image restoration process is specifically accomplished by solving the restoration objective function. Taking the image after response non-uniformity correction as input, a physical model-constrained optimization method is adopted. The optimization objectives include data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiation degradation physical consistency. The data consistency constraint ensures that the restored image is consistent with the observed image after the current degradation operator is applied. The edge preservation, residual salt-and-pepper noise sparsity, and radiation degradation physical consistency are used as regularization constraints to establish the restoration objective function. The gradient-based iterative algorithm is used to solve the problem, minimizing the restoration objective function to obtain the restored image.

[0112] At this point, with constraints of data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiative degradation physical consistency, image restoration is performed on the image after response nonuniformity correction to obtain the restored image. Specifically, this includes: constructing a restoration objective function based on data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiative degradation physical consistency; and optimizing the restoration objective function using a bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, and the image after response nonuniformity correction as inputs to obtain the restored image corresponding to the minimum restoration objective function.

[0113] The objective function for restoration is: (19) (20) In the above formula, To restore the objective function, The image to be restored; The image after processing the restored image using the degradation operator, The degradation operator is composed of degradation parameters from the degradation parameter library, including bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map and bias correction map; In response to the non-uniformity-corrected image, it is used as the currently observed image; For data consistency items; The weights corresponding to the edge preservation term; For edge-preserving terms; The weights corresponding to the sparsity terms of residual salt-and-pepper noise; This represents the residual salt-and-pepper noise sparsity term (e.g., L1 norm). The weights corresponding to the physical consistency term of radiation degradation; For radiation degradation physical consistency term (constrained restoration image and dose) (This is consistent with the statistical patterns of degradation). The goal is to restore the image when the objective function is minimized.

[0114] The edge preservation term employs isotropic smoothed total variation to preserve edges while suppressing noise: (twenty one) In equation (21), and These are the first-order difference operators in the horizontal and vertical directions, respectively; To restore pixels in the image Pixel values; To avoid a smoothing constant that is not differentiable at zero.

[0115] The residual salt-and-pepper noise sparsity term is weighted by the L1 norm of the salt-and-pepper noise probability map, so that residual impulses are suppressed only at high-probability locations: (twenty two) In equation (22), This is a probability plot of salt-and-pepper noise. For pixels in the salt-and-pepper noise probability map Pixel values; To restore pixels in the image neighborhood The median filtering result.

[0116] The residual of the restored image constrained by the radiation degradation physical consistency term, after being positively applied by the current degradation operator, satisfies the noise statistics at this dose, meaning the variance of the whitening residual should be 1: (twenty three) In equation (23), To process pixels in the restored image using a degradation operator Pixel values; In response to pixels in the image after non-uniformity correction Pixel values; For dosage The following is the pixel-wise noise standard deviation given by the degradation parameter library, which is the dose. Pixels in each frame of the degraded image The standard deviation of the pixel values. This term ensures that the restoration result is neither underfitted (excessively large residuals) nor overfitted (the residuals are less than the noise level and the noise is treated as a signal during restoration).

[0117] Optionally, the image restoration in this embodiment can also employ a combination of physical models and deep networks. The degraded images acquired at different doses and radiation-free reference images are used to form a training dataset. The inputs to the deep network include the degraded image, dose, dose rate, sensor temperature, exposure time, bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, and bias correction map. The outputs are the restored image, uncertainty map, and residual noise map.

[0118] The deep network adopts an encoder-decoder (U-Net type) structure with skip connections: the encoder extracts multi-scale features of the degraded image, and the graph priors such as bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map and bias correction map are concatenated with the degraded image by channel and input into the encoder. Scalar conditions such as dose, dose rate, sensor temperature and exposure time are normalized and fully connected and then mapped to modulate the features of each scale of the encoder by channel. The decoder fuses the multi-scale features through skip connections, and the three output branches give the restored image, uncertainty map and residual noise map respectively.

[0119] The loss function during training is: (twenty four) In equation (24), Total loss; This refers to the reconstruction error term (such as L1 norm or L2 norm) between the restored image and the non-radioactive reference image. For the weighting coefficients corresponding to the physical consistency term; This is the physical consistency term, which is the deviation between the restored image and the observed image after the current degradation operator has processed it in the forward direction. These are the weighting coefficients corresponding to the edge preservation term; For edge preservation, the L1 norm of the gradient domain constrains the edge position and intensity; These are the weighting coefficients corresponding to the sparsity term of the residual salt-and-pepper noise. This represents the sparsity term of the residual salt-and-pepper noise. These are the weighting coefficients corresponding to the uncertainty estimation term; The uncertainty estimate is expressed as a negative log-likelihood term. The uncertainty plot of the constrained deep network prediction is consistent with the statistical characteristics of the actual restored residual.

[0120] The expression for the reconstruction error term is: (25) In equation (25), Î represents the restored image output by the deep network; This is a non-radioactive reference image.

[0121] The expression for the physical consistency term is: (26) In equation (26), This is the image after processing the restored image Î output by the deep network using the degradation operator.

[0122] The expression for the edge preservation term is: (27) In equation (27), The gradient of the restored image Î output by the deep network; The gradient of the non-radioactive reference image.

[0123] The expression for the sparsity term of residual salt-and-pepper noise is: (28) In equation (28), To restore the image output by the deep network The image obtained after median filtering.

[0124] The expression for the uncertainty estimate is: (29) In equation (29), Reconstructed image output by a deep network Medium pixel Pixel values; For radiation-free reference images Medium pixel Pixel values; The uncertainty map output by the deep network. Uncertainty map output by deep network Medium pixel The pixel value.

[0125] After training, the original camera image (which is a degraded image), dose, dose rate, sensor temperature, exposure time, bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, and bias correction map can be directly input into the trained deep network to output the restored image.

[0126] (2) Distortion correction After obtaining the restored image, dose-adaptive geometric distortion correction is performed based on the obtained restored image and the current geometric parameters (i.e., camera intrinsic parameters and distortion parameters), and the restored and corrected image is output.

[0127] In this embodiment, distortion correction is performed on the restored image based on camera intrinsic parameters and distortion parameters to obtain a restored and corrected image.

[0128] Specifically, based on the current dose call or interpolation (when the current dose is between two calibrated doses, the camera intrinsic parameters and distortion parameters can be obtained by linear interpolation formula), the camera intrinsic parameters and distortion parameters (including radial distortion parameters and tangential distortion parameters) are obtained. Based on the camera intrinsic parameters and distortion parameters, the denoised and restored image is remapped and geometrically corrected. First, the pixel coordinates are normalized according to the current camera intrinsic parameters to obtain the distortion coordinates. Then, the radial distortion and tangential distortion models (including distortion parameters) are substituted to calculate the distortion-free coordinates. Remapping is completed by interpolation resampling to obtain the distortion-free restored and corrected image. At the same time, the final restored and corrected image, degradation parameters, image acquisition parameters, and restoration quality indicators can be output to the host computer or monitoring system.

[0129] The distorted coordinates are represented as: (30) In equation (30), ( , ( ) represents the x and y coordinates of the distorted coordinates; , ) represents the pixel coordinates of the restored image; , () are the coordinates of the main point; , These are the equivalent focal lengths in the x and y directions, respectively. The principal point coordinates and the equivalent focal lengths in the x and y directions constitute the camera's intrinsic parameters.

[0130] The above formula can be used to normalize pixel coordinates.

[0131] The radial and tangential distortion models are as follows: (31) (32) (33) In the above formula, ( , ( ) represents the x and y coordinates of the distorted coordinates; , () represents the x and y coordinates without distortion; , , For radial distortion parameters; Normalized radial distance; , This refers to the tangential distortion parameter.

[0132] The above formula yields the distortion-free coordinates, thus completing the remapping.

[0133] Dose-related distortion parameters can be expressed as: (34) In equation (34), the subscript and These represent the values ​​at the current dose and the values ​​at the radiation-free reference dose, respectively. This is the dose-related distortion parameter drift vector, obtained by fitting calibration data at each dose.

[0134] At this point, distortion correction is performed on the restored image based on camera intrinsic parameters and distortion parameters to obtain a restored and corrected image. Specifically, for each pixel in the restored image, the pixel coordinates are converted into the distorted coordinates of the pixel based on camera intrinsic parameters, and the distorted coordinates of the pixel are converted into the undistorted coordinates of the pixel based on distortion parameters to obtain the restored and corrected image.

[0135] This embodiment proposes a camera image restoration and correction method that combines the degradation mechanism of strong radiation imaging, experimental calibration data of different radiation doses, and physical model-constrained image restoration algorithms. It realizes bad pixel suppression, salt-and-pepper noise removal, dark current subtraction, response non-uniformity correction, and dose-adaptive geometric distortion correction in strong radiation environments. It belongs to the fields of image processing, machine vision, radiation environment perception and camera calibration technology, and is applicable to visual monitoring equipment in strong radiation environments.

[0136] This method establishes a physical model of camera imaging degradation under strong radiation environments, including a bad pixel generation model, a salt-and-pepper noise formation model, a dark current drift model, a response non-uniformity model, and a geometric distortion drift model. It constructs a strong radiation environment experimental acquisition system and acquires dark-field images, flat-field images, calibration plate images, and target scene images under different image acquisition parameters. It extracts bad pixel masks, salt-and-pepper noise probability maps, dark current maps, gain correction maps, bias correction maps, camera intrinsic parameters, and distortion parameters under different image acquisition parameters. A degradation parameter library is established, and corresponding degradation parameters are obtained from the library based on the current image acquisition parameters. Based on these degradation parameters, the original camera images acquired under strong radiation environments are subjected to bad pixel repair, salt-and-pepper noise suppression, dark current subtraction, response non-uniformity correction, and geometric distortion correction to obtain restored and corrected images. This method can improve the image quality and visual measurement reliability of cameras under strong radiation environments and is applicable to scenarios such as nuclear power, space exploration, high-energy physics experiments, and radioactive environment monitoring.

[0137] This method proposes a model for the formation mechanism of camera defects and salt-and-pepper noise under strong radiation environments, rather than simply using traditional filtering algorithms. It constructs an experimental calibration data system under different radiation doses, establishes a mapping relationship between dose, defective pixels, noise, response drift, and distortion parameters, and establishes dose-related defective pixel masks, salt-and-pepper noise probability maps, dark current maps, gain correction maps, bias correction maps, camera intrinsic parameters, and distortion parameters. It can dynamically select degradation parameters based on the current dose, dose rate, sensor temperature, and exposure time, and proposes a physically constrained image restoration method that preserves edges, textures, and measurement features while denoising. Simultaneously, it achieves dose-adaptive camera geometric distortion correction under strong radiation environments, thereby improving image quality under strong radiation conditions, effectively suppressing defective pixels, salt-and-pepper noise, and dark current drift, improving the reliability of visual measurements, enhancing the usability of edge, corner, and texture features, and possessing dose-adaptive capabilities to adapt to different radiation dose rates and cumulative dose scenarios. It balances physical interpretability and algorithm performance, facilitates engineering deployment and parameter traceability, and can be extended to CMOS cameras, CCD cameras, industrial cameras, infrared cameras, and radiation-resistant cameras.

[0138] This application also provides an application scenario in which the above-described camera image restoration and correction method under strong radiation environments is applied. Specifically, the camera image restoration and correction method under strong radiation environments provided in this embodiment can be applied in image processing scenarios. Image processing scenarios include a processing stage and a display stage. The processing stage is used to process the original camera image to obtain a restored and corrected image, and the display stage is used to show the restored and corrected image to the user for easier decision-making. The camera image restoration and correction method under strong radiation environments provided in this embodiment belongs to the processing stage.

[0139] In one exemplary embodiment, a camera image restoration and correction device under strong radiation environment is provided, including an image acquisition module, a radiation dose acquisition module, an experimental calibration data management module, a bad pixel identification module, a salt-and-pepper noise detection module, a physical model constraint denoising module, a response non-uniformity correction module, a geometric distortion correction module, an online update module, and a result output module. All modules work together to implement the steps in the above method embodiment.

[0140] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a method for restoring and correcting camera images under strong radiation conditions.

[0141] Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0143] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0144] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations and be authorized by the owner of the corresponding device.

[0146] Any references to memory, databases, or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0147] The databases involved in the various embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases.

[0148] The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.

[0149] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for restoring and correcting camera images under strong radiation conditions, characterized in that, include: Acquire raw camera images and image acquisition parameters; The image acquisition parameters are the dose, dose rate, sensor temperature, and exposure time corresponding to the camera capturing original camera images in a strong radiation environment. Based on the degradation parameter library, the degradation parameters corresponding to the image acquisition parameters are determined; the degradation parameter library includes degradation parameter sample values ​​corresponding to each image acquisition parameter sample value among multiple image acquisition parameter sample values, and the degradation parameters include bad pixel mask, salt-and-pepper noise probability map, dark current map, gain correction map, bias correction map, camera intrinsic parameters and distortion parameters; Based on the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, and the bias correction map, the original camera image is restored to obtain a restored image. Based on the camera intrinsic parameters and the distortion parameters, the restored image is distorted to obtain a restored and corrected image.

2. The method for restoring and correcting camera images under strong radiation conditions according to claim 1, characterized in that, Methods for establishing a degradation parameter library include: Acquire the dark field image, flat field image, calibration plate image, and target scene image corresponding to each of the multiple image acquisition parameter sample values; For each of the image acquisition parameter sample values, a bad pixel mask and dark current map corresponding to the image acquisition parameter sample value are determined based on the dark field image; a gain correction map and bias correction map corresponding to the image acquisition parameter sample value are determined based on the flat field image; camera intrinsic parameters and distortion parameters corresponding to the image acquisition parameter sample value are determined based on the calibration board image; and a salt-and-pepper noise probability map corresponding to the image acquisition parameter sample value is determined based on the target scene image, thereby determining the degradation parameter sample value corresponding to the image acquisition parameter sample value. The degradation parameter sample value corresponding to each of the multiple image acquisition parameter sample values ​​is stored to obtain the degradation parameter library.

3. The method for restoring and correcting camera images under strong radiation conditions according to claim 1, characterized in that, Based on the degradation parameter library, the degradation parameters corresponding to the image acquisition parameters are determined, specifically including: Determine whether the image acquisition parameters are the same as the sample values ​​of image acquisition parameters in the degradation parameter library; If so, the degradation parameter sample value corresponding to the same image acquisition parameter sample value as the image acquisition parameter is taken as the degradation parameter corresponding to the image acquisition parameter; If not, when the image acquisition parameter is located between two image acquisition parameter sample values ​​in the degradation parameter library, interpolation is performed on the degradation parameter sample values ​​corresponding to the two image acquisition parameter sample values ​​to obtain the degradation parameter corresponding to the image acquisition parameter.

4. The method for restoring and correcting camera images under strong radiation conditions according to claim 1, characterized in that, Based on the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, and the bias correction map, the original camera image is restored to obtain a restored image, specifically including: Based on the bad pixel mask, bad pixels in the original camera image are determined. The pixel values ​​of the pixels in the neighborhood corresponding to the bad pixels are weighted and summed to obtain the repaired pixel values. The pixel values ​​of the bad pixels in the original camera image are replaced with the repaired pixel values ​​to obtain the image after bad pixel repair. Based on the salt-and-pepper noise probability map, salt-and-pepper noise points in the image after bad pixel repair are determined. The pixel values ​​of the pixels in the neighborhood corresponding to the salt-and-pepper noise points are weighted and summed to obtain the suppressed pixel values. The pixel values ​​of the salt-and-pepper noise points in the image after bad pixel repair are replaced with the suppressed pixel values ​​to obtain the image after salt-and-pepper noise suppression. The difference between the image after salt-and-pepper noise suppression and the dark current map is calculated to obtain the image after dark current subtraction. The dark current subtraction image is processed based on the gain correction map and the bias correction map to obtain the response non-uniformity correction image; The image after response nonuniformity correction is restored by constraining data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiative degradation physical consistency, to obtain the restored image.

5. The method for restoring and correcting camera images under strong radiation conditions according to claim 4, characterized in that, Determining salt-and-pepper noise points in the image after bad pixel repair based on the salt-and-pepper noise probability map specifically includes: for each pixel in the image after bad pixel repair, determining the median deviation threshold of the pixel based on the salt-and-pepper noise probability map, and using the salt-and-pepper noise point detection formula based on the median deviation threshold of the pixel to determine whether the pixel is a salt-and-pepper noise point, so as to determine the salt-and-pepper noise points in the image after bad pixel repair. The formula for detecting salt and pepper noise points is as follows: ； in, For pixels The salt-and-pepper noise point detection result, if the value is 1, is a salt-and-pepper noise point; Pixels in the image after bad pixel repair Pixel values; This is the saturation discrimination margin; This represents the pixel saturation value. Pixels in the image after bad pixel repair Corresponding neighborhood The median pixel value of an inner pixel; Pixels in the image after bad pixel repair With dose The changing median deviation threshold.

6. The method for restoring and correcting camera images under strong radiation conditions according to claim 4, characterized in that, The dark current subtraction image is processed based on the gain correction map and the bias correction map to obtain a response non-uniformity corrected image. Specifically, the dark current subtraction image is processed using a response non-uniformity correction formula based on the gain correction map and the bias correction map to obtain a response non-uniformity corrected image. The formula for correcting non-uniformity of response is: ； in, In response to pixels in the image after non-uniformity correction Pixel values; For pixels in the image after dark current subtraction Pixel values; For dosage Pixels in the offset correction image below Pixel values; For dosage Pixels in the gain correction image below The pixel value.

7. The method for restoring and correcting camera images under strong radiation conditions according to claim 4, characterized in that, Constrained by data consistency, edge preservation, residual salt-and-pepper noise sparsity, and radiative degradation physical consistency, image restoration is performed on the image after response non-uniformity correction to obtain the restored image, specifically including: Based on data consistency, edge preservation, residual salt-and-pepper noise sparsity, and physical consistency of radiative degradation, a restoration objective function is constructed. Using the bad pixel mask, the salt-and-pepper noise probability map, the dark current map, the gain correction map, the bias correction map, and the image after response non-uniformity correction as input, the restoration objective function is optimized and solved to obtain the restored image corresponding to the minimum restoration objective function.

8. The method for restoring and correcting camera images under strong radiation conditions according to claim 1, characterized in that, Based on the camera intrinsic parameters and the distortion parameters, distortion correction is performed on the restored image to obtain a restored and corrected image, specifically including: For each pixel in the restored image, based on the camera intrinsic parameters, the pixel coordinates of the pixel are converted into the distorted coordinates of the pixel, and based on the distortion parameters, the distorted coordinates of the pixel are converted into the undistorted coordinates of the pixel, thus obtaining the restored and corrected image.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the camera image restoration and correction method under strong radiation environment as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the camera image restoration and correction method under strong radiation environment as described in any one of claims 1-8.

Images