CCD image processing method, device and storage medium

Abstract

This specification discloses a CCD image processing method, device, and storage medium. In this method, a server can split the image data to be processed into sub-image data to be processed and perform charge transfer inefficiency removal processing on each sub-image data to be processed using a pre-trained charge transfer inefficiency removal model. This method can effectively remove charge transfer inefficiencies caused by semiconductor material defects or impurities. This method can effectively remove false signals caused by charge transfer inefficiency in CCD images, thereby accurately correcting pixel values and improving overall image quality.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a CCD image processing method, device, and storage medium. Background Art

[0002] A charge-coupled device (CCD) is an optical detector whose surface is composed of numerous tiny photosensitive cells (pixels or picture elements) arranged in a two-dimensional array. When light strikes the CCD surface, each photosensitive cell generates a specific amount of charge based on the intensity of the light received. This process is based on the photoelectric effect, where the energy of a photon is absorbed by a semiconductor material, stimulating the generation of electron-hole pairs. The more intense the light, the greater the charge generated. The charge generated by each photosensitive cell is temporarily stored within the cell until, under the influence of a periodic voltage pulse, each photosensitive cell transfers its accumulated charge to the next adjacent photosensitive cell. The adjacent photosensitive cell then transfers its accumulated charge to the next photosensitive cell (typically row-by-row or column-by-column), until the charge is transferred to registers located at the edge of the CCD surface. The registers convert the charge into an electrical signal whose intensity corresponds to the intensity of the light received by each photosensitive cell. For example, the stronger the light received by a photosensitive cell, the more charge it generates, and the stronger the readout signal. Then, the electrical signal corresponding to each photosensitive unit can be converted into the brightness value of each pixel point that constitutes the CCD image through a computer or other processing equipment, thereby obtaining a CCD image.

[0003] However, during the CCD imaging process, the quality of the obtained CCD image may be poor due to the problem of charge transfer inefficiency (CTI). That is, because defects or impurities in the semiconductor material also have the ability to capture charges, the charges accumulated in a photosensitive unit cannot be fully transferred to the register. These captured charges will cause the problem of charge transfer inefficiency.

[0004] Therefore, how to remove the influence of the charge transfer inefficiency problem in CCD images to improve the quality of CCD images is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a CCD image processing method, device and storage medium to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a CCD image processing method, including:

[0008] Acquiring image data to be processed collected by a charge coupled device;

[0009] Splitting the image data to be processed into sub-image data to be processed; wherein each sub-image data to be processed is used to represent the pixel value of a row or a column of pixels in the image data to be processed;

[0010] For each sub-image data to be processed, the sub-image data to be processed is input into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed;

[0011] The processed sub-image data are spliced together to obtain the target image data.

[0012] This specification provides a CCD image processing device, including:

[0013] An acquisition module, used for acquiring image data to be processed collected by a charge coupled device;

[0014] a splitting module, configured to split the image data to be processed into sub-image data to be processed; wherein each sub-image data to be processed is used to represent the pixel values of a row or a column of pixels in the image data to be processed;

[0015] a processing module, configured to input each sub-image data to be processed into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed;

[0016] The output module is used to splice the processed sub-image data to obtain the target image data.

[0017] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned CCD image processing method is implemented.

[0018] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned CCD image processing method when executing the program.

[0019] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0020] In the CCD image processing method provided in this specification, the image data to be processed collected by the charge coupled device is first obtained, and the image data to be processed is split into sub-image data to be processed, wherein each sub-image data to be processed is used to represent the pixel value of a row or a column of pixel points in the image data to be processed. For each sub-image data to be processed, the sub-image data to be processed is input into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed, and the processed sub-image data are spliced to obtain the target image data.

[0021] As can be seen from the above method, the server can effectively remove charge transfer inefficiency caused by semiconductor material defects or impurities by splitting the image data to be processed into sub-image data to be processed and performing charge transfer inefficiency removal processing on each sub-image data to be processed using a pre-trained charge transfer inefficiency removal model. This method can effectively remove the false signals caused by charge transfer inefficiency in CCD images, thereby accurately correcting pixel values and improving the overall image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0023] Figure 1 A schematic flow chart of a CCD image processing method provided in this specification;

[0024] Figure 2 This is a schematic diagram of the pixel value distribution of the astronomical images taken by the Hubble telescope provided in this manual;

[0025] Figure 3 A schematic diagram of the preprocessing process of the image data to be processed provided in this specification;

[0026] Figure 4 This is a schematic diagram of the effect of the unilateral scaling normalization process provided in this manual;

[0027] Figure 5 This is a schematic diagram of the effect of the bilateral scaling normalization process provided in this specification;

[0028] Figure 6 A schematic diagram of the charge transfer inefficiency removal model provided in this specification;

[0029] Figure 7A The relative error distribution histogram obtained based on the test set provided in this specification;

[0030] Figure 7B A schematic diagram of the target image data provided in this specification;

[0031] Figure 8 This is a schematic diagram of the CCD image processing speed comparison provided in this manual;

[0032] Figure 9 A schematic diagram of a CCD image processing device provided in this specification;

[0033] Figure 10 This is a schematic diagram of an electronic device for CCD image processing provided in this specification. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0035] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0036] Charge transfer efficiency (CTE) refers to how much charge is correctly transferred from one photosensitive cell to another during the CCD imaging process. The ideal charge transfer efficiency during CCD imaging is 100%. However, because defects or impurities in semiconductor materials can also capture charge, this efficiency often falls short of ideal in practical applications. Furthermore, since the CCD imaging process determines the pixel value of each photosensitive cell in the final image based on the amount of charge accumulated by that cell, the accuracy of the resulting image is often low due to inefficient charge transfer.

[0037] Based on this, in the fields of remote sensing, astronomy, biology, medicine, etc., there are often strict requirements for the instructions of CCD images. Therefore, in these fields, it is particularly important to eliminate the charge transfer inefficiency (CTI) in CCD images (eliminating the negative impact of charge transfer inefficiency on CCD imaging).

[0038] For example, in astronomical imaging applications, cosmic rays carry a large number of high-energy, high-speed particles, which can damage CCDs and increase the number of defects in semiconductors, resulting in a severe CTI effect. Furthermore, fields requiring astronomical imaging have extremely high requirements for image sensor performance, making CTI a particularly prominent issue in these fields.

[0039] Figure 1 The following is a flow chart of a CCD image processing method provided in this specification, comprising the following steps:

[0040] S101: Acquire image data to be processed collected by a charge coupled device.

[0041] In this specification, the execution entity used to implement the CCD image processing method can refer to a designated device such as a server set up in a data processing center or an image analysis platform, or it can refer to a terminal device such as a desktop computer, a laptop computer, a professional graphics workstation, etc. For the sake of ease of description, the CCD image processing method provided in this specification is explained below using the server as an example of the execution entity.

[0042] In this specification, the server can perform charge transfer inefficiency (CTI) removal processing on the CCD image data collected by the charge coupled device using a pre-trained charge transfer inefficiency removal model to obtain processed image data, and then perform tasks based on the processed image data.

[0043] Before this, the server may first obtain CCD image data collected by the charge coupled device as image data to be processed.

[0044] It should be noted that the above-mentioned image data to be processed may be two-dimensional matrix data for representing the pixel value of each pixel constituting the CCD image.

[0045] S102: Split the image data to be processed into sub-image data to be processed; wherein each sub-image data to be processed is used to represent the pixel value of a row or a column of pixel points in the image data to be processed.

[0046] In actual application scenarios, during the CCD imaging process, each photosensitive unit transfers its accumulated charge to the next adjacent photosensitive unit one by one in the form of row-by-row transfer or column-by-column transfer, and then the adjacent photosensitive unit transfers the accumulated charge of this unit to the next photosensitive unit until it is transferred to the register set at the edge of the CCD surface, and converted into an electrical signal through the register. Then, based on the electrical signal converted by the register, the pixel value of the pixel point corresponding to the photosensitive unit in the generated CCD image can be determined.

[0047] Therefore, in the image data to be processed, the pixel values of each row or column of pixels are affected differently by the inefficiency of charge transfer. Therefore, after obtaining the image data to be processed, the server can split the image data to be processed into each sub-image data to be processed, and then process each sub-image data to be processed to obtain the processed sub-image data.

[0048] Each sub-image data to be processed is used to represent the pixel values of a row or a column of pixel points in the image data to be processed.

[0049] It should be noted that for scientific image data, there is often a problem of large and uneven distribution of values, such as Figure 2 shown.

[0050] Figure 2 This is a schematic diagram of the pixel value distribution of the astronomical images taken by the Hubble telescope provided in this manual.

[0051] exist Figure 2 In the figure, the horizontal axis represents different pixel values, and the vertical axis represents the number of pixel points in the image data whose pixel values are the pixel values corresponding to the horizontal axis. Figure 2 It can be seen that in astronomical images taken by the Hubble telescope, a large number of background pixel values are concentrated in a very small range, while a small number of values are distributed over a large range. This makes the distribution of pixel values extremely uneven even after high-precision processing using traditional normalization methods such as z-score.

[0052] Based on this, in this specification, the server can perform variable scaling normalization processing on each sub-image data to be processed according to the predicted variable scaling normalization strategy to obtain each pre-processed sub-image data to be processed, as shown in the following example: Figure 3 shown.

[0053] Figure 3 This is a schematic diagram of the preprocessing process of the image data to be processed provided in this specification.

[0054] Combine Figure 3 It can be seen that after dividing the data to be processed into sub-image data to be processed, the server can also perform variable scaling normalization processing on each sub-image data to be processed according to the predicted variable scaling normalization strategy to obtain each pre-processed sub-image data to be processed, and perform image processing on each pre-processed sub-image data to be processed, so as to obtain the final processed sub-image data.

[0055] The variable scaling normalization strategy includes one of a unilateral variable scaling normalization strategy and a bilateral variable scaling normalization strategy. The unilateral variable scaling normalization strategy or the bilateral variable scaling normalization strategy can be selected based on the distribution of pixel values of each pixel in the image data to be processed. For example, if the impact of negative values in the image data to be processed (such as pixel values of overexposed or underexposed pixels) does not need to be considered, the unilateral variable scaling normalization strategy can be selected; otherwise, the bilateral variable scaling normalization strategy can be selected.

[0056] Specifically, if the server adopts a unilateral variable scaling normalization strategy to perform variable scaling normalization processing on each sub-image data to be processed to obtain each pre-processed sub-image data to be processed, the steps may include:

[0057] Step 1: The server may clip the pixel value of each pixel according to the valid range vmin and vmax of the pixel value of each pixel contained in the image data to be processed, that is, the value smaller than vmin is set to be equal to vmin, and the value greater than vmax is set to be equal to vmax.

[0058] Step 2: Perform unilateral scaling normalization processing on the pixel value of each pixel point contained in the sub-image data to be processed to obtain the pre-processed sub-image data to be processed. For details, please refer to the following formula:

[0059]

[0060] In the above formula, That is the pixel value of the pixel before scaling normalization. That is, the pixel value of the pixel before the scaling normalization process. The a and add values are used to adjust the shape and skip the rapid change interval of the logarithmic function. They can be set according to actual needs. Figure 4 shown.

[0061] Figure 4 This is a schematic diagram of the effect after unilateral scaling normalization processing provided in this manual.

[0062] exist Figure 4 In the figure, part a shows the correspondence between the pixel number and pixel value before unilateral scaling normalization. Part b shows the correspondence between the pixel number and pixel value after unilateral scaling normalization. The horizontal axis is the pixel number in the sub-image to be processed, and the vertical axis is the pixel value.

[0063] Combining the above-mentioned image part a and image part b, it can be seen that the server can perform unilateral scaling normalization processing on the sub-image data to be processed, so that the densely distributed value interval becomes sparse, and the sparse value interval becomes dense.

[0064] Furthermore, if the server adopts a bilateral variable scaling normalization strategy to perform variable scaling normalization processing on each sub-image data to be processed to obtain each pre-processed sub-image data to be processed, the specific steps may include:

[0065] Step 1: The server may clip the pixel value of each pixel according to the valid range vmin and vmax of the pixel value of each pixel contained in the image data to be processed, that is, the value smaller than vmin is set to be equal to vmin, and the value greater than vmax is set to be equal to vmax.

[0066] Step 2: Set the median value, that is, the value med corresponding to the position with the most dense value distribution. For each pixel in the sub-image data to be processed, adjust the pixel value of the pixel according to the median value med. For details, refer to the following formula:

[0067] If the pixel value x of the pixel is greater than med:

[0068]

[0069] If the pixel value x of the pixel is less than med:

[0070]

[0071] Step 3: Perform bilateral scaling normalization processing on the pixel value of each pixel point contained in the sub-image data to be processed to obtain the pre-processed sub-image data to be processed. For details, please refer to the following formula:

[0072]

[0073]

[0074]

[0075] In the above formula, That is the pixel value of the pixel before scaling and normalization. That is, the pixel value of the pixel before the scaling normalization process. The a and add values are used to adjust the shape and skip the rapid change interval of the logarithmic function. They can be set according to actual needs. Figure 5 shown.

[0076] Figure 5 This is a schematic diagram of the effect of bilateral scaling normalization processing provided in this manual.

[0077] exist Figure 5 In the figure, image c shows the correspondence between pixel numbers and pixel values before bilateral scaling normalization. Image d shows the correspondence between pixel numbers and pixel values after bilateral scaling normalization. The horizontal axis is the pixel number in the sub-image to be processed, and the vertical axis is the pixel value.

[0078] Combining the above images in part c and part d, it can be seen that the server can perform bilateral scaling normalization processing on the sub-image data to be processed, so that the densely distributed value intervals become sparse, and the sparse value intervals become dense.

[0079] S103: For each sub-image data to be processed, input the sub-image data to be processed into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed.

[0080] Furthermore, the server may input each preprocessed sub-image data to be processed into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the preprocessed sub-image data to be processed.

[0081] Of course, the server may also directly input each sub-image data to be processed into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed.

[0082] In actual application scenarios, in order to improve the processing efficiency of CCD images, the server can also divide the sub-image data to be processed into batch data sets, and then, for each batch data set, each sub-image data to be processed contained in the batch data set can be input into a pre-trained charge transfer inefficiency removal model, so that the charge transfer inefficiency removal model can perform parallel processing on each sub-image data to be processed contained in the batch data set, and obtain the processed sub-image data corresponding to each sub-image data to be processed contained in the batch data set.

[0083] The size of the batch data set can be set according to actual needs. For example, if the size of each batch data set is batch_size, the server can divide the w sub-image data to be processed into W / / batch_size batch data sets, that is, W / / batch_size batch data sets of shape H batch_size 1's matrix data.

[0084] It should be noted that the above-mentioned charge transfer inefficiency removal model can only be deployed on a server for use with CCD image data after training. The method for training the above-mentioned charge transfer inefficiency removal model can be to obtain sample image data and split the sample image data into sample sub-image data. For each sample sub-image data, the sample sub-image data is input into the charge transfer inefficiency removal model to be trained to obtain processed sample sub-image data corresponding to the sample sub-image data. The processed sample sub-image data can then be concatenated to obtain processed sample image data. A target loss value can be determined based on the deviation between the processed sample image data and the true image data corresponding to the sample image data. The charge transfer inefficiency removal model to be trained is then trained with minimizing the target loss value as the optimization goal, thereby obtaining a trained charge transfer inefficiency removal model.

[0085] The greater the deviation between the processed sample image data and the true value image data corresponding to the sample image data, the greater the determined target loss value.

[0086] In actual application scenarios, the server can also determine the target loss value based on the deviation between the processed sample image data and the true image data corresponding to the sample image data, and use minimizing the target loss value as the optimization goal to train the charge transfer inefficiency removal model to obtain an alternative charge transfer inefficiency removal model, and repeat the above steps at least once to obtain each alternative charge transfer inefficiency removal model.

[0087] Among them, different alternative charge transfer inefficiency removal models are obtained by training the charge transfer inefficiency removal model to be trained using different sample image data.

[0088] Furthermore, for each alternative charge transfer inefficiency removal model, the preset verification image data can be input into the alternative charge transfer inefficiency removal model to determine the evaluation loss value of the alternative charge transfer inefficiency removal model based on the deviation between the processed verification image data output by the alternative charge transfer inefficiency removal model for the verification image data and the true value verification image data corresponding to the verification image data.

[0089] On this basis, the server can select the alternative charge transfer inefficiency removal model with the smallest evaluation loss value from the alternative charge transfer inefficiency removal models according to the evaluation loss value of each alternative charge transfer inefficiency removal model as the trained charge transfer inefficiency removal model.

[0090] It should be noted that in practical applications, each sub-image data to be processed is often processed in parallel as a batch data set. In this case, the charge transfer inefficiency removal model for each sub-image data to be processed may be affected by the characteristics of other sub-image data to be processed, resulting in reduced accuracy of the processed sub-image data.

[0091] Therefore, the aforementioned charge transfer inefficiency removal model can also be composed of a main model and a conditional fusion model. In this case, the server can input the identification information (such as row ID or column ID) and timestamp of each sub-image data to be processed into the conditional fusion model to obtain modulation parameters. The server can also input the sub-image data to be processed and the modulation parameters into the main model of the pre-trained charge transfer inefficiency removal model to obtain the processed sub-image data corresponding to the sub-image data to be processed.

[0092] From the above content, it can be seen that the server can generate modulation parameters based on the identification information and timestamp of each sub-image data to be processed by introducing a conditional fusion model. These parameters can reflect the specific characteristics of each sub-image data to be processed (such as the defects in the semiconductor material where each photosensitive unit corresponding to the sub-image data to be processed is located), thereby providing a personalized processing strategy for each sub-image data to be processed to solve the problem of feature difference retention and interference that may exist between different sub-image data to be processed in batch processing. Specifically, Figure 6 shown.

[0093] Figure 6 Schematic diagram of the charge transfer inefficiency removal model provided in this specification.

[0094] Combine Figure 6 It can be seen that the main model in the charge transfer inefficiency removal model includes: at least one intermediate processing layer, wherein the intermediate processing layer includes: an intermediate convolution layer and an activation layer.

[0095] In practical applications, the main model described above may further include: a first convolutional layer, at least one intermediate processing layer, and a second convolutional layer. The intermediate processing layer may include: an intermediate convolutional layer, a batch normalization layer, and an activation layer. If the hidden layer variable dimension of the main model described above is D_hidden, then the first convolutional layer has 1 input channel and D_hidden output channels. The second convolutional layer has D_hidden input channels and 1 output channel. The input and output dimensions of each intermediate convolutional layer are both D_hidden.

[0096] Specifically, the first convolutional layer maps the input single-channel image data (i.e., the sub-image data to be processed) into a higher-dimensional feature space, thereby extracting the initial features of the sub-image data. It has D_hidden convolution kernels, each responsible for extracting a specific feature of the input image, outputting D_hidden feature maps. These feature maps capture information such as local texture and edges in the sub-image data to be processed, providing a rich feature foundation for subsequent processing.

[0097] The intermediate processing layer is used to further process and optimize D_hidden feature maps.

[0098] Among them, the middle convolution layer continues to extract local features. Through the design of multiple convolution kernels, it can extract more complex features from the input D_hidden feature maps.

[0099] In the above content, the calculation process of the i-th output feature map can refer to the following formula:

[0100]

[0101] In the above formula, I is the input feature map, is the calculated i-th feature map, is the i-th convolution kernel, is the i-th bias.

[0102] Furthermore, the batch normalization layer is used to normalize the output of the intermediate convolutional layer, eliminate internal covariate shift, accelerate the model training process and improve the stability of the model.

[0103] Activation layers (such as ReLU) are used to introduce nonlinear factors, enabling the model to learn more complex feature mapping relationships.

[0104] The second convolutional layer maps the D_hidden-dimensional feature map output by the intermediate processing layer back to a single-channel output, generating the final processed sub-image data. This layer uses a single convolution kernel to fuse the extracted complex features to produce the final processed sub-image data.

[0105] During this process, the server can input the sub-image data to be processed and the modulation parameters into the main model of the pre-trained charge transfer inefficiency removal model, so that each intermediate processing layer of the main model performs linear transformation modulation on the intermediate feature map output by the intermediate convolution layer in the intermediate processing layer according to the modulation parameters to obtain the modulated intermediate feature map, and input it into the activation layer to obtain the processed sub-image data corresponding to the sub-image data to be processed.

[0106] Specifically, the conditional fusion model can input the identification information and timestamp of the input sub-image data to be processed in each sub-image data to be processed into the conditional encoder of the conditional fusion model to obtain the modulation parameters beta and alpha output by the conditional fusion model. The dimensions of the modulation parameters beta and alpha here are both N*D_hidden. Furthermore, in the process of processing each intermediate processing layer of the main model, the intermediate feature map output by the batch normalization layer can be linearly transformed and modulated before being input into the activation layer. Among them, if the j-th dimension of the intermediate feature map output by the batch normalization layer before the convolution layer of the i-th intermediate processing layer is F_i_j, then the j-th dimension of the modulated intermediate feature map obtained after modulation is FM_i_j. For details, please refer to the following formula:

[0107]

[0108] Furthermore, the server can also splice the sub-image data to be processed input into the main model into the feature map output by the above-mentioned second convolutional layer in a residual connection manner as the target feature map, thereby forcing the charge transfer inefficiency removal model to learn the difference part of the output data to reduce the complexity of the charge transfer inefficiency removal model learning.

[0109] Furthermore, the server may perform inverse scaling and normalization processing on the target feature map to obtain processed sub-image data corresponding to the sub-image data to be processed.

[0110] It should be noted that the inverse scaling normalization processing here is the same as the above-mentioned variable scaling normalization processing, except that the input and output are reversed, and this specification will not describe it in detail.

[0111] S104: Splicing the processed sub-image data to obtain target image data.

[0112] In this specification, after obtaining each processed sub-image data, the server may splice each processed sub-image data to obtain target image data, and may perform a task according to the target image data.

[0113] These tasks can be determined based on the specific application scenario. For example, in remote sensing, target image data can be used for tasks such as land use classification, environmental monitoring, and crop growth assessment. By removing the effects of charge transfer inefficiency (CTI), the processed images can more accurately reflect surface features and changes. For example, high-resolution remote sensing imagery can be used to monitor regional changes such as deforestation, urban expansion, or natural disasters, providing more reliable data support for environmental protection and resource management.

[0114] Another example: In astronomical imaging applications, target image data can be used for tasks such as galaxy classification, star observation, and cosmic background radiation research. Because cosmic ray damage to CCDs can lead to severe CTI effects, images after removing CTI can more clearly reveal the details and structure of celestial objects. For example, in deep-space observations, processed images can be used to study the morphology and evolution of distant galaxies or discover new celestial bodies such as asteroids and comets.

[0115] From the above method, it can be seen that the server can effectively remove the charge transfer inefficiency problem caused by semiconductor material defects or impurities by splitting the image data to be processed into each sub-image data to be processed and performing charge transfer inefficiency removal processing on each sub-image data to be processed through a pre-trained charge transfer inefficiency removal model. This method can remove the false signal caused by charge transfer inefficiency in CCD images, thereby accurately correcting the pixel value to improve the overall quality of the image. Specifically, Figure 7A 、 Figure 7B shown.

[0116] Figure 7A This is a relative error distribution histogram obtained based on the test set provided in this specification.

[0117] from Figure 7A As can be seen from the relative error distribution histogram generated by processing a test set of 100 4096*4096 images, the relative error values of the image data processed by the above CCD image processing method are closer to 0. This demonstrates that the above method effectively removes the influence of charge transfer inefficiency (CTI), significantly reducing the difference between the processed image and the ideal image. The closer the relative error is to 0, the better the image restoration effect and the more significant the improvement in image quality.

[0118] Figure 7B This is a schematic diagram of the target image data provided in this specification.

[0119] exist Figure 7B In the image data, the left image data is the image data to be processed, the middle image data is the target image data, and the right image data is the true value image data corresponding to the image data to be processed. Figure 7B It can be seen from the figure that the above CCD image processing method can remove the false signal caused by the inefficient charge transfer in the CCD image, thereby accurately correcting the pixel value.

[0120] In addition, the above method can be used to process the image data to obtain the target image data, which can effectively improve the speed of CCD image processing. Figure 8 shown.

[0121] from Figure 8 It can be seen that when a graphics card with a graphics card model of Rt*2080Ti is used to execute the CCD image processing method provided in this manual, the time taken to perform CCD image processing is 1.19 seconds per image. When a graphics card with a graphics card model of V100 is used to execute the CCD image processing method provided in this manual, the time taken to perform CCD image processing is 0.8 seconds per image. In the prior art, the time taken to perform high-precision CCD image processing using the SimpleCTI algorithm is 1990.375 seconds per image, and the time taken to perform low-precision CCD image processing is 19.647 seconds per image. The time taken to perform high-precision CCD image processing using the Arctic algorithm is 10161.74 seconds per image, and the time taken to perform low-precision CCD image processing is 271.078 seconds per image. It can be seen that the above-mentioned CCD image processing method can not only effectively remove the false signals caused by the inefficient charge transfer of the CCD image, thereby accurately correcting the pixel value. It can also effectively improve the speed of CCD image processing.

[0122] The above is one or more implementations of the CCD image processing method of this specification. Based on the same idea, this specification also provides a corresponding CCD image processing device, such as Figure 9 shown.

[0123] Figure 9 A schematic diagram of a CCD image processing device provided in this specification, including:

[0124] An acquisition module 901 is used to acquire image data to be processed collected by a charge coupled device;

[0125] A splitting module 902 is configured to split the image data to be processed into sub-image data to be processed; wherein each sub-image data to be processed is used to represent the pixel values of a row or a column of pixels in the image data to be processed;

[0126] The processing module 903 is configured to input each sub-image data to be processed into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed;

[0127] The output module 904 is used to stitch the processed sub-image data together to obtain target image data.

[0128] Optionally, the processing module 903 is specifically used to divide the sub-image data to be processed into batch data sets; for each batch data set, each sub-image data to be processed contained in the batch data set is input into a pre-trained charge transfer inefficiency removal model, so that the charge transfer inefficiency removal model performs parallel processing on each sub-image data to be processed contained in the batch data set, and obtains processed sub-image data corresponding to each sub-image data to be processed contained in the batch data set.

[0129] Optionally, the processing module 903 is specifically used to perform variable scaling normalization processing on each sub-image data to be processed according to the predicted variable scaling normalization strategy to obtain each preprocessed sub-image data to be processed; the variable scaling normalization strategy includes: one of: a unilateral variable scaling normalization strategy and a bilateral variable scaling normalization strategy; for each preprocessed sub-image data to be processed, the preprocessed sub-image data to be processed is input into a pre-trained charge transfer inefficiency removal model to obtain the processed sub-image data corresponding to the preprocessed sub-image data to be processed.

[0130] Optionally, the apparatus further includes: a training module 905;

[0131] The training module 905 is specifically used to obtain sample image data and split the sample image data into sample sub-image data; for each sample sub-image data, input the sample sub-image data into the charge transfer inefficiency removal model to be trained to obtain processed sample sub-image data corresponding to the sample sub-image data; splice the processed sample sub-image data to obtain processed sample image data; determine the target loss value based on the deviation between the processed sample image data and the true value image data corresponding to the sample image data, and train the charge transfer inefficiency removal model to be trained with minimizing the target loss value as the optimization goal to obtain the trained charge transfer inefficiency removal model.

[0132] Optionally, the training module 905 is specifically used to determine a target loss value based on the deviation between the processed sample image data and the true image data corresponding to the sample image data, and train the charge transfer inefficiency removal model to be trained with minimizing the target loss value as the optimization goal to obtain an alternative charge transfer inefficiency removal model; repeat the above steps at least once to obtain each alternative charge transfer inefficiency removal model; wherein different alternative charge transfer inefficiency removal models are obtained by training the charge transfer inefficiency removal model to be trained using different sample image data; input the preset verification image data into each alternative charge transfer inefficiency removal model to obtain an evaluated loss value of each alternative charge transfer inefficiency removal model; and select a trained charge transfer inefficiency removal model from each alternative charge transfer inefficiency removal model based on the evaluated loss value.

[0133] Optionally, the charge transfer inefficiency removal model includes: a main model and a conditional fusion model;

[0134] The processing module 903 is specifically used to, for each sub-image data to be processed, input the identification information and timestamp of the sub-image data to be processed in each sub-image data to be processed into the conditional fusion model to obtain modulation parameters; input the sub-image data to be processed and the modulation parameters into the main model of the pre-trained charge transfer inefficiency removal model to obtain the processed sub-image data corresponding to the sub-image data to be processed.

[0135] Optionally, the main model includes: at least one intermediate processing layer; the intermediate processing layer includes: an intermediate convolution layer and an activation layer;

[0136] The processing module 903 is specifically used to input the sub-image data to be processed and the modulation parameters into the main model of the pre-trained charge transfer inefficiency removal model, so that each intermediate processing layer of the main model performs linear transformation modulation on the intermediate feature map output by the intermediate convolution layer in the intermediate processing layer according to the modulation parameters, obtains the modulated intermediate feature map, and inputs it into the activation layer to obtain the processed sub-image data corresponding to the sub-image data to be processed.

[0137] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 A CCD image processing method is provided.

[0138] This manual also provides Figure 10 The one shown corresponds to Figure 1 Schematic diagram of the electronic equipment. Figure 10As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0139] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0141] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0142] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A CCD image processing method, characterized in that: include: Acquiring image data to be processed collected by a charge coupled device; Splitting the image data to be processed into sub-image data to be processed; wherein each sub-image data to be processed is used to represent the pixel value of a row or a column of pixels in the image data to be processed; For each sub-image data to be processed, inputting identification information and a timestamp of the sub-image data to be processed in each sub-image data to be processed into a conditional fusion model included in a pre-trained charge transfer inefficiency removal model to obtain a modulation parameter; Inputting the sub-image data to be processed and the modulation parameters into the main model included in the charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed; The processed sub-image data are spliced together to obtain the target image data.

2. The method according to claim 1, wherein For each sub-image data to be processed, the sub-image data to be processed is input into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed, specifically including: Dividing the sub-image data to be processed into batch processing data sets; For each batch data set, each sub-image data to be processed contained in the batch data set is input into a pre-trained charge transfer inefficiency removal model, so that the charge transfer inefficiency removal model performs parallel processing on each sub-image data to be processed contained in the batch data set, and obtains processed sub-image data corresponding to each sub-image data to be processed contained in the batch data set.

3. The method according to claim 1, wherein For each sub-image data to be processed, inputting the sub-image data to be processed into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed, the method further includes: According to the predicted variable scaling normalization strategy, variable scaling normalization processing is performed on each sub-image data to be processed to obtain each pre-processed sub-image data to be processed; the variable scaling normalization strategy includes: a unilateral variable scaling normalization strategy and a bilateral variable scaling normalization strategy; For each sub-image data to be processed, the sub-image data to be processed is input into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed, specifically including: For each preprocessed sub-image data to be processed, the preprocessed sub-image data to be processed is input into a pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the preprocessed sub-image data to be processed.

4. The method according to claim 1, wherein The method further includes: training a charge transfer inefficiency removal model, specifically including: Acquire sample image data, and split the sample image data into sample sub-image data; For each sample sub-image data, input the sample sub-image data into the charge transfer inefficiency removal model to be trained to obtain processed sample sub-image data corresponding to the sample sub-image data; splicing the processed sample sub-image data to obtain processed sample image data; According to the deviation between the processed sample image data and the true image data corresponding to the sample image data, a target loss value is determined, and the charge transfer inefficiency removal model to be trained is trained with minimizing the target loss value as the optimization goal to obtain a trained charge transfer inefficiency removal model.

5. The method according to claim 4, wherein Determining a target loss value based on a deviation between the processed sample image data and true image data corresponding to the sample image data, and training the to-be-trained charge transfer inefficiency removal model with minimizing the target loss value as an optimization goal, thereby obtaining a trained charge transfer inefficiency removal model, specifically comprising: determining a target loss value based on a deviation between the processed sample image data and true image data corresponding to the sample image data, and training the to-be-trained charge transfer inefficiency removal model with minimizing the target loss value as an optimization goal to obtain an alternative charge transfer inefficiency removal model; Repeat the above steps at least once to obtain candidate charge transfer inefficiency removal models; wherein different candidate charge transfer inefficiency removal models are obtained by training the to-be-trained charge transfer inefficiency removal model using different sample image data; Inputting preset verification image data into each candidate charge transfer inefficiency removal model to obtain an evaluation loss value of each candidate charge transfer inefficiency removal model; According to the evaluation loss value, a trained charge transfer inefficiency removal model is selected from each candidate charge transfer inefficiency removal model.

6. The method according to claim 1, wherein The main model includes: at least one intermediate processing layer; the intermediate processing layer includes: an intermediate convolution layer and an activation layer; Inputting the sub-image data to be processed and the modulation parameters into the main model of the pre-trained charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed specifically includes: The sub-image data to be processed and the modulation parameters are input into the main model of the pre-trained charge transfer inefficiency removal model, so that each intermediate processing layer of the main model performs linear transformation modulation on the intermediate feature map output by the intermediate convolution layer in the intermediate processing layer according to the modulation parameters to obtain the modulated intermediate feature map, and inputs the modulated intermediate feature map into the activation layer to obtain the processed sub-image data corresponding to the sub-image data to be processed.

7. A CCD image processing device, characterized in that: include: An acquisition module, used for acquiring image data to be processed collected by a charge coupled device; a splitting module, configured to split the image data to be processed into sub-image data to be processed; wherein each sub-image data to be processed is used to represent the pixel values of a row or a column of pixels in the image data to be processed; a processing module configured to input, for each sub-image data to be processed, identification information and a timestamp of the sub-image data to be processed into a conditional fusion model included in a pre-trained charge transfer inefficiency removal model to obtain modulation parameters; and input the sub-image data to be processed and the modulation parameters into a main model included in the charge transfer inefficiency removal model to obtain processed sub-image data corresponding to the sub-image data to be processed; The output module is used to splice the processed sub-image data to obtain the target image data.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images