A method, apparatus and optical detection system for low-resolution image processing with few photons

Abstract

This invention belongs to the field of photon imaging technology and relates to a method, apparatus, and optical detection system for processing low-resolution images with few photons. It involves constructing a low-resolution image dataset of Gaussian spots based on a Poisson detection model; constructing a low-resolution subpixel image processing model, which includes an initial convolutional module, a first depthwise convolutional module, a first upsampling convolutional module, a second depthwise convolutional module, a second upsampling convolutional module, and a fully connected layer, sequentially connected along the forward propagation direction; inputting the low-resolution image with few photons into the low-resolution subpixel image processing model; constructing an image processing loss function based on the feature map output by the model; training the low-resolution image dataset with few photons until the image processing loss function is minimized, thus obtaining a trained low-resolution subpixel image processing model; and obtaining a high-precision centroid estimate based on the high-resolution image output by the model.

Description

Technical Field

[0001] This invention relates to the field of photon imaging technology, and in particular to a low-resolution image processing method, apparatus and optical detection system with few photons. Background Technology

[0002] In recent years, with the increasing frequency of human activities in aerospace and space exploration, optical-based space target detection and tracking technologies have flourished in laser communication and remote sensing applications. The space targets involved include cooperative targets such as communication terminals and known celestial bodies, as well as non-cooperative targets such as space debris and unknown celestial bodies. Optical-based space target detection and tracking refers to using an optical detection system to acquire images, obtaining the positional information of light spots in the images, and returning this positional information to the optical detection system to achieve target localization and tracking. Therefore, accurately obtaining the positional information of light spots in images is crucial for the accuracy of target localization and tracking.

[0003] Existing space target detection and tracking methods typically utilize the centroid method and maximum likelihood estimation to obtain the location information of light spots in images. The centroid method first binarizes the acquired image to highlight the light spot region, then uses an edge detection algorithm to further identify the edges of the light spot, and finally identifies and extracts the pixels in the light spot region. Based on the brightness and position of all pixels in the light spot region, the centroid position of the light spot is calculated, and this centroid position is returned to the optical detection system as the location information of the light spot in the image. The maximum likelihood estimation method sets a light spot brightness distribution model, constructs a likelihood function to describe the center position and intensity of the light spot based on the acquired image and the set light spot brightness distribution model, and finds the parameters that maximize the likelihood function through numerical optimization methods. Based on the optimized parameters, the location information of the light spot is extracted and returned to the optical detection system.

[0004] Although both the centroid method and the maximum likelihood estimation method can obtain the position information of light spots based on images, as the distance of laser communication increases and the targets detected by remote sensing become smaller, spatial targets exhibit obvious dim characteristics. In this case, the images acquired by the optical detection system are low-resolution images with few photons. These images have low resolution and blurry light spot edges. When directly using the centroid method to calculate the centroid position of low-resolution images with few photons, it is impossible to accurately identify the light spot region, resulting in a large error in the calculated centroid position. On the other hand, when using the maximum likelihood estimation method to obtain the light spot position information of low-resolution images with few photons, it cannot accurately describe the actual distribution of the blurry light spot, resulting in an inaccurate light spot brightness distribution model. Consequently, the accuracy of the obtained light spot position information is low, affecting the accuracy of target positioning and tracking.

[0005] For the reasons mentioned above, when acquiring the spot position of a low-resolution image with few photons, it is necessary to process the image to improve its resolution, thereby improving the accuracy of the spot position and the precision of target localization and tracking. However, due to the extremely low signal-to-noise ratio of low-resolution images with few photons, the effective information in the image is extremely limited. Existing image processing models often require a good signal-to-noise ratio and sufficient pixel information to effectively improve the image resolution. In addition, due to the difficulty in labeling low-resolution images with few photons, existing technologies cannot obtain sufficient and highly accurate datasets of such images. This also results in image processing models not having enough data for training, thus failing to effectively improve the resolution of low-resolution images with few photons and failing to solve the problem of low target localization and tracking accuracy caused by inaccurate spot position information.

[0006] In summary, existing image processing methods cannot effectively improve the resolution of low-resolution images with few photons, resulting in inaccurate spot position information and failing to meet the technical requirements for high-precision positioning and aiming of spatial point targets. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the image processing methods in the prior art cannot effectively improve the resolution of low-resolution images with few photons, resulting in inaccurate spot position information and thus failing to meet the technical requirements for high-precision positioning and aiming of spatial point targets.

[0008] To address the aforementioned technical problems, this invention provides a low-resolution image processing method with few photons, comprising:

[0009] A low-resolution image dataset of Gaussian spots with few photons was constructed based on the Poisson detection model.

[0010] A few-photon subpixel image processing model is constructed; the few-photon subpixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module, and a fully connected layer, which are sequentially connected in series along the forward propagation direction;

[0011] The low-resolution images of few photons in the low-resolution image dataset are input into the low-resolution subpixel image processing model, and an image processing loss function is constructed based on the feature map output by the low-resolution subpixel image processing model.

[0012] The few-photon subpixel image processing model is trained using few-photon low-resolution images from the few-photon low-resolution image dataset until the image processing loss function is minimized, thus obtaining the trained few-photon subpixel image processing model.

[0013] Preferably, the low-resolution image dataset of Gaussian spots constructed based on the Poisson detection model includes:

[0014] Obtain the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under different preset spot center positions;

[0015] Substituting the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system at different preset spot center positions into the Poisson detection model, we obtain the photon count probability detected by each single-photon detector in the single-photon detector array at different preset spot center positions.

[0016] Based on the photon count probability detected by each single photon detector in the single photon detector array at each preset spot center position, a low-resolution image with few photons is generated, and the preset spot center position is used as the actual centroid position of the spot in the low-resolution image with few photons.

[0017] Based on all the generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained.

[0018] Preferably, the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under different preset spot center positions is expressed as follows:

[0019]

[0020] Where, λ s (x,y) represents the signal beam intensity on the surface of the single-photon detector array in the optical detection system at the preset spot center position; I0 represents the peak intensity; ρ represents the half-height of the spot; (x0,y0) represents the preset spot center position; (x,y) represents any position of the spot.

[0021] The probability of photon count detected by each single-photon detector in the single-photon detector array under different preset spot center positions is expressed as follows:

[0022]

[0023] Wherein, P({Z m =z m}) represents the photon count probability detected by the m-th single-photon detector in the single-photon detector array at the preset spot center position; z m A represents the photon count detected by the m-th single-photon detector; m represents the photosensitive area of ​​the m-th single-photon detector; M represents the number of single-photon detectors in the single-photon detector array.

[0024] Preferably, the initial convolutional module, the first upsampling convolutional module, and the second upsampling convolutional module each include at least one convolutional kernel;

[0025] The first depthwise convolutional module includes at least two convolutional kernels connected in series along the forward propagation direction;

[0026] The second depthwise convolutional module includes at least five convolutional kernels connected in series along the forward propagation direction.

[0027] Preferably, constructing the image processing loss function based on the feature map output by the few-photon sub-pixel image processing model includes:

[0028] The predicted centroid position of the light spot in the feature map is obtained using the centroid method or the maximum likelihood estimation method;

[0029] An image processing loss function is constructed based on the predicted centroid position of the light spot and the actual centroid position of the light spot in the low-resolution image with few photons corresponding to the feature map.

[0030] Preferably, the image processing loss function is expressed as:

[0031] Loss=CLoss(pred,targ)+0.1*MSELoss(pred,targ)-(pred-0.5) 2 ,

[0032]

[0033]

[0034] Where Loss represents the image processing loss function; CLoss(pred, targ) represents the error between the predicted centroid position and the actual centroid position of the spot; pred represents the feature map output by the few-photon subpixel image processing model; (px_i, py_j) represents the coordinates of the pixel in the i-th row and j-th column of pred; targ represents the few-photon low-resolution image; (tx_i, ty_j) represents the coordinates of the pixel in the i-th row and j-th column of targ; N represents the number of rows of pixels in the few-photon low-resolution image / feature map; J represents the number of columns of pixels in the few-photon low-resolution image / feature map; MSELoss(pred, targ) represents the root mean square error between the predicted centroid position and the actual centroid position of the spot; (Cp_x, Cp_y) represents the predicted centroid position of the spot; (Ct_x, Ct_y) represents the actual centroid position of the spot.

[0035] Preferably, after obtaining the trained few-photon subpixel image processing model, the process further includes:

[0036] Acquire low-resolution images with few photons, and input the low-resolution images with few photons into a trained low-photon subpixel image processing model to output feature maps;

[0037] The centroid position of the light spot in the feature image is obtained using the centroid method or the maximum likelihood estimation method, and the centroid position of the light spot is sent to the control module of the optical detection system so that the control module can control the fast-reflecting mirror through the fast-reflecting mirror drive control module to realize the localization and tracking of the light spot in the low-resolution image with few photons.

[0038] Preferably, when the resolution of the low-resolution image with few photons is 32*32,

[0039] The initial convolutional module includes a 5*5 convolutional kernel;

[0040] The first depthwise convolutional module includes two 3*3 convolutional kernels connected in series along the forward propagation direction;

[0041] The first upsampling convolution module includes a 5*5 convolution kernel;

[0042] The second depthwise convolutional module includes five 3*3 convolutional kernels connected in series along the forward propagation direction;

[0043] The second upsampling convolution module includes a 5*5 convolution kernel.

[0044] The present invention also provides a few-photon low-resolution image processing apparatus, comprising:

[0045] The dataset building module is used to construct a low-resolution image dataset of Gaussian spots with few photons based on the Poisson detection model.

[0046] The model building module is used to construct a few-photon subpixel image processing model; the few-photon subpixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module, and a fully connected layer, which are sequentially connected in the forward propagation direction.

[0047] The loss function construction module is used to input the low-resolution images of the low-photon low-resolution image dataset into the low-photon subpixel image processing model, and construct an image processing loss function based on the feature map output by the low-photon subpixel image processing model.

[0048] The model training and acquisition module is used to train the few-photon subpixel image processing model using few-photon low-resolution images in the few-photon low-resolution image dataset until the value of the image processing loss function is minimized, thereby obtaining the trained few-photon subpixel image processing model.

[0049] The present invention also provides an optical detection system, comprising:

[0050] A telescope is used to collect light signals;

[0051] A quick-reflecting mirror is used to adjust the light signal collected by the telescope;

[0052] A single-photon detector array is used to receive optical signals adjusted by a fast mirror and convert the received optical signals into electrical signals.

[0053] An analog-to-digital converter is used to receive electrical signals generated by a single-photon detector array and convert the electrical signals into digital signals.

[0054] The data processing module is used to convert the digital signal generated by the analog-to-digital converter into a low-resolution image with few photons, and to process the low-resolution image with few photons using the above-mentioned low-resolution image processing method, to obtain the centroid position of the light spot in the processed low-resolution image, and to send the centroid position of the light spot to the control module.

[0055] A control module is used to generate control commands based on the centroid position of the light spot and send them to the fast-reflecting mirror drive control module.

[0056] The fast-reflecting mirror drive control module is used to control the angle and displacement of the fast-reflecting mirror based on the control commands, thereby realizing the positioning and tracking of the light spot in the low-resolution image with few photons.

[0057] The low-resolution image processing method for few photons provided in this application considers that under low illumination conditions, the number of photons reaching the detector often follows a Poisson distribution. Using a Poisson model to simulate the imaging process under few photon conditions can reflect the uncertainty of photon counting in actual observations. Therefore, this application uses a low-resolution image dataset of Gaussian spots constructed based on the Poisson detection model to train a low-resolution sub-pixel image processing model for few photons. The model then includes an initial convolutional module, a first depth convolutional module, a first upsampling convolutional module, a second depth convolutional module, a second upsampling convolutional module, and a fully connected layer, sequentially connected along the forward propagation direction. First, the initial convolutional module extracts potential effective information from the low-resolution image for few photons. Then, the first depth convolutional module extracts low-level features of the image, and the first upsampling convolutional module extracts low-level features. The first upsampling convolution module improves the spatial resolution of the image. Then, the second deep convolution module can extract deep features of the image and filter out unnecessary noise generated during the upsampling process, retaining the main features of the signal. The second upsampling convolution module is used again to further improve the spatial resolution of the image. Finally, a fully connected layer is used to integrate the extracted features. This can filter out noise in the image as much as possible and extract effective features under conditions of extremely low signal-to-noise ratio and limited effective information, thereby achieving high-resolution image reconstruction. By training the few-photon subpixel image processing model until the image processing loss function is minimized, a well-trained few-photon subpixel image processing model is obtained. This can effectively improve the resolution of low-resolution images with few photons, thereby obtaining more accurate spot position information and achieving high-precision positioning and aiming of spatial point targets. Attached Figure Description

[0058] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0059] Figure 1 Flowchart of the low-resolution image processing method with few photons provided in this application;

[0060] Figure 2 A schematic diagram of an optical detection system and a few-photon subpixel image processing model provided in this application embodiment;

[0061] Figure 3 This is a schematic diagram of the image processing results provided in an embodiment of this application; wherein, Figure 3 (a) in the image is the original high-resolution image of the light spot. Figure 3 (b) in the middle is Figure 2 The image shown is a low-resolution image with few photons acquired by the optical detection system. Figure 3 (c) in the middle is Figure 2 The super-resolution image output by the few-photon subpixel image processing model shown;

[0062] Figure 4 This application provides a schematic diagram comparing the root mean square error of the spot center position obtained using different methods under different noise photon counts and cumulative pulse counts in embodiments of this application; wherein... Figure 4 (a) in the figure is a schematic diagram comparing the root mean square error of the spot center position obtained by different methods under different noise photon numbers. Figure 4 (b) in the figure is a schematic diagram comparing the root mean square error of the spot center position obtained by different methods under different cumulative pulse numbers;

[0063] Figure 5 This is a schematic diagram of the structure of the low-resolution image processing device with few photons provided in this application. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0065] Please see Figure 1 , Figure 1 The flowchart of the low-resolution image processing method with few photons provided in this application is as follows:

[0066] S10: Construct a low-resolution image dataset of Gaussian spots with few photons based on the Poisson detection model;

[0067] S20: Construct a few-photon subpixel image processing model; the few-photon subpixel image processing model includes an initial convolutional module, a first depth convolutional module, a first upsampling convolutional module, a second depth convolutional module, a second upsampling convolutional module, and a fully connected layer, which are sequentially connected along the forward propagation direction;

[0068] S30: Input the few-photon subpixel images from the few-photon low-resolution image dataset into the few-photon low-resolution image processing model, and construct the image processing loss function based on the feature map output by the few-photon subpixel image processing model;

[0069] S40: Train the few-photon subpixel image processing model using few-photon low-resolution images from the few-photon low-resolution image dataset until the image processing loss function is minimized, thus obtaining the trained few-photon subpixel image processing model.

[0070] The low-resolution image processing method for few photons provided in this application considers that under low illumination conditions, the number of photons reaching the detector often follows a Poisson distribution. Using a Poisson model to simulate the imaging process under few photon conditions can reflect the uncertainty of photon counting in actual observations. Therefore, this application uses a low-resolution image dataset of Gaussian spots constructed based on the Poisson detection model to train a low-resolution sub-pixel image processing model for few photons. The model then includes an initial convolutional module, a first depth convolutional module, a first upsampling convolutional module, a second depth convolutional module, a second upsampling convolutional module, and a fully connected layer, sequentially connected along the forward propagation direction. First, the initial convolutional module extracts potential effective information from the low-resolution image for few photons. Then, the first depth convolutional module extracts low-level features of the image, and the first upsampling convolutional module extracts low-level features. The first upsampling convolution module improves the spatial resolution of the image. Then, the second deep convolution module can extract deep features of the image and filter out unnecessary noise generated during the upsampling process, retaining the main features of the signal. The second upsampling convolution module is used again to further improve the spatial resolution of the image. Finally, a fully connected layer is used to integrate the extracted features. This can filter out noise in the image as much as possible and extract effective features under conditions of extremely low signal-to-noise ratio and limited effective information, thereby achieving high-resolution image reconstruction. By training the few-photon subpixel image processing model until the image processing loss function is minimized, a well-trained few-photon subpixel image processing model is obtained. This can effectively improve the resolution of low-resolution images with few photons, thereby obtaining more accurate spot position information and achieving high-precision positioning and aiming of spatial point targets.

[0071] Specifically, when constructing a low-resolution image dataset with few photons, it is assumed that the intensity distribution of the signal beam incident on the surface of the single-photon detector array satisfies a Gaussian distribution, so that the intensity distribution of the signal beam on the surface of the single-photon detector array in the optical detection system can be obtained under different preset spot center positions.

[0072] Specifically, the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under different preset spot center positions is expressed as follows:

[0073]

[0074] Where, λ s (x,y) represents the signal beam intensity on the surface of the single-photon detector array in the optical detection system at the preset spot center position; I0 represents the peak intensity; ρ represents the half-height of the spot; (x0,y0) represents the preset spot center position; (x,y) represents any position of the spot.

[0075] Furthermore, since the response of a single-photon detector is a Poisson process, by substituting the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system at different preset spot center positions into the Poisson detection model, the photon count probability detected by each single-photon detector in the single-photon detector array at different preset spot center positions can be obtained.

[0076] Specifically, the photon count probability detected by each single-photon detector in the single-photon detector array under different preset spot center positions is expressed as follows:

[0077]

[0078] Wherein, P({Z m =z m}) represents the photon count probability detected by the m-th single-photon detector in the single-photon detector array at the preset spot center position; z m A represents the photon count detected by the m-th single-photon detector; m The pixel area of ​​the m-th single-photon detector is represented by ; M represents the number of single-photon detectors in the single-photon detector array.

[0079] Based on the photon count probability detected by each single photon detector in the single photon detector array at each preset spot center position, a low-resolution image with few photons is generated, and the preset spot center position is used as the actual centroid position of the spot in the low-resolution image with few photons.

[0080] Based on all the generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained.

[0081] The structure of each module in the few-photon subpixel image processing model will affect the image processing effect. In some embodiments of this application, the initial convolution module, the first upsampling convolution module and the second upsampling convolution module each include at least one convolution kernel.

[0082] The first depthwise convolutional module includes at least two convolutional kernels connected in series along the forward propagation direction;

[0083] The second depthwise convolutional module includes at least five convolutional kernels connected in series along the forward propagation direction.

[0084] Specifically, the convolutional kernels in the initial convolutional module convolve the low-resolution image with few photons to extract its basic features, resulting in a first feature map. The first depth convolutional module uses at least two convolutional kernels to gradually extract the depth features of the first feature map, resulting in a second feature map. The concatenated arrangement of the convolutional kernels increases the nonlinear expressive power of the model. The convolutional kernels in the first upsampling convolutional module deconvolve the second feature map, unfolding it to retain richer contextual information and outputting a high-resolution third feature map. After obtaining the third feature map, multiple convolutional kernels in the second depth convolutional module are used again to extract its depth features, outputting a fourth feature map. The multiple convolutional kernels are still concatenated to increase the nonlinear expressive power of the model. Finally, the convolutional kernels in the second upsampling convolutional module deconvolve the fourth feature map again, unfolding it once more to output a higher-resolution fifth feature map.

[0085] In a specific example, when the resolution of the low-resolution image with few photons is 32*32, the initial convolutional module includes a 5*5 convolutional kernel; the first depth convolutional module includes two 3*3 convolutional kernels connected in series along the forward propagation direction; the first upsampling convolutional module includes a 5*5 convolutional kernel; the second depth convolutional module includes five 3*3 convolutional kernels connected in series along the forward propagation direction; the second upsampling convolutional module includes a 5*5 convolutional kernel; and the feature map output by the low-photon subpixel image processing model has a resolution of 160*160.

[0086] Furthermore, in step S30, the image processing loss function is constructed based on the feature map output by the few-photon sub-pixel image processing model, including:

[0087] The predicted centroid position of the light spot in the feature map can be obtained using the centroid method or the maximum likelihood estimation method.

[0088] An image processing loss function is constructed based on the predicted centroid position of the light spot and the actual centroid position of the light spot in the low-resolution image with few photons corresponding to the feature map.

[0089] Specifically, in some embodiments of this application, the constructed image processing loss function is expressed as:

[0090] Loss=CLoss(pred,targ)+0.1*MSELoss(pred,targ)-(pred-0.5) 2 ,

[0091]

[0092]

[0093] Where Loss represents the image processing loss function; CLoss(pred, targ) represents the error between the predicted centroid position and the actual centroid position of the spot; pred represents the feature map output by the few-photon subpixel image processing model; (px_i, py_j) represents the coordinates of the pixel in the i-th row and j-th column of pred; targ represents the few-photon low-resolution image; (tx_i, ty_j) represents the coordinates of the pixel in the i-th row and j-th column of targ; N represents the number of rows of pixels in the few-photon low-resolution image / feature map; J represents the number of columns of pixels in the few-photon low-resolution image / feature map; MSELoss(pred, targ) represents the root mean square error between the predicted centroid position and the actual centroid position of the spot; (Cp_x, Cp_y) represents the predicted centroid position of the spot; (Ct_x, Ct_y) represents the actual centroid position of the spot.

[0094] Specifically, the formula for calculating the position of the centroid is expressed as follows:

[0095]

[0096]

[0097] Furthermore, after obtaining the trained few-photon subpixel image processing model, the acquired few-photon low-resolution images can be input into the model for processing to improve the resolution of the few-photon low-resolution images, thereby enabling the localization and tracking of targets in the images.

[0098] Specifically, after obtaining the trained few-photon subpixel image processing model, the following steps are also included:

[0099] Acquire low-resolution images with few photons and input them into a trained low-photon subpixel image processing model to output feature maps;

[0100] The centroid position of the light spot in the feature map is obtained by using the centroid method or the maximum likelihood estimation method, and the centroid position of the light spot is sent to the control module of the optical detection system so that the control module can control the fast reflection mirror through the fast reflection mirror to realize the localization and tracking of the light spot in the low-resolution image with few photons.

[0101] Based on the few-photon subpixel image processing method provided in the above embodiments, this application also provides an optical detection system, including:

[0102] A telescope is used to collect light signals;

[0103] A quick-reflecting mirror is used to adjust the light signal collected by the telescope;

[0104] A single-photon detector array is used to receive optical signals adjusted by a fast mirror and convert the received optical signals into electrical signals.

[0105] An analog-to-digital converter is used to receive electrical signals generated by a single-photon detector array and convert them into digital signals.

[0106] The data processing module is used to convert the digital signal generated by the analog-to-digital converter into a low-resolution image with few photons, and to process the low-resolution image with few photons using the aforementioned low-resolution subpixel image processing method, to obtain the centroid position of the light spot in the processed low-resolution image, and to send the centroid position of the light spot to the control module.

[0107] The control module is used to generate control commands based on the centroid position of the light spot and send them to the fast-reflecting mirror drive control module.

[0108] The fast-reflecting mirror drive control module is used to control the angle and displacement of the fast-reflecting mirror based on control commands, thereby realizing the positioning and tracking of light spots in low-resolution images with few photons.

[0109] The effectiveness of the few-photon low-resolution image processing method provided in this application is verified through specific examples below, such as... Figure 2 The figure shows the optical detection system and the few-photon subpixel image processing model provided in this embodiment. The left side of the figure is a schematic diagram of the optical detection system structure, and the right side is a schematic diagram of the few-photon subpixel image processing model structure. This model is deployed on the data processing module of the optical detection system.

[0110] In this embodiment, the few-photon subpixel image processing model includes an initial convolution module with a size of 5*5 convolution kernels, which perform convolution operations on a few-photon low-resolution image with a resolution of 32*32 to generate a 64-dimensional first feature map.

[0111] The first depthwise convolutional module includes two concatenated convolutional kernels, each with a size of 3*3. The first convolutional kernel convolves the 64-dimensional first feature map to generate a new 64-dimensional first feature map, and the second convolutional kernel convolves the new first feature map to generate a 25-dimensional second feature map.

[0112] The first upsampling convolutional module includes a 5*5 convolutional kernel, which upsamples the second feature map and outputs a third feature map with a size of 161*161*1.

[0113] The second depthwise convolutional module includes five cascaded 3x3 convolutional kernels. The first kernel convolves the third feature map to generate a 32-dimensional feature map. The second kernel convolves the 32-dimensional feature map to generate a 64-dimensional feature map. The third kernel convolves the 64-dimensional feature map to generate a new 64-dimensional feature map. The fourth kernel convolves the new 64-dimensional feature map to generate a 128-dimensional feature map. The fifth kernel convolves the 128-dimensional feature map to generate a 128-dimensional fourth feature map.

[0114] The second upsampling convolution module includes a 5*5 convolution kernel, which upsamples the fourth feature map and outputs a fifth feature map with a size of 160*160*1.

[0115] The fully connected layer reshapes the fifth feature map, outputting a feature map with a resolution of 160*160.

[0116] like Figure 3 The image shown is a schematic diagram of the image processing results provided in this embodiment; wherein, Figure 3 (a) in the image is the original high-resolution image of the light spot. Figure 3 (b) in the image is a low-resolution image with few photons acquired by the optical detection system described above. Figure 3 (c) in the above-mentioned few-photon subpixel image processing model is the super-resolution image output.

[0117] Based on the above images, Figure 3 In (a) of the diagram, the centroid of the light spot is located at (-0.71, 0.75). This embodiment utilizes the centroid method directly based on... Figure 3 The centroid of the light spot obtained in (b) is (-0.34, 0.43). The maximum likelihood estimation method is directly based on... Figure 3 The centroid position of the light spot obtained in (b) is (-0.64, 0.74). Using the centroid method or maximum likelihood estimation method based on... Figure 3 The centroid positions of the light spots obtained in (c) are all approximately equal to (-0.704, 0.765). It can be seen that after processing the image using the few-photon subpixel image processing model provided in this embodiment, the resolution of the image is greatly improved, and the centroid positions of the obtained light spots are closer to the centroid positions of the light spots in the original image.

[0118] like Figure 4 The diagram shows a comparison of the root mean square error of the spot center position obtained by different methods under different noise photon numbers and cumulative pulse numbers provided in this embodiment. Figure 4 (a) in the figure is a schematic diagram comparing the root mean square error of the spot center position obtained by different methods under different noise photon numbers. Figure 4(b) in the figure is a schematic diagram comparing the root mean square error of the center position of the light spot obtained by different methods under different cumulative pulse numbers; the CNN super-resolution method in the figure is to process the image using the few photon sub-pixel image processing model provided in this embodiment, and then use the centroid method or the maximum likelihood estimation method to obtain the centroid position of the light spot based on the processed image.

[0119] from Figure 4 As can be seen, after processing the image using the model provided in this embodiment, the error of the centroid position of the obtained spot is much smaller than that of obtaining the centroid position of the spot in a low-resolution image with few photons by directly using the centroid method or the maximum likelihood estimation method. This shows that the low-resolution image processing method with few photons provided in this embodiment can effectively improve the resolution of low-resolution images with few photons.

[0120] Furthermore, to verify the effectiveness of the few-photon subpixel image processing model provided in this embodiment, this embodiment changed the number of convolution kernels in the above-mentioned few-photon subpixel image processing model, and provided comparison results of few-photon subpixel image processing models with different numbers of convolution kernels on the processing of a 32*32 low-resolution few-photon image, as shown in Table 1:

[0121] Table 1

[0122] Number of convolutional network layers 7 8 9 10 11 Extracting positional error 0.00152 0.000697 0.000261 0.000259 0.000264

[0123] As can be seen from the data in Table 1, when the resolution of the low-resolution image with few photons is 32*32, the few-photon subpixel image processing model obtained by reducing the number of convolution kernels cannot effectively improve the image resolution, resulting in a large error in the centroid position of the light spot obtained based on the processed image. However, after processing the image using the few-photon subpixel image processing model obtained by increasing the number of convolution kernels, the error in the centroid position of the obtained light spot is not significantly reduced. This indicates that the few-photon subpixel image processing model provided in this embodiment improves the image resolution while using a smaller number of convolution kernels, reducing the model parameters and taking into account the model's running speed.

[0124] The above embodiments demonstrate that, for low-resolution images with few photons at a resolution of 32×32, the few-photon subpixel image processing model provided in this embodiment can achieve a subpixel accuracy of 0.2 pixels. This model is used for super-resolution centroid extraction of a 4×4 pixel extremely weak Gaussian point, and performs even better under conditions of poor signal-to-noise ratio. Its performance is far superior to existing methods that directly use centroid determination methods to extract the centroid position of light spots. It can be applied to subpixel centroid extraction in long-distance single-photon deep space communication, which is beneficial to improving the positioning accuracy of beacon lights and increasing communication distance. For space-based systems, it can also effectively reduce the demand for laser energy resources by the payload.

[0125] Based on the few-photon low-resolution image processing method provided in the above embodiments, this application also provides a few-photon low-resolution image processing apparatus, such as... Figure 5 As shown, the device specifically includes:

[0126] Dataset construction module 10 is used to construct a low-resolution image dataset of Gaussian spots with few photons based on the Poisson detection model.

[0127] The model building module 20 is used to build a few-photon subpixel image processing model. The few-photon subpixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module, and a fully connected layer, which are connected in series along the forward propagation direction.

[0128] The loss function construction module 30 is used to input the low-resolution images of the low-photon low-resolution image dataset into the low-photon subpixel image processing model, and construct the image processing loss function based on the feature map output by the low-photon subpixel image processing model.

[0129] The model training and acquisition module 40 is used to train the few-photon subpixel image processing model using few-photon low-resolution images in the few-photon low-resolution image dataset until the value of the image processing loss function is minimized, thus obtaining the trained few-photon subpixel image processing model.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for processing low-resolution images with few photons, characterized in that, include: A low-resolution image dataset with few photons based on the Poisson detection model is constructed, specifically including: The signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system is obtained under different preset spot center positions, and it is expressed as follows: ， in, This indicates the intensity of the signal beam on the surface of the single-photon detector array in the optical detection system at the preset center position of the light spot. Indicates peak intensity; Indicates the half-height of the light spot; Indicates the preset center position of the light spot; This indicates any position of the light spot; Substituting the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system at different preset spot center positions into the Poisson detection model, we obtain the photon count probability detected by each single-photon detector in the single-photon detector array at different preset spot center positions, which is expressed as: ， ， in, This represents the probability of photon count detected by the m-th single-photon detector in the single-photon detector array at the preset spot center position. This represents the photon count value detected by the m-th single-photon detector; This represents the photosensitive area of ​​the m-th single-photon detector; This indicates the number of single-photon detectors in the single-photon detector array; Based on the photon count probability detected by each single photon detector in the single photon detector array at each preset spot center position, a low-resolution image with few photons is generated, and the preset spot center position is used as the actual centroid position of the spot in the low-resolution image with few photons. Based on all the generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained. A few-photon subpixel image processing model is constructed; the few-photon subpixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module, and a fully connected layer, which are sequentially connected in series along the forward propagation direction; The low-resolution images of few photons in the low-resolution image dataset are input into the low-resolution subpixel image processing model, and an image processing loss function is constructed based on the feature map output by the low-resolution subpixel image processing model. The few-photon subpixel image processing model is trained using few-photon low-resolution images from the few-photon low-resolution image dataset until the image processing loss function is minimized, thus obtaining the trained few-photon subpixel image processing model.

2. The low-resolution image processing method with few photons according to claim 1, characterized in that, The initial convolutional module, the first upsampling convolutional module, and the second upsampling convolutional module each include at least one convolutional kernel; The first depthwise convolutional module includes at least two convolutional kernels connected in series along the forward propagation direction; The second depthwise convolutional module includes at least five convolutional kernels connected in series along the forward propagation direction.

3. The low-resolution image processing method with few photons according to claim 1, characterized in that, The image processing loss function is constructed based on the feature map output by the few-photon sub-pixel image processing model, including: The predicted centroid position of the light spot in the feature map is obtained using the centroid method or the maximum likelihood estimation method; An image processing loss function is constructed based on the predicted centroid position of the light spot and the actual centroid position of the light spot in the low-resolution image with few photons corresponding to the feature map.

4. The low-resolution image processing method with few photons according to claim 3, characterized in that, The image processing loss function is expressed as: ， ， ， in, Represents the image processing loss function; This represents the error between the predicted centroid position of the light spot and the actual centroid position of the light spot; This represents the feature map output by the few-photon subpixel image processing model. express The Middle Line number The coordinates of the column pixels; This represents a low-resolution image with few photons. express The Middle Line number The coordinates of the column pixels; This indicates the number of rows of pixels in a low-resolution image / feature map with few photons. This indicates the number of columns of pixels in a low-resolution image / feature map with few photons. The root mean square of the error between the predicted centroid position and the actual centroid position of the light spot; Indicates the predicted centroid position of the light spot; This indicates the actual centroid location of the light spot.

5. The low-resolution image processing method with few photons according to claim 1, characterized in that, After obtaining the trained few-photon subpixel image processing model, the following is also included: Acquire low-resolution images with few photons, and input the low-resolution images with few photons into a trained low-photon subpixel image processing model to output feature maps; The centroid position of the light spot in the feature image is obtained using the centroid method or the maximum likelihood estimation method, and the centroid position of the light spot is sent to the control module of the optical detection system so that the control module can control the fast-reflecting mirror through the fast-reflecting mirror drive control module to realize the localization and tracking of the light spot in the low-resolution image with few photons.

6. The low-resolution image processing method with few photons according to claim 1, characterized in that, When the resolution of a low-resolution image with few photons is 32*32, The initial convolutional module includes a 5*5 convolutional kernel; The first depthwise convolutional module includes two 3*3 convolutional kernels connected in series along the forward propagation direction; The first upsampling convolution module includes a 5*5 convolution kernel; The second depthwise convolutional module includes five 3*3 convolutional kernels connected in series along the forward propagation direction; The second upsampling convolution module includes a 5*5 convolution kernel.

7. A low-resolution image processing device with few photons, characterized in that, include: The dataset construction module is used to construct a low-resolution image dataset of Gaussian spots with few photons based on the Poisson detection model. Specifically, it includes: The signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system is obtained under different preset spot center positions, and it is expressed as follows: ， in, This indicates the intensity of the signal beam on the surface of the single-photon detector array in the optical detection system at the preset center position of the light spot. Indicates peak intensity; Indicates the half-height of the light spot; Indicates the preset center position of the light spot; This indicates any position of the light spot; Substituting the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system at different preset spot center positions into the Poisson detection model, we obtain the photon count probability detected by each single-photon detector in the single-photon detector array at different preset spot center positions, which is expressed as: ， ， in, This represents the probability of photon count detected by the m-th single-photon detector in the single-photon detector array at the preset spot center position. This represents the photon count value detected by the m-th single-photon detector; This represents the photosensitive area of ​​the m-th single-photon detector; This indicates the number of single-photon detectors in the single-photon detector array; Based on the photon count probability detected by each single photon detector in the single photon detector array at each preset spot center position, a low-resolution image with few photons is generated, and the preset spot center position is used as the actual centroid position of the spot in the low-resolution image with few photons. Based on all the generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained. The model building module is used to construct a few-photon subpixel image processing model; the few-photon subpixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module, and a fully connected layer, which are sequentially connected in the forward propagation direction. The loss function construction module is used to input the low-resolution images of the low-photon low-resolution image dataset into the low-photon subpixel image processing model, and construct an image processing loss function based on the feature map output by the low-photon subpixel image processing model. The model training and acquisition module is used to train the few-photon subpixel image processing model using few-photon low-resolution images in the few-photon low-resolution image dataset until the value of the image processing loss function is minimized, thereby obtaining the trained few-photon subpixel image processing model.

8. An optical detection system, characterized in that, include: A telescope is used to collect light signals; A quick-reflecting mirror is used to adjust the light signal collected by the telescope; A single-photon detector array is used to receive optical signals adjusted by a fast mirror and convert the received optical signals into electrical signals. An analog-to-digital converter is used to receive electrical signals generated by a single-photon detector array and convert the electrical signals into digital signals. A data processing module is used to convert the digital signal generated by the analog-to-digital converter into a low-resolution image with few photons, and to process the low-resolution image with few photons using the low-resolution image processing method according to any one of claims 1-6, to obtain the centroid position of the light spot in the processed low-resolution image, and to send the centroid position of the light spot to the control module. A control module is used to generate control commands based on the centroid position of the light spot and send them to the fast-reflecting mirror drive control module. The fast-reflecting mirror drive control module is used to control the angle and displacement of the fast-reflecting mirror based on the control commands, thereby realizing the positioning and tracking of the light spot in the low-resolution image with few photons.

Images