Less-photon low-resolution image processing method and device and optical detection system

Abstract

The invention belongs to the technical field of photon imaging, and relates to a few-photon low-resolution image processing method and device and an optical detection system. Constructing a few-photon low-resolution image data set of Gaussian light spots according to a Poisson detection model; constructing a few-photon sub-pixel image processing model, wherein the model comprises an initial convolution module, a first deep convolution module, a first up-sampling convolution module, a second deep convolution module, a second up-sampling convolution module and a full connection layer which are sequentially connected in series along the forward propagation direction; inputting the low-photon low-resolution image into at least a photon sub-pixel image processing model, and constructing an image processing loss function based on a feature map output by the low-photon sub-pixel image processing model; and training the few-photon sub-pixel image processing model by using the few-photon low-resolution image data set until the value of the image processing loss function is minimum, and obtaining the trained few-photon sub-pixel image processing model, thereby obtaining high-precision centroid estimation based on a high-resolution image output by the model.

Description

Technical Field

[0001] The present invention relates to the field of photon imaging technology, and in particular to a method and device for processing a few-photon low-resolution image, and an optical detection system. Background Art

[0002] In recent years, with the increasing frequency of human activities in aerospace and space exploration, optical-based space target detection and tracking technology has been vigorously developed in laser communication and remote sensing detection applications. The space targets involved include both cooperative targets such as communication terminals and known celestial bodies, as well as non-cooperative targets such as space debris and unknown celestial bodies. Optical-based space target detection and tracking refers to the use of optical detection systems to collect images, obtain the position information of light spots in the image, and return the position information to the optical detection system to achieve target positioning and tracking. Therefore, how to accurately obtain the position information of light spots in the image is crucial to the positioning and tracking accuracy of the target.

[0003] Existing space target detection and tracking usually use the centroid method and maximum likelihood estimation method to obtain the position information of the light spot in the image. The centroid method first binarizes the collected image to highlight the light spot area, and then uses the edge detection algorithm to further identify the edge of the light spot. Finally, the pixels in the light spot area are identified and extracted, and the centroid position of the light spot is calculated according to the brightness and position of all pixels in the light spot area. The centroid position is returned to the optical detection system as the position information of the light spot in the image; the maximum likelihood estimation method sets the light spot brightness distribution model, constructs a likelihood function for describing the center position and intensity of the light spot based on the collected image and the set light spot brightness distribution model, and uses a numerical optimization method to find the parameters that maximize the likelihood function, thereby extracting the position information of the light spot based on the optimized parameters, and returning the position information to the optical detection system.

[0004] Although both the centroid method and the maximum likelihood estimation method can obtain the position information of the light spot based on the image, as the distance of laser communication becomes longer and longer, the target of remote sensing detection becomes smaller and smaller, and the space target presents obvious dark and weak characteristics. In this case, the image collected by the optical detection system is a low-photon low-resolution image. The resolution of this type of image is low and the edge of the light spot is blurred. When the centroid method is used directly to calculate the centroid position of the low-photon low-resolution image, the light spot area cannot be accurately identified, resulting in a large error in the calculated centroid position; and when the maximum likelihood estimation method is used to obtain the light spot position information of the low-photon low-resolution image, the actual distribution of the blurred light spot cannot be accurately described, resulting in an inaccurate set light spot brightness distribution model, which makes the acquired light spot position information less accurate, affecting the accuracy of target positioning and tracking.

[0005] Based on the above reasons, when obtaining the spot position of a few-photon low-resolution image, the few-photon low-resolution image needs to be processed to improve the resolution of the image, thereby improving the accuracy of the spot position and the accuracy of target positioning and tracking. However, due to the extremely low signal-to-noise ratio of the few-photon low-resolution image, the effective information in the image is extremely limited, and the image processing model in the prior art often requires a good signal-to-noise ratio and sufficient pixel information to effectively improve the resolution of the image; in addition, since it is difficult to label the few-photon low-resolution image, it is impossible to obtain sufficient and high-precision few-photon low-resolution image data sets in the prior art, which also results in the image processing model being unable to obtain sufficient data for training, thereby being unable to effectively improve the resolution of the few-photon low-resolution image, and being unable to solve the problem of low target positioning and tracking accuracy caused by inaccurate acquired spot position information.

[0006] In summary, the existing image processing methods cannot effectively improve the resolution of low-photon low-resolution images, resulting in inaccurate spot position information, which cannot meet the technical requirements of high-precision positioning and tracking of spatial point targets. Summary of the invention

[0007] To this end, the technical problem to be solved by the present invention is to overcome the problem that the image processing method in the prior art cannot effectively improve the resolution of the low-photon low-resolution image, resulting in inaccurate spot position information obtained, thus failing to meet the technical requirements of high-precision positioning and tracking of spatial point targets.

[0008] In order to solve the above technical problems, the present invention provides a method for processing a few-photon low-resolution image, comprising:

[0009] Construct a few-photon low-resolution image dataset of Gaussian spot based on Poisson detection model;

[0010] Constructing a few-photon sub-pixel image processing model; the few-photon sub-pixel image processing model comprises an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module and a fully connected layer which are sequentially connected in series along a forward propagation direction;

[0011] Inputting the few-photon low-resolution image in the few-photon low-resolution image data set into the few-photon sub-pixel image processing model, and constructing an image processing loss function based on the feature map output by the few-photon sub-pixel image processing model;

[0012] The few-photon low-resolution images in the few-photon low-resolution image data set are used to train the few-photon sub-pixel image processing model until the value of the image processing loss function is minimized, thereby obtaining a trained few-photon sub-pixel image processing model.

[0013] Preferably, constructing a few-photon low-resolution image data set of Gaussian spot according to the Poisson detection model includes:

[0014] Obtain the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under different preset spot center positions;

[0015] Substituting the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under the different preset light spot center positions into the Poisson detection model, respectively, to obtain the photon counting probability detected by each single-photon detector in the single-photon detector array under the different preset light spot center positions;

[0016] Based on the photon counting probability detected by each single-photon detector in the single-photon detector array at each preset light spot center position, a few-photon low-resolution image is generated, and the preset light spot center position is used as the actual centroid position of the light spot of the few-photon low-resolution image;

[0017] Based on all generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained.

[0018] Preferably, under different preset light spot center positions, the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system is expressed as:

[0019]

[0020] Among them, λ s (x, y) represents the signal beam intensity on the surface of the single-photon detector array in the optical detection system at the preset center position of the light spot; I0 ​​represents the peak intensity; ρ represents the half height of the light spot; (x0, y0) represents the preset center position of the light spot; (x, y) represents any position of the light spot;

[0021] Under different preset spot center positions, the probability of photon counting detected by each single-photon detector in the single-photon detector array is expressed as:

[0022]

[0023] Among them, P({Z m =z m}) represents the probability of photon counting detected by the mth single-photon detector in the single-photon detector array at the preset spot center position; z m A represents the photon count value detected by the mth single-photon detector; m represents the pixel sensitive area of ​​the mth single-photon detector; M represents the number of single-photon detectors in the single-photon detector array.

[0024] Preferably, the initial convolution module, the first upsampling convolution module and the second upsampling convolution module each include at least one convolution kernel;

[0025] The first depth convolution module includes at least two convolution kernels connected in series in a forward propagation direction;

[0026] The second deep convolution module includes at least five convolution kernels connected in series in a forward propagation direction.

[0027] Preferably, constructing an image processing loss function based on the feature map output by the few-photon sub-pixel image processing model comprises:

[0028] Obtaining the predicted centroid position of the light spot in the feature map by using the centroid method or the maximum likelihood estimation method;

[0029] An image processing loss function is constructed based on the predicted centroid position of the light spot and the actual centroid position of the light spot in the few-photon low-resolution image corresponding to the feature map.

[0030] Preferably, the image processing loss function is expressed as:

[0031] Loss=CLoss(pred,targ)+0.1*MSELoss(pred,targ)-(pred-0.5) 2 ,

[0032]

[0033]

[0034] Among them, Loss represents the image processing loss function; CLoss(pred,targ) represents the error between the predicted centroid position of the light spot and the actual centroid position of the light spot; pred represents the feature map output by the few-photon sub-pixel image processing model; (px_i,py_j) represents the coordinates of the pixel point in the i-th row and j-th column in pred; targ represents the few-photon low-resolution image; (tx_i,ty_j) represents the coordinates of the pixel point in the i-th row and j-th column in targ; N represents the number of rows of pixels in the few-photon low-resolution image / feature map; J represents the number of columns of pixels in the few-photon low-resolution image / feature map; MSELoss(pred,targ) represents the root mean square error between the predicted centroid position of the light spot and the actual centroid position of the light spot; (Cp_x,Cp_y) represents the predicted centroid position of the light spot; (Ct_x,Ct_y) represents the actual centroid position of the light spot.

[0035] Preferably, after obtaining the trained few-photon sub-pixel image processing model, the method further includes:

[0036] Collecting a few-photon low-resolution image, and inputting the few-photon low-resolution image into a trained few-photon sub-pixel image processing model, and outputting a feature map;

[0037] The centroid position of the light spot in the feature image is obtained by using the centroid method or the maximum likelihood estimation method, and the centroid position of the light spot is sent to the control module in the optical detection system, so that the control module controls the fast mirror through the fast mirror driving control module to achieve positioning and tracking of the light spot in the low-photon low-resolution image.

[0038] Preferably, when the resolution of the few-photon low-resolution image is 32*32,

[0039] The initial convolution module includes a convolution kernel with a size of 5*5;

[0040] The first depth convolution module includes two convolution kernels of size 3*3 which are connected in series in the forward propagation direction;

[0041] The first upsampling convolution module includes a convolution kernel with a size of 5*5;

[0042] The second depth convolution module includes five convolution kernels of size 3*3 which are serially connected in sequence along the forward propagation direction;

[0043] The second upsampling convolution module includes a convolution kernel with a size of 5*5.

[0044] The present invention also provides a low-photon low-resolution image processing device, comprising:

[0045] A dataset construction module is used to construct a few-photon low-resolution image dataset of Gaussian spot based on a Poisson detection model;

[0046] A model building module, used to build a few-photon sub-pixel image processing model; the few-photon sub-pixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module and a fully connected layer which are sequentially connected in series along a forward propagation direction;

[0047] A loss function construction module, used for inputting the few-photon low-resolution image in the few-photon low-resolution image data set into the few-photon sub-pixel image processing model, and constructing an image processing loss function based on a feature map output by the few-photon sub-pixel image processing model;

[0048] The model training and acquisition module is used to train the few-photon sub-pixel image processing model using the few-photon low-resolution images in the few-photon low-resolution image data set until the value of the image processing loss function is minimized, thereby obtaining a trained few-photon sub-pixel image processing model.

[0049] The present invention also provides an optical detection system, comprising:

[0050] A telescope, used to collect optical signals;

[0051] A fast-reflecting mirror, used to adjust the light signal collected by the telescope;

[0052] A single-photon detector array is used to receive the optical signal adjusted by the fast mirror and convert the received optical signal into an electrical signal;

[0053] an analog-to-digital converter, used for receiving the electrical signal generated by the single-photon detector array and converting the electrical signal into a digital signal;

[0054] A data processing module, used for converting the digital signal generated by the analog-to-digital converter into a low-photon low-resolution image, and processing the low-photon low-resolution image using the above-mentioned low-photon low-resolution image processing method, obtaining the centroid position of the light spot of the processed low-photon low-resolution image, and sending the centroid position of the light spot to the control module;

[0055] A control module, used for generating a control instruction based on the centroid position of the light spot and sending the control instruction to a fast mirror drive control module;

[0056] The fast-reflection mirror driving control module is used to control the angle and displacement of the fast-reflection mirror based on the control instruction, so as to realize the positioning and tracking of the light spot in the few-photon low-resolution image.

[0057] The few-photon low-resolution image processing method provided in the present application takes into account that under low-light conditions, the number of photons reaching the detector often obeys the Poisson distribution. The use of the Poisson model to simulate the imaging process under few-photon conditions can reflect the uncertainty of photon counting in actual observations. Therefore, the present application constructs a few-photon low-resolution image dataset of Gaussian spot based on the Poisson detection model to train the few-photon sub-pixel image processing model; then constructs the few-photon sub-pixel image processing model, which includes an initial convolution module, a first deep convolution module, a first upsampling convolution module, a second deep convolution module, a second upsampling convolution module and a fully connected layer connected in series along the forward propagation direction. The initial convolution module is first used to extract potential effective information from the few-photon low-resolution image, and then the first deep convolution module is used to extract low-level features of the image, and the first upsampling convolution module is used to extract low-level features of the image. The upsampling convolution module improves the spatial resolution of the image. After that, the second deep convolution module can not only extract the deep features of the image, but also filter out the unnecessary noise generated in the upsampling process, retain the main features of the signal, and use the second upsampling convolution module to further improve the spatial resolution of the image. Finally, the fully connected layer is used to integrate the extracted features. It can filter the noise in the image as much as possible when the signal-to-noise ratio is extremely low and the effective information is limited, extract the effective features, and realize the reconstruction of the high-resolution image; by training the few-photon sub-pixel image processing model until the value of the image processing loss function is minimized, the trained few-photon sub-pixel image processing model is obtained, which can effectively improve the resolution of the few-photon low-resolution image, and then obtain more accurate spot position information, and realize high-precision positioning and tracking of spatial point targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0059] Figure 1 Flow chart of the low-photon low-resolution image processing method provided in this application;

[0060] Figure 2 A schematic diagram of the structure of an optical detection system and a few-photon sub-pixel image processing model provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of the image processing results provided in the embodiment of the present application; wherein, Figure 3 (a) is the original high-resolution image of the light spot. Figure 3 (b) in Figure 2 The optical detection system shown here collects a few-photon, low-resolution image. Figure 3 (c) in Figure 2 The super-resolution image output by the few-photon sub-pixel image processing model shown;

[0062] Figure 4 This is a schematic diagram comparing the root mean square error of the spot center position obtained using different methods under different noise photon numbers and cumulative pulse numbers provided in the embodiment of the present application; wherein, Figure 4 (a) is a schematic diagram showing the comparison of the root mean square error of the spot center position obtained using different methods under different noise photon numbers. Figure 4 (b) is a schematic diagram showing the comparison of the root mean square error of the spot center position obtained using different methods under different cumulative pulse numbers;

[0063] Figure 5 This is a schematic diagram of the structure of the low-photon low-resolution image processing device provided in this application. DETAILED DESCRIPTION

[0064] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0065] See also Figure 1 , Figure 1 The present invention provides a flowchart of a method for processing a few-photon low-resolution image, which specifically includes:

[0066] S10: Construct a few-photon low-resolution image dataset of Gaussian spot based on the Poisson detection model;

[0067] S20: construct a few-photon sub-pixel image processing model; the few-photon sub-pixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module and a fully connected layer which are sequentially connected in series along a forward propagation direction;

[0068] S30: inputting the few-photon sub-pixel image in the few-photon low-resolution image dataset into the few-photon low-resolution image processing model, and constructing an image processing loss function based on the feature map output by the few-photon sub-pixel image processing model;

[0069] S40: The few-photon low-resolution images in the few-photon low-resolution image dataset are used to train the few-photon sub-pixel image processing model until the value of the image processing loss function is minimized, thereby obtaining a trained few-photon sub-pixel image processing model.

[0070] The few-photon low-resolution image processing method provided in the present application takes into account that under low-light conditions, the number of photons reaching the detector often obeys the Poisson distribution. The use of the Poisson model to simulate the imaging process under few-photon conditions can reflect the uncertainty of photon counting in actual observations. Therefore, the present application constructs a few-photon low-resolution image dataset of Gaussian spot based on the Poisson detection model to train the few-photon sub-pixel image processing model; then constructs the few-photon sub-pixel image processing model, which includes an initial convolution module, a first deep convolution module, a first upsampling convolution module, a second deep convolution module, a second upsampling convolution module and a fully connected layer connected in series along the forward propagation direction. The initial convolution module is first used to extract potential effective information from the few-photon low-resolution image, and then the first deep convolution module is used to extract low-level features of the image, and the first upsampling convolution module is used to extract low-level features of the image. The upsampling convolution module improves the spatial resolution of the image. After that, the second deep convolution module can not only extract the deep features of the image, but also filter out the unnecessary noise generated in the upsampling process, retain the main features of the signal, and use the second upsampling convolution module to further improve the spatial resolution of the image. Finally, the fully connected layer is used to integrate the extracted features. It can filter the noise in the image as much as possible when the signal-to-noise ratio is extremely low and the effective information is limited, extract the effective features, and realize the reconstruction of the high-resolution image; by training the few-photon sub-pixel image processing model until the value of the image processing loss function is minimized, the trained few-photon sub-pixel image processing model is obtained, which can effectively improve the resolution of the few-photon low-resolution image, and then obtain more accurate spot position information, and realize high-precision positioning and tracking of spatial point targets.

[0071] Specifically, when constructing a few-photon low-resolution image dataset, it is assumed that the intensity distribution of the signal beam incident on the surface of the single-photon detector array satisfies the Gaussian distribution, so that the intensity distribution of the signal beam on the surface of the single-photon detector array in the optical detection system under different preset spot center positions can be obtained;

[0072] Specifically, under different preset spot center positions, the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system is expressed as:

[0073]

[0074] Among them, λ s (x, y) represents the signal beam intensity on the surface of the single-photon detector array in the optical detection system at the preset center position of the light spot; I0 ​​represents the peak intensity; ρ represents the half height of the light spot; (x0, y0) represents the preset center position of the light spot; (x, y) represents any position of the light spot;

[0075] Furthermore, since the response of the single-photon detector is a Poisson process, the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system at different preset light spot center positions is substituted into the Poisson detection model, and the probability of photon counting detected by each single-photon detector in the single-photon detector array at different preset light spot center positions can be obtained;

[0076] Specifically, under different preset spot center positions, the probability of photon counting detected by each single-photon detector in the single-photon detector array is expressed as:

[0077]

[0078] Among them, P({Z m =z m}) represents the probability of photon counting detected by the mth single-photon detector in the single-photon detector array at the preset spot center position; z m A represents the photon count value detected by the mth single-photon detector; m represents the pixel photosensitive area of ​​the mth single-photon detector; M represents the number of single-photon detectors in the single-photon detector array;

[0079] Based on the photon counting probability detected by each single-photon detector in the single-photon detector array at each preset light spot center position, a few-photon low-resolution image is generated, and the preset light spot center position is used as the actual centroid position of the light spot of the few-photon low-resolution image;

[0080] Based on all generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained.

[0081] The structure of each module in the few-photon sub-pixel image processing model will affect the image processing effect. In some embodiments of the present application, the initial convolution module, the first upsampling convolution module, and the second upsampling convolution module each include at least one convolution kernel;

[0082] The first depth convolution module includes at least two convolution kernels connected in series in the forward propagation direction;

[0083] The second depth convolution module includes at least five convolution kernels connected in series in the forward propagation direction.

[0084] Specifically, the convolution kernel in the initial convolution module convolves the few-photon low-resolution image to extract the basic features of the few-photon low-resolution image to obtain a first feature map; the first deep convolution module uses at least two convolution kernels to gradually extract the deep features of the first feature map to obtain a second feature map, and the convolution kernels are connected in series to increase the nonlinear expression ability of the model; the convolution kernel in the first upsampling convolution module deconvolves the second feature map, expands the second feature map to retain richer contextual information, and outputs a high-resolution third feature map; after obtaining the third feature map, the multiple convolution kernels in the second deep convolution module are used again to extract its deep features, and a fourth feature map is output. The multiple convolution kernels are still connected in series to increase the nonlinear expression ability of the model; finally, the convolution kernel in the second upsampling convolution module is used again to deconvolve the fourth feature map, expand the fourth feature map again, and output a fifth feature map with higher resolution.

[0085] In a specific example, when the resolution of the few-photon low-resolution image is 32*32, the initial convolution module includes a convolution kernel of size 5*5; the first deep convolution module includes two convolution kernels of size 3*3 connected in series along the forward propagation direction; the first upsampling convolution module includes a convolution kernel of size 5*5; the second deep convolution module includes five convolution kernels of size 3*3 connected in series along the forward propagation direction; the second upsampling convolution module includes a convolution kernel of size 5*5, and the resolution of the feature map output by the few-photon sub-pixel image processing model is 160*160.

[0086] Furthermore, in step S30, constructing an image processing loss function based on the feature map output by the few-photon sub-pixel image processing model includes:

[0087] The predicted centroid position of the light spot in the feature map is obtained by using the centroid method or the maximum likelihood estimation method;

[0088] The image processing loss function is constructed based on the predicted centroid position of the light spot and the actual centroid position of the light spot in the few-photon low-resolution image corresponding to the feature map.

[0089] Specifically, in some embodiments of the present application, the constructed image processing loss function is expressed as:

[0090] Loss=CLoss(pred,targ)+0.1*MSELoss(pred,targ)-(pred-0.5) 2 ,

[0091]

[0092]

[0093] Among them, Loss represents the image processing loss function; CLoss(pred,targ) represents the error between the predicted centroid position of the light spot and the actual centroid position of the light spot; pred represents the feature map output by the few-photon sub-pixel image processing model; (px_i,py_j) represents the coordinates of the pixel point in the i-th row and j-th column in pred; targ represents the few-photon low-resolution image; (tx_i,ty_j) represents the coordinates of the pixel point in the i-th row and j-th column in targ; N represents the number of rows of pixels in the few-photon low-resolution image / feature map; J represents the number of columns of pixels in the few-photon low-resolution image / feature map; MSELoss(pred,targ) represents the root mean square error between the predicted centroid position of the light spot and the actual centroid position of the light spot; (Cp_x,Cp_y) represents the predicted centroid position of the light spot; (Ct_x,Ct_y) represents the actual centroid position of the light spot.

[0094] Specifically, the calculation formula of the center of mass position is expressed as:

[0095]

[0096]

[0097] Furthermore, after obtaining the trained few-photon sub-pixel image processing model, the collected few-photon low-resolution images can be input into the model for processing to improve the resolution of the few-photon low-resolution images, thereby locating and tracking the targets in the images.

[0098] Specifically, after obtaining the trained few-photon sub-pixel image processing model, the following steps are also included:

[0099] Collect a few-photon low-resolution image, input the few-photon low-resolution image into a trained few-photon sub-pixel image processing model, and output a feature map;

[0100] The centroid position of the light spot in the feature image is obtained by using the centroid method or the maximum likelihood estimation method, and the centroid position of the light spot is sent to the control module in the optical detection system, so that the control module controls the fast mirror through the fast mirror driving control module to achieve positioning and tracking of the light spot in the low-photon low-resolution image.

[0101] Based on the few-photon sub-pixel image processing method provided in the above embodiment, the embodiment of the present application further provides an optical detection system, including:

[0102] A telescope, used to collect optical signals;

[0103] A fast-reflecting mirror, used to adjust the light signal collected by the telescope;

[0104] A single-photon detector array is used to receive the optical signal adjusted by the fast mirror and convert the received optical signal into an electrical signal;

[0105] An analog-to-digital converter, used for receiving the electrical signal generated by the single-photon detector array and converting the electrical signal into a digital signal;

[0106] A data processing module is used to convert the digital signal generated by the analog-to-digital converter into a low-photon low-resolution image, and use the above-mentioned low-photon sub-pixel image processing method to process the low-photon low-resolution image, obtain the centroid position of the light spot of the processed low-photon low-resolution image, and send the centroid position of the light spot to the control module;

[0107] A control module, used for generating a control instruction based on the center of mass position of the light spot and sending the control instruction to the fast reflection mirror drive control module;

[0108] The fast-reflection mirror drive control module is used to control the angle and displacement of the fast-reflection mirror based on control instructions, thereby achieving positioning and tracking of the light spot in the low-photon low-resolution image.

[0109] The effectiveness of the few-photon low-resolution image processing method provided in the embodiment of the present application is verified by a specific example below. Figure 2 The figure shows an optical detection system and a few-photon sub-pixel image processing model provided in this embodiment. The left side of the figure is a schematic diagram of the structure of the optical detection system, and the right side is a schematic diagram of the structure of the few-photon sub-pixel image processing model, which is deployed on a data processing module in the optical detection system;

[0110] In the few-photon sub-pixel image processing model of this embodiment, the initial convolution module includes a convolution kernel with a size of 5*5, and performs a convolution operation on the few-photon low-resolution image with a resolution of 32*32 to generate a 64-dimensional first feature map;

[0111] The first depth convolution module includes two convolution kernels of size 3*3 connected in series. The first convolution kernel convolves the 64-dimensional first feature map to generate a new 64-dimensional first feature map, and the second convolution kernel convolves the new first feature map to generate a 25-dimensional second feature map.

[0112] The first upsampling convolution module includes a convolution kernel of size 5*5, upsamples the second feature map, and outputs a third feature map of size 161*161*1;

[0113] The second depth convolution module includes five convolution kernels of size 3*3 connected in series. The first convolution kernel convolves the third feature map to generate a 32-dimensional feature map. The second convolution kernel convolves the 32-dimensional feature map to generate a 64-dimensional feature map. The third convolution kernel convolves the 64-dimensional feature map to generate a new 64-dimensional feature map. The fourth convolution kernel convolves the new 64-dimensional feature map to generate a 128-dimensional feature map. The fifth convolution kernel convolves the 128-dimensional feature map to generate a 128-dimensional fourth feature map.

[0114] The second upsampling convolution module includes a convolution kernel with a size of 5*5, upsamples the fourth feature map, and outputs a fifth feature map with a size of 160*160*1;

[0115] The fully connected layer reshapes the fifth feature map and outputs a feature map with a resolution of 160*160.

[0116] like Figure 3 FIG. 1 is a schematic diagram of the image processing result provided by this embodiment; wherein, Figure 3 (a) is the original high-resolution image of the light spot. Figure 3 (b) is a low-resolution image with few photons collected by the above optical detection system. Figure 3 (c) in FIG. 1 is a super-resolution image output by the above-mentioned few-photon sub-pixel image processing model.

[0117] Based on the above image, Figure 3 The centroid position of the light spot in (a) is (-0.71, 0.75). This embodiment uses the centroid method directly based on Figure 3 The centroid position of the light spot obtained in (b) is (-0.34, 0.43), and the maximum likelihood estimation method is directly based on Figure 3 The centroid position of the light spot obtained in (b) is (-0.64, 0.74). The centroid method or maximum likelihood estimation method is used based on Figure 3 The centroid positions of the light spots obtained in (c) are approximately equal to (-0.704, 0.765). It can be seen that after the image is processed using the few-photon sub-pixel image processing model provided in this embodiment, the resolution of the image is greatly improved, so that the centroid position of the light spot obtained is closer to the centroid position of the light spot in the original image.

[0118] like Figure 4 The figure is a schematic diagram showing a comparison of the root mean square errors of the center positions of the light spots obtained using different methods under different noise photon numbers and cumulative pulse numbers provided in this embodiment; Figure 4 (a) is a schematic diagram showing the comparison of the root mean square error of the spot center position obtained using different methods under different noise photon numbers. Figure 4(b) is a schematic diagram of the comparison of the root mean square error of the center position of the light spot obtained by different methods under different cumulative pulse numbers; the CNN super-resolution method in the figure uses the few-photon sub-pixel image processing model provided in this embodiment to process the image, and then uses the centroid method or the maximum likelihood estimation method to obtain the centroid position of the light spot based on the processed image;

[0119] from Figure 4 It can be seen that after the image is processed using the model provided in this embodiment, the error of the center of mass position of the obtained light spot is much smaller than the center of mass position of the light spot obtained by directly using the center of mass method or the maximum likelihood estimation method to obtain the few-photon low-resolution image, indicating that the few-photon low-resolution image processing method provided in this embodiment can effectively improve the resolution of the few-photon low-resolution image.

[0120] Furthermore, in order to verify the effectiveness of the few-photon sub-pixel image processing model provided in this embodiment, this embodiment changes the number of convolution kernels of the above-mentioned few-photon sub-pixel image processing model, and provides a comparison result of the few-photon sub-pixel image processing models with different numbers of convolution kernels for processing a few-photon low-resolution image with a resolution of 32*32, as shown in Table 1:

[0121] Table 1

[0122] Convolutional network layers 7 8 9 10 11 Extracting position error 0.00152 0.000697 0.000261 0.000259 0.000264

[0123] It can be seen from the data in Table 1 that when the resolution of the few-photon low-resolution image is 32*32, the few-photon sub-pixel image processing model obtained by reducing the number of convolution kernels cannot effectively improve the resolution of the image, resulting in a large error in the center of mass position of the light spot obtained based on the processed image. After the image is processed by the few-photon sub-pixel image processing model obtained by increasing the number of convolution kernels, the error in the center of mass position of the obtained light spot is not significantly reduced, indicating that the few-photon sub-pixel image processing model provided in this embodiment improves the resolution of the image while using a smaller number of convolution kernels, reducing the parameters of the model and taking into account the running speed of the model.

[0124] The above embodiments show that for a 32×32 resolution low-photon low-resolution image, the low-photon sub-pixel image processing model provided in this embodiment can achieve a sub-pixel accuracy of 0.2 pixels, which is used for super-resolution centroid extraction of an extremely weak Gaussian point of 4×4 pixels in size, especially under conditions of poor signal-to-noise ratio. Its performance is much better than the existing method of directly using the centroid method to extract the centroid position of the light spot, and can be used for sub-pixel centroid extraction in long-distance single-photon deep space communication, which is beneficial to improving the positioning accuracy of beacon light and increasing the communication distance. For space-based systems, it can also effectively reduce the payload's demand for laser energy resources.

[0125] Based on the few-photon low-resolution image processing method provided in the above embodiment, the present application embodiment also provides a few-photon low-resolution image processing device, such as Figure 5 As shown, the device specifically includes:

[0126] A data set construction module 10 is used to construct a low-photon low-resolution image data set of a Gaussian spot according to a Poisson detection model;

[0127] A model building module 20 is used to build a few-photon sub-pixel image processing model; the few-photon sub-pixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module and a fully connected layer which are sequentially connected in series along a forward propagation direction;

[0128] A loss function construction module 30 is used to input the few-photon low-resolution image in the few-photon low-resolution image data set into the at least one photon sub-pixel image processing model, and construct an image processing loss function based on a feature map output by the few-photon sub-pixel image processing model;

[0129] The model training and acquisition module 40 is used to train the few-photon sub-pixel image processing model using the few-photon low-resolution images in the few-photon low-resolution image data set until the value of the image processing loss function is minimized, thereby obtaining a trained few-photon sub-pixel image processing model.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A method for processing a few-photon low-resolution image, characterized in that: include: Construct a few-photon low-resolution image dataset of Gaussian spot based on Poisson detection model; Constructing a few-photon sub-pixel image processing model; the few-photon sub-pixel image processing model comprises an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module and a fully connected layer which are sequentially connected in series along a forward propagation direction; Inputting the few-photon low-resolution image in the few-photon low-resolution image data set into the few-photon sub-pixel image processing model, and constructing an image processing loss function based on the feature map output by the few-photon sub-pixel image processing model; The few-photon low-resolution images in the few-photon low-resolution image data set are used to train the few-photon sub-pixel image processing model until the value of the image processing loss function is minimized, thereby obtaining a trained few-photon sub-pixel image processing model.

2. The method for processing a few-photon low-resolution image according to claim 1, characterized in that: The few-photon low-resolution image datasets constructed based on the Poisson detection model for Gaussian spots include: Obtain the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under different preset spot center positions; Substituting the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system under the different preset light spot center positions into the Poisson detection model, respectively, to obtain the photon counting probability detected by each single-photon detector in the single-photon detector array under the different preset light spot center positions; Based on the photon counting probability detected by each single-photon detector in the single-photon detector array at each preset light spot center position, a few-photon low-resolution image is generated, and the preset light spot center position is used as the actual centroid position of the light spot of the few-photon low-resolution image; Based on all generated few-photon low-resolution images and the actual centroid positions of their spots, a few-photon low-resolution image dataset of Gaussian spots is obtained.

3. The method for processing a few-photon low-resolution image according to claim 2, characterized in that: Under different preset spot center positions, the signal beam intensity distribution on the surface of the single-photon detector array in the optical detection system is expressed as: Among them, λ s (x, y) represents the signal beam intensity on the surface of the single-photon detector array in the optical detection system at the preset center position of the light spot; I0 ​​represents the peak intensity; ρ represents the half height of the light spot; (x0, y0) represents the preset center position of the light spot; (x, y) represents any position of the light spot; Under different preset spot center positions, the probability of photon counting detected by each single-photon detector in the single-photon detector array is expressed as: Among them, P({Z m =z m }) represents the probability of photon counting detected by the mth single-photon detector in the single-photon detector array at the preset spot center position; z m A represents the photon count value detected by the mth single-photon detector; m represents the pixel sensitive area of ​​the mth single-photon detector; M represents the number of single-photon detectors in the single-photon detector array.

4. The method for processing a few-photon low-resolution image according to claim 1, characterized in that: The initial convolution module, the first upsampling convolution module and the second upsampling convolution module each include at least one convolution kernel; The first depth convolution module includes at least two convolution kernels connected in series in a forward propagation direction; The second deep convolution module includes at least five convolution kernels connected in series in a forward propagation direction.

5. The method for processing a few-photon low-resolution image according to claim 1, characterized in that: Constructing an image processing loss function based on the feature map output by the few-photon sub-pixel image processing model includes: Obtaining the predicted centroid position of the light spot in the feature map by using the centroid method or the maximum likelihood estimation method; An image processing loss function is constructed based on the predicted centroid position of the light spot and the actual centroid position of the light spot in the few-photon low-resolution image corresponding to the feature map.

6. The method for processing a few-photon low-resolution image according to claim 5, characterized in that: The image processing loss function is expressed as: Loss=CLoss(pred,targ)+0.1*MSELoss(pred,targ)-(pred-0.5) 2 , Among them, Loss represents the image processing loss function; CLoss(pred,targ) represents the error between the predicted centroid position of the light spot and the actual centroid position of the light spot; pred represents the feature map output by the few-photon sub-pixel image processing model; (px_i,py_j) represents the coordinates of the pixel point in the i-th row and j-th column in pred; targ represents the few-photon low-resolution image; (tx_i,ty_j) represents the coordinates of the pixel point in the i-th row and j-th column in targ; N represents the number of rows of pixels in the few-photon low-resolution image / feature map; J represents the number of columns of pixels in the few-photon low-resolution image / feature map; MSELoss(pred,targ) represents the root mean square error between the predicted centroid position of the light spot and the actual centroid position of the light spot; (Cp_x,Cp_y) represents the predicted centroid position of the light spot; (Ct_x,Ct_y) represents the actual centroid position of the light spot.

7. The method for processing a few-photon low-resolution image according to claim 1, characterized in that: After obtaining the trained few-photon sub-pixel image processing model, it also includes: Collecting a few-photon low-resolution image, and inputting the few-photon low-resolution image into a trained few-photon sub-pixel image processing model, and outputting a feature map; The centroid position of the light spot in the feature image is obtained by using the centroid method or the maximum likelihood estimation method, and the centroid position of the light spot is sent to the control module in the optical detection system, so that the control module controls the fast mirror through the fast mirror driving control module to achieve positioning and tracking of the light spot in the low-photon low-resolution image.

8. The method for processing a few-photon low-resolution image according to claim 1, characterized in that: When the resolution of the few-photon low-resolution image is 32*32, The initial convolution module includes a convolution kernel with a size of 5*5; The first depth convolution module includes two convolution kernels of size 3*3 which are connected in series in the forward propagation direction; The first upsampling convolution module includes a convolution kernel with a size of 5*5; The second depth convolution module includes five convolution kernels of size 3*3 which are serially connected in sequence along the forward propagation direction; The second upsampling convolution module includes a convolution kernel with a size of 5*5.

9. A low-photon low-resolution image processing device, characterized in that: include: A dataset construction module is used to construct a few-photon low-resolution image dataset of Gaussian spot based on a Poisson detection model; A model building module, used to build a few-photon sub-pixel image processing model; the few-photon sub-pixel image processing model includes an initial convolution module, a first depth convolution module, a first upsampling convolution module, a second depth convolution module, a second upsampling convolution module and a fully connected layer which are sequentially connected in series along a forward propagation direction; A loss function construction module, used for inputting the few-photon low-resolution image in the few-photon low-resolution image data set into the few-photon sub-pixel image processing model, and constructing an image processing loss function based on a feature map output by the few-photon sub-pixel image processing model; The model training and acquisition module is used to train the few-photon sub-pixel image processing model using the few-photon low-resolution images in the few-photon low-resolution image data set until the value of the image processing loss function is minimized, thereby obtaining a trained few-photon sub-pixel image processing model.

10. An optical detection system, characterized in that: include: A telescope, used to collect optical signals; A fast-reflecting mirror, used to adjust the light signal collected by the telescope; A single-photon detector array is used to receive the optical signal adjusted by the fast mirror and convert the received optical signal into an electrical signal; an analog-to-digital converter, used for receiving the electrical signal generated by the single-photon detector array and converting the electrical signal into a digital signal; A data processing module, used for converting the digital signal generated by the analog-to-digital converter into a low-photon low-resolution image, and processing the low-photon low-resolution image using the low-photon low-resolution image processing method according to any one of claims 1 to 8, obtaining the centroid position of the light spot of the processed low-photon low-resolution image, and sending the centroid position of the light spot to the control module; A control module, used for generating a control instruction based on the centroid position of the light spot and sending the control instruction to a fast mirror drive control module; The fast-reflection mirror driving control module is used to control the angle and displacement of the fast-reflection mirror based on the control instruction, so as to realize the positioning and tracking of the light spot in the few-photon low-resolution image.

Images