Computational ghost imaging reconstruction method and device under low sampling rate, equipment and medium

Abstract

The application relates to a low-sampling-rate computational ghost imaging reconstruction method, device, equipment and medium. The method comprises the following steps: randomly initializing a generated illumination speckle and loading the illumination speckle into a spatial light modulator to control a light field of laser irradiation, irradiating the spatial light modulator with the laser to determine a modulated light beam, irradiating the modulated light beam to a target object and passing through one or more media, and capturing light intensity values after the target object by a bucket detector; inputting the light intensity values into a second visual converter model, training by using a self-supervised method to determine an optimized illumination speckle, reloading the optimized illumination speckle into the spatial light modulator to remodulate the light field of laser irradiation until an improved visual converter neural network reaches a convergence state; and inputting the light intensity values of the optimized illumination speckle into the second visual converter model to determine a reconstructed image of the target object. The application can significantly improve the picture quality of the computational ghost imaging reconstruction under a low sampling rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer images, in particular to a low-sampling-rate computational ghost imaging reconstruction method, a corresponding device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Ghost imaging is an imaging method that uses a single photodetector to obtain information about a target scene. It does not use a large array of pixel detectors as in traditional imaging, but uses a bucket detector without spatial resolution to capture a two-dimensional image. Computational ghost imaging preloads a series of speckles on a spatial light modulator. When the spatial light modulator is illuminated by a laser, the modulated light is reflected to the object. The light passing through the object is detected by the bucket detector. The light intensity value is correlated with the speckle to reconstruct the target image. Computational ghost imaging has higher detection efficiency and sensitivity, and has attracted much attention in recent years.

[0003] At present, deep learning has also been applied to ghost imaging because it can solve various challenging problems in different fields. By iteratively optimizing network parameters, the reconstruction quality of ghost imaging at low sampling rates has been improved. However, at very low sampling rates, the reconstruction quality of ghost imaging methods is not high, and the reconstruction quality of deep learning-based ghost imaging methods is not ideal.

[0004] In view of the above, the present application is made to solve the problem of the reconstruction quality of ghost imaging methods at very low sampling rates not being high and the reconstruction quality of deep learning-based ghost imaging methods not being ideal in the prior art. SUMMARY

[0005] The present application aims to solve the above problems and provides a low-sampling-rate computational ghost imaging reconstruction method, a corresponding device, an electronic device and a computer readable storage medium.

[0006] To achieve the various purposes of the present application, the present application adopts the following technical solutions:

[0007] A low-sampling-rate computational ghost imaging reconstruction method is proposed to achieve one of the purposes of the present application, comprising:

[0008] In response to a low-sampling-rate computational ghost imaging reconstruction instruction, randomly initialize and generate an illumination speckle and load it into a spatial light modulator to control the light field of laser illumination. The spatial light modulator is illuminated by a laser to determine the modulated light beam. The modulated light beam is illuminated to a target object and passes through one or more media, and the light intensity value after passing through the target object is captured by a bucket detector.

[0009] adding an encoder, a bottleneck layer and a decoder to a preset first visual transformer model, fusing a coordinate attention mechanism into each visual transformer block of the first visual transformer model, and fusing deep convolutions of multiple scales into a feedforward network of the first visual transformer model to construct a second visual transformer model;

[0010] inputting the light intensity value into the second visual transformer model and training the second visual transformer model in a self-supervised manner to determine an optimized illumination speckle, and reloading the optimized illumination speckle to the spatial light modulator to remodulate a light field of laser irradiation until the second visual transformer model reaches a convergent state;

[0011] inputting the light intensity value corresponding to the optimized illumination speckle into the second visual transformer model trained to the convergent state to determine a reconstructed image of the target object to complete the computational ghost imaging reconstruction under a low sampling rate.

[0012] Optionally, the step of inputting the light intensity value corresponding to the optimized illumination speckle into the second visual transformer model trained to the convergent state to determine a reconstructed image of the target object to complete the computational ghost imaging reconstruction under a low sampling rate, comprises:

[0013] processing the light intensity value after passing through the target object to convert it into a two-dimensional feature map;

[0014] using the encoder to reduce the spatial resolution of the spatial features to capture information of different scales and levels;

[0015] using the bottleneck layer to integrate the abstract features from the high layer of the encoder;

[0016] the decoder restores the low-resolution feature map to the resolution of the input image, fuses the corresponding features in the encoder through a skip connection to determine a reconstructed image of the target object.

[0017] Optionally, the step of inputting the light intensity value corresponding to the optimized illumination speckle into the second visual transformer model trained to the convergent state to determine a reconstructed image of the target object to complete the computational ghost imaging reconstruction under a low sampling rate, comprises:

[0018] fusing the coordinate attention mechanism into each visual transformer block, using the coordinate attention mechanism to pool the two-dimensional feature map according to the X direction and the Y direction respectively, transforming the generated feature map, and performing a splicing operation;

[0019] segmenting along the spatial dimension, using a 1x1 convolution to perform a dimension increasing operation, and combining an activation function to obtain a weighted feature map.

[0020] Optionally, the light intensity value corresponding to the optimized illumination speckle is input into the second visual converter model trained to a convergent state to determine the reconstructed image of the target object, so as to complete the step of computing the ghost imaging reconstruction under the low sampling rate, comprising:

[0021] The deep convolution of multiple scales is fused into the feedforward network, four deep convolutions of different scales are inserted between two linear layers in the feedforward network to learn features of different scales, each deep convolution processes one quarter of the channels, and the features are reshaped into one-dimensional format through pointwise convolution.

[0022] Optionally, the light intensity value is input into the second visual converter model and trained by using a self-supervised method to determine the optimized illumination speckle, and the optimized illumination speckle is reloaded to the spatial light modulator to re-modulate the light field of the laser irradiation until the second visual converter model reaches a convergent state, comprising:

[0023] In each iteration of the training process of the second visual converter model, the laser irradiation illumination speckle causes the bucket detector to obtain the light intensity value after the target object;

[0024] The light intensity value after the target object is input into the second visual converter model to obtain a reconstructed image through a mapping matrix, an encoder, a bottleneck layer and a decoder;

[0025] A preset loss function is used to calculate the mean square error between the reconstructed image and the original image, the network is optimized by backward propagation, the network parameters and the illumination speckle are optimized, the computer re-loads the optimized illumination speckle to the spatial light modulator, and the above steps are repeated until a preset iteration number is reached, and the training of the second visual converter model is completed.

[0026] Optionally, the expression of the illumination speckle is

[0027] The expression of the light intensity value after the target object is obtained for the i-th time is:

[0028]

[0029] Wherein, D(x, y) represents the target object, represents the medium, I i is the intensity value of light propagation, B i is the light intensity value after the target object.

[0030] Optionally, the first visual converter model is a basic model of the visual converter model, and the second visual converter model is an improved visual converter model.

[0031] The low-sampling-rate computational ghost imaging reconstruction device provided by another object of the application comprises:

[0032] The light intensity value acquisition module is configured to respond to the low-sampling-rate computational ghost imaging reconstruction instruction, randomly initialize the illumination speckle and load it into the spatial light modulator to control the light field of laser irradiation, irradiate the spatial light modulator with laser to determine the modulated light beam, irradiate the modulated light beam to the target object and pass through one or more media, and capture the light intensity value after the target object by the bucket detector.

[0033] The second visual converter construction module is configured to add an encoder, a bottleneck layer and a decoder in the preset first visual converter model, fuse the coordinate attention mechanism into each visual converter block of the first visual converter model, and fuse the depth convolution of multiple scales into the feedforward network of the first visual converter model to construct a second visual converter model.

[0034] The second visual converter training module is configured to input the light intensity value into the second visual converter model and train by using a self-supervised method to determine an optimized illumination speckle, and load the optimized illumination speckle into the spatial light modulator again to remodulate the light field of laser irradiation until the second visual converter model reaches a convergence state.

[0035] The ghost imaging reconstruction module is configured to input the light intensity value corresponding to the optimized illumination speckle into the second visual converter model trained to the convergence state, determine a reconstructed image of the target object, and complete the low-sampling-rate computational ghost imaging reconstruction.

[0036] The electronic device provided by another object of the application comprises a central processor and a memory, and the central processor is used to call and run a computer program stored in the memory to perform the steps of the low-sampling-rate computational ghost imaging reconstruction method.

[0037] The computer readable storage medium provided by another object of the application stores a computer program implemented according to the low-sampling-rate computational ghost imaging reconstruction method in the form of computer readable instructions, and the computer program performs the steps included in the corresponding method when called and run by a computer.

[0038] Compared with the prior art, the present application aims at the problems in the prior art, such as low reconstruction quality of the ghost imaging method at an extremely low sampling rate and unsatisfactory reconstruction quality of the ghost imaging method based on deep learning, and includes but is not limited to the following beneficial effects:

[0039] Firstly, the application can enhance channel attention. According to the strong correlation between ghost imaging measurement values, the coordinate attention mechanism is combined into the visual transformer block, effectively improving the imaging quality.

[0040] Secondly, the application inserts deep convolution of different scales between two linear layers in the feedforward network, performs feature fusion at multiple scales, and effectively transmits information and features between multiple feature layers.

[0041] Thirdly, the improved visual transformer-based computational ghost imaging reconstruction method of the application is trained by simultaneously optimizing the illumination speckle and the visual transformer model. The improved visual transformer combines local and global information and multi-scale feature fusion, effectively improving the reconstruction efficiency and quality of the computational ghost imaging picture under extremely low sampling rate. BRIEF DESCRIPTION OF DRAWINGS

[0042] The above and / or additional aspects and advantages of the application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0043] Figure 1 is an exemplary network architecture used by the low-sampling-rate computational ghost imaging reconstruction method of the application;

[0044] Figure 2 is an exemplary network architecture for computational ghost imaging reconstruction in the embodiments of the application;

[0045] Figure 3 is a flowchart of the improved visual transformer-based computational ghost imaging reconstruction method in the embodiments of the application;

[0046] Figure 4 is a principle diagram of the improved visual transformer;

[0047] Figure 5 is a principle diagram of each visual transformer block in the improved visual transformer;

[0048] Figure 6 is a principle diagram of the feedforward network in the visual transformer block;

[0049] Figure 7 is a principle diagram of the low-sampling-rate computational ghost imaging reconstruction device in the embodiments of the application;

[0050] Figure 8 is a structural schematic diagram of the computer device in the embodiments of the application. DETAILED DESCRIPTION

[0051] Embodiments of the present application are described below in the context of example embodiments, which are shown in the drawings, wherein like or similar designations are used to indicate like or similar elements or elements having the same or similar function throughout the several views. The embodiments described below are merely examples, which are used to explain the present application and are not to be construed as limiting the present application.

[0052] Those skilled in the art will understand that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and / or "including," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can be present. In addition, the use of "connection" or "coupling" herein also includes wireless connection or wireless coupling. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0053] Those skilled in the art will appreciate that unless otherwise indicated, as used herein, all terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0054] Those skilled in the art will appreciate that the term "client", "terminal", "terminal device" as used herein encompasses both devices that are solely wireless signal receivers, devices that are solely wireless signal receivers without transmit capability, and devices that have receive and transmit hardware enabling two-way communications over a two-way communications link. Such devices can include cellular or other communication devices with or without a single-line or multiple-line display capabilities or a multi-line display, Personal Communications Service (PCS) devices that can combine a voice and data function, Personal Digital Assistants (PDAs) that can include a radio frequency receiver, pagers, Internet / Intranet access, web browsers, organizers, calendars, and / or a Global Positioning System (GPS) receiver. Further, the term "client", "terminal", "terminal device" as used herein can be portable, transportable, installed in a vehicle (aeronautical, maritime, and / or land) or adapted for and / or configured for local and / or distributed operation on Earth and / or any other location in space. The term "client", "terminal", "terminal device" as used herein can also be a communication terminal, a web appliance, a music / video player terminal, such as a PDA, a M ID (Mobile Internet Device) and / or a mobile phone with music / video player function, a smart television, a set-top box, and the like.

[0055] The term "server", "client", "service node" and the like as used herein refers to a hardware device having the equivalent capability of a personal computer, which is a hardware device having a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device, and the like necessary components disclosed by the Von Neumann principle. A computer program is stored in the memory, the central processing unit loads the program stored in the external storage into the memory and runs it, executes the instructions in the program, and interacts with the input and output devices, thereby completing a specific function.

[0056] It should be noted that the concept of "server" in the present application can also be extended to the case of a server cluster. According to the principle of network deployment understood by those skilled in the art, the servers should be logically divided, and in physical space, these servers can be independent of each other but can be called through an interface, or can be integrated into a physical computer or a computer cluster. Those skilled in the art should understand this variation and should not be restricted by the implementation of the network deployment of the present application.

[0057] One or more technical features of the present application, unless explicitly specified, can be deployed on a server and accessed by remotely calling the online service interface provided by the server, or can be directly deployed and run on a client to implement access.

[0058] The neural network model referred to or possibly referred to in the present application, unless explicitly specified, can be deployed on a remote server and remotely called at the client, or can be deployed on a client with sufficient device capability and directly called at the client. In some embodiments, when it is run on the client, its corresponding intelligence can be obtained through transfer learning, so as to reduce the requirement for client hardware running resources and avoid excessive occupation of client hardware running resources.

[0059] The various data involved in the present application, unless explicitly specified, can be remotely stored on a server or stored on a local terminal device, as long as it is suitable for being called by the technical solutions of the present application.

[0060] Those skilled in the art should know that the various methods of the present application, although based on the same concept and described to present commonality between them, are independently executable unless otherwise specified. Similarly, for each embodiment disclosed in the present application, it is based on the same inventive concept, so the same concept is understood to be equivalent, and although the concept is expressed differently, it is only a suitable transformation for convenience.

[0061] Unless it is explicitly stated that the embodiments disclosed in the present application are mutually exclusive, the technical features involved in each embodiment can be combined flexibly to construct new embodiments, as long as such combination does not deviate from the spirit of the present application and can meet the needs of the prior art or solve some deficiencies in the prior art. For this variation, those skilled in the art should know.

[0062] Please refer to Figures 1 to 6 The low sampling rate calculation ghost imaging reconstruction method of the present application includes, in one embodiment:

[0063] Step S10, in response to the low sampling rate calculation ghost imaging reconstruction instruction, randomly initialize the generated illumination speckle and load it into the spatial light modulator to control the light field of laser irradiation, the laser irradiates the spatial light modulator to determine the modulated light beam, and the modulated light beam is irradiated to the target object and passes through one or more media, and the light intensity value after passing through the target object is captured by the bucket detector.

[0064] The computing terminal device can respond to the low sampling rate calculation ghost imaging reconstruction instruction, randomly initialize the generated illumination speckle and load it into the spatial light modulator to control the light field of laser irradiation, the laser irradiates the spatial light modulator to determine the modulated light beam, and the modulated light beam is irradiated to the target object and passes through one or more media, and the light intensity value after passing through the target object is captured by the bucket detector.

[0065] Specifically, please refer to Figure 2 The illumination speckle is a binary illumination speckle image, and a random binary speckle image can be generated using a computer program. The value of each pixel point is 0 or 1, which can be realized by a random number generation function in a programming language such as Python. Save the generated binary image in a suitable image format (such as PNG or T I FF) for loading into the spatial light modulator; the laser source includes a laser diode and a solid-state laser. The laser beam passes through the spatial light modulator (SLM), which modulates the laser beam according to the loaded binary speckle image, thereby generating a modulated light field. This modulation can be modulation of light intensity or phase modulation of light;

[0066] Further, the modulated light beam is irradiated to the target object. The light beam may pass through one or more media, such as lenses, optical fibers or other optical elements, after passing through the target object, affecting the propagation and characteristics of the light, and finally measuring the light intensity through the bucket detector.

[0067] In some embodiments, ghost imaging is an imaging method based on speckle imaging technology. Unlike traditional imaging techniques, ghost imaging does not rely on direct optical images, but reconstructs images by statistically analyzing speckle light fields. In ghost imaging, the light emitted by the light source passes through a random scattering medium (such as fog or frosted glass), forming a light field with specific speckle patterns. These speckle patterns vary depending on the imaging object.

[0068] In some embodiments, the illumination speckle is a specific implementation of a speckle pattern, which is commonly used to describe the speckle effect caused by the coherence of the light source and the lighting conditions. Illumination speckle is a speckle pattern formed under specific lighting conditions (such as using a coherent light source), which is commonly used to describe the speckle effect produced when light is irradiated onto an object, which affects the clarity and quality of the image.

[0069] In some embodiments, a spatial light modulator (SLM) is an optical device for modulating the spatial distribution of light. It can control the intensity, phase, or polarization state of light, common types include liquid crystal spatial light modulators (LC-SLM) and micromirror arrays (MEMS-SLM).

[0070] In some embodiments, the expression of the illumination speckle is

[0071] The expression of the light intensity value after passing through the target object for the i-th time is:

[0072]

[0073] where D(x, y) represents the target object, represents the medium, I i is the intensity value of light propagation, B i the light intensity value after passing through the target object.

[0074] Step S20, add an encoder, a bottleneck layer and a decoder in the preset first visual transformer model, fuse the coordinate attention mechanism into each visual transformer block of the first visual transformer model, and fuse multiple scales of deep convolution into the feedforward network of the first visual transformer model to construct a second visual transformer model.

[0075] After the light intensity value after passing through the target object is captured by the bucket detector, an encoder, a bottleneck layer and a decoder are added in the preset first visual transformer model, the coordinate attention mechanism is fused into each visual transformer block of the first visual transformer model, and multiple scales of deep convolution are fused into the feedforward network of the first visual transformer model to construct a second visual transformer model.

[0076] Specifically, the first visual converter model is the base model for visual converter models, and the second visual converter model is an improved visual converter model. The base model for visual converter models is a deep learning model for computer vision tasks. It is based on the Transformer architecture, which was originally used in natural language processing. The main features of the visual converter include: image patch partitioning: dividing the image into fixed-size patches, each patch being flattened into a one-dimensional vector; linear embedding: transforming the flattened image patches into high-dimensional feature vectors through linear layers; positional encoding: adding positional encoding to represent the position of each image patch, preserving spatial information; self-attention mechanism: capturing long-distance dependencies between image patches through the self-attention mechanism of the converter; classification head: outputting the final result through a classification head, such as image classification or prediction for other visual tasks.

[0077] Furthermore, the second visual converter model adds an encoder, a bottleneck layer, and a decoder to the first visual converter model, integrates a coordinate attention mechanism into each visual converter block of the first visual converter model, and integrates multi-scale depth convolutions into the feedforward network of the first visual converter model to construct the second visual converter model.

[0078] For more details, please refer to Figure 4 An encoder, bottleneck layer, and decoder structure are added to the preset first visual converter model. The light intensity value after passing through the target object is processed and converted into a two-dimensional feature map. The encoder reduces the spatial resolution of the spatial features and captures information at different scales and levels. The bottleneck layer integrates the high-level abstract features from the encoder. Finally, the decoder restores the low-resolution feature map to the resolution of the input image. The image is then fused with the corresponding features in the encoder through skip connections to finally reconstruct the image.

[0079] Integrate the coordinate attention mechanism into each visual converter block. Specifically, such as... Figure 5 As shown, the coordinate attention mechanism pools the feature map along both the X and Y directions, transforms the generated feature maps, and then concatenates them. Next, it segments along the spatial dimensions, performs dimensionality upscaling using 1×1 convolutions, and combines this with an activation function to obtain a weighted feature map.

[0080] Multiple scales of depthwise convolutions are fused into the feedforward network. Specifically, such as... Figure 6As shown, four depthwise convolutions of different scales (1×1, 3×3, 5×5, and 7×7) are inserted between two linear layers in the feedforward network to learn features at different scales, with each convolution processing one-quarter of the channels. Finally, the features are reshaped into a one-dimensional format via point-to-point convolutions. Because there is a strong correlation between each ghost image measurement, this method can simultaneously utilize local and global information, activating more pixels and thus better recovering high-frequency information in the image.

[0081] Step S30: Input the light intensity value into the second visual converter model and train it using a self-supervised method to determine the optimized illumination speckle. Reload the optimized illumination speckle into the spatial light modulator to remodulate the light field of the laser illumination until the second visual converter model reaches a convergence state.

[0082] After constructing the second visual converter model, the light intensity value is input into the second visual converter model, and a self-supervised method is used for training to determine the optimized illumination speckle. The optimized illumination speckle is then reloaded into the spatial light modulator to remodulate the light field of the laser illumination until the second visual converter model reaches a convergence state.

[0083] For further details, please refer to Figure 3 The steps include: inputting the light intensity value into the second visual converter model and training it using a self-supervised method to determine the optimized illumination speckle; reloading the optimized illumination speckle into the spatial light modulator to remodulate the laser-illuminated light field; and continuing until the second visual converter model reaches convergence.

[0084] Step S301: In each iteration of the training process of the second vision converter model, the laser illumination speckle enables the barrel detector to obtain the light intensity value after passing the target object.

[0085] Step S302: The light intensity value after passing through the target object is input into the second visual converter model, and the reconstructed image is obtained through the mapping matrix, encoder, bottleneck layer and decoder;

[0086] Step S303: Calculate the mean square error between the reconstructed image and the original image using a preset loss function, perform backpropagation on the network to optimize network parameters and illumination speckle, reload the optimized illumination speckle into the spatial light modulator, repeat the above steps until the preset number of iterations is reached, and complete the training of the second visual converter model.

[0087] Step S40: Input the light intensity value corresponding to the optimized illumination speckle into the second visual converter model that has been trained to convergence state to determine the reconstructed image of the target object, so as to complete the computational ghost imaging reconstruction under low sampling rate.

[0088] After the second visual converter model is trained to a convergent state, the light intensity value corresponding to the optimized illumination speckle is input into the second visual converter model that has been trained to a convergent state to determine the reconstructed image of the target object, so as to complete the computational ghost imaging reconstruction under low sampling rate.

[0089] Furthermore, the light intensity value corresponding to the optimized illumination speckle is input into the second visual converter model that has been trained to a convergent state to determine the reconstructed image of the target object, thereby completing the step of computational ghost imaging reconstruction under low sampling rate, including:

[0090] Step S401: Process the light intensity value after passing through the target object and convert it into a two-dimensional feature map;

[0091] Step S402: Use the encoder to reduce the spatial resolution of spatial features and capture information at different scales and levels;

[0092] Step S403: Integrate the abstract features from the higher levels of the encoder using the bottleneck layer;

[0093] Step S404: The decoder restores the low-resolution feature map to the resolution of the input image, and fuses it with the corresponding features in the encoder through skip connections to determine the reconstructed image of the target object.

[0094] Furthermore, the light intensity value corresponding to the optimized illumination speckle is input into the second visual converter model that has been trained to a convergent state to determine the reconstructed image of the target object, thereby completing the step of computational ghost imaging reconstruction under low sampling rate, including:

[0095] Step S4001: Integrate the coordinate attention mechanism into each visual converter block, use the coordinate attention mechanism to pool the two-dimensional feature map according to the X and Y directions respectively, transform the generated feature map, and perform a stitching operation.

[0096] Step S4002: Segment along the spatial dimension, perform dimensionality increase operation using 1×1 convolution, and obtain weighted feature maps by combining activation functions.

[0097] Furthermore, the light intensity value corresponding to the optimized illumination speckle is input into the second visual converter model that has been trained to a convergent state to determine the reconstructed image of the target object, thereby completing the step of computational ghost imaging reconstruction at a low sampling rate, including:

[0098] Multiple scales of deep convolutions are fused into a feedforward network. Four different scales of deep convolutions are inserted between two linear layers in the feedforward network to learn features at different scales. Each deep convolution processes a quarter of the channels, and the features are reshaped into a one-dimensional format through point-to-point convolutions.

[0099] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems in the prior art such as low reconstruction quality of ghost imaging methods at extremely low sampling rates and unsatisfactory reconstruction quality of deep learning-based ghost imaging methods. This application includes, but is not limited to, the following beneficial effects:

[0100] Firstly, this application can enhance channel attention. Based on the strong correlation between ghost imaging measurements, the coordinate attention mechanism is combined into the vision converter block, which effectively improves the imaging quality.

[0101] Secondly, this application inserts depthwise convolutions of different scales between two linear layers in the feedforward network to perform feature fusion at multiple scales, thereby effectively transferring information and features between multiple feature layers.

[0102] Third, the computational ghost imaging reconstruction method based on the improved visual converter in this application is trained by simultaneously optimizing the illumination speckle and visual converter models. The improved visual converter combines local and global information and multi-scale feature fusion, which effectively improves the reconstruction efficiency and reconstruction quality of computational ghost imaging images at extremely low sampling rates.

[0103] Please see Figure 7This application provides a computational ghost imaging reconstruction device for low sampling rate, comprising a light intensity value acquisition module 1100, a second visual converter construction module 1200, a second visual converter training module 1300, and a ghost imaging reconstruction module 1400, all for the purposes of this application. The light intensity value acquisition module 1100 is configured to, in response to a computational ghost imaging reconstruction command for low sampling rate, randomly initialize and generate illumination speckle and load it into a spatial light modulator to control the light field of laser illumination. The laser illuminates the spatial light modulator to determine the modulated beam. The modulated beam is then irradiated onto a target object and, after passing through one or more media, the light intensity value after passing through the target object is captured by a bucket detector. The second visual converter construction module 1200 is configured to add an encoder, a bottleneck layer, and a decoder to a preset first visual converter model, integrate a coordinate attention mechanism into each visual converter block of the first visual converter model, and integrate depth convolutions of multiple scales into the first visual converter model. In the feedforward network of the visual converter model, a second visual converter model is constructed; the second visual converter training module 1300 is configured to input the light intensity value into the second visual converter model and train it using a self-supervised method to determine the optimized illumination speckle, and reload the optimized illumination speckle into the spatial light modulator to remodulate the light field of the laser illumination until the second visual converter model reaches a convergent state; the ghost imaging reconstruction module 1400 is configured to input the light intensity value corresponding to the optimized illumination speckle into the second visual converter model that has been trained to a convergent state to determine the reconstructed image of the target object, so as to complete the computational ghost imaging reconstruction at a low sampling rate.

[0104] Based on any embodiment of this application, please refer to Figure 8 Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 8 The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, they enable the processor to implement a computational ghost imaging reconstruction method at a low sampling rate. The processor of the computer device provides computational and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When these computer-readable instructions are executed by the processor, they enable the processor to execute the computational ghost imaging reconstruction method at a low sampling rate of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0105] In this embodiment, the processor is used to execute... Figure 7 The system contains the specific functions of each module and its sub-modules. The memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the computational ghost imaging reconstruction device at low sampling rates of this application. The server can call the server's program code and data to execute the functions of all sub-modules.

[0106] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the computational ghost imaging reconstruction method at low sampling rates described in any embodiment of this application.

[0107] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the computational ghost imaging reconstruction method under low sampling rate described in any embodiment of this application.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0109] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

[0110] In summary, the computational ghost imaging reconstruction method based on the improved visual converter proposed in this application is trained by simultaneously optimizing the illumination speckle and visual converter models. The improved visual converter combines local and global information and multi-scale feature fusion, which effectively improves the reconstruction efficiency and reconstruction quality of computational ghost imaging images at extremely low sampling rates.

Claims

1. A method of computed ghost imaging reconstruction at low sampling rates, characterized in that, The method comprises the following steps: In response to the instruction of the low sampling rate computational ghost imaging reconstruction, randomly initialize the generated illumination speckle and load it into a spatial light modulator to control the light field of laser irradiation, the spatial light modulator is irradiated by the laser to determine the modulated light beam, the modulated light beam is irradiated to the target object and passes through one or more media, and the light intensity value after passing through the target object is captured by a bucket detector; Add an encoder, a bottleneck layer and a decoder to a preset first visual converter model, fuse a coordinate attention mechanism into each visual converter block of the first visual converter model, and fuse multiple scales of depth convolution into a feedforward network of the first visual converter model to construct a second visual converter model; Input the light intensity value into the second visual converter model and train it in a self-supervised manner to determine an optimized illumination speckle, and reload the optimized illumination speckle into the spatial light modulator to remodulate the light field of laser irradiation until the second visual converter model reaches a convergence state; Input the light intensity value corresponding to the optimized illumination speckle into the second visual converter model trained to the convergence state to determine a reconstructed image of the target object to complete the low sampling rate computational ghost imaging reconstruction.

2. The method of claim 1, wherein, The step of inputting the light intensity value corresponding to the optimized illumination speckle into the second visual converter model trained to the convergence state to determine a reconstructed image of the target object to complete the low sampling rate computational ghost imaging reconstruction comprises: Processing the light intensity value after passing through the target object to convert it into a two-dimensional feature map; Using the encoder to reduce the spatial resolution of the spatial features and capture information of different scales and levels; Using the bottleneck layer to integrate the abstract features from the high layer of the encoder; The decoder restores the low-resolution feature map to the resolution of the input image, fuses the corresponding features in the encoder through the jump connection, and determines the reconstructed image of the target object.

3. The method of claim 2, wherein, The step of inputting the light intensity value corresponding to the optimized illumination speckle into the second visual converter model trained to the convergence state to determine a reconstructed image of the target object to complete the low sampling rate computational ghost imaging reconstruction comprises: Fusing the coordinate attention mechanism into each visual converter block, using the coordinate attention mechanism to pool the two-dimensional feature map according to the X direction and the Y direction respectively, transforming the generated feature map, and performing a splicing operation; Along the spatial dimension, a 1x1 convolution is used for dimension lifting, and an activation function is used to obtain a weighted feature map.

4. The method of claim 3, wherein, The step of inputting the light intensity value corresponding to the optimized illumination speckle into the second visual converter model trained to the convergence state to determine a reconstructed image of the target object to complete the low sampling rate computational ghost imaging reconstruction comprises: Fusing multiple scales of depth convolution into the feedforward network, inserting four depth convolutions of different scales between two linear layers in the feedforward network to learn features of different scales, each depth convolution processing one quarter of the channels, and remodeling the features into one-dimensional format through pointwise convolution.

5. The method of claim 1, wherein, The light intensity value is input into the second visual converter model, and a self-supervised method is used for training to determine an optimized illumination speckle, and the optimized illumination speckle is reloaded to the spatial light modulator to re-modulate the light field of laser irradiation until the second visual converter model reaches a convergent state, including: In each iteration of the second visual converter model training process, the laser irradiation illumination speckle causes the bucket detector to obtain a light intensity value after the target object; The light intensity value after the target object is input into the second visual converter model to obtain a reconstructed image after passing through a mapping matrix, an encoder, a bottleneck layer and a decoder; A preset loss function is used to calculate the mean square error of the reconstructed image and the original image, and the network is optimized in a reverse propagation manner to optimize the network parameters and the illumination speckle, and the computer re-loads the optimized illumination speckle to the spatial light modulator, and the above steps are repeated until a preset iteration number is reached, and the training of the second visual converter model is completed.

6. The method of claim 1, wherein, The expression of the illumination speckle is The expression of the light intensity value after the target object is obtained for the i-th time is: where D(x, y) represents the target object, represents the medium, I i is the intensity value of light propagation, B i is the intensity value of light after passing through the target object.

7. The method of claim 1 to 6, wherein, The first visual converter model is a basic model of the visual converter model, and the second visual converter model is an improved visual converter model.

8. An apparatus for computed ghost imaging reconstruction at low sampling rates, characterized in that It includes: A light intensity value acquisition module is configured to respond to a computed ghost imaging reconstruction instruction under a low sampling rate, randomly initialize an illumination speckle and load it into a spatial light modulator to control a light field of laser irradiation, irradiate the spatial light modulator with laser to determine a modulated light beam, irradiate the target object with the modulated light beam and pass through one or more media, and capture a light intensity value after the target object by a bucket detector; A second visual converter construction module is configured to add an encoder, a bottleneck layer and a decoder in a preset first visual converter model, fuse a coordinate attention mechanism into each visual converter block of the first visual converter model, and fuse multiple scales of depth convolution into a feedforward network of the first visual converter model to construct a second visual converter model; A second visual converter training module is configured to input the light intensity value into the second visual converter model and use a self-supervised method for training to determine an optimized illumination speckle, and reload the optimized illumination speckle to the spatial light modulator to re-modulate the light field of laser irradiation until the second visual converter model reaches a convergent state; A ghost imaging reconstruction module is configured to input the light intensity value corresponding to the optimized illumination speckle into the second visual converter model trained to the convergent state to determine a reconstructed image of the target object to complete the computed ghost imaging reconstruction under the low sampling rate.

9. An electronic device comprising a central processing unit and a memory, characterized in that The central processing unit is configured to call and run a computer program stored in the memory to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program implemented according to the method of any one of claims 1 to 7 in the form of computer readable instructions, and when the computer program is called and run by a computer, the steps included in the corresponding method are performed.

Images