Adaptive single-pixel imaging method, system, apparatus, medium, and device

Abstract

The present application relates to the technical field of single-pixel imaging, and discloses a self-adaptive single-pixel imaging method, system, device, medium and equipment. The method comprises the following steps: acquiring pre-perception information of a target to be measured and generating a spatial prior graph, and then generating a self-adaptive coding pattern, wherein the proportion of high-resolution coding components at different spatial positions changes adaptively with a weight value; projecting the coding pattern and collecting measurement data, combining the pre-perception information to generate complete measurement data and a corresponding complete coding matrix; solving a physical inverse problem based on the complete data to obtain an initial reconstructed image; inputting the initial reconstructed image and the spatial prior graph into a neural network to obtain residual correction data and output a target reconstructed image. The above method realizes efficient configuration of measurement resources, effective recovery of target region details, and ensures the physical consistency of the target reconstructed image and the actual measurement data without increasing the total number of measurements.

Description

Technical Field

[0001] This invention relates to the field of single-pixel imaging technology, and in particular to an adaptive single-pixel imaging method, system, device, medium, and equipment. Background Technology

[0002] Single-pixel imaging projects coded patterns sequentially onto the target using a spatial light modulator, and a single-pixel detector collects the integral response of the target to each pattern to obtain a one-dimensional measurement sequence. This sequence is then combined with the coding matrix and a reconstruction algorithm to reconstruct a two-dimensional image. Compared to array detector imaging, single-pixel imaging significantly reduces hardware costs in the non-visible light band and can utilize high-speed modulators and high-sensitivity point detectors to achieve imaging capabilities in specific scenarios. Existing single-pixel imaging methods are mainly divided into three categories: The first category is fixed coding methods, which use preset patterns such as random coding, Hadamard coding, Fourier coding, or DCT coding for measurement. The coding is simple to generate and the measurement matrix is ​​easy to analyze. Combining total variation, sparse regularization, or low-frequency reconstruction algorithms can yield stable reconstruction results. The second category is adaptive coding methods, which first acquire a rough preview image and then adjust subsequent measurement patterns based on the preview results, concentrating measurement resources on the region of interest. The third category is neural network reconstruction methods, which utilize neural networks to learn image priors, improving reconstruction speed and visual quality under low sampling rate conditions.

[0003] However, all three methods mentioned above have inherent drawbacks. Fixed encoding methods measure according to a preset order, unable to adjust measurement allocation based on the target space content. When the target area is small or important details are concentrated in a local area, the limited number of measurements may be allocated to areas with low information value, affecting the reconstruction quality of the target area. Adaptive encoding methods can adjust subsequent measurements based on preview results, but their effectiveness is limited by the preview image quality, and subsequent reconstruction still relies on traditional physical reconstruction algorithms, making it difficult to fully recover details under low measurement budgets. Neural network reconstruction methods, if only used as a post-acquisition processing method, may lack a clear physical correspondence between their output and the actual measurement data. Current technologies lack imaging schemes that can coordinate all of these aspects.

[0004] Therefore, a single-pixel imaging scheme that can simultaneously coordinate measurement resource allocation, physical constraint reconstruction, and neural network refinement urgently needs to be studied. Summary of the Invention

[0005] In view of this, this application provides an adaptive single-pixel imaging method, system, device, medium and apparatus, the main purpose of which is to solve the technical problems in the prior art that single-pixel imaging schemes cannot allocate measurement resources according to the target spatial content and cannot balance detail recovery and physical consistency under low measurement budget.

[0006] According to a first aspect of the present invention, an adaptive single-pixel imaging method is provided, comprising: Acquire pre-sensing information of the target to be tested, generate a spatial prior map based on the pre-sensing information, and generate an adaptive coding pattern based on the spatial prior map, wherein the proportion of high-resolution coding components at different spatial locations in the adaptive coding pattern changes adaptively with the weight value of the spatial prior map. The adaptive coding pattern is projected onto the target to be measured, measurement data of the target to be measured is collected, and the measurement data is combined with the pre-sensing information to generate complete measurement data and the corresponding complete coding matrix. Based on the complete measurement data and the complete encoding matrix, the physical inverse problem is solved to obtain the initial reconstructed image; The initial reconstructed image and the spatial prior image are input into a pre-trained neural network to obtain residual correction data, and the target reconstructed image is output based on the initial reconstructed image and the residual correction data.

[0007] Optionally, the step of acquiring pre-sensing information of the target to be tested and generating a spatial prior map based on the pre-sensing information includes: projecting an coded pattern onto the target to be tested and collecting measurement data corresponding to the coded pattern as pre-sensing information; generating a preview image based on the pre-sensing information; and generating a spatial prior map based on the preview image, wherein the spatial prior map includes weight values ​​corresponding to multiple spatial locations.

[0008] Optionally, generating an adaptive coding pattern based on the spatial prior map includes: acquiring a preset high-resolution coding pattern and a preset low-resolution coding pattern; performing an upsampling operation on the low-resolution coding pattern to ensure that the spatial resolution of the low-resolution coding pattern is consistent with the spatial resolution of the high-resolution coding pattern; performing pixel-by-pixel weighted fusion of the high-resolution coding pattern and the low-resolution coding pattern based on the weight values ​​corresponding to multiple spatial locations in the spatial prior map to generate an adaptive coding pattern, wherein the spatial location with a larger weight value has a higher proportion of high-resolution coding components in the adaptive coding pattern; and performing normalization and decorrelation processing on the adaptive coding pattern.

[0009] Optionally, the step of solving the physical inverse problem based on the complete measurement data and the complete encoding matrix to obtain the initial reconstructed image includes: constructing an objective function, wherein the objective function includes a data consistency term and a regularization term, the data consistency term is used to constrain the initial reconstructed image to be consistent with the complete measurement data after projection onto the complete encoding matrix, and the regularization term is used to constrain the spatial characteristics of the initial reconstructed image; iteratively solving the optimal solution of the objective function, and using the optimal solution as the initial reconstructed image.

[0010] Optionally, the step of inputting the initial reconstructed image and the spatial prior image into a pre-trained neural network to obtain residual correction data includes: constructing auxiliary input features, wherein the auxiliary input features include at least one of the back projection results of the complete measurement data, the edge features of the initial reconstructed image, and the multi-scale representation of the spatial prior image; concatenating the initial reconstructed image, the spatial prior image, and the auxiliary input features into an input tensor, and using the pre-trained neural network to perform feature extraction and mapping on the input tensor to output residual correction data.

[0011] Optionally, the step of outputting the target reconstructed image based on the initial reconstructed image and the residual correction data includes: synthesizing the initial reconstructed image and the residual correction data to obtain a synthesized result, and performing a measurement consistency check on the synthesized result; when the synthesized result passes the consistency check, using the synthesized result as the target reconstructed image; when the synthesized result fails the consistency check, adjusting the amplitude of the residual correction data to obtain updated residual correction data, and re-performing the synthesis operation on the initial reconstructed image and the updated residual correction data to obtain an updated reconstructed image, wherein the deviation between the updated reconstructed image and the complete measurement data after projection onto the complete coding matrix is ​​less than a preset threshold.

[0012] According to a second aspect of the present invention, an adaptive single-pixel imaging system is provided, comprising: A light source, used to provide illumination; A spatial light modulator is disposed in the output light path of the light source and is used to load an encoded pattern and modulate the light field; A single-pixel detector is used to acquire the integral response of the target under test to the coded pattern and output measurement data; A data acquisition controller is connected to the spatial light modulator and the single-pixel detector, respectively, and is used to synchronize the loading of the coded pattern of the spatial light modulator with the data acquisition of the single-pixel detector. A processor, connected to the acquisition controller, is configured to execute the adaptive single-pixel imaging method as described in any one of claims 1 to 6; A memory, connected to the processor, is used to store the coded pattern, the measurement data, and the computer program.

[0013] According to a third aspect of the present invention, an adaptive single-pixel imaging device is provided, comprising: The encoding generation module is used to acquire pre-sensing information of the target to be tested, generate a spatial prior map based on the pre-sensing information, and generate an adaptive encoding pattern based on the spatial prior map, wherein the proportion of high-resolution encoding components at different spatial locations in the adaptive encoding pattern adapts to the weight value of the spatial prior map. The projection acquisition module is used to project the adaptive coding pattern onto the target to be measured, acquire the measurement data of the target to be measured, and combine the measurement data with the pre-sensing information to generate complete measurement data and the corresponding complete coding matrix. The image reconstruction module is used to solve the physical inverse problem based on the complete measurement data and the complete encoding matrix to obtain the initial reconstructed image; The image output module is used to input the initial reconstructed image and the spatial prior image into a pre-trained neural network to obtain residual correction data, and output the target reconstructed image based on the initial reconstructed image and the residual correction data.

[0014] According to a fourth aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described adaptive single-pixel imaging method.

[0015] According to a fifth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described adaptive single-pixel imaging method.

[0016] This invention provides an adaptive single-pixel imaging method, system, device, medium, and equipment. First, a spatial prior map is generated by acquiring pre-sensing information of the target object. An adaptive coding pattern is then generated based on the spatial prior map, causing the proportion of high-resolution coded components at different spatial locations in the coding pattern to adaptively change with weight values. This allows for the rational allocation of a limited total number of measurements according to the target spatial content. When the target area is small or important details are concentrated in a local area, measurement resources can adaptively tilt towards such areas, avoiding wasting measurement attempts on areas with low information value. This improves the reconstruction quality of both the target and detail areas within the same total measurement budget. Second, the pre-sensing data and adaptive measurement data are combined into complete measurement data and a complete coding matrix. An initial reconstruction is obtained by solving the physical inverse problem based on the complete data. By embedding a physical model into the reconstruction process, the influence of preview image quality fluctuations on the initial reconstruction result is effectively suppressed. Even when the preview image resolution is low or the boundaries are inaccurate, the solution of the physical inverse problem can still ensure the physical consistency between the reconstructed image and the actual measurement data, providing a reliable foundation for subsequent refinement. Then, the initial reconstructed image and the spatial prior image are jointly input into a pre-trained neural network, so that the neural network only outputs residual correction data, rather than directly generating the reconstructed image. The target reconstructed image is synthesized from the initial reconstructed image and the residual correction data. This limits the learning ability of the neural network to making detailed corrections to the physical reconstruction results, rather than generating images independently of the measurement data, reducing the risk of the neural network output deviating from the actual measurement data and ensuring a clear physical correspondence between the output results and the acquired data. In summary, this application, through the synergistic effect of pre-sensor-guided adaptive measurement allocation, physical inverse problem-constrained initial reconstruction, and neural network residual correction, achieves efficient allocation of measurement resources, effective restoration of target region details, and ensures the physical consistency between the target reconstructed image and the actual measurement data without increasing the total number of measurements.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart illustrating an adaptive single-pixel imaging method provided by an embodiment of the present invention is shown; Figure 2A flowchart illustrating another adaptive single-pixel imaging method provided by an embodiment of the present invention is shown; Figure 3 This diagram illustrates the adaptive coding generation in an adaptive single-pixel imaging method provided by an embodiment of the present invention. Figure 4 This diagram illustrates the physical constraint neural network reconstruction process in an adaptive single-pixel imaging method provided by an embodiment of the present invention. Figure 5 The image shows a comparison of reconstruction quality test results for an adaptive single-pixel imaging method provided in an embodiment of the present invention. Figure 6 The image shows a visual comparison of the reconstruction effect of an adaptive single-pixel imaging method provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of an adaptive single-pixel imaging system provided by an embodiment of the present invention is shown; Figure 8 A schematic diagram of the structure of an adaptive single-pixel imaging device provided in an embodiment of the present invention is shown; Figure 9 A schematic diagram of the device structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0019] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0020] This application provides an adaptive single-pixel imaging method, such as... Figure 1 As shown, the method includes the following steps: 101. Obtain the pre-sensing information of the target to be tested, generate a spatial prior map based on the pre-sensing information, and generate an adaptive coding pattern based on the spatial prior map. The proportion of high-resolution coding components at different spatial locations in the adaptive coding pattern changes adaptively with the weight value of the spatial prior map.

[0021] Among them, the pre-sensing information refers to the measurement data and intermediate processing results obtained by projecting a small number of coded patterns onto the target and performing single-pixel detection, which are used to initially understand the spatial distribution of the target; the spatial prior map refers to a two-dimensional map generated based on the pre-sensing information, which contains the weight values ​​of multiple spatial locations. The weight values ​​are used to represent the level of coding precision required for the corresponding spatial location in subsequent adaptive coding; the adaptive coding pattern refers to the coding pattern formed by performing pixel-by-pixel weighted fusion of the preset high-resolution coding pattern and the low-resolution coding pattern according to the weight value distribution of the spatial prior map. The proportion of high-resolution coding components at different spatial locations changes with the weight values. The magnitude of the weight values ​​reflects the importance or level of detail required for the corresponding location in the entire imaging area.

[0022] Specifically, before conducting formal single-pixel measurements, this application first acquires pre-sensing information about the target to be measured, using a very small number of measurements to obtain a rough spatial distribution clue of the target. Based on this, a spatial prior map is generated according to the pre-sensing information, and the importance of each spatial location is quantified in the form of weight values. When generating the adaptive coding pattern, the weight values ​​of the spatial prior map are used as the basis to perform pixel-by-pixel weighted fusion of the preset high-resolution coding pattern and the low-resolution coding pattern, so that the coding fineness of different spatial locations in the adaptive coding pattern changes adaptively with the weight values. The spatial location with a higher weight value has a corresponding modulation pattern containing richer high-resolution spatial information, which can obtain more detailed responses in subsequent detection. The spatial location with a lower weight value has a corresponding modulation pattern mainly composed of low-resolution background information, which only needs to ensure the overall structure of the imaging area.

[0023] In this embodiment, the measurement budget can be allocated based on the target spatial content before imaging. By using a spatial prior map, limited measurement resources are guided to the target area or the area requiring detail, avoiding the drawback of fixed encoding which consumes measurement attempts in areas with low information value. This improves the efficiency of measurement resource utilization without changing the total number of measurements. At the same time, the generation of the adaptive encoding pattern does not require complex online optimization calculations. It can be achieved simply by fusing the preset high-resolution encoding pattern with the low-resolution encoding pattern according to the weight values. This reduces the computational load and is easy to implement in hardware. Furthermore, it provides measurement data-level assurance for solving the subsequent physical inverse problem, laying a data foundation for the effective recovery of the target area's edges and details.

[0024] 102. Project the adaptive coding pattern onto the target to be measured, collect the measurement data of the target to be measured, combine the measurement data with the pre-sensing information, and generate complete measurement data and the corresponding complete coding matrix.

[0025] Here, projection refers to the process of loading and projecting the coded pattern onto the surface of the target under test by a spatial light modulator in the manner of light field modulation; measurement data refers to the one-dimensional response signal sequence obtained by integrating and sampling the light field reflected or transmitted by the target under test during each coded pattern projection by a single pixel detector; complete measurement data refers to the one-dimensional measurement vector composed of the measurement data collected in the pre-sensing stage and the measurement data collected in the adaptive measurement stage; and complete coding matrix refers to the two-dimensional matrix composed of the coded pattern projected in the pre-sensing stage and the adaptive coding pattern, where each row of the matrix corresponds to the coded pattern used in a measurement, and the number of rows in the matrix is ​​equal to the total number of measurements.

[0026] Specifically, after generating the adaptive coding pattern, this application synchronizes the spatial light modulator and the single-pixel detector through the acquisition controller, controls the spatial light modulator to load each set of adaptive coding patterns in sequence and project them onto the target to be measured, and the single-pixel detector performs integral sampling on the light field response of the target to be measured after each projection to obtain the measurement data corresponding to each coding pattern. Since the measurement data of the pre-sensing stage has been obtained in the aforementioned steps, this application merges the measurement data of the pre-sensing stage with the measurement data acquired in the adaptive measurement stage into complete measurement data. At the same time, the coding matrix used in the pre-sensing stage and the adaptive coding matrix are combined into a complete coding matrix. The number of rows of the complete measurement data and the complete coding matrix are consistent and correspond one-to-one according to the measurement order, together constituting the complete input for subsequent physical reconstruction.

[0027] In this embodiment, the measurement data from the pre-sensing stage and the measurement data from the adaptive stage are incorporated into the same reconstruction framework. This ensures that the target spatial distribution information obtained in the pre-sensing stage is not discarded when a new stage of measurement begins, but rather participates in the subsequent reconstruction together with the adaptive measurement data, effectively utilizing the data information obtained from the entire measurement budget. At the same time, since the complete coding matrix fully records the coding patterns used in all measurements, the projection relationship between the measurement data and the reconstructed image can be accurately established during the solution of the physical inverse problem, ensuring the physical consistency between the subsequent reconstruction results and all actual measurement data, and avoiding information loss or reconstruction distortion caused by using only a portion of the measurement data for reconstruction.

[0028] 103. Solve the physical inverse problem based on complete measurement data and complete coding matrix to obtain the initial reconstructed image.

[0029] The physical inverse problem refers to the mathematical solution process of reversing the target image based on the forward physical model of single-pixel imaging, given the encoding matrix and measurement data. Specifically, in the forward physical model, the measurement data collected by the single-pixel detector is equal to the result of linearly projecting the encoding matrix onto the target image and adding noise. The physical inverse problem is the process of reversing the target image based on the obtained measurement data and the known encoding matrix. The data consistency term is a constraint in the objective function, used to measure the degree of deviation between the candidate reconstructed image after projection onto the encoding matrix and the measured data. The smaller its value, the better the reconstructed image matches the physical measurement process. The regularization term is an additional penalty term in the objective function, used to introduce constraints on the prior of the image into the solution process to alleviate the underdetermined problem caused by the number of measurements being less than the number of pixels. The initial reconstructed image refers to the reconstruction result obtained by solving the above physical inverse problem, which is constrained by physical measurements but not refined by a neural network. The image retains the physical basis consistent with the actual measurement data, but may have edge blurring or missing details under low measurement budget.

[0030] Specifically, after obtaining complete measurement data and a complete encoding matrix, this application obtains an initial reconstructed image by constructing and solving a physical inverse problem. Specifically, an objective function containing a data consistency term and a regularization term is constructed. The data consistency term is used to constrain the reconstructed image to be consistent with the complete measurement data after projection onto the complete encoding matrix, and the regularization term is used to constrain the spatial smoothness or sparsity of the reconstructed image to solve the underdetermined problem. The objective function is solved by an iterative optimization algorithm, and the optimal solution of the objective function is output as the initial reconstructed image. The obtained initial reconstructed image provides a physically consistent foundation for subsequent neural network refinement. The spatial structure of the initial reconstructed image has established a correspondence with the actual measurement data at the physical level.

[0031] In this embodiment, the initial reconstructed image is obtained by solving the physical inverse problem. This ensures that the reconstruction result is subject to the dual physical constraints of the complete encoding matrix and complete measurement data at the source, guaranteeing a clear physical correspondence between the initial reconstructed image and the actual acquired data. This provides a reliable data foundation for subsequent neural network refinement. At the same time, the introduction of regularization terms allows for the acquisition of stable solutions even under underdetermined conditions where the number of measurements is less than the number of pixels. This avoids the numerical instability caused by direct inversion. Compared to directly inputting measurement data into the neural network for end-to-end reconstruction, this application constructs an initial solution with physical consistency through the physical inverse problem. This allows the neural network to make minor corrections based on physical reasonableness, effectively reducing the risk of the neural network output deviating from the actual measurement data.

[0032] 104. Input the initial reconstructed image and spatial prior image into the pre-trained neural network to obtain residual correction data, and output the target reconstructed image based on the initial reconstructed image and residual correction data.

[0033] Among them, residual correction data refers to the correction amount output by the neural network to compensate for the difference between the initial reconstructed image and the target real image. The correction amount has the same spatial dimension as the initial reconstructed image, and each pixel value represents the adjustment range that needs to be increased or decreased at the corresponding position. Measurement consistency verification refers to the process of comparing the reconstructed image after being projected through the complete encoding matrix with the complete measurement data to determine whether the deviation between the two meets the preset threshold requirements. It is used to verify whether the target reconstructed image still meets the physical measurement constraints. Input tensor refers to the multi-dimensional data array formed by splicing the initial reconstructed image, spatial prior image and auxiliary input features along the channel dimension, which is used as the input data format of the neural network. Pre-trained neural network refers to a deep neural network model that has learned the image prior and residual mapping relationship in advance through training samples.

[0034] Specifically, after obtaining the initial reconstructed image, this application inputs the initial reconstructed image and the spatial prior image into a pre-trained neural network. The input of the neural network also includes constructed auxiliary input features, which include at least one of the back projection results of the complete measurement data, the edge features of the initial reconstructed image, and the multi-scale representation of the spatial prior image. The neural network performs feature extraction and mapping on the input initial reconstructed image, spatial prior image, and auxiliary input features, and outputs residual correction data. Finally, the residual correction data and the initial reconstructed image are synthesized to obtain the target reconstructed image. Preferably, after synthesis, a measurement consistency check is performed. When the consistency check fails, the amplitude of the residual correction data is adjusted and resynthesized so that the deviation between the target reconstructed image projected by the complete encoding matrix and the complete measurement data is less than a preset threshold.

[0035] In this embodiment, the neural network only needs to predict residual correction data instead of directly generating a complete reconstructed image. The network learning objective is reduced from generating an image from scratch to correcting local biases in an existing image, which reduces the learning difficulty of the network and is beneficial for obtaining stable mapping performance when training samples are limited. At the same time, since the target reconstructed image is obtained by superimposing residual correction data on the initial reconstructed image, and the initial reconstructed image itself already has physical consistency, even if the neural network prediction has biases, the final output is still based on physical constraints and will not produce false structures that are seriously inconsistent with the actual measurement data. In addition, by introducing a measurement consistency verification step, it is further ensured that the output reconstructed image remains highly consistent with the complete measurement data after being projected by the encoding matrix, thereby improving the reliability of the imaging results.

[0036] This invention provides an adaptive single-pixel imaging method. First, a spatial prior map is generated by acquiring pre-sensing information of the target object. An adaptive coding pattern is then generated based on this prior map, allowing the proportion of high-resolution coded components at different spatial locations in the coding pattern to adaptively change with weight values. This enables the limited total number of measurements to be rationally allocated according to the target spatial content. When the target area is small or important details are concentrated in a local area, measurement resources can be adaptively tilted towards such areas, avoiding wasting measurement attempts on areas with low information value. This improves the reconstruction quality of both the target and detail areas within the same total measurement budget. Second, the pre-sensing data and adaptive measurement data are combined into complete measurement data and a complete coding matrix. An initial reconstructed image is obtained by solving the physical inverse problem based on the complete data. The physical model is embedded in the reconstruction process, effectively suppressing the impact of preview image quality fluctuations on the initial reconstruction results. Even when the preview image resolution is low or the boundaries are inaccurate, the solution of the physical inverse problem still ensures the physical consistency between the reconstructed image and the actual measurement data, providing a reliable foundation for subsequent refinement. Then, the initial reconstructed image and the spatial prior map are jointly input into a pre-trained neural network, causing the neural network to output only residual correction data instead of directly generating the reconstructed image. The target reconstructed image is synthesized from the initial reconstructed image and the residual correction data. This limits the learning ability of the neural network to making detailed corrections to the physical reconstruction results, rather than generating images independently of the measurement data, reducing the risk of the neural network output deviating from the actual measurement data and ensuring a clear physical correspondence between the output results and the acquired data. In summary, this application, through the synergistic effect of pre-sensor-guided adaptive measurement allocation, physical inverse problem-constrained initial reconstruction, and neural network residual correction, achieves efficient allocation of measurement resources, effective restoration of target region details, and ensures the physical consistency between the target reconstructed image and the actual measurement data without increasing the total number of measurements.

[0037] Another adaptive single-pixel imaging method provided in this application, such as Figure 2 As shown, it specifically includes: 201. Divide the total measurement budget into the first phase measurement budget and the second phase measurement budget.

[0038] In this embodiment, before imaging begins, the total measurement budget is first divided into a first-stage budget and a second-stage budget. The total measurement budget, denoted by M, refers to the total number of measurements allowed throughout the imaging process. The first-stage budget, denoted by M1, refers to the number of measurements allocated to the pre-sensing stage, and the second-stage budget, denoted by M2, refers to the number of measurements allocated to the adaptive measurement stage, satisfying M = M1 + M2. The first-stage budget is used to acquire pre-sensing information of the target, obtaining a rough spatial distribution clue of the target by projecting a small number of coded patterns, providing a basis for subsequent adaptive measurements. The second-stage budget is used to perform adaptive coded measurements, generating adaptive measurements based on the spatial prior map. The system adapts to the coded pattern and collects corresponding measurement data. The measurement data from the two stages together constitute the complete measurement data, and the coded matrices from the two stages together constitute the complete coded matrix, which are used together for subsequent physical inverse problem solving and neural network refinement. In one implementation, the number of pixels in the image to be reconstructed is N=4096, i.e., 64×64 pixels, and the total number of measurements is M=512. The budget M1 for the first stage can be 32, 64, or 128, and the budget M2 for the second stage is the remaining number of measurements, i.e., 480, 448, or 384. The specific values ​​of the budget for each stage can be flexibly determined according to the target category, target movement speed, noise level, spatial light modulator refresh rate, and the sampling time allowed by the system.

[0039] The above budget allocation scheme, while keeping the total number of measurements constant, enables the pre-sensing phase and the adaptive measurement phase to work together. Pre-sensing measurement data and adaptive measurement data jointly participate in subsequent reconstruction, and all measurement budgets are effectively utilized, avoiding the waste of measurement time in areas with low information value.

[0040] 202. Project the first-stage coded pattern and collect pre-sensing measurement data.

[0041] Specifically, a coded pattern is projected onto the target to be measured, and the measurement data corresponding to the coded pattern is collected as pre-sensing information.

[0042] In this embodiment, during the pre-sensing stage, the acquisition controller controls the spatial light modulator to sequentially load the first-stage coded pattern. The single-pixel detector performs integral sampling on the target response after each projection to obtain measurement data y1. The first-stage coded pattern can be one or more combinations of low-frequency Hadamard coding, low-frequency DCT coding, random coding, and edge enhancement coding. All of the above coding types can obtain the overall contour, low-frequency structure, or coarse edge information of the target with fewer measurements, meeting the need for rapid acquisition of spatial distribution clues in the pre-sensing stage, without requiring the spatial resolution or visual quality of the final image to be achieved. Among them, low-frequency Hadamard coding and low-frequency DCT coding can reconstruct the main contour of the target with fewer measurements by prioritizing the acquisition of low-frequency components of the image. Random coding has good versatility and is suitable for target scenes without prior information. Edge enhancement coding focuses on extracting the boundary information of the target, which is beneficial for the weight assignment of edge regions in the subsequent spatial prior image. The above coding types can be flexibly selected or combined according to the actual application scenario to balance the acquisition efficiency and accuracy of pre-sensing information.

[0043] 203. Generate a preview image based on the pre-sensing measurement data.

[0044] Specifically, a preview image is generated based on the pre-perceived information.

[0045] In this embodiment, a preview image p is generated based on the measurement data y1 and the first-stage encoding matrix A1. The preview image can be obtained through low-frequency inverse transform, back projection, sparse reconstruction, total variational reconstruction, or lightweight neural network coarse reconstruction, etc., which can quickly obtain the spatial distribution overview of the target with a small amount of measurement data. The preview image does not need to reach the spatial resolution or visual quality of the final image. Its main function is to provide spatial location clues for the subsequent generation of spatial prior maps. The target location area, edge concentration area, or texture-rich area in the preview image are the key areas that need to be encoded at high resolution. For structural foreground targets, the preview image can present the target outline, edges, and main strokes. For natural images, the preview image can present low-frequency structure and preliminary texture distribution. The method of generating the preview image can be flexibly selected according to the actual application scenario. Under the condition that computing resources and time allow, a method with higher reconstruction quality can be selected. In scenarios with high real-time requirements, a method with less computation can be selected.

[0046] 204. Determine the weight map of the regions that require priority high-resolution measurement based on the preview image.

[0047] Specifically, a spatial prior map is generated based on the preview image, wherein the spatial prior map includes weight values ​​corresponding to multiple spatial locations.

[0048] In this embodiment of the application, the weight map generation module generates a spatial prior map based on the preview image p. In this step, the spatial prior map is the region weight map. The weight value of each spatial location in the region weight map is used to represent the degree of need for high-resolution measurement in the subsequent adaptive coding. The higher the weight value, the more high-resolution coding modulation is needed at this location in the second stage. Correspondingly, the proportion of high-resolution coding components retained in the adaptive coding pattern is also higher. The value range of the region weight map can be [0,1], or it can be mapped to other preset ranges through normalization. The region weight map can be obtained by one or more of the following methods: grayscale distribution of the preview image, edge intensity, gradient magnitude, local texture, low-frequency residual, saliency estimation, foreground probability, or neural network output.

[0049] In one implementation, edge detection and local energy calculation are first performed on the preview image, and then the results are smoothed, normalized, and thresholded to obtain a region weight map. This method has low computational cost and is suitable for scenarios with high real-time requirements. In another implementation, the preview image is input into a lightweight neural network, which directly outputs a region weight map. This method can comprehensively utilize the multi-dimensional features of the preview image and is suitable for scenarios with high requirements for the accuracy of the weight map. The above generation methods can be flexibly selected or combined according to the actual application scenario.

[0050] 205. Generate a second-stage adaptive coding pattern that integrates regional coding and background coding according to regional weights.

[0051] Specifically, a preset high-resolution encoding pattern and a preset low-resolution encoding pattern are obtained. The low-resolution encoding pattern is upsampled to ensure that the spatial resolution of the low-resolution encoding pattern is consistent with that of the high-resolution encoding pattern. Based on the weight values ​​corresponding to multiple spatial locations in the spatial prior image, the high-resolution encoding pattern and the low-resolution encoding pattern are weighted and fused pixel by pixel to generate an adaptive encoding pattern. The spatial location with the larger weight value has a higher proportion of high-resolution encoding components in the adaptive encoding pattern. The adaptive encoding pattern is then normalized and decorrelated.

[0052] In this embodiment, as Figure 3 As shown, the second-stage adaptive coding pattern is generated based on the spatial prior map, i.e., the region weight map. The specific process is to first generate a high-resolution coding pattern corresponding to the key region. Then generate a low-resolution coded pattern corresponding to the background area. Since the spatial resolution of the low-resolution coded pattern is lower than that of the high-resolution coded pattern, an upsampling operation U needs to be performed on the low-resolution coded pattern first to make its spatial resolution consistent with that of the high-resolution coded pattern. Then, according to the weight values ​​corresponding to multiple spatial locations in the spatial prior image, the high-resolution coded pattern and the upsampled low-resolution coded pattern are weighted and fused pixel by pixel to generate the adaptive coded pattern a. The fusion process can be represented as:

[0053] In the formula, 'a' represents the adaptive coding pattern. This represents the weight values ​​in the spatial prior graph. h represents a high-resolution coded pattern. This represents a low-resolution coded pattern, and U represents an upsampling operation.

[0054] Therefore, spatial locations with larger weight values ​​correspond to a higher proportion of high-resolution coded components in the adaptive coding pattern. That is, high-resolution spatial modulation is retained in high-weight regions, while low-resolution background modulation is used in low-weight regions. To avoid excessive correlation between different coding rows after spatial prior map modulation, which would affect the stability of subsequent physical reconstruction, normalization and decorrelation processing are also required for the fused adaptive coding pattern. The decorrelation processing can be achieved by removing the mean, row decorrelation constraints, or condition number constraints. Considering that when using a spatial light modulator that can only display non-negative intensities, the coding pattern containing positive and negative values ​​can be split into positive and negative patterns, projected separately, and the equivalent coding response can be obtained through differential measurement. The above coding generation method effectively reduces the correlation between rows of the coding matrix while ensuring adaptability, providing a coding matrix with better numerical stability for solving the subsequent physical inverse problem.

[0055] 206. Project the second-stage adaptive coding pattern and form complete measurement data and complete coding matrix.

[0056] In this embodiment, the second-stage adaptive coding pattern is loaded sequentially, and the second-stage measurement data y2 is collected. Then, the pre-sensing measurement data y1 and the second-stage measurement data y2 are merged into complete measurement data y, and the first-stage coding matrix A1 and the second-stage coding matrix A2 are combined into complete coding matrix AA. The complete measurement data y and the complete coding matrix A have the same number of rows and correspond one-to-one according to the measurement order, together forming the complete input for subsequent physical reconstruction and neural network refinement. Since the pre-sensing measurement data and the second-stage measurement data both come from the same total measurement budget, the measurement data of both stages are used for subsequent reconstruction. There are no additional independent measurement branches collected, and all measurement data are effectively utilized, avoiding the waste of measurement resources.

[0057] 207. Solve for the initial reconstructed image based on the complete coding matrix and complete measurement data to meet physical constraints.

[0058] Specifically, an objective function is constructed, which includes a data consistency term and a regularization term. The data consistency term is used to constrain the initial reconstructed image to be consistent with the complete measurement data after projection onto the complete coding matrix, and the regularization term is used to constrain the spatial characteristics of the initial reconstructed image. The optimal solution of the objective function is solved iteratively, and the optimal solution is used as the initial reconstructed image.

[0059] In this embodiment, as Figure 4 As shown, the physical inverse problem is solved based on the complete measurement data y and the complete encoding matrix A to obtain the initial reconstructed image with physical constraints. .

[0060] In one implementation, the physical inverse problem is expressed as:

[0061] In the formula, This represents the image to be solved. This represents a data consistency term, used to ensure that the image to be solved, after being projected onto the complete encoding matrix, remains consistent with the complete measurement data. This represents the regional weighted total variation regularization term. For regularization parameters, this term is used to constrain the spatial smoothness of the image to be solved. The region-weighted total variational regularization term sets the constraint strength for different spatial locations based on the spatial prior map. The smoothness strength corresponding to regions with higher weight values ​​in the spatial prior map is lower than that corresponding to regions with lower weight values. For example, the smoothness strength corresponding to the target region or detail region is lower than that corresponding to the background region, thereby preserving more spatial variation at the target edge and local details. This represents a non-negativity constraint or pixel range constraint, used to ensure the physical plausibility of the reconstruction result. The physical constraints apply to the initial reconstructed image. The optimal solution to the above objective function can be obtained by using the alternating direction multiplier method, the proximal gradient method, the iterative threshold method, the primal dual algorithm, or the iterative solution of the differentiable optimization layer.

[0062] The initial reconstructed image obtained by solving the physical inverse problem not only maintains physical consistency with the complete measurement data, but also retains more edges and details in the target region, providing a physically based and structurally reliable initial solution for subsequent neural network refinement.

[0063] 208. Input the physical constraint initial reconstruction image, preview image, region weight map, and measurement features obtained from complete measurement data into the neural network to obtain residual correction results or refined images.

[0064] Specifically, auxiliary input features are constructed, wherein the auxiliary input features include at least one of the back projection results of the complete measurement data, the edge features of the initial reconstructed image, and the multi-scale representation of the spatial prior map; the initial reconstructed image, the spatial prior map, and the auxiliary input features are concatenated into an input tensor, and a pre-trained neural network is used to extract and map features from the input tensor to output residual correction data.

[0065] In this embodiment, auxiliary input features are first constructed, and then concatenated with the physical constraint initial reconstructed image, preview image, and spatial prior image to form the input tensor of the neural network. Measurement features can include various types such as back projection results of complete measurement data, normalized measurement vectors, encoding matrix statistics, edge maps of the physical constraint initial reconstructed image, edge maps of the preview image, multi-scale representations of the spatial prior image, low-frequency residual maps, or difference maps between different reconstruction results. The neural network can adopt structures such as U-Net, residual network, multi-scale convolutional network, attention network, Transformer network, or physical unfolding network. The input of the neural network includes physical consistency information provided by the physical constraint initial reconstructed image, spatial distribution prior information provided by the spatial prior image, and physical measurement process information provided by the measurement features. The fusion of multi-dimensional information enables the network to more accurately identify defects in the initial reconstructed image, thereby predicting more accurate residual correction results. Preferably, the neural network outputs the residual correction result r instead of directly generating a complete image, so that the neural network only needs to perform local correction based on the initial reconstructed image, reducing the learning difficulty of the network, while ensuring that the final reconstructed image is always based on the physical constraint initial reconstructed image, avoiding the network output from deviating from the actual measurement data.

[0066] Furthermore, during neural network training, a training sample set is constructed. Each training sample includes the target ground truth image, pre-sensory measurement data, adaptive measurement data, preview image, spatial prior map, physically constrained initial reconstructed image, and measurement features. The training loss includes at least one of pixel reconstruction loss, structural similarity loss, edge consistency loss, physical constraint preservation loss, measurement consistency loss, and high-frequency residual loss. Pixel reconstruction loss is used to constrain the output image to be close to the ground truth image in the pixel domain; structural similarity loss is used to constrain the image structure; edge consistency loss is used to constrain the edges of the target region; physical constraint preservation loss is used to limit the network output from deviating excessively from the physically constrained initial reconstructed image; and measurement consistency loss is used to constrain the output image to be consistent with the complete measurement data after projection onto the complete encoding matrix. Through the joint constraints of the above multi-dimensional loss functions, the neural network can learn the ability to perform residual correction on the initial reconstructed image while maintaining physical consistency.

[0067] 209. Based on the initial reconstructed image and residual correction results or thinned image according to physical constraints, output the target reconstructed image.

[0068] Specifically, the initial reconstructed image and residual correction data are synthesized to obtain a synthesized result, and the synthesized result is subjected to measurement consistency verification. When the synthesized result passes the consistency verification, it is used as the target reconstructed image. When the synthesized result fails the consistency verification, the amplitude of the residual correction data is adjusted to obtain updated residual correction data, and the initial reconstructed image and updated residual correction data are re-synthesized to obtain an updated reconstructed image. The deviation between the updated reconstructed image and the complete measurement data after projection through the complete coding matrix is ​​less than a preset threshold.

[0069] In this embodiment, the image output module will initially reconstruct the image based on physical constraints. The target reconstructed image is obtained by synthesizing the residual correction result r from the neural network output. for:

[0070] To reduce the risk of the final reconstructed image deviating excessively from the physical constraints of the initial reconstructed image, the residual correction result r can also be subject to amplitude constraints, edge consistency constraints, or measurement consistency recorrection.

[0071] In one implementation, a measurement consistency check is performed on the synthesized result, i.e., it is determined whether the deviation between the synthesized result after projection onto the complete encoding matrix and the complete measurement data is less than a preset threshold. When the consistency check fails, the amplitude of the residual correction data is adjusted and the image is resynthesized until the deviation between the updated reconstructed image after projection onto the complete encoding matrix and the complete measurement data is less than the preset threshold, thereby ensuring that the final reconstructed image still maintains physical consistency with the actual measurement data after being refined by the neural network. In another implementation, the refined image directly output by the neural network can also be used as the target reconstructed image output. Through the above amplitude limitation and check mechanism, the correction amplitude of the neural network to the initial reconstructed image is limited to a physically acceptable range, effectively reducing the risk of the pure data-driven neural network output deviating from the actual measurement data, so that the final reconstructed image can enjoy the detail enhancement capabilities of the neural network while still being based on physical measurements.

[0072] In a structural foreground image test case, the image size is 64×64, and the total number of measurements is 512. The first stage obtains a preview image and generates a spatial prior map, i.e., a region weight map. The second stage generates an adaptive coding pattern based on the spatial prior map. Then, the initial reconstructed image of physical constraints is obtained through region-weighted total variational solving. The neural network predicts the residual correction result based on this initial reconstructed image, such as... Figure 5As shown, on 100 test samples, the peak signal-to-noise ratio (PSNR) of random compressed sensing total variational reconstruction is approximately 19.38 dB, and the PNR of the target region is approximately 18.47 dB; the PNR of low-frequency orthogonal coding total variational reconstruction is approximately 23.40 dB, and the PNR of the target region is approximately 21.48 dB; the PNR of physical constraint reconstruction, i.e., the initial physical constraint reconstruction image, is approximately 22.95 dB, and the PNR of the target region is approximately 22.66 dB; the PNR of the final reconstruction result of this application is approximately 27.89 dB, and the PNR of the target region is approximately 28.13 dB. Therefore, the test results show that compared to the result using only the initial physical constraint reconstruction image, the final reconstructed image has a significant improvement in both overall quality and target region quality. This indicates that the region adaptive measurement in this invention actually changes the second-stage coding pattern, tilting measurement resources towards the target region, and that neural network refinement can effectively correct residual artifacts and missing local details in physical reconstruction.

[0073] Furthermore, such as Figure 6 As shown, in visual comparison of the same target, the fixed-encoding reconstruction result can recover the general outline of the target, but there is blurring or breakage at the endpoints, intersections, and fine line structures. The pre-sensing map is obtained from the first-stage pre-sensing measurement data through coarse reconstruction, back projection, low-frequency reconstruction, or attention decoding. It is used to provide spatial cues for the approximate location of the target, edge regions, or regions requiring detail, and can be further used to generate the spatial prior map required for the second-stage adaptive encoding. The physical constraint reconstruction result is obtained by solving the physical inverse problem from the complete encoding matrix and complete measurement data. It can retain the main structure consistent with the actual measurement data, but there may still be edge blurring and insufficient local details under low measurement budget. The reconstruction result of this invention further refines the result with neural network residuals based on the physical constraint reconstruction result, making the target boundary and local structure closer to the real target.

[0074] In a natural image test case, the spatial prior map is constructed from low-frequency residuals, local textures, and edge strength. The spatial prior map is not limited to a semantically meaningful foreground segmentation map, but rather used to represent regions requiring detailed information that are difficult to fully express through low-frequency reconstruction. Test results show that this application can improve both the overall peak signal-to-noise ratio (PSNR) and the target region PSNR. This test case demonstrates that the present invention is applicable not only to structural foreground images but also to various imaging scenarios where the preview image can locate the target region or regions requiring detailed information.

[0075] In an extended implementation, when the target under test has known translational, rotational, or affine motion during the measurement process, the processor performs coordinate compensation on the coded pattern or coded matrix according to the motion parameters, and then solves the physical constraint initial reconstruction image and the target reconstruction image based on the compensated coded matrix. This implementation does not require increasing the number of measurements, but incorporates the known motion model into the complete coded matrix, so that the measurement data can be used for physical reconstruction and neural network refinement under a unified reference coordinate, effectively expanding the applicability of the present invention in dynamic scenes.

[0076] like Figure 7 As shown, the adaptive single-pixel imaging system provided by the present invention includes a light source, a spatial light modulator, a target to be measured, a single-pixel detector, an acquisition controller, a processor, and a memory.

[0077] Specifically, the light source provides the illumination beam, and its type can be selected according to the actual imaging scenario, including but not limited to lasers, light-emitting diodes, or broadband light sources. The spatial light modulator is positioned in the output light path of the light source to load the coded pattern and spatially modulate the incident light field. Its specific form can be a digital micromirror device, a liquid crystal spatial light modulator, or other programmable spatial light modulators. The single-pixel detector is used to acquire the integral response of the target to the coded pattern and output one-dimensional measurement data. Its type can be an integrating detector such as a photodiode, avalanche photodiode, photomultiplier tube, infrared unit detector, or terahertz detector, with the specific type determined according to the operating wavelength and sensitivity requirements. The acquisition controller is connected to both the spatial light modulator and the single-pixel detector to synchronize the coded pattern loading timing of the spatial light modulator with the data acquisition timing of the single-pixel detector, ensuring... Each coded pattern projection is precisely synchronized with the corresponding integral sampling. The processor is connected to the acquisition controller and is used to execute the aforementioned adaptive single-pixel imaging method, including outputting the coded pattern to the spatial light modulator, receiving the measurement data output by the single-pixel detector, performing operations such as pre-sensing information acquisition, spatial prior map generation, adaptive coded pattern generation, solving the physical inverse problem, and neural network residual correction. The memory is connected to the processor and is used to store the coded pattern, measurement data, model parameters, and computer program. The above system structure organically integrates optoelectronic devices such as the light source, spatial light modulator, and single-pixel detector with computing resources such as the processor and memory, forming a complete imaging hardware platform. This provides a physical basis for executing the aforementioned adaptive single-pixel imaging method, enabling pre-sensing-guided adaptive measurement and physically constrained neural reconstruction to be realized in a real physical system.

[0078] like Figure 7As shown, the processor further includes a data receiving module, a preview reconstruction module, a weight map generation module, an encoding generation module, an encoding output module, a physical reconstruction module, a neural reconstruction module, and an image output module. The data receiving module receives measurement data output by a single-pixel detector and transmits the measurement data to the preview reconstruction module and the physical reconstruction module. The preview reconstruction module generates a preview image based on the measurement data from the pre-sensing stage and the first-stage encoding matrix. The weight map generation module generates a spatial prior map based on the preview image, the spatial prior map containing weight values ​​corresponding to multiple spatial locations. The encoding generation module generates an adaptive encoding pattern based on the spatial prior map, specifically including acquiring a preset high-resolution encoding pattern and a low-resolution encoding pattern, performing pixel-by-pixel weighted fusion according to the weight values ​​of the spatial prior map, and generating an adaptive encoding pattern. The encoding output module uses... The generated coded pattern is sent to the spatial light modulator for loading via the acquisition controller; the physical reconstruction module solves the physical inverse problem based on the complete measurement data and the complete coding matrix to obtain the initial reconstructed image; the neural reconstruction module inputs the initial reconstructed image, preview image, spatial prior map, and measurement features into a pre-trained neural network to obtain residual correction data; the image output module outputs the final reconstructed image based on the initial reconstructed image and the residual correction data; each of these modules can be a computer program module executed by the processor, or it can be implemented by a graphics processor, a field-programmable gate array, an application-specific integrated circuit, or a combination thereof; through the coordinated work of the above modules, the processor can complete the entire imaging process from measurement data reception, preview reconstruction, weight map generation, adaptive coding generation, coding output, physical reconstruction, neural network refinement to image output.

[0079] Furthermore, as Figure 1 In terms of specific implementation of the method, this application provides an adaptive single-pixel imaging device, such as... Figure 8 As shown, the device includes: an encoding generation module 301, a projection acquisition module 302, an image reconstruction module 303, and an image output module 304.

[0080] The encoding generation module 301 is used to acquire the pre-sensing information of the target to be tested, generate a spatial prior map based on the pre-sensing information, and generate an adaptive encoding pattern based on the spatial prior map. The proportion of high-resolution encoding components at different spatial locations in the adaptive encoding pattern changes adaptively with the weight value of the spatial prior map. The projection acquisition module 302 is used to project the adaptive coding pattern onto the target to be measured, acquire the measurement data of the target to be measured, and combine the measurement data with the pre-sensing information to generate complete measurement data and the corresponding complete coding matrix. Image reconstruction module 303 is used to solve the physical inverse problem based on complete measurement data and complete coding matrix to obtain an initial reconstructed image; The image output module 304 is used to input the initial reconstructed image and the spatial prior image into a pre-trained neural network to obtain residual correction data, and output the target reconstructed image based on the initial reconstructed image and the residual correction data.

[0081] In specific application scenarios, the encoding generation module 301 is specifically used to project an encoded pattern onto the target to be tested, collect the measurement data corresponding to the encoded pattern as pre-sensing information, generate a preview image based on the pre-sensing information, and generate a spatial prior map based on the preview image, wherein the spatial prior map includes weight values ​​corresponding to multiple spatial locations.

[0082] In specific application scenarios, the encoding generation module 301 is also used to acquire a preset high-resolution encoding pattern and a preset low-resolution encoding pattern, perform upsampling operation on the low-resolution encoding pattern to make the spatial resolution of the low-resolution encoding pattern consistent with the spatial resolution of the high-resolution encoding pattern; based on the weight values ​​corresponding to multiple spatial locations in the spatial prior map, the high-resolution encoding pattern and the low-resolution encoding pattern are weighted and fused pixel by pixel to generate an adaptive encoding pattern, wherein the spatial location with the larger the weight value has a higher proportion of the high-resolution encoding component in the adaptive encoding pattern; and perform normalization and decorrelation processing on the adaptive encoding pattern.

[0083] In specific application scenarios, the image reconstruction module 303 is specifically used to construct an objective function, which includes a data consistency term and a regularization term. The data consistency term is used to constrain the initial reconstructed image to be consistent with the complete measurement data after projection onto the complete coding matrix, and the regularization term is used to constrain the spatial characteristics of the initial reconstructed image. The optimal solution of the objective function is solved iteratively, and the optimal solution is used as the initial reconstructed image.

[0084] In specific application scenarios, the image output module 304 is specifically used to construct auxiliary input features, wherein the auxiliary input features include at least one of the back projection results of the complete measurement data, the edge features of the initial reconstructed image, and the multi-scale representation of the spatial prior map; the initial reconstructed image, the spatial prior map, and the auxiliary input features are concatenated into an input tensor, and a pre-trained neural network is used to extract and map features from the input tensor to output residual correction data.

[0085] In specific application scenarios, the image output module 304 is specifically used to synthesize the initial reconstructed image and the residual correction data to obtain the synthesized result, and to perform measurement consistency verification on the synthesized result; when the synthesized result passes the consistency verification, the synthesized result is used as the target reconstructed image; when the synthesized result fails the consistency verification, the amplitude of the residual correction data is adjusted to obtain updated residual correction data, and the initial reconstructed image and the updated residual correction data are re-synthesized to obtain an updated reconstructed image, wherein the deviation between the updated reconstructed image and the complete measurement data after projection through the complete coding matrix is ​​less than a preset threshold.

[0086] It should be noted that other corresponding descriptions of the functional units involved in the adaptive single-pixel imaging device provided in this embodiment can be found in [reference]. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.

[0087] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described adaptive single-pixel imaging method.

[0088] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), including several instructions to enable a computer device (such as a personal computer, server, or network device) to execute the adaptive single-pixel imaging method for various implementation scenarios of this application.

[0089] Based on the above, Figure 1 and Figure 2 The method shown, and Figure 8 The adaptive single-pixel imaging device embodiment shown above, in order to achieve the above objectives, such as... Figure 9 As shown, this embodiment also provides a physical device for adaptive single-pixel imaging. This device includes a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device. The various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory to perform the adaptive single-pixel imaging method described in the above embodiment.

[0090] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0091] Those skilled in the art will understand that the adaptive single-pixel imaging physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0092] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs to be identified. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the technical solution of this application, firstly, a spatial prior map is generated by acquiring the pre-sensing information of the target to be measured, and an adaptive coding pattern is generated based on the spatial prior map. The proportion of high-resolution coding components at different spatial locations in the coding pattern changes adaptively with the weight value, so that the limited total number of measurements can be reasonably allocated according to the target spatial content. When the target area is small or important details are concentrated in a local area, the measurement resources can be adaptively tilted towards such areas, avoiding the consumption of measurement times in areas with low information value, thereby improving the reconstruction quality of the target area and detail area under the same total measurement budget; secondly, the pre-sensing data and adaptive measurement data are combined into complete measurement data and a complete coding matrix, and the initial reconstructed image is obtained by solving the physical inverse problem based on the complete data. The embedded reconstruction process effectively suppresses the impact of preview image quality fluctuations on the initial reconstruction result. Even with low preview image resolution or inaccurate boundaries, the solution of the physical inverse problem still ensures the physical consistency between the reconstructed image and the actual measurement data, providing a reliable foundation for subsequent refinement. Then, the initial reconstructed image and spatial prior image are input into a pre-trained neural network, causing the network to output only residual correction data instead of directly generating the reconstructed image. The target reconstructed image is synthesized from the initial reconstructed image and the residual correction data. This limits the neural network's learning ability to perform detailed corrections on the physical reconstruction result, rather than generating images independently of the measurement data. This reduces the risk of the neural network output deviating from the actual measurement data and ensures a clear physical correspondence between the output and the acquired data. In summary, this application, through the synergistic effect of pre-sensory guided adaptive measurement allocation, physical inverse problem-constrained initial reconstruction, and neural network residual correction, achieves efficient allocation of measurement resources, effective restoration of target region details, and ensures the physical consistency between the target reconstructed image and the actual measurement data without increasing the total number of measurements.

[0094] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0095] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. An adaptive single-pixel imaging method, characterized in that, include: Acquire pre-sensing information of the target to be tested, generate a spatial prior map based on the pre-sensing information, and generate an adaptive coding pattern based on the spatial prior map, wherein the proportion of high-resolution coding components at different spatial locations in the adaptive coding pattern changes adaptively with the weight value of the spatial prior map. The adaptive coding pattern is projected onto the target to be measured, measurement data of the target to be measured is collected, and the measurement data is combined with the pre-sensing information to generate complete measurement data and the corresponding complete coding matrix. Based on the complete measurement data and the complete encoding matrix, the physical inverse problem is solved to obtain the initial reconstructed image; The initial reconstructed image and the spatial prior image are input into a pre-trained neural network to obtain residual correction data, and the target reconstructed image is output based on the initial reconstructed image and the residual correction data.

2. The method according to claim 1, characterized in that, The step of acquiring pre-perception information of the target to be tested and generating a spatial prior map based on the pre-perception information includes: Project an coded pattern onto the target to be measured, and collect the measurement data corresponding to the coded pattern as pre-sensing information; Generate a preview image based on the pre-sensing information; A spatial prior map is generated based on the preview image, wherein the spatial prior map includes weight values ​​corresponding to multiple spatial locations.

3. The method according to claim 1 or 2, characterized in that, The step of generating an adaptive coding pattern based on the spatial prior map includes: Obtain a preset high-resolution encoding pattern and a preset low-resolution encoding pattern, and perform an upsampling operation on the low-resolution encoding pattern to make the spatial resolution of the low-resolution encoding pattern consistent with the spatial resolution of the high-resolution encoding pattern; Based on the weight values ​​corresponding to multiple spatial locations in the spatial prior map, the high-resolution coding pattern and the low-resolution coding pattern are weighted and fused pixel by pixel to generate an adaptive coding pattern. The spatial location with a larger weight value has a higher proportion of high-resolution coding components in the adaptive coding pattern. The adaptive coding pattern is then normalized and decorrelated.

4. The method according to claim 1, characterized in that, The process of solving the physical inverse problem based on the complete measurement data and the complete encoding matrix to obtain the initial reconstructed image includes: Construct an objective function, wherein the objective function includes a data consistency term and a regularization term, the data consistency term is used to constrain the initial reconstructed image to be consistent with the complete measurement data after being projected by the complete coding matrix, and the regularization term is used to constrain the spatial characteristics of the initial reconstructed image; The optimal solution to the objective function is obtained iteratively, and the optimal solution is used as the initial reconstructed image.

5. The method according to claim 1, characterized in that, The step of inputting the initial reconstructed image and the spatial prior map into a pre-trained neural network to obtain residual correction data includes: Construct auxiliary input features, wherein the auxiliary input features include at least one of the back projection results of the complete measurement data, the edge features of the initial reconstructed image, and the multi-scale representation of the spatial prior map; The initial reconstructed image, the spatial prior image, and the auxiliary input features are concatenated into an input tensor. A pre-trained neural network is then used to extract and map features from the input tensor, and residual correction data is output.

6. The method according to claim 1, characterized in that, The step of outputting the target reconstructed image based on the initial reconstructed image and the residual correction data includes: The initial reconstructed image and the residual correction data are synthesized to obtain a synthesized result, and the consistency of the synthesized result is verified by measurement. When the synthesis result passes the consistency check, the synthesis result is used as the target reconstructed image; When the synthesis result fails the consistency check, the amplitude of the residual correction data is adjusted to obtain updated residual correction data, and the synthesis operation is re-executed on the initial reconstructed image and the updated residual correction data to obtain an updated reconstructed image. The deviation between the updated reconstructed image and the complete measurement data after projection by the complete coding matrix is ​​less than a preset threshold.

7. An adaptive single-pixel imaging system, characterized in that, include: A light source, used to provide illumination; A spatial light modulator is disposed in the output light path of the light source and is used to load an encoded pattern and modulate the light field; A single-pixel detector is used to acquire the integral response of the target under test to the coded pattern and output measurement data; A data acquisition controller is connected to the spatial light modulator and the single-pixel detector, respectively, and is used to synchronize the loading of the coded pattern of the spatial light modulator with the data acquisition of the single-pixel detector. A processor, connected to the acquisition controller, is configured to execute the adaptive single-pixel imaging method as described in any one of claims 1 to 6; A memory, connected to the processor, is used to store the coded pattern, the measurement data, and the computer program.

8. An adaptive single-pixel imaging device, characterized in that, include: The encoding generation module is used to acquire pre-sensing information of the target to be tested, generate a spatial prior map based on the pre-sensing information, and generate an adaptive encoding pattern based on the spatial prior map, wherein the proportion of high-resolution encoding components at different spatial locations in the adaptive encoding pattern adapts to the weight value of the spatial prior map. The projection acquisition module is used to project the adaptive coding pattern onto the target to be measured, acquire the measurement data of the target to be measured, and combine the measurement data with the pre-sensing information to generate complete measurement data and the corresponding complete coding matrix. The image reconstruction module is used to solve the physical inverse problem based on the complete measurement data and the complete encoding matrix to obtain the initial reconstructed image; The image output module is used to input the initial reconstructed image and the spatial prior image into a pre-trained neural network to obtain residual correction data, and output the target reconstructed image based on the initial reconstructed image and the residual correction data.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images