Reconfigurable computational imaging method
By introducing reconfigurable coding operators and electro-computation reconstruction algorithms into computational imaging technology, the acquisition and reconstruction of light field information are dynamically controlled, solving the problems of data explosion and system solidification in traditional imaging technology, and realizing efficient and flexible multi-dimensional light field information processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2026-01-30
- Publication Date
- 2026-06-05
AI Technical Summary
Traditional computational imaging technology faces an explosion of data volume when acquiring high-resolution and multi-dimensional light field information, resulting in excessive pressure on sensor bandwidth, computing power and storage system. Furthermore, the system cannot be dynamically adjusted according to different imaging tasks, which limits its applicability and flexibility.
By determining the target encoding operator and the electro-computation reconstruction algorithm from the preset task configuration library based on the target imaging task, a dynamically reconfigurable imaging system is constructed to realize the collaborative design of optical computing and electro-computation, and to dynamically control the acquisition and reconstruction of light field information.
It breaks through the information throughput bottleneck, realizes efficient light field information acquisition and reconstruction under limited sensor capabilities, adapts to different task requirements, and improves the system's flexibility and applicability.
Smart Images
Figure CN122156667A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computational imaging technology, and in particular to a reconfigurable computational imaging method. Background Technology
[0002] With the development of imaging technology, the demand for high-resolution and multi-dimensional light field information is increasing, and traditional optical imaging paradigms are gradually facing bottlenecks. In particular, when a single dimension (such as spatial resolution) is increased to the tens of millions of pixels level, or when multiple dimensions of light field information need to be acquired simultaneously, the amount of data generated grows exponentially, putting enormous pressure on sensor bandwidth, computing power, and storage systems. As a result, traditional imaging systems are unable to meet current needs in terms of efficiency and flexibility.
[0003] Furthermore, to alleviate data throughput pressure and expand imaging dimensions, computational imaging technology has emerged. This technology can achieve compressed information acquisition by introducing specific encodings into the optical path (e.g., microlens arrays for angle encoding, filter arrays for spectral encoding, polarizer arrays for polarization encoding, and diffractive optical elements for phase encoding), and then reconstruct the target light field from the encoded measurements using computational algorithms. However, the aforementioned computational imaging schemes rely on pre-designed optical encoding structures and corresponding static reconstruction algorithms. Once the optical encoding devices are manufactured, their encoding characteristics are fixed, preventing the system's information perception mode from dynamically adjusting to different imaging tasks. This limits the system's applicability in variable environments or diverse tasks. On the computational reconstruction end, using fixed optimization models or pre-trained networks means that when imaging conditions, task objectives, or hardware states change, the algorithm lacks the ability to coordinate with the front-end optical encoding process, reducing overall system performance. Therefore, a reconfigurable computational imaging method is urgently needed to achieve task-oriented, dynamically adjustable, and efficient light field information acquisition and reconstruction, solving the throughput bottleneck and system fixation problems in related technologies. Summary of the Invention
[0004] This disclosure aims to at least partially address one of the technical problems in the related art.
[0005] Therefore, the first objective of this disclosure is to propose a reconfigurable computational imaging method. Based on the target imaging task, the method determines the corresponding target encoding operator and target electro-computation reconstruction algorithm from a preset task configuration library, and obtains the target light field information through the target encoding operator and target electro-computation reconstruction algorithm. Thus, through the collaborative design of optical computing and electro-computation, an imaging system with dynamic reconfigurability can be constructed, realizing efficient light field information acquisition and reconstruction that is task-oriented and dynamically adjustable, and solving the problems of throughput bottleneck and system solidification.
[0006] To achieve the above objectives, a first aspect of this disclosure proposes a reconfigurable computational imaging method, the method comprising:
[0007] The multidimensional light field is modeled as a mathematical function containing multiple dimensions; Based on the target imaging task, the corresponding target configuration parameters are determined from the preset task configuration library. The target configuration parameters include target encoding operators and target electrical computation reconstruction algorithms. The mathematical function corresponding to the multidimensional light field is mapped to the sensor through the target encoding operator to obtain the sensor sampling signal; Based on the sensor sampling signal, the target light field information is obtained through the target electrical calculation and reconstruction algorithm.
[0008] The reconfigurable computational imaging method of this invention may also have the following additional technical features: Optionally, determining the corresponding target configuration parameters from a preset task configuration library based on the target imaging task includes: If the target imaging task is ultrafast imaging, then the target encoding operator is a first encoding matrix that enhances the time dimension, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge. If the target imaging task is medical image analysis, then the target encoding operator is to improve the encoding accuracy of the spectral or polarization dimension, and the target electrical computation reconstruction algorithm is a high-precision regularized reconstruction algorithm. If the target imaging task is environmental perception, then the target encoding operator is a second encoding matrix that guides the balancing of spatial and angular dimensions, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angular prior knowledge.
[0009] Optionally, the first encoding matrix, the second encoding matrix, and the third encoding matrix are linear codes; the step of mapping the mathematical function corresponding to the multidimensional light field to the sensor through the target encoding operator to obtain the sensor sampling signal includes: based on the target encoding operator, mapping the mathematical function corresponding to the multidimensional light field to the sensor through a calculation formula to obtain the sensor sampling signal, wherein the calculation formula is: C = (Light) Let be the operator function, Light be the mathematical function corresponding to the multidimensional light field, and C be the sensor sampling signal.
[0010] Optionally, when the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge, obtaining the target light field information based on the sensor sampling signal through the target electrical computation reconstruction algorithm includes: inputting the sensor sampling signal into the target Transformer network to obtain the target light field information.
[0011] Optionally, the method further includes: Obtain a training dataset containing a large number of multidimensional light field scenes; Constructing sparse prior constraints The hybrid loss function is given by β, where β is the constraint coefficient, L is the regularization operator obtained based on the data prior, and Light is the estimated light field output by the network. The initial Transformer network is trained based on the training dataset and the hybrid loss function to obtain the target Transformer network.
[0012] Optionally, when the target electrical calculation and reconstruction algorithm is a high-precision regularized reconstruction algorithm, the step of obtaining the target optical field information based on the sensor sampling signal through the target electrical calculation and reconstruction algorithm includes: An optimization objective function is constructed based on the Tikhonov regularization principle. The optimization objective function includes a data fidelity term for the sensor sampled signal and an L2 norm regularization term for the multidimensional light field. Solve the optimization objective function to obtain the target light field information.
[0013] Optionally, the optimization objective function is: min Light || C - × (Light) 2 2 + , in, This is a regularization coefficient used to balance data fidelity and noise suppression strength. This represents the channel weighting coefficient.
[0014] To achieve the above objectives, a second aspect of this disclosure provides a reconfigurable computational imaging system, comprising: The modeling module is used to model multidimensional light fields as mathematical functions containing multiple dimensions; The determination module is used to determine the corresponding target configuration parameters from a preset task configuration library based on the target imaging task. The target configuration parameters include a target encoding operator and a target electrical computation reconstruction algorithm. The sampling module is used to map the mathematical function corresponding to the multidimensional light field to the sensor through the target encoding operator to obtain the sensor sampling signal; The reconstruction module is used to obtain the target light field information based on the sensor sampling signal and the target electro-reconstruction algorithm.
[0015] To achieve the above objectives, a third aspect of this disclosure provides a computer storage medium storing computer-executable instructions; when executed by a processor, the computer-executable instructions can realize the reconfigurable computational imaging method as described in the second aspect.
[0016] In summary, the reconfigurable computational imaging method provided in this disclosure includes: determining corresponding target configuration parameters from a preset task configuration library based on the target imaging task; the target configuration parameters including a target encoding operator and a target electro-computational reconstruction algorithm; modeling the multidimensional light field as a mathematical function containing multiple dimensions; mapping the mathematical function corresponding to the multidimensional light field to a sensor through the target encoding operator to obtain a sensor sampling signal; and obtaining the target light field information through the target electro-computational reconstruction algorithm based on the sensor sampling signal. This disclosure can determine the corresponding target encoding operator and target electro-computational reconstruction algorithm from a preset task configuration library based on the target imaging task, and obtain the target light field information through these two algorithms. Therefore, through the collaborative design of optical and electro-computation, an imaging system with dynamic reconfigurability can be constructed, achieving efficient light field information acquisition and reconstruction that is task-oriented and dynamically adjustable, solving the problems of throughput bottlenecks and system solidification.
[0017] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description
[0018] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which: Figure 1 This is a schematic flowchart of a reconfigurable computational imaging method provided in an embodiment of this disclosure; Figure 2 A schematic diagram showing the results of a reconfigurable computational imaging method provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a reconfigurable computational imaging system provided in an embodiment of this disclosure. Detailed Implementation
[0019] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.
[0020] In related technologies, computational imaging requires the acquisition of information in different dimensions (such as spectrum and angle) by connecting multiple independent dedicated encoding modules in series or in parallel. This cascading approach not only increases the system's size, complexity, and light energy loss, but also makes it difficult to design the information coupling between different dimensions, hindering collaborative optimization within a unified framework and necessitating sacrifices in spatial resolution or imaging efficiency.
[0021] Furthermore, the limitations of the aforementioned technologies collectively lead to the common contradiction in the current field of computational imaging: "high-dimensional information explosion – inefficient reconstruction capability." This limits the performance of existing systems in many cutting-edge application scenarios. For example, in ultrafast scientific imaging, it is difficult to simultaneously achieve extremely high temporal resolution and sufficiently high spatial / spectral resolution; in biomedical imaging or remote sensing, it is difficult to efficiently and flexibly acquire high-quality spatial, spectral, and polarization information under limited hardware constraints; in dynamic scenarios such as autonomous driving and industrial inspection, the system struggles to adaptively switch to the optimal imaging mode in real time according to changes in the environment and task. Traditional fixed-architecture computational imaging systems cannot simultaneously meet the requirements of high throughput, high flexibility, and high reconstruction quality in these scenarios.
[0022] In summary, existing computational imaging technologies suffer from throughput bottlenecks and system rigidity issues, making it impossible to achieve efficient acquisition and reconstruction of light field information that is task-oriented and dynamically adjustable.
[0023] The reconfigurable computational imaging method and system of this disclosure will be described in detail below with reference to specific embodiments.
[0024] Figure 1 This disclosure provides a reconfigurable computational imaging method according to an embodiment of the present disclosure. For example... Figure 1 As shown, the method may include the following steps: Step 101: Model the multidimensional light field as a mathematical function containing multiple dimensions.
[0025] In one embodiment of this disclosure, a multidimensional light field can be modeled as a mathematical function Light(x, y, z, θ, φ, λ, S, t) containing multiple dimensions, where (x, y, z) represents space, (θ, φ) represents angle, λ represents spectrum, S represents polarization, and t represents time.
[0026] Step 102: Determine the corresponding target configuration parameters from the preset task configuration library based on the target imaging task. The target configuration parameters include the target encoding operator and the target electrical calculation and reconstruction algorithm.
[0027] In one embodiment of this disclosure, the preset task configuration library includes encoding operators and electro-computation reconstruction algorithms corresponding to imaging tasks. In another embodiment of this disclosure, the encoding operators and electro-computation reconstruction algorithms corresponding to different imaging tasks are also different.
[0028] Specifically, in one embodiment of this disclosure, the method for determining the corresponding target configuration parameters from a preset task configuration library based on the target imaging task may include: if the target imaging task is ultrafast imaging, the target encoding operator is a first encoding matrix that enhances the temporal dimension, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge; if the target imaging task is medical image analysis, the target encoding operator is a second encoding matrix, and the target electrical computation reconstruction algorithm is a high-precision regularized reconstruction algorithm; if the target imaging task is environmental perception, the target encoding operator is a third encoding matrix that guides the balance of spatial and angular dimensions, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge.
[0029] In one embodiment of this disclosure, the first, second, and third encoding matrices can be linear encodings. Furthermore, in one embodiment of this disclosure, before determining the encoding matrices corresponding to different imaging tasks, a predefined set of reconfigurable encoding matrices {A1, A2, ..., A...} can be generated. n}, where each encoding matrix A i A specific weighting strategy for the dimensions of light field information.
[0030] In one embodiment of this disclosure, the encoding strategy can be dynamically determined based on the imaging task requirements. Specifically, in one embodiment, a strategy selection parameter k can be generated based on the resolution requirements of the current imaging task for different dimensions of the light field, and a corresponding encoding matrix A can be selected from a set of reconstructable encoding matrices. The value of k determines the priority of weight allocation for spatial, angular, and spectral dimensions.
[0031] In one embodiment of this disclosure, when the target imaging task is ultrafast imaging, the target imaging task focuses on temporal resolution. In this case, the strategy selection parameter k makes the first encoding matrix A 1 Increase the weighting of the time dimension (t) to prioritize preserving the temporal details of the light field; when the target imaging task is medical image analysis, which focuses on spectral resolution, the strategy selection parameter k makes the second encoding matrix A 2Increase the spectral dimension ( The weight allocation prioritizes the extraction of spectral features of the light field; when the target imaging task is environmental perception, the focus is on spatial resolution, and the strategy selection parameter k makes the third encoding matrix A... 3 Increase the weighting of spatial dimensions (x, y, z) to prioritize the preservation of spatial details of the light field.
[0032] In one embodiment of this disclosure, the optical path modulation characteristics are directly changed by controlling the physical state of the tunable metasurface, tunable interference cavity, liquid crystal spatial light modulator, or electro-optic modulator, so that the transfer function of the physical optical modulation system dynamically matches different coding matrices, thereby obtaining a set of reconfigurable coding matrices.
[0033] In another embodiment of this disclosure, an optimal encoding matrix A that matches the task requirements is solved in the digital domain using an optimization algorithm, thereby obtaining a set of reconfigurable encoding matrices.
[0034] Step 103: The mathematical function corresponding to the multidimensional light field is mapped to the sensor through the target encoding operator to obtain the sensor sampling signal.
[0035] In one embodiment of this disclosure, after obtaining the target encoding operator through the above steps, the mathematical function corresponding to the multidimensional light field can be mapped to the sensor through the target encoding operator to obtain the sensor sampling signal.
[0036] In one embodiment of this disclosure, the method of mapping the mathematical function corresponding to the multidimensional light field to the sensor using a target encoding operator to obtain the sensor sampling signal may include: based on the target encoding operator, mapping the mathematical function corresponding to the multidimensional light field to the sensor using a calculation formula to obtain the sensor sampling signal, wherein the calculation formula may be: C = (Light) Let be the operator function, Light be the mathematical function corresponding to the multidimensional light field, and C be the sensor sampling signal.
[0037] In one embodiment of the present invention, the target encoding operator can be a linear encoding. And, in one embodiment of the present invention, when the target encoding operator is a linear encoding, the above... The operator function can be A Light, that is, C = A Light, where A is the high-dimensional encoding matrix corresponding to the target encoding operator. This represents multidimensional tensor multiplication, Light is the mathematical function corresponding to the multidimensional light field, and C is the sensor sampling signal.
[0038] Step 104: Based on the sensor sampling signal, obtain the target light field information through the target electrical calculation and reconstruction algorithm.
[0039] In one embodiment of this disclosure, after obtaining the sensor sampling signal through the above steps, the target light field information can be obtained based on the sensor sampling signal through a target electrical calculation and reconstruction algorithm.
[0040] Specifically, in one embodiment of this disclosure, when the target electrical calculation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge, the above-mentioned method for obtaining target light field information based on sensor sampling signals through the target electrical calculation reconstruction algorithm may include: inputting sensor sampling signals into the target Transformer network to obtain target light field information.
[0041] In one embodiment of this disclosure, the target Transformer network is obtained through training. Furthermore, in one embodiment of this disclosure, the training method for the target Transformer network may include the following steps: Step 1: Obtain a training dataset containing a large number of multi-dimensional light field scenes; Step 2, Constructing fused sparse prior constraint terms The hybrid loss function is given by β, where β is the constraint coefficient, L is the regularization operator obtained based on the data prior, and Light is the estimated light field output by the network. Step 3: Train the initial Transformer network based on the training dataset and the hybrid loss function to obtain the target Transformer network.
[0042] In one embodiment of this disclosure, the target Transformer network described above can accurately and efficiently reconstruct multidimensional light fields by learning a large number of sparse priors of scenes and data, and learn different priors for different tasks (such as high-resolution spectrum, high-resolution space, high-resolution light field (viewpoint) etc.) to adaptively meet the perception requirements of various tasks.
[0043] Furthermore, in one embodiment of this disclosure, when the target electrical calculation reconstruction algorithm is a high-precision regularized reconstruction algorithm, the above-mentioned method for obtaining target light field information based on sensor sampling signals through the target electrical calculation reconstruction algorithm may include: constructing an optimization objective function based on the Tikhonov regularization principle, wherein the optimization objective function includes a data fidelity term of the sensor sampling signal and an L2 norm regularization term of the multidimensional light field, solving the optimization objective function, and obtaining the target light field information.
[0044] In one embodiment of this disclosure, the above-mentioned optimization objective function can be: min Light || C - × (Light) 2 2 + , in, This is a regularization coefficient used to balance data fidelity and noise suppression strength. This represents the channel weighting coefficient.
[0045] In one embodiment of this disclosure, a closed-loop framework from light field acquisition to task-driven reconstruction is formed through the unified modeling of optical and electrical computing, enabling dynamic adjustment of imaging capabilities based on the environment and task. Compared with traditional methods, this method overcomes the information throughput bottleneck, achieving efficient perception of high-dimensional light fields with limited sensor capabilities; it avoids system capability rigidity, flexibly adapting to different task requirements; and it establishes a unified modeling framework at the theoretical level, independent of specific hardware implementation, possessing broad adaptability and promotion potential.
[0046] In one embodiment of this disclosure, the above-described solution can be applied to transient capture in ultrafast scientific imaging, multimodal diagnosis in medical imaging, hyperspectral information acquisition in remote sensing, and real-time environmental perception in intelligent manufacturing and autonomous driving. Furthermore, in one embodiment of this disclosure, the reconfigurable computational imaging method provides a novel path to overcome the bottlenecks of traditional imaging technologies, playing a crucial role in meeting future diverse and high-performance imaging needs.
[0047] This disclosure provides a reconfigurable computational imaging method. The method includes determining corresponding target configuration parameters from a pre-set task configuration library based on the target imaging task. These target configuration parameters include a target encoding operator and a target electro-computation reconstruction algorithm. A multi-dimensional light field is modeled as a mathematical function containing multiple dimensions. The mathematical function corresponding to the multi-dimensional light field is mapped to a sensor using the target encoding operator to obtain a sensor sampling signal. Based on the sensor sampling signal, the target light field information is obtained through the target electro-computation reconstruction algorithm. This disclosure can determine the corresponding target encoding operator and target electro-computation reconstruction algorithm from a pre-set task configuration library based on the target imaging task, and obtain the target light field information through these two algorithms. Therefore, through the collaborative design of optical and electro-computation, an imaging system with dynamic reconfigurability can be constructed, achieving efficient light field information acquisition and reconstruction that is task-oriented and dynamically adjustable, solving the problems of throughput bottlenecks and system solidification.
[0048] Figure 2 This is a schematic diagram comparing the results of a reconfigurable computational imaging method proposed in an embodiment of this disclosure. Figure 2 As shown, the yellow path represents the computational imaging process in related technologies, while the blue path below represents the computational imaging process of this disclosure. Figure 2 As can be seen, this disclosure expands the number of channels for coded sensing through reconfigurable coding operators, provides richer light field information, realizes a coded light field sensing paradigm for computational imaging, flattens light field information, and achieves efficient extraction of cross-dimensional full light field features.
[0049] Figure 3 This is a schematic diagram of the structure of a reconfigurable computational imaging system proposed in an embodiment of this disclosure. Figure 3 As shown, the system may include: Modeling module 301 is used to model the multidimensional light field as a mathematical function containing multiple dimensions; The determination module 302 is used to determine the corresponding target configuration parameters from the preset task configuration library based on the target imaging task. The target configuration parameters include the target encoding operator and the target electrical calculation and reconstruction algorithm. The sampling module 303 is used to map the mathematical function corresponding to the multidimensional light field to the sensor through the target encoding operator to obtain the sensor sampling signal; The reconstruction module 304 is used to obtain the target light field information based on the sensor sampling signal and the target electrical calculation reconstruction algorithm.
[0050] In one embodiment of this disclosure, the determining module 302 is specifically used for: If the target imaging task is ultrafast imaging, then the target encoding operator is the first encoding matrix with enhanced time dimension, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge. If the target imaging task is medical image analysis, then the target encoding operator is the second encoding matrix, and the target electrical computation reconstruction algorithm is a high-precision regularized reconstruction algorithm. If the target imaging task is environmental perception, then the target encoding operator is a third encoding matrix that guides the balance of spatial and angular dimensions, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angular prior knowledge.
[0051] In one embodiment of this disclosure, the sampling module 303 is specifically used for: Based on the target encoding operator, the mathematical function corresponding to the multidimensional light field is mapped to the sensor through a calculation formula to obtain the sensor sampling signal. The calculation formula is as follows: C = (Light) Let be the operator function, Light be the mathematical function corresponding to the multidimensional light field, and C be the sensor sampling signal.
[0052] In one embodiment of this disclosure, when the target electrical calculation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge, the reconstruction module 304 is specifically used to: input the sensor sampling signal into the target Transformer network to obtain the target light field information.
[0053] In one embodiment of this disclosure, the system is further configured to: Obtain a training dataset containing a large number of multidimensional light field scenes; Constructing sparse prior constraints The hybrid loss function is given by β, where β is the constraint coefficient, L is the regularization operator obtained based on the data prior, and Light is the estimated light field output by the network. The initial Transformer network is trained based on the training dataset and the hybrid loss function to obtain the target Transformer network.
[0054] In one embodiment of this disclosure, when the target electrical calculation reconstruction algorithm is a high-precision regularized reconstruction algorithm, the reconstruction module 304 is specifically used for: The objective function is constructed based on the Tikhonov regularization principle. The objective function includes a data fidelity term for the sensor sampled signal and an L2 norm regularization term for the multidimensional light field. Solve the objective function to obtain the target light field information.
[0055] In one embodiment of this disclosure, the above-mentioned optimization objective function is: min Light || C - × (Light) 2 2 + , in, This is a regularization coefficient used to balance data fidelity and noise suppression strength. This represents the channel weighting coefficient.
[0056] This disclosure provides a reconfigurable computational imaging system. The system determines corresponding target configuration parameters from a pre-set task configuration library based on the target imaging task. These parameters include a target encoding operator and a target electro-computation reconstruction algorithm. The system models the multi-dimensional light field as a mathematical function containing multiple dimensions. The target encoding operator maps the mathematical function corresponding to the multi-dimensional light field to a sensor, obtaining a sensor sampling signal. Based on the sensor sampling signal, the target light field information is obtained through the target electro-computation reconstruction algorithm. This disclosure can determine the corresponding target encoding operator and target electro-computation reconstruction algorithm from a pre-set task configuration library based on the target imaging task, and obtain the target light field information through these two algorithms. Therefore, through the collaborative design of optical and electro-computation, an imaging system with dynamic reconfigurability can be constructed, achieving efficient light field information acquisition and reconstruction that is task-oriented and dynamically adjustable, solving the problems of throughput bottlenecks and system solidification.
[0057] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0058] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.
[0059] This disclosure is intended to provide implementation schemes for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.
[0060] The acquisition, transmission, storage, use, and processing of data in this disclosed technical solution all comply with the relevant provisions of national laws and regulations.
[0061] It should be noted that in the embodiments disclosed herein, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary and are intended only to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used such solutions.
[0062] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0063] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0064] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.
[0065] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.
[0066] It should be understood that various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0067] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.
[0068] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0069] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.
Claims
1. A reconfigurable computational imaging method, characterized in that, The method includes: The multidimensional light field is modeled as a mathematical function containing multiple dimensions; Based on the target imaging task, the corresponding target configuration parameters are determined from the preset task configuration library. The target configuration parameters include target encoding operators and target electrical computation reconstruction algorithms. The mathematical function corresponding to the multidimensional light field is mapped to the sensor through the target encoding operator to obtain the sensor sampling signal; Based on the sensor sampling signal, the target light field information is obtained through the target electrical calculation and reconstruction algorithm.
2. The method according to claim 1, characterized in that, The step of determining the corresponding target configuration parameters from a preset task configuration library based on the target imaging task includes: If the target imaging task is ultrafast imaging, then the target encoding operator is a first encoding matrix that enhances the time dimension, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge. If the target imaging task is medical image analysis, then the target encoding operator is the second encoding matrix, and the target electrical computation reconstruction algorithm is a high-precision regularized reconstruction algorithm. If the target imaging task is environmental perception, then the target encoding operator is a third encoding matrix that guides the balance of spatial and angular dimensions, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angular prior knowledge.
3. The method according to claim 1, characterized in that, The step of mapping the mathematical function corresponding to the multidimensional light field to the sensor using the target encoding operator to obtain the sensor sampling signal includes: based on the target encoding operator, mapping the mathematical function corresponding to the multidimensional light field to the sensor using a calculation formula to obtain the sensor sampling signal, wherein the calculation formula is: C = (Light), where, Let be the operator function, Light be the mathematical function corresponding to the multidimensional light field, and C be the sensor sampling signal.
4. The method according to claim 2, characterized in that, When the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge, the step of obtaining the target light field information based on the sensor sampling signal through the target electrical computation reconstruction algorithm includes: inputting the sensor sampling signal into the target Transformer network to obtain the target light field information.
5. The method according to claim 4, characterized in that, The method further includes: Obtain a training dataset containing a large number of multidimensional light field scenes; Constructing sparse prior constraints The hybrid loss function is given by β, where β is the constraint coefficient, L is the regularization operator obtained based on the data prior, and Light is the estimated light field output by the network. The initial Transformer network is trained based on the training dataset and the hybrid loss function to obtain the target Transformer network.
6. The method according to claim 2, characterized in that, When the target electrical calculation and reconstruction algorithm is a high-precision regularized reconstruction algorithm, the step of obtaining the target optical field information based on the sensor sampling signal through the target electrical calculation and reconstruction algorithm includes: An optimization objective function is constructed based on the Tikhonov regularization principle. The optimization objective function includes a data fidelity term for the sensor sampled signal and an L2 norm regularization term for the multidimensional light field. Solve the optimization objective function to obtain the target light field information.
7. The method according to claim 6, characterized in that, The optimization objective function is: min Light || C - × (Light) ||2 2 + , in, This is a regularization coefficient used to balance data fidelity and noise suppression strength. This represents the channel weighting coefficient.
8. A reconfigurable computational imaging system, characterized in that, The system includes: The modeling module is used to model multidimensional light fields as mathematical functions containing multiple dimensions; The determination module is used to determine the corresponding target configuration parameters from a preset task configuration library based on the target imaging task. The target configuration parameters include a target encoding operator and a target electrical computation reconstruction algorithm. The sampling module is used to map the mathematical function corresponding to the multidimensional light field to the sensor through the target encoding operator to obtain the sensor sampling signal; The reconstruction module is used to obtain the target light field information based on the sensor sampling signal and the target electro-reconstruction algorithm.
9. The system according to claim 7, characterized in that, The determining module is specifically used for: If the target imaging task is ultrafast imaging, then the target encoding operator is a first encoding matrix that enhances the time dimension, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angle prior knowledge. If the target imaging task is medical image analysis, then the target encoding operator is to improve the encoding accuracy of the spectral or polarization dimension, and the target electrical computation reconstruction algorithm is a high-precision regularized reconstruction algorithm. If the target imaging task is environmental perception, then the target encoding operator is a second encoding matrix that guides the balancing of spatial and angular dimensions, and the target electrical computation reconstruction algorithm is a reconstruction algorithm based on a deep learning reconstruction network with spatial-angular prior knowledge.
10. A computer storage medium, wherein, The computer storage medium stores computer-executable instructions; when the computer-executable instructions are executed by the processor, they can implement the method as described in claims 1-7.