Imaging system and method of designing the same

Abstract

The application discloses an imaging system and a design method thereof. The design method comprises the following steps: obtaining first imaging information and first target information, wherein the first imaging information is obtained by imaging a first target object by an imaging system, and the first target information represents actual information of the first target object; determining a loss value based on the first imaging information and the first target information; updating a focus position of an optical element, or updating a design of a processing module, or updating both the focus position of the optical element and the design of the processing module based on the loss value until the loss value meets a preset numerical condition, and obtaining design information of the imaging system. The design method provided by the application can adapt to imaging and processing under complex tasks, and the generated information is comprehensive and accurate.

Description

Technical Field

[0001] This application relates to the field of optical imaging technology, and in particular to an imaging system and its design method. Background Technology

[0002] With the rapid development of imaging technology, target detection and image processing have been widely applied in various industries, including autonomous driving, security systems, and smart grids. However, traditional imaging systems still have many limitations when dealing with complex scenes.

[0003] For example, traditional imaging systems typically have a single focal point, leading to rigidity and limited capabilities: they cannot differentiate between different regions or semantic content in an image, thus failing to meet the needs of visual privacy protection or multi-task imaging. Furthermore, since privacy protection and multi-task imaging require processing modules to participate more in computation, using existing technologies in the design will lead to error accumulation, making it impossible to obtain comprehensive and accurate imaging information. Summary of the Invention

[0004] One of the purposes of this application is to provide a design method for an imaging system to solve the technical problems of imaging systems designed by existing methods being unable to adapt to imaging and processing under complex tasks, as well as the lack of or inaccuracy of imaging information.

[0005] One of the purposes of this application is to provide an imaging system.

[0006] To achieve one of the above objectives, one embodiment of this application provides a design method for an imaging system. The imaging system includes: an optical element for receiving incident light from a target object and focusing it at at least two locations in the image plane to generate an intermediate signal; and a processing module for generating imaging information based on the intermediate signal. The design method includes: obtaining first imaging information and first target information, wherein the first imaging information is obtained by the imaging system imaging a first target object, and the first target information represents the actual information of the first target object; determining a loss value based on the first imaging information and the first target information; updating the focusing position of the optical element, or updating the design of the processing module, or updating both the focusing position of the optical element and the design of the processing module, until the loss value meets a preset numerical condition, thereby obtaining the design information of the imaging system.

[0007] Optionally, the design method includes: determining the point spread function based on the focusing position of the optical element, and determining the first imaging information based on the point spread function and the information of the first target object.

[0008] Optionally, the design method includes: determining a point spread function corresponding to each sub-unit in the optical element, wherein the sub-unit in the optical element corresponds to the focal position of the optical element, and updating the point spread function to update the focal position of the optical element.

[0009] Optionally, the design method includes: determining the point spread function based on the focal position coordinates of the optical element, the light intensity weight of the corresponding focal position, and the basis function.

[0010] Optionally, the basis functions are determined based on the Dirac function, or the basis functions are determined based on the Fraunhofer diffraction function.

[0011] Optionally, the point spread function satisfy: , These are the focal position coordinates. This corresponds to the light intensity weight at the focus position coordinates. It is a scale parameter.

[0012] Optionally, the loss value is determined based on the task loss value and the regularization term, wherein the task loss value is determined based on the task performed by the imaging system, and the regularization term is determined based on the target distribution constraints at the focus position.

[0013] Optionally, the regularization term includes at least one of the following: sparsity regularization term; smoothness regularization term; minimum spacing regularization term.

[0014] Optionally, the sparsity regularization term satisfies: , It is the first regularization parameter. These are the coordinates of the corresponding focus position. The light intensity weight; the smoothness regularization term satisfies: , It is the second regularization parameter. These are the coordinates of the first focal position. These are the coordinates of the second focusing position; the minimum spacing regularization term satisfies: , It is the third regularization parameter. These are the coordinates of the first focal position. These are the coordinates of the second focal position. It is the minimum distance between the coordinates of the first focal position and the coordinates of the second focal position.

[0015] Optionally, the design method includes at least one of the following: adjusting the arrangement of sub-units in the optical element to update the focal position of the optical element, adjusting the position of the sub-units in the optical element to update the focal position of the optical element, and adjusting the nanostructure of the sub-units in the optical element to update the focal position of the optical element, wherein the nanostructure in the sub-units is used to form a metasurface at the optical element.

[0016] To achieve one of the above objectives, one embodiment of this application provides an imaging system, comprising: an optical element for receiving incident light from a target object and focusing it at at least two locations in an image plane to generate an intermediate signal; and a processing module for generating imaging information based on the intermediate signal. The imaging system includes one of the following: the focusing position of the optical element is determined based on a loss value, the loss value is determined based on first imaging information and first target information, the first imaging information is obtained by the imaging system imaging the first target object, the first target information represents the actual information of the first target object, the design of the processing module is determined based on the loss value, the loss value is determined based on the first imaging information and the first target information, the first imaging information is obtained by the imaging system imaging the first target object, the first target information represents the actual information of the first target object, and the design information of the imaging system includes both the focusing position of the optical element and the design of the processing module.

[0017] Optionally, the optical element includes at least two sub-units, and the sub-units of the optical element correspond to the focusing positions of the optical element.

[0018] Optionally, a nanostructure is provided at the sub-unit of the optical element, the nanostructure being used to form a metasurface at the optical element.

[0019] Optionally, the imaging system includes at least one of the following: the focal position of the optical element is determined by configuring the shape of the nanostructure; the focal position of the optical element is determined by configuring the size of the nanostructure; or the focal position of the optical element is determined by configuring the arrangement of the nanostructure.

[0020] Optionally, the processing module is used to implement at least one algorithm model, which is used to achieve the task of at least one imaging system.

[0021] Optionally, the imaging system includes at least one of the following: a constraint setting module for constraining the focus position of the optical element; a discriminator module for identifying the authenticity of intermediate signals input to the processing module; an iterative optimization module for updating the focus position of the optical element and the design of the processing module, or both, by implementing a gradient descent method; and a hardware mapping module for constructing a mapping relationship between the nanostructures in the optical element and the focus position.

[0022] Compared with existing technologies, the imaging system design method provided in this application, by updating at least two focusing positions of optical elements in the image plane, can adapt to task requirements by adjusting the focusing positions when facing complex imaging tasks, thus broadening the application scenarios of the imaging system in multi-task and multi-region imaging; by updating the focusing positions of optical elements and / or the design of processing modules, an end-to-end joint optimization mechanism is established to avoid error accumulation and significantly improve the matching accuracy between imaging information and actual target information, thereby designing a higher-performance imaging system.

[0023] Based on the assumption of weak invariance, the technical solution provided in this application regards the focal position coordinates corresponding to the diffusion function of each sub-unit or each point as an intermediate hidden variable from the micro-nano structure parameters (e.g., the shape, size and arrangement of the nanostructure) to the imaging, opening up a new way of thinking for end-to-end design and overcoming the limitations and errors of end-to-end design based on angular spectrum approximation in the prior art. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the imaging system provided in this application.

[0025] Figure 2 This is a schematic diagram of the focusing position of the optical element provided in this application.

[0026] Figure 3 This is a schematic diagram of the imaging system provided in this application at different distances Z.

[0027] Figure 4 This is a schematic diagram of the design method of the imaging system provided in this application. Detailed Implementation

[0028] The present application will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present application, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of this application.

[0029] It should be noted that the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0030] Furthermore, the terms "first," "second," and "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. There is no necessary correlation between the terms "first," "second," and "third," etc.; for example, the inclusion of "second" in one embodiment provided in this application does not necessarily mean that "first" is included in that embodiment, and so on.

[0031] Imaging system

[0032] This application provides an imaging system 100, such as Figure 1 and Figure 2 As shown.

[0033] Imaging system 100 includes optical element 11. Optical element 11 is used to receive incident light from target object T0. Optical element 11 is focused at at least two locations in image plane I0 to generate intermediate signals.

[0034] The intermediate signal can be an optical signal generated by the optical element 11 focusing the incident light, or it can be an electrical signal generated after the optical signal undergoes photoelectric conversion. The component used to realize photoelectric conversion can be a photosensitive element 13; the imaging system 100 can include a photosensitive element 13. The image plane I0 can be located on the surface of the photosensitive element 13 used to receive signals from the optical element 11; the image plane I0 can be the surface of the photosensitive element 13 close to the optical element 11.

[0035] The photosensitive element 13 can be CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled Device).

[0036] Optical element 11 focuses the incident light signal at at least two focusing positions in the image plane I0. The signals corresponding to the at least two focusing positions can generate a unified intermediate signal, or a first intermediate signal, a second intermediate signal, etc., can be output separately.

[0037] The incident light from the target object T0 can be light that the target object T0 reflects or refracts into the optical element 11. The light reflected or refracted by the target object T0 can be natural light or light output by the light-emitting element; the imaging system 100 may include the light-emitting element.

[0038] The imaging system 100 includes a processing module 12. The processing module 12 is used to generate imaging information based on an intermediate signal. The intermediate signal is generated based on the output of the optical element 11.

[0039] The imaging information can be light field distribution information, electric field distribution information, or image data information. The imaging information can be determined through simulation or through actual imaging operations.

[0040] As mentioned above, the intermediate signal can be the optical signal output of the optical element 11, or it can be an electrical signal generated by photoelectric conversion of the optical signal output of the optical element 11.

[0041] In one embodiment, the focusing position of the optical element 11 is determined based on a loss value. The loss value is determined based on first imaging information and first target information. The first imaging information is obtained by the imaging system 100 imaging a first target object. The first target information represents the actual information of the first target object.

[0042] The focusing position of the optical element 11 can also be configured according to any of the design methods provided below.

[0043] The loss value can specifically be the difference between the first imaging information and the first target information.

[0044] The first target object is any target object used in designing the imaging system 100. The target object actually imaged by the imaging system 100 is the same as the target object corresponding to the first target information. The first target object has known first target information.

[0045] The process of imaging the first target object by the imaging system 100 can be an actual imaging operation or a simulation process.

[0046] The process of determining the focal position of optical element 11 based on the loss value can be achieved by adjusting the focal position, thereby reducing the difference between the updated first imaging information and the first target information. The focal position affects the content of the first imaging information, and the first target information reflects the actual condition of the first target object. Therefore, the focal position corresponding to the converged loss value can constitute a better design for optical element 11.

[0047] In one embodiment, the processing module 12 is designed based on a loss value. The loss value is determined based on first imaging information and first target information. The first imaging information is obtained by an imaging system imaging a first target object. The first target information represents the actual information of the first target object.

[0048] The design of processing module 12 can also be configured according to any of the design methods provided below.

[0049] The design of processing module 12 can be the instructions, programs, or parameter configurations executed by processing module 12. Processing module 12 can be used to implement the algorithm model; the design of processing module 12 can include the parameter and weight configuration of the algorithm model.

[0050] The process of determining the design of the processing module 12 based on the loss value can be achieved by adjusting the above-mentioned configuration of the processing module 12, thereby reducing the difference between the updated first imaging information and the first target information. The above configuration affects the content of the first imaging information, and the first target information reflects the actual condition of the first target object. Therefore, the design of the processing module 12 corresponding to the converged loss value can constitute a better design of the processing module 12.

[0051] In one embodiment, the design information of the imaging system 100 is determined based on a loss value. The loss value is determined based on first imaging information and first target information. The first imaging information is obtained by the imaging system imaging a first target object. The first target information represents the actual information of the first target object.

[0052] The design information for the imaging system 100 can also be configured according to any of the design methods provided below.

[0053] The design information for the imaging system 100 includes both the focusing position of the optical element 11 and the design of the processing module 12.

[0054] The process of determining the design information of the imaging system 100 based on the loss value can be achieved by adjusting the design information to reduce the difference between the updated first imaging information and the first target information. The design information affects the content of the first imaging information, and the first target information reflects the actual condition of the first target object. Therefore, the design information corresponding to the converged loss value can constitute a better design of the imaging system 100.

[0055] Thus, the imaging system 100 provided in this application can maintain relative consistency between the imaging result and the target while achieving multifocal imaging. It can adapt to imaging under complex tasks such as multi-region and multi-task scenarios, and is especially suitable for application scenarios such as optical encryption, thus having a wider range of applications. Furthermore, when optical components and processing modules are designed in a unified manner, error accumulation can be effectively avoided, enabling the imaging system 100 to have higher imaging accuracy.

[0056] In one embodiment, the optical element 11 includes at least two sub-units. For example, Figure 2 It includes the first subunit Z11 and the second subunit Z12.

[0057] The sub-units can be distinguished from each other by differences in structure or shape, or they can be formed by pre-dividing the optical element 11.

[0058] The sub-unit can be used to enable the optical element 11 to focus at at least two locations. The optical element 11 may include a substrate and the sub-unit, with the sub-unit disposed on one surface of the substrate. The sub-unit and the substrate may be structurally distinct. Each sub-unit in the optical element 11 may also be considered as a single unit, with each sub-unit including the substrate and a functional part for achieving focusing.

[0059] The sub-unit can establish an optical field response between the incident and emitted light to achieve optical field manipulation. This optical field response relationship can be expressed by a point spread function (PSF) or a modulation transfer function (MTF). The optical field response may include phase delay, focusing, etc.

[0060] Each subunit can be controlled independently, thereby achieving zoned control of the optical element 11 and local response adjustment. In this way, the optical element 11 effectively realizes discrete optical path control during the imaging process.

[0061] The optical element 11 may include m*m sub-units; the sub-units in the optical element 11 are arranged in an array. Each sub-unit may be a 10μm*10μm square. Each sub-unit in the optical element 11 can be regarded as an independent optical modulation unit, used to construct a specific optical field response.

[0062] exist Figure 2 In the illustrated embodiment, the optical element comprises 2×2 sub-units, specifically including a first sub-unit Z11, a second sub-unit Z12, a third sub-unit Z13, and a fourth sub-unit Z14. The four sub-units can be arranged in an array and have the same or different sizes.

[0063] In one embodiment, the sub-units of the optical element 11 correspond to the focusing positions of the optical element 11.

[0064] For example, the incident light is modulated by the first sub-unit Z11 and focused at a first focusing position, having the first focusing position coordinates; the first focusing position is located in the first region Z21 in the image plane I0.

[0065] For example, the incident light is modulated by the second sub-unit Z12 and focused at the second focusing position, having the second focusing position coordinates; the second focusing position is located in the second region Z22 in the image plane I0.

[0066] For example, the incident light is modulated by the third subunit Z13 and focused at the third focusing position, having the coordinates of the third focusing position; the third focusing position is located in the third region Z23 in the image plane I0.

[0067] For example, the incident light is modulated by the fourth sub-unit Z14 and focused at the fourth focusing position, having the coordinates of the fourth focusing position; the fourth focusing position is located in the fourth region Z24 in the image plane I0.

[0068] Thus, when configuring optical element 11 for light field control, the task of multi-point focusing is distributed among different sub-units, which alleviates the structural configuration pressure of each sub-unit, reduces the structural design complexity of the sub-unit to a certain extent, improves the accuracy, and increases the possibility of manufacturing. If multi-sub-unit configuration is not performed, the overall structure of optical element 11 needs to be specially configured to achieve multi-point focusing, which greatly reduces the feasibility of manufacturing.

[0069] The correspondence between several sub-units and several focal positions can be one-to-one, one sub-unit can correspond to multiple focal positions, or multiple sub-units can correspond to one focal position.

[0070] For example, incident light passing through the first sub-unit Z11 can produce a single focused position in the first region Z21; or, incident light passing through the first sub-unit Z11 can produce two focused positions at two different locations in the first region Z21; or, incident light passing through the first sub-unit Z11 can produce one focused position in the first region Z21 and another focused position in the second region Z22; or, incident light passing through the first sub-unit Z11 can produce one focused position in the first region Z21, and incident light passing through the second sub-unit Z12 can produce one focused position in the first region Z21.

[0071] For example, the Each sub-unit corresponds to the focal position coordinates. Focus position coordinates Within the image plane I0; focus position coordinates Specifically, it can be located on one surface of the photosensitive element 13. ; Focus position coordinates The imaging system 100 can be updated and optimized during the design process.

[0072] When each sub-unit corresponds to a focal position, m=N.

[0073] In one embodiment, a nanostructure is provided at a subunit of the optical element 11.

[0074] The nanostructure at each sub-unit can be pre-fixed and adjusted during the design process. Other physical parameters of each sub-unit itself can also be pre-fixed and adjusted during the design process.

[0075] The imaging system provided in this application has several advantages. First, since the optical element 11 is divided into at least two sub-units, the focusing position coordinates corresponding to the sub-units can be directly adjusted during the design of the imaging system 100, without having to adjust the shape, phase, and other parameters of each nanostructure in the optical element 11 one by one. This significantly reduces the search space and improves training stability. Second, since the updating and optimization targets are the sub-units, it is equivalent to performing partitioned optimization on the optical element 11, which greatly reduces the computational load. Only each sub-unit needs to be optimized, without having to optimize each nanostructure one by one. Third, the fabrication of the nanostructures is mainly carried out through sub-unit exposure (e.g., through electron beam lithography). Dividing the optical element 11 into multiple sub-units can match the manufacturing process and improve the yield.

[0076] The nanostructure is used to form a metasurface at the optical element 11.

[0077] The optical element 11 may include a reconfigurable metasurface; the reconfigurable metasurface may actively change the physical properties of its nanostructures through external stimuli.

[0078] Based on this, optical element 11 can be a metasurface optical element.

[0079] The metasurface optical element refers to an artificial layered material with a size smaller than or approximately equal to the wavelength, which can be regarded as a two-dimensional counterpart of metamaterials. The metasurface optical element can achieve the control of the polarization, phase, amplitude, frequency, propagation mode, and other characteristics of electromagnetic waves through surface subwavelength microstructure units (or metastructure units), thereby realizing characteristics such as beam shaping, beam deflection, superlensing, super holography, optical rotation, anti-reflection and anti-reflection.

[0080] Meanwhile, the metasurface optical element is a subwavelength optical element, which is suitable for current micrometer-scale sensor architectures. Its fabrication process is compatible with mature semiconductor sensor technology, making it highly practical and economical.

[0081] Specifically, the metasurface optical element includes a substrate and multiple microstructure units arranged in an array on the substrate. Each microstructure unit has a nanostructure at its center and / or vertex. The microstructure unit can be a structural unit centered on each nanostructure after the metasurface optical element is divided. Each period of nanostructures constitutes a microstructure unit. The microstructure unit is a close-packed pattern, such as a regular square, regular hexagon, or sector. Each period contains one nanostructure, and the vertex and / or center of the microstructure unit can have a nanostructure. In the case of a regular hexagonal microstructure unit, at least one nanostructure is located at each vertex and center of the hexagon. Similarly, the same applies to sector and square microstructure units.

[0082] The substrate of the metasurface optical element can be selected from materials with similar refractive indices, such as silicon dioxide, BF33, silicon, and polymethyl methacrylate. The nanostructure can be selected from materials such as monomeric silicon (c-Si), polycrystalline silicon (p-Si), amorphous silicon (a-Si), compound semiconductors (such as GaN, GaP, GaAs, SiC, etc.), TiO2, Si3N4, AlSb, AlAs, AlGaAs, AlGaInP, BP, ZnGeP2, and other suitable materials, as well as combinations of the above materials.

[0083] Specifically, nanostructures can be configured as polarization-dependent or polarization-independent structures. Depending on the application, the nanostructure units can be configured as either polarization-dependent or polarization-independent structures. Examples of polarization-independent structures include cylindrical, square prisms, cross-shaped prisms, and square prisms with circular holes. Examples of polarization-dependent structures include elliptical cylinders, rectangular prisms, and hexagonal prisms. Nanostructures can be positive or negative structures. For example, the shapes of nanostructures include cylinders, hollow cylinders, square prisms, and hollow square prisms.

[0084] The metasurface optical element may also include a protective layer covering the nanostructure. The material of the protective layer may be any material with a low refractive index and absorption coefficient in the visible or near-infrared band, such as: silicon dioxide (SiO2), spin-coated glass (SOG), or polymers such as polymethyl methacrylate (PMMA), polydimethylsiloxane (PDMS), polymethylpentene (PMP), and combinations of the above materials, or it may be air (i.e., no protective layer is provided).

[0085] In one embodiment, optical element 11 may be a refractive optical element. In another embodiment, optical element 11 may include a refractive optical element. The refractive optical element may replace the metasurface optical element in the above embodiments. The refractive optical element may include, but is not limited to, lenses or prisms made of materials such as optical glass, optical plastics, and optical crystals.

[0086] In one embodiment, optical element 11 may be a diffractive optical element. In another embodiment, optical element 11 may include a diffractive optical element. The diffractive optical element may be used to replace the metasurface optical element in the above embodiments. The diffractive optical element may include, but is not limited to, two-step or multi-step diffractive optical elements, gratings, Dammann gratings, metasurfaces, holograms, diffusers, phase masks, intensity masks, spatial light modulators, etc.

[0087] In one embodiment, the optical element 11 may be a scattering medium element. In another embodiment, the optical element 11 may include a scattering medium element. The scattering medium element may be used to replace the metasurface optical element in the above embodiments. The scattering medium element may include, but is not limited to, frosted glass.

[0088] When configuring the optical element 11, at least one of the following can be included: for multiple optical elements 11, different combination methods of optical elements 11 and parameters such as the spacing between each optical element 11 can be configured; for the metasurface optical element, parameters such as the arrangement period, material, shape, size, and position coordinates of its nanostructure can be configured; for the refractive optical element, parameters such as refractive index and radius of curvature can be configured; for the diffractive optical element, parameters such as its focal length characteristics, phase function of the diffraction surface, radial radius at the abrupt change of each ring zone of the diffraction surface, ring zone depth of the diffraction surface, and diffraction efficiency can be configured.

[0089] The nanostructure can be a silicon nanopillar, a metal resonator, or the like.

[0090] By controlling the shape, size, and arrangement of each nanostructure, the phase, amplitude, and polarization of incident light can be precisely modulated, thereby customizing the propagation characteristics of the light field (such as focusing, deflection, vortex beams, etc.).

[0091] In one embodiment, the focusing position of the optical element 11 is determined by configuring the shape of the nanostructure.

[0092] For example, the first focusing position can be configured by adjusting the shape of the nanostructure at the first subunit Z11.

[0093] The shapes of the nanostructures in each subunit can be the same or different; the nanostructures in different subunits can be different.

[0094] In one embodiment, the focusing position of the optical element 11 is determined by configuring the size of the nanostructure.

[0095] For example, the first focusing position can be configured by adjusting the size of the nanostructure at the first subunit Z11.

[0096] The nanostructures in each subunit can have the same or different sizes; the nanostructures in different subunits can also be different.

[0097] In one embodiment, the focusing position of the optical element 11 is determined by configuring the arrangement of the nanostructures.

[0098] For example, the first focusing position can be configured by adjusting the arrangement of the nanostructures at the first subunit Z11.

[0099] The arrangement of nanostructures in each subunit can be the same or different; the nanostructures in different subunits can be different.

[0100] The arrangement of the nanostructures may include the overall phase distribution of the nanostructures, or the spacing between adjacent nanostructures, etc.

[0101] In one embodiment, the processing module 12 is used to implement at least one algorithm model.

[0102] In one embodiment, the algorithm model is used to implement at least one task of the imaging system.

[0103] The imaging information generated by the processing module 12 can be imaging information that directly reflects the shape of the target object, or it can be other data information determined after analyzing the shape of the target object.

[0104] The algorithm model may include an input layer, multiple hidden layers, and an output layer.

[0105] The input layer is used to receive input signals corresponding to intermediate signals; specifically, it can be an electrical signal generated after the intermediate signal is converted by photoelectric conversion; when the processing module 12 is used to analyze the shape of the target object, the input layer can be used to receive image data information corresponding to the target object.

[0106] The hidden layer is used for feature extraction and data processing to achieve the task corresponding to the algorithm model.

[0107] The output layer is used to generate the imaging information.

[0108] Processing module 12 can be used to implement various algorithm models. These various algorithm models can be used to achieve different tasks.

[0109] The processing module 12 may also include a base network for extracting image features; the output of the base network may be linked to the input of an algorithm model for implementing a specific task.

[0110] The imaging information generated by the processing module 12 should correspond to the target information. For example, when the imaging information is image data generated by the imaging system 100 imaging the target object, the target information should be image data obtained by the imaging system 100 through clear imaging of the target object. When the imaging information is feature information determined by the imaging system 100 through shape analysis of the target object, the target information can be corresponding target feature information determined by manual or machine annotation.

[0111] The aforementioned imaging information can be used to construct a dataset, and the aforementioned target information can be used to construct a training set corresponding to that dataset.

[0112] In one embodiment, the imaging system 100 includes a constraint setting module. The constraint setting module is used to constrain the focusing position of the optical element 11.

[0113] For example, the constraint setting module can set a specific region or focus point, constraining the focus position of the optical element 11 during the focus position update process, ensuring that it always falls within the specific region or is close to the focus point. This is equivalent to incorporating prior knowledge into the design update process, guiding the optimization and update direction, making the multi-point focusing results more realistic, or meeting specific needs such as encryption and privacy, and accelerating the convergence speed.

[0114] For example, the constraint setting module constrains the first focusing position corresponding to the first sub-unit Z11, so that it is always within the range of the first region Z21.

[0115] In one embodiment, the imaging system 100 includes a discriminator module. The discriminator module is used to distinguish the authenticity of intermediate signals input to the processing module 12.

[0116] The discriminator module can extract depth-sensing features from the input image through successive convolutional layers.

[0117] The discriminator module can be used to balance the privacy protection requirements and feature extraction capabilities of the imaging system 100.

[0118] In one embodiment, the imaging system 100 includes an iterative optimization module.

[0119] In one specific embodiment, the iterative optimization module is used to update the focus position of the optical element 11 by implementing a gradient descent method.

[0120] In one specific embodiment, the iterative optimization module is used to update the design of the processing module by implementing a gradient descent method.

[0121] In one specific embodiment, the iterative optimization module is used to update both the focusing position of the optical element and the design of the processing module by implementing a gradient descent method.

[0122] The iterative optimization module can also be configured to perform parallel optimization of multiple initial designs. The iterative optimization module can also be configured to use simulated annealing or genetic algorithms for auxiliary searching to avoid getting trapped in local optima and to obtain a better design combination of optical element 11 and processing module 12.

[0123] In one embodiment, the imaging system 100 includes a hardware mapping module. The hardware mapping module is used to construct a mapping relationship between nanostructures in optical elements and focusing positions.

[0124] For example, after obtaining a better focusing position, the hardware mapping module can map the coordinates of the focusing position back to the design of the nanostructure in the optical element, which facilitates the subsequent implementation of the structure of the optical element 11 for the focusing position.

[0125] The hardware mapping module can also be configured to further fine-tune the nanostructure design corresponding to the focusing position based on constraints such as manufacturing processes. This can improve the actual performance and manufacturing yield of the final designed optical element 11.

[0126] Furthermore, the distance Z between the optical element 11 and the image plane I0 can also be adjusted accordingly. When the spatial dispersion of multiple focal positions at the image plane I0 is high, the imaging information generated by the imaging system 100 may appear out of focus and unclear. On the one hand, the distance Z between the optical element 11 and the image plane I0 can be adjusted (e.g., reduced) according to the target display effect set by the task to make the imaging information clearer. On the other hand, this out-of-focus state can also be used to achieve physical-level privacy protection without the need for additional optical elements or digital post-processing.

[0127] like Figure 3 As shown, the dispersion of the light spot varies when the distance Z between the optical element 11 and the image plane I0 is different. Specifically, for example, the light spot dispersion is the lowest and the light spot is the most concentrated when the distance Z = 1.50e-0.3m.

[0128] Imaging system design methods

[0129] This application provides a design method for an imaging system, such as Figure 4 As shown.

[0130] The imaging system can be constructed according to any of the technical solutions in this application. The design method provided in this application is used for designing the imaging system.

[0131] In one embodiment, the imaging system 100 as Figure 1 and Figure 2 Configuration. Imaging system 100 includes an optical element 11. The optical element 11 receives incident light from a target object T0. The optical element 11 focuses at at least two locations in the image plane I0 to generate an intermediate signal. Imaging system 100 includes a processing module 12. The processing module 12 generates imaging information based on the intermediate signal. The focusing position of the optical element 11 is determined based on a loss value; or the design of the processing module 12 is determined based on the loss value; or the design information of the imaging system 100 is determined based on the loss value, the design information of the imaging system 100 including both the focusing position of the optical element 11 and the design of the processing module 12. The loss value is determined based on first imaging information and first target information. The first imaging information is obtained by imaging the first target object by imaging system 100. The first target information represents the actual information of the first target object.

[0132] The design method provided in this application includes the following steps.

[0133] Step S1: Obtain first imaging information and first target information.

[0134] The first imaging information is obtained by imaging the first target object by the imaging system.

[0135] The first target information represents the actual information of the first target object.

[0136] Step S2: Determine the loss value based on the first imaging information and the first target information.

[0137] Step S3: Based on the loss value, update the focus position of the optical element, or update the design of the processing module, or update both the focus position of the optical element and the design of the processing module, until the loss value meets the preset numerical conditions, and obtain the design information of the imaging system.

[0138] Thus, by configuring the optical elements to focus at multiple focal points, it is beneficial to achieve complex tasks such as optical encryption. Furthermore, by using a loss value determined by the imaging information and target information, the imaging result maintains consistency with the actual target object, thereby balancing adaptability to complex tasks and the accuracy of the imaging results. Since the loss value is used to update the design of the optical elements and / or processing modules, this constitutes an end-to-end training method. This effectively avoids the accumulation of errors caused by combining separately trained optical elements and processing modules, improving the overall reliability of the system from the source of system design.

[0139] The components, data information or other concepts involved in the design method provided in this application can be interpreted using other solutions in this application, or can be expanded based on other solutions in this application.

[0140] In one embodiment, the design method includes step S11: determining the point spread function based on the focusing position of the optical element.

[0141] In one embodiment, the design method includes step S12: determining the first imaging information based on the point spread function and information about the first target object.

[0142] The above steps may be included in step S1, or set before step S1.

[0143] Each focal position of an optical element can correspond to a set of point spread functions. The point spread function is used to construct the relationship between imaging information and focal position. Therefore, when the difference between imaging information and target information (e.g., expressed by the loss value) does not meet the requirements, the point spread function can be affected by adjusting the focal position coordinates, thereby adjusting the imaging information to approach the target information.

[0144] In step S12, specifically, the information of the first target object may be convolved with the point spread function to determine the first intermediate information corresponding to the intermediate signal, and the first imaging information may be determined based on the first intermediate information.

[0145] The information of the first target object may be a sample image containing the first target object; the first intermediate information may be an intermediate image corresponding to the first target object generated by an imaging system.

[0146] The information of the first target object is defined as follows: The point spread function of the corresponding optical element is defined as follows: Then the first intermediate information It can satisfy: .

[0147] In other embodiments, the first intermediate information can also be obtained directly through imaging by the imaging system.

[0148] In one embodiment, the design method includes the step of determining a point spread function corresponding to each sub-unit in the optical element.

[0149] This step can be included in step S11.

[0150] The sub-units in the optical element correspond to the focusing positions of the optical element.

[0151] In this embodiment, each sub-unit in the optical element can correspond to a focal position, and the optical field response relationship between the sub-unit and the focal position can be expressed by a point spread function. Thus, there is a one-to-one correspondence between the sub-unit, the point spread function, and the focal position.

[0152] In one embodiment, the design method includes the step of updating the point spread function to update the focus position of the optical element.

[0153] This step can be included in step S3.

[0154] In this embodiment, each sub-unit of the optical element has a point spread function, which corresponds to the focus position. The point spread function describes the optical field response relationship, and the imaging information is related to the optical field response relationship. Based on this, the focus position can be configured by updating the point spread function (specifically, by adjusting the parameters in the point spread function).

[0155] In one embodiment, the point spread function can be the final output of the imaging system design method provided in this application; in another embodiment, the design information of the imaging system is the updated point spread function; in yet another embodiment, the design information of the imaging system includes the updated point spread function.

[0156] In one embodiment, the point spread function can be updated based on the loss value. More specifically, optimization methods such as gradient descent can be used to update the point spread function with the goal of converging the loss value or making its value meet preset conditions.

[0157] In one embodiment, the final output of the imaging system design method provided in this application can be other parameters; specifically, for example, the structural information of optical elements. Based on this, the steps may further include: determining the structural information of the optical elements according to the updated point spread function.

[0158] The structural information of the optical element may include at least one of the following: the arrangement of sub-units in the optical element, the position of the sub-units, and the nanostructure of the sub-units.

[0159] The process of updating the point spread function and determining the structural information of the optical element based on the updated point spread function can also be summarized as the process of adjusting the structural information of the optical element.

[0160] In some embodiments, multiple sub-units are densely distributed in the optical element, and the point spread function of the optical element as a whole can also be regarded as a continuous point spread function.

[0161] In one embodiment, the design method includes the step of: determining the point spread function based on the focal position coordinates of the optical element, the light intensity weight of the corresponding focal position, and the basis function.

[0162] This step can be included in step S11.

[0163] Each focusing position of an optical element can have a corresponding point spread function. Since the optical element is used to modulate incident light to focus at least two focusing positions, there are at least two point spread functions corresponding to the optical element.

[0164] When an optical element comprises several sub-units, each sub-unit corresponding to a focal position, each sub-unit can have a corresponding point spread function.

[0165] The optical element has an overall point spread function. The overall point spread function can be determined based on the overall optical field response of the optical element, or it can be determined by superimposing multiple point spread functions corresponding to each focusing position, or it can be determined by superimposing multiple point spread functions corresponding to each sub-unit.

[0166] In one embodiment, the point spread function satisfy:

[0167] ,

[0168] These are the focal position coordinates. This corresponds to the light intensity weight at the focus position coordinates. It is either a basis function corresponding to each focus position or a basis function corresponding to each sub-unit.

[0169] In this embodiment, the point spread function corresponding to the entire optical element can specifically be the superposition of the contributions of all sub-units. In this embodiment, the optical element can be regarded as a combination of several sub-units, each of which has its own characteristics and contributions. These characteristics and contributions are expressed through its own point spread function. The point spread functions of the sub-units are superimposed to produce the point spread function of the entire optical element. The point spread function corresponding to the entire optical element can express the overall effect of the sub-units.

[0170] The basis function can be the point spread function of the sub-unit under aberration-free and noise-free conditions.

[0171] In one embodiment, the basis functions are determined based on the Dirac function.

[0172] For example, .

[0173] This can improve computation speed and simplify the analysis process, making it suitable for scenarios where the imaging system has a relatively large aperture or a very short wavelength, and where diffraction effects can be relatively weakened or ignored.

[0174] In one embodiment, the basis functions are determined based on Fraunhofer diffraction functions.

[0175] For example, . For the sub-unit aperture size, The incident light wavelength, This represents the distance light travels.

[0176] Thus, it has higher fidelity and physical accuracy, and can be used to simulate or analyze the performance of real imaging systems, reflecting the fundamental physical limits determined by the above parameters.

[0177] In one embodiment, the point spread function satisfy:

[0178] ,

[0179] These are the focal position coordinates. This corresponds to the light intensity weight at the focus position coordinates. It is a scale parameter.

[0180] The scale parameter can be a spatial stretching parameter associated with the optical characteristics of the imaging device.

[0181] The values ​​of the scale parameters are obtained through calibration or theoretical models.

[0182] For any embodiment of this application It can also correspond to the first Light intensity weight of each sub-unit.

[0183] For any embodiment of this application, the light intensity weight can be determined based on the coordinates of the beam center.

[0184] For example, the coordinates of the beam center are defined as follows: Then the light intensity weight It can satisfy:

[0185] .

[0186] In one embodiment, the loss value is determined based on the task loss value and a regularization term.

[0187] In one embodiment, the task loss value is determined based on the task performed by the imaging system.

[0188] The imaging system can be used to perform traditional imaging tasks. In this embodiment, the task loss value can be specifically set based on generating a target that approximates the original information; the task loss value may include mean squared error loss and perception loss, etc.

[0189] The imaging system can be used to perform encryption tasks. In this embodiment, the task loss value can be set based on the goal of being unable to recover the original information; the task loss value may include structural similarity loss, edge smoothing loss, and privacy adversarial loss, etc.

[0190] The imaging system can also be used to perform at least one of the following tasks: recognition, classification, anomaly detection, semantic segmentation, and behavior recognition. The recognition task can be designed with objectives such as improving positioning accuracy and accurately determining the category; the task loss value can include cross-entropy loss, localization loss, etc. The classification task can be designed with the goal of analyzing global semantic features; the task loss value can include cross-entropy loss, etc.

[0191] Define the first intermediate information as The first imaging information determined after processing can be Define the task loss value as... Then the task loss value It can satisfy: ; This is the primary target information.

[0192] In one embodiment, the regularization term is determined based on the target distribution constraints of the focus position.

[0193] The regularization term and the task loss value can be superimposed to determine the overall loss value.

[0194] Define the regular expression as Then the overall loss value It can satisfy: . It is the regularization parameter of the regularization term, used to control the overall strength of the regularization term.

[0195] In one embodiment, the regularization term includes a sparse regularization term. This can encourage focused locations to concentrate on key regions.

[0196] In one embodiment, the regularization term includes a smoothing regularization term. This allows the coordinate changes of adjacent sub-cells to be gradual.

[0197] In one embodiment, the regularization term includes a minimum spacing regularization term. This prevents the focal positions corresponding to sub-units from overlapping.

[0198] In one embodiment, the sparsity regularization term satisfies: , It is the first regularization parameter. These are the coordinates of the corresponding focus position. Light intensity weight.

[0199] In one embodiment, the smoothness regularization term satisfies: , It is the second regularization parameter. These are the coordinates of the first focal position. These are the coordinates of the second focusing position. Here, the first focusing position and the second focusing position are adjacent.

[0200] In one embodiment, the minimum spacing regularization term satisfies: , It is the third regularization parameter. These are the coordinates of the first focal position. These are the coordinates of the second focal position. It is the minimum distance between the coordinates of the first focal position and the coordinates of the second focal position. Here, the first focal position and the second focal position can be any two focal positions.

[0201] In one embodiment, the regularization term includes the sum of the sparsity regularization term, the smoothness regularization term, and the minimum spacing regularization term.

[0202] In this embodiment, the regularization term It can satisfy:

[0203] .

[0204] In one embodiment, the design method includes the step of updating the focus position of the optical element based on a gradient descent method.

[0205] In one embodiment, the design method includes the step of updating the design of the processing module based on the gradient descent method.

[0206] In one embodiment, the design method includes the steps of updating both the focus position of the optical element and the design of the processing module based on a gradient descent method.

[0207] In one embodiment, the design method specifically updates the focusing position of the optical element by adjusting the structure of the optical element or the structural information of the optical element.

[0208] At this time, the structure of the optical element or its structural information can be the final output of the design method of the imaging system provided in this application; in one embodiment, the design information of the imaging system is the adjusted structure of the optical element or its structural information; in another embodiment, the design information of the imaging system includes the adjusted structure of the optical element or its structural information.

[0209] In one embodiment, the design method includes the step of adjusting the arrangement of sub-units in an optical element to update the focusing position of the optical element.

[0210] This step can be included in step S3.

[0211] In one embodiment, this step may specifically include: adjusting the arrangement of sub-units in the optical element based on the loss value. In another embodiment, this step may specifically include: determining the arrangement of sub-units in the optical element based on the point spread function.

[0212] In one embodiment, the design method includes the step of: adjusting the position of a sub-unit in an optical element based on a loss value to update the focus position of the optical element.

[0213] This step can be included in step S3.

[0214] In one embodiment, this step may specifically include: adjusting the position of the sub-units in the optical element based on the loss value. In another embodiment, this step may specifically include: determining the position of the sub-units in the optical element based on the point spread function.

[0215] In one embodiment, the design method includes the step of: adjusting the nanostructure of sub-units in an optical element based on a loss value to update the focusing position of the optical element.

[0216] This step can be included in step S3.

[0217] The nanostructures in the subunits are used to form metasurfaces at optical elements.

[0218] In one embodiment, this step may specifically include: adjusting the nanostructure of the sub-units in the optical element based on the loss value. In another embodiment, this step may specifically include: determining the nanostructure of the sub-units in the optical element based on the point spread function.

[0219] In this application, the optical element is divided into at least two sub-units, and therefore the nanostructures disposed in the optical element are also divided accordingly. Each group of nanostructures corresponding to each sub-unit can be adjusted separately to adapt to the adjustment requirements of the focusing position. The sub-units can be regarded as discrete and independent units in terms of structure or function. Therefore, the design method provided in this application also has discrete characteristics, which can reduce the search space and improve the stability of training.

[0220] Based on this, the design method or imaging system provided in the preferred embodiments of this application has advantages over the prior art in the following aspects.

[0221] First, in terms of the number of trainable parameters, existing technologies require optimizing the physical parameters (such as shape and phase) of millions of nanostructures, while this application only requires optimizing... Each focus coordinate parameter (i.e.) The coordinate pairs corresponding to each sub-unit This avoids the need for independent modeling of each nanostructure, and the search space is compressed by several orders of magnitude.

[0222] Second, in terms of physical feasibility, existing technologies require the fabrication of complex subwavelength structures and FDTD (Finite-Difference Time-Domain) or diffraction calculations during the design and manufacturing process. In contrast, the embodiments of this application are based on point spread function, and the optical field response can be calculated and fixed in advance and directly called during the inference stage, reducing real-time computing overhead.

[0223] Third, regarding the differentiability of joint training, existing technologies are difficult to embed training into discontinuous structures and are limited by the non-differentiability of traditional complex variable operations. However, this application can optimize parameters through automatic differentiation (Autograd, such as gradient backpropagation in PyTorch / TensorFlow) to achieve end-to-end training. When the processing module uses a deep neural network (DNN), joint training between the two can also be more natural.

[0224] Fourth, regarding the dependence on optical modeling, this application can directly generate the optical field by superimposing discrete point spread functions, without the need for traditional frequency domain simulations such as angular spectrum / RCWA.

[0225] In summary, the imaging system and its design method provided in this application, by updating at least two focusing positions of optical elements in the image plane, can adapt to task requirements by adjusting the focusing positions when facing complex imaging tasks, thus broadening the application scenarios of the imaging system in multi-task and multi-region imaging; by updating the focusing positions of optical elements and / or the design of processing modules, an end-to-end joint optimization mechanism is established to avoid error accumulation and significantly improve the matching accuracy between imaging information and actual target information, thereby designing an imaging system with better performance.

[0226] Based on the assumption of weak invariance, the technical solution provided in this application regards the focal position coordinates corresponding to the diffusion function of each sub-unit or each point as an intermediate hidden variable from the micro-nano structure parameters (e.g., the shape, size and arrangement of the nanostructure) to the imaging, opening up a new way of thinking for end-to-end design and overcoming the limitations and errors of end-to-end design based on angular spectrum approximation in the prior art.

[0227] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0228] The detailed descriptions listed above are merely specific descriptions of feasible implementation methods of this application and are not intended to limit the scope of protection of this application. All equivalent implementation methods or modifications made without departing from the spirit of the art of this application should be included within the scope of protection of this application.

Claims

1. A method for designing an imaging system, the imaging system comprising: an optical element configured to receive incident light from a target object, focus the incident light at at least two locations in an image plane to generate intermediate signals, and a processing module configured to generate imaging information based on the intermediate signals, the method comprising: obtaining first imaging information and first target information, the first imaging information being generated by imaging a first target object with the imaging system, and the first target information representing actual information of the first target object, determining a loss value based on a difference between the first imaging information and the first target information, and updating a focus position of the optical element, or both the focus position of the optical element and a design of the processing module, based on the loss value, until the loss value satisfies a predetermined numerical condition, to obtain a design of the imaging system.

2. The method of claim 1, wherein the method comprises at least one of: adjusting an arrangement of subunits in the optical element to update the focus position of the optical element, adjusting a position of subunits in the optical element to update the focus position of the optical element, and adjusting a nanostructure of subunits in the optical element to update the focus position of the optical element, the nanostructure of the subunits being configured to form a metasurface at the optical element.

3. The method of claim 1, wherein the method comprises: determining a point spread function based on the focus position of the optical element, and determining the first imaging information based on the point spread function and information of the first target object.

4. The method of claim 3, wherein the method comprises: determining a point spread function corresponding to each subunit in the optical element, the subunits in the optical element corresponding to the focus position of the optical element respectively, and updating the point spread function to update the focus position of the optical element.

5. The method of claim 4, wherein the method comprises: determining the point spread function based on a focus position coordinate of the optical element, an intensity weight corresponding to the focus position, and a basis function.

6. The method of claim 5, wherein the basis function is determined based on a Dirac function, or the basis function is determined based on a Fraunhofer diffraction function.

7. The method of claim 1, wherein the method comprises: determining the loss value based on a task loss value and a regularization term, the task loss value being determined based on a task performed by the imaging system, and the regularization term being determined based on a target distribution constraint of the focus position.

8. The method of claim 7, wherein the regularization term comprises at least one of: a sparsity regularization term, a smoothness regularization term, and a minimum distance regularization term.

9. The method of claim 1, wherein the imaging system comprises: an optical element configured to receive incident light from a target object, focus the incident light at at least two locations in an image plane to generate intermediate signals, and a processing module configured to generate imaging information based on the intermediate signals, and the imaging system comprises at least one of: ​ ​ ​ 2. The design method of claim 1, wherein ​ ​ ​ 3. The method of designing according to claim 2, wherein, ​ ​ ​ 4. The method of claim 2, wherein, ​ ​ ​ ​ ​ ​ The point spread function satisfies: ， is a focus position coordinate, is a light intensity weight corresponding to the focus position coordinate, is a scale parameter. ​ ​ ​ ​ 8. The design method of claim 7, wherein, ​ ​ ​ ​ ​ The sparsity regularizer satisfies: , is a first regularization parameter, is a light intensity weight corresponding to a focus position coordinate . The smoothness regularizer satisfies: , is a second regularization parameter, is a first focus position coordinate, is a second focus position coordinate; The minimum distance regularizer satisfies: , is a third regularization parameter, is a first focus position coordinate, is a second focus position coordinate, is a minimum distance between the first focus position coordinate and the second focus position coordinate.

10. An imaging system characterized by, ​ ​ ​ ​ The focusing position of the optical element is determined based on a loss value aiming to reduce the difference between the first imaging information and the first target information, the loss value being determined based on the difference between the first imaging information and the first target information, the first imaging information being obtained by imaging the first target object by the imaging system, the first target information representing the actual information of the first target object, The design information of the imaging system is determined based on a loss value aiming to reduce the difference between the first imaging information and the first target information, the loss value being determined based on the difference between the first imaging information and the first target information, the first imaging information being obtained by imaging the first target object by the imaging system, the first target information representing the actual information of the first target object, the design information of the imaging system including both the focusing position of the optical element and the design of the processing module; The focusing position of the optical element is updated according to at least one of the following: adjusting the arrangement of sub-units in the optical element, adjusting the position of sub-units in the optical element, adjusting the nanostructure of sub-units in the optical element, the nanostructure of the sub-units being used to constitute a metasurface at the optical element.

11. The imaging system of claim 10, wherein: the optical element includes at least two sub-units, the sub-units of the optical element correspond to the focusing position of the optical element respectively.

12. The imaging system of claim 10, wherein, at least one of the following: the focusing position of the optical element is determined by configuring the shape of the nanostructure; the focusing position of the optical element is determined by configuring the size of the nanostructure; the focusing position of the optical element is determined by configuring the arrangement of the nanostructure.

13. The imaging system of claim 11, wherein: the processing module is configured to implement at least one algorithmic model, the algorithmic model is configured to implement at least one task of the imaging system.

14. The imaging system of claim 11, wherein, at least one of the following: a constraint setting module configured to constrain the focusing position of the optical element, a discriminator module configured to discriminate the authenticity of the intermediate signal input to the processing module, an iterative optimization module configured to update the focusing position of the optical element, the design of the processing module, or both the focusing position of the optical element and the design of the processing module by implementing a gradient descent method, a hardware mapping module configured to construct a mapping relationship between the nanostructure in the optical element and the focusing position.

Images