Camera array based far field fourier ptychographic imaging method and system

Abstract

This application discloses a long-range Fourier stacked imaging method and system based on a camera array, relating to the field of optical imaging technology. The invention employs a spatially parallel multi-camera array approach, achieving spatial sampling instead of temporal sampling, effectively improving imaging speed. Furthermore, precise programmable variable-angle illumination is achieved through a spatial light modulator to ensure that the frequency bands of adjacent sub-cameras overlap in Fourier space. Finally, a macroscopic Fourier stacked reconstruction algorithm integrates multiple small-aperture cameras into an equivalent large-aperture camera. This invention enables a more compact and lightweight overall system, and significantly improves data acquisition efficiency and imaging speed without sacrificing reconstructed image quality.

Description

Technical Field

[0001] This invention belongs to the field of optical imaging technology, and particularly relates to a long-range Fourier stacked imaging method and system based on a camera array. Background Technology

[0002] Fourier Ptychographic Microscopy (FPM) is a fusion of three major technological fields: phase retrieval algorithms, synthetic aperture imaging, and computational imaging. Its core idea is to overcome the traditional optical diffraction limit through frequency domain information synthesis. Macroscopic Fourier Ptychographic Microscopy (Macro-FPM) is an extension of traditional Fourier Ptychographic Microscopy to the macroscopic scale. Its core challenge lies in solving the unique physical limitations of the macroscopic scale.

[0003] This technology is currently still in the laboratory research stage, but it provides a potential technical path for long-distance, non-contact, high-resolution macroscopic imaging.

[0004] Currently, the mainstream long-distance Fourier stacked imaging methods at home and abroad can be divided into two categories: scanning Fourier stacked imaging and camera array Fourier stacked imaging based on deep learning.

[0005] Existing scanning Fourier layering techniques can be broadly categorized into two types: camera-scanning Fourier layering and laser-scanning Fourier layering. The principle of camera-scanning Fourier layering is as follows: Under coherent illumination, the object light from the target propagates through the far field to the camera's aperture plane. Assuming this process satisfies the Fraunhofer diffraction condition, the camera's aperture plane can be considered the target object's spectral plane. A high-precision displacement stage is used to move the entire camera, allowing the camera aperture to perform a layered scan on the object's spectral plane, enabling information from different positions in the object's spectrum to pass through. This yields multiple low-resolution images required for the Fourier layered reconstruction algorithm. Phase retrieval is then used to perform interference on the image plane, synthesizing a large-scale spectrum of the target, thereby improving the target's resolution. The basic principle of the laser scanning Fourier stacking method is similar to that of the original Fourier stacking microscopy, except that the method of generating multi-angle coherent illumination differs. One method involves using variable-angle laser illumination to achieve a relative displacement between the target spectrum and the camera aperture, thereby acquiring low-resolution images of different spectral passbands required for high-resolution reconstruction. Then, phase retrieval is used to perform interference on the image plane to synthesize a large-scale spectrum of the target, thus improving the target's resolution.

[0006] Existing deep learning-based camera array Fourier layer stacking techniques utilize camera arrays to acquire low-resolution images corresponding to discrete small-aperture spectral passbands of objects. These images are used as prior knowledge, and deep learning techniques are then combined to estimate the overall large-aperture spectral components, thereby reconstructing high-resolution images. This approach aims to improve imaging speed, system robustness, and practical application capabilities.

[0007] The drawbacks of existing technologies are that scanning Fourier layered imaging methods require high-precision displacement devices and suffer from low data acquisition efficiency and limited imaging speed. While deep learning-based camera array Fourier layered imaging methods improve imaging speed, they rely on large amounts of training data for model building, and their generalization ability still needs improvement. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a long-range Fourier stacked imaging method and system based on a camera array, achieving a highly compact, high-speed imaging system with a large equivalent aperture and active illumination imaging capabilities.

[0009] The first aspect discloses a long-range Fourier layered imaging method based on a camera array, the method comprising:

[0010] Acquire low-resolution images of the target in different spectral passbands acquired by a camera array, wherein the camera array includes multiple sub-cameras that acquire images synchronously;

[0011] In the current iteration, the target spectrum information and camera aperture function obtained from the previous iteration are used to calculate the [number of iterations]. The camera The spectrum corresponding to the low-resolution image acquired in the second acquisition, where... This indicates the number of times the sub-camera array acquires low-resolution images;

[0012] According to the The camera The low-resolution image acquired in the second acquisition is used as an amplitude constraint to update the image light field amplitude, wherein the image light field is obtained by performing an inverse Fourier transform on the spectrum of the corresponding low-resolution image.

[0013] In the corresponding frequency domain, the target spectral information and camera aperture function are updated using the updated image light field;

[0014] Traverse the target low-resolution image sequence and camera array to complete one iteration update process;

[0015] Repeat the iterations until convergence to obtain a high-resolution super-diffraction image of the target. The convergence condition is that the mean square error of the target spectral information between two adjacent iterations is less than a preset minimum threshold.

[0016] The second aspect discloses a long-range Fourier stacked imaging system based on a camera array, which applies the long-range Fourier stacked imaging method based on a camera array disclosed in any of the claims of the first aspect. The system includes: a point light source, a spatial light modulator, an illumination lens, a target, and an imaging system, wherein the imaging system includes a camera array composed of multiple sub-cameras.

[0017] The spatial light modulator is used to encode and modulate the illumination laser emitted by the point light source to obtain different illumination light fields, and to illuminate the target through the different illumination light fields;

[0018] The camera array is used to acquire low-resolution images of targets in different spectral passbands.

[0019] As can be seen from the above technical solutions, the present invention has the following beneficial effects:

[0020] This invention employs a spatially parallel multi-camera array approach, achieving spatial sampling instead of temporal sampling, effectively improving imaging speed. Furthermore, precise programmable variable-angle illumination is achieved through a spatial light modulator, ensuring that the frequency bands of adjacent sub-cameras overlap in Fourier space. Finally, a macroscopic Fourier stacked reconstruction algorithm is used to integrate multiple sub-cameras into an equivalent large-aperture camera. This invention enables a more compact and lightweight overall system, and significantly improves data acquisition efficiency and imaging speed without sacrificing reconstructed image quality. Attached Figure Description

[0021] Figure 1 The overall flowchart of the long-range Fourier stacked imaging method based on camera array provided by the present invention is shown.

[0022] Figure 2 The diagram shows the effect comparison, where (a) is the imaging scene of three optically rough diffuse reflection targets, (b1–b3) are single original images of all targets, and (c1–c3) are the corresponding high-resolution reconstructed images.

[0023] Figure 3 A schematic diagram of the long-range Fourier stacked imaging system based on a camera array provided by the present invention. Detailed Implementation

[0024] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be thorough and complete.

[0025] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," "up," "down," and similar expressions used herein are for illustrative purposes only and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0027] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The term "and / or" as used herein includes any and all combinations of one or more of the related listed items.

[0028] In one embodiment, the present invention provides a long-range Fourier stacked imaging method based on a camera array, such as... Figure 1 As shown, the specific steps include:

[0029] S101. Acquire low-resolution images of the target in different spectral passbands acquired by the camera array, wherein the camera array includes multiple sub-cameras and acquires images synchronously.

[0030] In this method, the illumination laser emitted from a point light source is encoded and modulated using a spatial light modulator to obtain different illumination light fields. These different fields are then used to illuminate the target, enabling the small-aperture camera array to acquire low-resolution images of targets with different spectral passbands. This illumination scheme provides precise variable-angle illumination to ensure that the frequency bands of adjacent sub-cameras overlap in Fourier space.

[0031] It should be noted that different spectral passbands of the target can be obtained using laser scanning methods. This invention preferably uses a pure phase spatial light modulator (SLM), but theoretically, other types of spatial light modulators can also be used to modulate the illumination light field required for macroscopic Fourier plane imaging, such as amplitude-type spatial light modulators like digital micromirror devices (DMDs). Additionally, reflective or transmissive pure phase spatial light modulators can also be used; this invention does not specifically limit these types.

[0032] In one embodiment, the spatial light modulator target surface is composed of a pixel block matrix. The illumination laser is encoded and modulated by moving phase pattern blocks on different pixel matrix blocks. The encoding and modulation includes translating the phase pattern blocks within a preset range on the spatial light modulator target surface. Different translation distances will modulate different illumination light fields. The phase pattern blocks are used to adjust the size of the illumination spot through phase encoding.

[0033] For example, the size of the phase pattern block is set to 400×400 pixels, with the center of the spatial light modulator (SLM) as the origin. It moves in 20-pixel steps along the X and Y axes, from a preset range of -40 pixels to 40 pixels. The preset range can be set according to actual needs, but it must be smaller than the illumination aperture to avoid limitations imposed by the aperture. (By only...) A single shot can acquire a 25×25 low-resolution image sequence for reconstruction, and the overlap between adjacent measurements can reach approximately 80% in the Fourier domain.

[0034] In one embodiment, the phase code corresponding to the phase pattern block is obtained by calculating the complex amplitude hologram using the GS algorithm, wherein the complex amplitude hologram is the light field of the target illuminated by the illumination laser without being irradiated by the spatial light modulator.

[0035] Specifically, the phase encoding of the phase pattern block is generated using Computer-Generated Holography (CGH). In CGH, it is typically necessary to convert the Complex Amplitude Hologram (CAH) into a Phase-Only Hologram (POH) for display. In fact, in the illumination system of this invention, the light field illuminating the target object itself can be considered as the CAH, while the moving phase-encoded mask is only a part of the POH. Since the CAH has amplitude constraints but no phase constraints, the POH can be calculated. This means that the size and shape of the illumination spot can be freely designed, and the corresponding POH can then be generated using iterative algorithms (such as the Gerchberg-Saxton (GS) algorithm, the Wirtinger algorithm, and non-convex optimization algorithms).

[0036] The specific steps include:

[0037] Acquire a complex amplitude hologram of the target illuminated by the illumination laser, and extract the phase of the complex amplitude hologram as the initial phase;

[0038] The initial phase is extracted from the CAH, as shown in the following formula (1):

[0039] (1)

[0040] in, The complex amplitude of the complex amplitude hologram. It is a complex argument function. Indicates the initial phase. This represents the initialized pure phase hologram (amplitude normalized to 1). It is the imaginary unit.

[0041] Next, from the holographic surface to the object surface: perform a Fourier transform, as shown in the following formula (2):

[0042] (2)

[0043] in, Indicates the first The complex amplitude distribution of the holographic plane in the next iteration. Two-dimensional Fourier transform operator, This represents the complex amplitude distribution of the object plane after transformation.

[0044] Furthermore, the amplitude of the object surface is constrained: the phase is preserved, and the amplitude is replaced with the target amplitude. (The amplitude of the object reconstructed by CAH) is shown in the following formula (3):

[0045] (3)

[0046] in, This indicates the extraction of phase information from the current object surface light field. This represents the complex amplitude of the object surface after constraint correction.

[0047] Furthermore, from the object surface to the holographic surface: perform an inverse Fourier transform, as shown in the following formula (4):

[0048] (4)

[0049] in, This represents the complex amplitude distribution of the corrected holographic surface. {} denotes the two-dimensional inverse Fourier transform operator.

[0050] Furthermore, holographic amplitude constraint is applied, with the amplitude forcibly set to 1 while preserving the phase, as shown in the following formula (5):

[0051] (5)

[0052] in, Indicates the first The complex amplitude distribution of the holographic plane in the next iteration. This represents the phase information of the complex amplitude distribution of the corrected holographic surface.

[0053] Repeat the iteration 2–5 times until the reconstruction error is small enough to achieve convergence, thus obtaining POH, i.e., phase encoding.

[0054] When using a pure phase spatial light modulator (SLM), the actual illumination field consists of three components: first, the light corresponding to the shifted phase pattern block loaded on the SLM; second, the light corresponding to other zero-phase regions of the SLM; and third, the unmodulated zero-order diffracted light. The latter two components converge into a central bright spot in the actual illumination. By adding a blazed grating phase to the shifted phase pattern block, this central bright spot can be separated from the illumination light on the SLM corresponding to the shifted phase pattern block. In practice, the illumination light corresponding to the shifted phase pattern block is generally used to illuminate the object so that subsequent analysis focuses only on this component of the light field.

[0055] Furthermore, the imaging end consists of multiple small-aperture cameras, which are ultimately integrated into an equivalent large-aperture camera. For example, a 5×5 camera array can acquire 25 low-resolution images simultaneously in each shot. The spectra of these images in Fourier space are adjacent but do not overlap, collectively forming the spectral passband of the camera array. Illumination at different angles causes the entire spectrum of the object to shift relative to the spectral passband of the camera array, thus ensuring that the acquired low-resolution images have sufficient overlap in the spectrum. For example, under different illumination conditions, the proposed camera array can achieve 80% spectral overlap and a synthetic aperture ratio (SAR) of 5 times with only 25 shots. The SAR is defined as the ratio of the synthetic aperture diameter to the lens aperture diameter. To achieve the same SAR and spectral overlap as the proposed camera array Fourier stacked imaging scheme, a single-camera scan typically requires 625 shots. Moreover, it is evident that, compared to a camera array, under the same spectral overlap conditions, the number of shots required by a single camera increases significantly with the increase in the required SAR value, exhibiting a near exponential growth trend.

[0056] In addition, it should be noted that for macroscopic Fourier stacked imaging of target objects with rough optical surfaces, far-field diffraction conditions are not necessary, which is a crucial theoretical component of the methodology of this invention.

[0057] The following will demonstrate, through a combination of theory and simulation experiments, that the target object will be coherently imaged directly by sub-cameras at different locations. This process can be formally expressed as the following formula (6):

[0058] (6)

[0059] in, and These represent the Fourier transform and the inverse Fourier transform, respectively. The distance from the target object to the imaging lens is represented by , and the second-order phase term introduced by free diffraction at this distance is denoted as . The aperture function of an imaging lens is expressed as: ,vector For frequency coordinates, in addition, Indicates the number of cameras in the array The position of each camera, vector These are spatial coordinates. Indicates the target object. Indicates the wavelength of the illumination laser. Indicates the first Aperture function of individual cameras Indicates the first Low-resolution images of the target captured by a single camera.

[0060] It should be noted that the image offset between the sub-cameras is not considered here. Assuming that these images have been registered, it can be seen from formula (6) that the purpose of using convergent illumination in camera scanning Fourier Ptychographic (FP) imaging is to eliminate the secondary phase term. Thus, the far-field condition is approximately satisfied. Only when this condition is satisfied will the lens aperture naturally become the aperture constraint on the Fourier plane of the target object. In fact, a virtual target can be defined as follows, as shown in the following formula (7):

[0061] (7)

[0062] Among them, virtual targets , Indicates the goal.

[0063] Furthermore, equation (6) can be rewritten in the form of the following equation (8):

[0064] (8)

[0065] This indicates that if camera scanning or camera arrays do not employ convergent illumination, the high-resolution information recovered by the reconstruction algorithm actually belongs to virtual targets. For objects with optically rough surfaces, their phase is typically random, and this phase information is usually not of interest. Instead, the primary concern is the intensity information of the imaged target object. For optically rough, diffuse-reflective objects, FP technology can be used to image virtual objects containing a quadratic phase term. The image intensity obtained by synthetic aperture restoration is similar to that of the real object. There is no significant difference between them. Furthermore, the Fourier spectra and small-aperture images corresponding to optically rough diffuse reflection objects with and without a second-phase term are very similar. Therefore, in this case, virtual objects... Setting the goal as high-resolution Fourier stack reconstruction is reasonable.

[0066] When the phase pattern block on the spatial light modulator is translated, the resulting illumination field can be described by the following formula (9) according to the properties of the Fourier transform:

[0067] (9)

[0068] in, The translation distance of the phase pattern block. Represents the spatial coordinates of the phase field. The illumination field, representing the change in illumination field due to modulation by the spatial light modulator, is the first... The illumination field obtained by secondary coding modulation Represents the illumination light field. Indicates the translation distance of the phase pattern block The corresponding phase field, and The system coefficients representing the overall lighting and imaging are shown in formulas (10) and (11) below:

[0069] (10)

[0070] (11)

[0071] in, This indicates the distance between the spatial light modulator and the illumination lens. Indicates the wavelength of the illumination laser. The imaginary unit, This indicates the distance between the point light source and the illumination lens. This indicates the distance from the illumination lens to the imaging plane, i.e., the illumination distance.

[0072] When modulated light illuminates the target object, a complete imaging model can be derived by combining equations (6) and (9):

[0073] (12)

[0074] in, Indicates the first The camera in the Low-resolution images captured when illumination light is modulated using a spatial light modulator. For the first Aperture function of individual cameras.

[0075] Subsequently, a virtual target is redefined as follows (13):

[0076] (13)

[0077] The imaging system directly detects the overall complex field of the object under illumination, denoted as . It is related to virtual targets There is a quadratic phase term. Therefore, based on the previous analysis, when dealing with optically rough diffuse reflective objects, virtual targets should be used. Setting high-resolution FP reconstruction as the target is reasonable.

[0078] Substituting equation (13) into equation (12), the entire forward imaging model can be expressed as the following formula (14):

[0079] (14)

[0080] in, It is a virtual object Fourier spectrum, The estimated result representing the target spectrum information is then shifted. The result.

[0081] Based on the forward imaging model derived from the analysis, it can be seen that the reconstruction algorithm required by this invention is essentially aimed at solving the phase recovery problem of Fourier stacking technology, while being constrained by the constraint of overlapping spectra of different sub-apertures.

[0082] In one embodiment, it further includes: initial estimation of target spectral information and camera aperture function.

[0083] The initial estimate of the target spectrum information is obtained by the following formula (15):

[0084] (15)

[0085] in, For image space coordinates, For spectral space coordinates, Indicates the first The second shoot Low-resolution images acquired by a small-aperture camera This indicates that the average value is taken from all the acquired low-resolution images. This represents the inverse Fourier transform. This represents the target spectral information estimated initially.

[0086] In one embodiment, the initial estimate of the camera aperture function is given by formula (16):

[0087] (16)

[0088] in, This represents the initial estimated camera aperture function. For spectral space coordinates, This indicates the aperture size of the camera.

[0089] S102. In the current iteration, calculate the 1st iteration based on the target spectrum information and camera aperture function obtained in the previous iteration. The camera The spectrum corresponding to the low-resolution image acquired in the second acquisition, where... This indicates the number of times the sub-camera array acquires low-resolution images;

[0090] Specifically, calculate the first The camera Low-resolution images acquired in the second acquisition The corresponding spectrum is calculated using formula (17):

[0091] (17)

[0092] in, The estimation results represent the target's spectral information. This represents the spectrum corresponding to the low-resolution image. The estimated result represents the camera aperture function. Represents the spatial coordinates of the spectrum. For the first Translation distance of the phase pattern block during the next shot Indicates the illumination wavelength. This represents the system coefficient of the lighting system. Indicates the first The position of the individual camera Indicates the first The estimation results of the target spectrum information in the next iteration, then moved The result.

[0093] S103, according to the... The camera The low-resolution image acquired in the second acquisition is used as an amplitude constraint to update the image light field amplitude, wherein the image light field is obtained by performing an inverse Fourier transform on the spectrum of the corresponding low-resolution image.

[0094] Specifically, using the actual input of the first The camera The low-resolution image acquired in the second acquisition is used as an amplitude constraint to update the image light field amplitude; the update formula (18) is:

[0095] (18)

[0096] in, Represents the number of iterations. Indicates the first In the nth iteration The camera The light field corresponding to the low-resolution image acquired in the second acquisition. Representing the In the nth iteration The camera The light field corresponding to the low-resolution image acquired in the second acquisition is determined by the light field of the first acquisition. In the nth iteration The camera Spectrum corresponding to the low-resolution image acquired in the second acquisition It is derived from the inverse Fourier transform, which satisfies formula (19):

[0097] (19)

[0098] S104. In the corresponding frequency domain, update the target spectrum information and camera aperture function using the updated image light field.

[0099] Specifically, returning to the frequency domain, the updated image light field is used to simultaneously update the target spectrum information and the camera aperture function, and the update formulas (20) and (21) are:

[0100] (20)

[0101] (twenty one)

[0102] in, Representing the The estimation results of the target spectrum information in the next iteration. , Indicates conjugate computation. This indicates the calculation of the maximum value. Indicates the first The pupil function estimated in the next iteration Translation As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. As a result, Indicates the first The estimation results of the object's spectral information in the next iteration are then translated. As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. The result.

[0103] S105. Traverse the target low-resolution image sequence and camera array to complete one iteration update process;

[0104] Specifically, the target's spectral information is updated by using low-resolution images captured by each sub-camera each time, thus allowing the estimation of the target's wide-range spectrum.

[0105] S106. Repeat the iteration until convergence to obtain the target super-diffraction high-resolution image. The convergence condition is that the mean square error of the target spectral information in two adjacent iterations is less than a preset minimum threshold.

[0106] Specifically, during the repeated iterations until convergence, when two adjacent iterations are... and mean square error Less than the preset minimum threshold, which is usually set to 0. .

[0107] To verify the effectiveness of this invention, three optically rough diffuse reflection objects were selected for imaging experiments, including a mark on an optical beam expander (BE02-05-A, THORLABS), a 5 RMB banknote, and a miniature plastic lollipop. As shown in the figure, Figure 2 (a) shows the imaging scene of the three imaging targets. Figure 2 (b1-b3) show the original image capture results for each object, which are in high agreement with conventional imaging results obtained under uniform laser illumination conditions. Figure 2 (c1-c3) show the results obtained by reconstruction using the method provided by the present invention, which fully demonstrates the practical effect of the method.

[0108] Furthermore, for coherent imaging of optically rough, diffusely reflective objects, a small camera aperture not only leads to severe diffraction blurring but also produces large-sized imaging speckle patterns. This is evident in... Figure 2 In (b1-b3), it is evident that the details of the three objects are almost completely obscured by speckle noise and diffraction blur, making them difficult to discern. In coherent optical imaging systems employing conventional small-aperture configurations, two fundamental limitations arise for macroscopically diffuse objects: (1) significant spatial blurring caused by diffraction; and (2) the formation of large-scale speckle patterns. These artifacts... Figure 2 As demonstrated in (b1-b3), the unique features of an object are effectively masked by the superposition of speckle noise and diffraction effects, making structural details below the speckle correlation length visually indistinguishable. Using the FP reconstruction technique of the camera array of this invention, the theoretical synthetic aperture of the system is significantly enlarged compared to the original aperture. Therefore, the blurring effect caused by diffraction and the resulting speckle size are significantly reduced, thus contributing to the clear presentation of a large number of fine image details, such as... Figure 2 As shown in (c1-c3).

[0109] This invention employs a spatially parallel multi-camera array approach, enabling spatial sampling to replace temporal sampling and effectively improving imaging speed. Furthermore, the spectral scanning utilizes a spatial light modulator, eliminating the need for high-precision displacement devices. This results in a system without any moving mechanical parts, exhibiting high stability and effectively mitigating diffraction blurring caused by limited optical apertures in macroscopic imaging.

[0110] Furthermore, this invention, based on programmable illumination and a camera array, achieves precise programmable variable-angle illumination through a spatial light modulator, ensuring that the frequency bands of adjacent sub-cameras overlap in Fourier space. Finally, using a macroscopic Fourier stacked reconstruction algorithm, multiple small-aperture cameras are integrated into an equivalent large-aperture camera. Compared to direct imaging, the proposed camera array (FP) achieves approximately five times higher resolution, a figure perfectly consistent with theoretical values. Simultaneously, the reconstruction results are comparable to those obtained using typical camera scanning FP methods. This demonstrates that this invention enables a more compact and lightweight overall system, significantly improving data acquisition efficiency and imaging speed without sacrificing reconstructed image quality.

[0111] In one embodiment, the present invention also provides a long-range Fourier layered imaging system based on a camera array, applying the long-range Fourier layered imaging method based on a camera array described above, such as... Figure 3 As shown, the system includes: a point light source, a spatial light modulator, an illumination lens, a target, and an imaging system, wherein the imaging system includes a camera array composed of multiple sub-cameras;

[0112] The spatial light modulator is used to encode and modulate the illumination laser emitted by the point light source to obtain different illumination light fields, and to illuminate the target through the different illumination light fields;

[0113] The camera array is used to acquire low-resolution images of targets in different spectral passbands.

[0114] Specifically, when laser light is emitted from a point source laser, it passes through a polarizer and then enters the center of the target surface of the spatial light modulator. The phase pattern block loaded on the spatial light modulator is translated to modulate the incident light. The modulated outgoing light field first passes through an illumination lens, and then propagates through Fraunhofer diffraction to form an illumination light field at the conjugate position of the point source, illuminating the surface of the target object. Finally, a camera array is used to acquire low-resolution images of different spectral passbands of the target from a distance, and the reconstruction algorithm is combined to synthesize a large-scale spectrum of the target, thereby improving the target resolution.

[0115] This application also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to provide the camera array-based long-range Fourier stacked imaging method provided in the above-described method embodiments.

[0116] Furthermore, the electronic device may participate in or include the apparatus or system provided in the embodiments of this application. The electronic device may include one or more processors (processors may include, but are not limited to, processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs), memory for storing data, and transmission devices for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera.

[0117] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits can be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits can be a single, independent processing module, or integrated, in whole or in part, into any other element within the device (or mobile device). As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0118] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the above-mentioned data processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to electronic devices via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0119] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the device's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0120] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of an electronic device (or mobile device).

[0121] This application also provides a computer storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the long-range Fourier stacked imaging method based on a camera array provided in the above method embodiments.

[0122] Optionally, in this embodiment, the aforementioned computer storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the aforementioned storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0123] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer storage medium. The processor of an electronic device reads the computer instructions from the computer storage medium and executes the computer instructions, causing the electronic device to perform the long-range Fourier stacked imaging method based on a camera array provided in the above-described method embodiments.

[0124] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] It should be understood that the above description of the preferred embodiments is quite detailed and should not be construed as a limitation on the scope of protection of the present invention. Those skilled in the art can make substitutions or modifications under the guidance of the present invention without departing from the scope of protection of the claims of the present invention, and all such substitutions or modifications fall within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the appended claims.

Claims

1. A long-range Fourier layered imaging method based on a camera array, characterized in that, The method includes: Acquire low-resolution images of the target in different spectral passbands acquired by a camera array, wherein the camera array includes multiple sub-cameras that acquire images synchronously; In the current iteration, the target spectrum information and camera aperture function obtained from the previous iteration are used to calculate the [number of iterations]. The camera The spectrum corresponding to the low-resolution image acquired in the second acquisition, where... This indicates the number of times the camera array acquires low-resolution images; According to the The camera The low-resolution image acquired in the second acquisition is used as an amplitude constraint to update the image light field amplitude, wherein the image light field is obtained by performing an inverse Fourier transform on the spectrum of the corresponding low-resolution image. In the corresponding frequency domain, the target spectral information and camera aperture function are updated using the updated image light field; Traverse the target low-resolution image sequence and camera array to complete one iteration update process; Repeat the iterations until convergence to obtain a high-resolution super-diffraction image of the target. The convergence condition is that the mean square error of the target spectral information between two adjacent iterations is less than a preset minimum threshold.

2. The long-range Fourier stacked imaging method based on camera array according to claim 1, characterized in that, The method further includes: By encoding and modulating the illumination laser emitted by the point light source using a spatial light modulator, different illumination light fields are obtained. The target is then illuminated using these different illumination light fields, enabling the camera array to acquire low-resolution images of targets with different spectral passbands.

3. The long-range Fourier stacked imaging method based on a camera array according to claim 2, characterized in that, The spatial light modulator target surface is composed of a pixel block matrix. The illumination laser is encoded and modulated by moving phase pattern blocks on different pixel matrix blocks. The encoding and modulation includes translating the phase pattern blocks within a preset range on the spatial light modulator target surface. Different translation distances will modulate different illumination light fields. The phase pattern blocks are used to adjust the size of the illumination spot through phase encoding.

4. The long-range Fourier stacked imaging method based on a camera array according to claim 3, characterized in that, The phase code corresponding to the phase pattern block is obtained by calculating the complex amplitude hologram using the GS algorithm. The complex amplitude hologram is the light field of the target illuminated by the illumination laser without being irradiated by the spatial light modulator.

5. The long-range Fourier stacked imaging method based on a camera array according to claim 4, characterized in that, The method further includes: Initial estimation of target spectral information and camera aperture function.

6. The long-range Fourier stacked imaging method based on a camera array according to claim 5, characterized in that, The initial estimate of the camera aperture function is obtained by the following formula: （16） in, This represents the initial estimated camera aperture function. For spectral space coordinates, This indicates the aperture size of the camera.

7. The long-range Fourier stacked imaging method based on a camera array according to claim 4, characterized in that, The calculation of the first step is based on the target spectrum information obtained from the previous iteration and the camera aperture function. The camera The spectrum corresponding to the low-resolution image acquired in this acquisition includes: （17） in, Indicates the number of iterations. Represents the m-th iteration. The camera The spectrum corresponding to the low-resolution image acquired in the second acquisition. Represents the spatial coordinates of the spectrum. For the first Translation distance of the phase pattern block during the next acquisition Indicates the illumination wavelength. This represents the system coefficient of the lighting system. Indicates the first The position of the individual camera Indicates the first The aperture function corresponding to each camera express The result.

8. The long-range Fourier stacked imaging method based on a camera array according to claim 4, characterized in that, The target spectral information is updated using the updated image light field, obtained through the following formula: （20） in, Represents the number of iterations. Representing the The estimation results of the target spectrum information in the next iteration. , Represents the spatial coordinates of the spectrum. For the first Translation distance of the phase pattern block during the next shot Indicates the illumination wavelength. This represents the system coefficient of the lighting system. Indicates the first The position of the individual camera Indicates conjugate computation. This indicates the calculation of the maximum value. This represents the spectrum corresponding to the low-resolution image. The estimated result represents the camera aperture function. Indicates the first The pupil function estimated in the next iteration Translation As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. The result.

9. The long-range Fourier stacked imaging method based on a camera array according to claim 4, characterized in that, The camera aperture function is updated using the updated image light field, obtained by the following formula: （21） in, Represents the number of iterations. Representing the The estimation results of the camera aperture function in the next iteration. , The estimation results representing the object's spectral information. This represents the spectrum corresponding to the low-resolution image. Represents the spatial coordinates of the spectrum. For the first Translation distance of the phase pattern block during the next acquisition Indicates the illumination wavelength. This represents the system coefficient of the lighting system. Indicates the first The position of the individual camera Indicates conjugate computation. This indicates the calculation of the maximum value. Indicates the first The estimation results of the target spectrum information in the next iteration are then shifted. As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. As a result, Indicates the first The camera The low-resolution image acquired in the first acquisition was in the second acquisition. The estimated spectrum corresponding to the next iteration is then shifted. The result.

10. A long-range Fourier layered imaging system based on a camera array, applied to the long-range Fourier layered imaging method based on a camera array according to any one of claims 1-9, characterized in that, The system includes: a point light source, a spatial light modulator, an illumination lens, a target, and an imaging system, wherein the imaging system includes a camera array composed of multiple sub-cameras; The spatial light modulator is used to encode and modulate the illumination laser emitted by the point light source to obtain different illumination light fields, and to illuminate the target through the different illumination light fields; The camera array is used to acquire low-resolution images of targets in different spectral passbands.

Images