Multi-task integrated optical vision processing system and preparation method thereof

Abstract

The invention discloses a multi-task integrated optical visual processing system and a preparation method thereof. A diffractive optical metasurface is used for carrying out random phase modulation on an input light field, and an optical propagation assembly is used for carrying out linear propagation transformation on the modulated light field; the image sensor is used for carrying out light intensity detection on the processed output light field, obtaining light intensity distribution characteristics and converting the light intensity distribution characteristics into electric signals to be output; and the electronic processing module is used for decoding the electric signal to complete a classification or regression type visual task. According to the scheme, a general optical computing architecture based on random phase projection is provided, high-density and high-parallel optical random projection and a low-parameter electronic decoder are integrated, an optical part does not need to be trained, high-precision and high-energy-efficiency processing is achieved in various machine vision tasks, and the high-precision and high-energy-efficiency processing is achieved. And an innovative path is provided for breaking through the bottleneck of traditional electronic calculation in speed and power consumption and realizing the practicability of a large-scale optical intelligent processing system.

Description

Technical Field

[0001] This invention belongs to the fields of integrated circuits, artificial intelligence and neuromorphic computing technology, and specifically relates to a multi-task integrated optical vision processing system and its preparation method. Background Technology

[0002] In recent years, artificial intelligence technology has been widely applied in fields such as image recognition, object detection, and intelligent perception, placing higher demands on the computing power, energy efficiency, and parallel processing capabilities of computing systems. As the scale of deep learning models continues to expand, traditional electronic computing systems based on the von Neumann architecture are increasingly affected by factors such as limited storage bandwidth, increased energy consumption, and increased computational latency when executing large-scale neural network tasks, making it difficult to simultaneously achieve high speed and low power consumption.

[0003] To improve computational efficiency, existing technologies have proposed hardware implementation schemes based on neuromorphic computing, which enhance processing power by constructing brain-like parallel structures. However, existing electronic neuromorphic computing hardware is usually limited by device integration density, interconnect complexity, and power consumption constraints, resulting in limited achievable neuron scale and parallelism, making it difficult to meet the demands of complex visual tasks for large-scale parameter and high-throughput computation.

[0004] Optical computing, with its high-speed propagation characteristics and spatial parallel processing capabilities, is considered a potential technology for high-throughput information processing. In recent years, optical neural network schemes based on diffractive optical structures or optical metasurfaces have attracted attention. These schemes achieve linear or nonlinear mapping calculations within the optical domain by spatially modulating the light field. However, some existing optical neural network schemes often rely on phase distribution structures designed or trained for specific tasks, resulting in a high degree of coupling between the optical modulation structure and the specific application scenario. When the task type changes, it is often necessary to redesign or optimize the optical structure, increasing the complexity of system implementation and maintenance.

[0005] Furthermore, the aforementioned optical modulation structures are typically highly dependent on the precise control of nanostructure parameters. Deviations in structural dimensions, fluctuations in material parameters, or assembly errors during manufacturing can cause shifts in the optical mapping relationship, thereby affecting the stability of the calculation results.

[0006] On the other hand, some existing technologies reduce design complexity by introducing random scattering media or disordered structures to achieve feature mapping. However, such solutions still have shortcomings in terms of device scale controllability, integration, and coordination with electronic processing modules, making it difficult to achieve stable and scalable multitasking capabilities while ensuring high integration.

[0007] Therefore, while maintaining the high-speed parallel characteristics of optical computing, how to reduce the dependence of optical structures on precise design and training, and achieve general processing capabilities for various visual tasks, remains a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] To address the aforementioned problems, the present invention aims to provide a multi-task integrated optical vision processing system and its fabrication method, overcoming the limitations of existing optical neural networks and optical computing systems. These systems rely on optical modulation structures designed or trained for specific tasks, whose functions are highly dependent on specific application scenarios, are highly sensitive to manufacturing and system assembly errors, and often require redesign or adjustment of the optical structure when switching tasks or expanding the system. Furthermore, they struggle to achieve a balance between high parallel processing capabilities, system robustness, and multi-task versatility.

[0009] The specific technical solution for achieving the objective of this invention is as follows:

[0010] A multi-task integrated optical vision processing system includes at least one diffractive optical metasurface layer arranged along the optical path, an optical propagation component, an image sensor, and an electronic processing module.

[0011] The diffractive optical metasurface is used to perform random phase modulation on the input light field, and the optical propagation component is used to perform linear propagation transformation on the light field modulated by the diffractive optical metasurface.

[0012] The image sensor is used to detect the light intensity of the output light field after it has been processed by the diffractive optical metasurface and the optical propagation component, obtain the light intensity distribution characteristics, and convert them into electrical signal output.

[0013] The electronic processing module is used to decode the electrical signal to complete classification or regression visual tasks.

[0014] Furthermore, the diffractive optical metasurface includes multiple periodically distributed subwavelength-scale phase modulation units, which generate a phase delay for the incident light field.

[0015] The phase modulation unit is a dielectric nanostructure, and its phase modulation is achieved by adjusting at least one geometric parameter of the dielectric nanostructure, including height, lateral dimension, duty cycle or spatial orientation angle.

[0016] The geometric parameters of each phase modulation unit are configured in space according to a preset random distribution so that, under the combined action of the optical propagation of the optical propagation component and the light intensity detection of the image sensor, a fixed random feature mapping is formed on the input light field; the light intensity detection of the image sensor performs amplitude squaring operation on the modulated complex amplitude to form a light intensity signal, thereby introducing a nonlinear feature response.

[0017] The phase values ​​of each phase modulation unit follow a predetermined random distribution in space. The random distribution can be a uniform distribution, a discrete multi-level distribution, or other distribution forms that satisfy statistical randomness within a predetermined phase interval.

[0018] Furthermore, the height of the phase modulation unit is discrete, preferably distributed in the range of 100 nm to 500 nm.

[0019] Furthermore, the structural density of the phase modulation unit is not less than 10. 8 pcs / square centimeter

[0020] Furthermore, the optical propagation component is a Fourier transform lens or an equivalent optical component, which, through free space propagation and lens transformation, enables the modulated light field to form a complex amplitude distribution on the output plane that is linearly related to the input light field.

[0021] Furthermore, the electronic processing module is a decoding structure containing only a single fully connected layer or a single nonlinear mapping layer, thereby forming a photoelectric hybrid computing architecture in which random feature mapping is achieved by linear optical propagation and nonlinear light intensity detection, and in coordination with electronic decoding.

[0022] Furthermore, the diffractive optical metasurface and the optical propagation components together constitute a universal optical random projection front end. Without optimizing or reconfiguring for a specific task, the optical random projection front end can adapt to various types of machine vision tasks, including image classification tasks and key point regression tasks, by changing the parameter configuration of the electronic processing module.

[0023] Furthermore, the diffractive optical metasurface is provided in multiple sets, which are respectively arranged before and after the optical propagation component to form a cascaded random phase modulation structure. The random phase distributions of the multiple sets of diffractive optical metasurfaces are set independently to form a cascaded random mapping relationship during the propagation of the input light field, thereby changing and enriching the distribution form of the random feature space.

[0024] In addition, this solution also provides a method for preparing the diffractive optical metasurface in the above system, characterized by comprising the following steps:

[0025] Step 1: Provide a substrate with a surface coated with a photolithography material, and use femtosecond laser two-photon polymerization technology to perform direct writing processing on the photolithography material: solidify multiple phase modulation units that constitute the diffractive optical metasurface point by point or in parallel inside the photolithography material;

[0026] In the direct writing process, the incident laser is modulated using a spatial light modulator or a diffractive optical metasurface to generate a multifocal parallel exposure light field containing multiple focused spots.

[0027] Step 2: Focus the multi-focus parallel exposure light field onto the interior of the photolithography material for exposure; during the exposure process, a randomized scanning sequence is adopted for the preset processing area so that adjacent phase modulation units are exposed discontinuously in time, in order to suppress heat accumulation and cross-aggregation effects during the processing.

[0028] Step 3: After exposure, the photolithography material is developed to remove uncured areas, thereby forming a nanostructure array with a fixed random phase distribution within a predetermined area.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] (1) The diffractive optical metasurface in this scheme is used to apply spatial phase modulation to the input light field in the optical domain. The metasurface is composed of multiple subwavelength scale phase modulation units, and the phase value of each phase modulation unit follows a pre-set random distribution in space. The random distribution can be a uniform distribution, a discrete multi-level distribution, or other distribution forms that satisfy statistical randomness within a predetermined phase interval, and is not limited to a specific distribution type. The random phase distribution remains fixed after the device is fabricated and is not optimized or updated based on specific tasks or training data during system operation. By integrating high-density nano-unit structures in a millimeter-scale area, the diffractive optical metasurface can achieve high-dimensional, parallel random phase mapping in the optical domain;

[0031] (2) When the size or morphology of the local nanostructure of the diffractive optical metasurface in this scheme deviates during the fabrication process, the deviation is mainly manifested as a local phase perturbation in optics. After propagation in free space or lens transformation, this phase perturbation will not simply be superimposed into overall distortion, but will be manifested as a small change in the spatial frequency component in the output speckle pattern. Since the random projection is formed by the combined action of a large number of phase modulation units, the overall statistical characteristics of its output speckle intensity distribution are determined by the random mapping structure, rather than depending on the precise phase value of a single modulation unit. Therefore, the local structural deviation will not destroy the statistical stability of the overall mapping relationship. The back-end electronic processing module can adaptively compensate for the mapping offset introduced by the above-mentioned fixed perturbation by training and decoding the fixed speckle features, thereby ensuring that the system can still operate stably under the conditions of manufacturing error or propagation error.

[0032] (3) Under the combined action of random phase modulation, linear optical propagation, and light intensity detection in the multi-task integrated optical vision processing system, the input light field forms a speckle light intensity distribution with statistical random characteristics on the output plane. This speckle light intensity distribution is not meaningless noise, but a feature representation of the input information in a high-dimensional random projection space. While achieving feature mixing, it retains the discrimination information related to the original input. From a mathematical point of view, this process is equivalent to applying a fixed random feature mapping to the input signal, similar to the kernel function approximation method based on random features: the input signal is first mapped to a high-dimensional feature space, and then the electronic processing module performs linear or lightweight nonlinear decoding; the image sensor collects the speckle light intensity distribution after random projection and converts it into an electrical signal, and the electronic processing module decodes the electrical signal to complete the classification or regression task. Since the front end has completed the high-dimensional feature mapping in the optical domain, the back end only needs to perform low-complexity decoding to obtain the target output, thereby reducing the parameter scale and computational complexity of the electronic system;

[0033] Furthermore, since random projection is based on a statistically significant overall mapping relationship, its computational function does not depend on the precise parameters of a single phase modulation unit. Therefore, it has a natural tolerance for manufacturing errors, material fluctuations, assembly errors, and propagation disturbances. These errors manifest as disturbances in the mapping as a whole, without altering the statistical structure of the random feature space, thus ensuring the stability of the system in actual manufacturing and deployment environments.

[0034] (4) The optical random projection front end of the present invention remains fixed during operation. For different visual tasks, only the parameters of the electronic processing module need to be trained or replaced, so that multiple visual tasks such as image classification, target recognition and key point regression can be realized on the same optical hardware platform, thereby improving the system's versatility and task expansion capability.

[0035] Furthermore, since the random projection output is in the form of speckle, it is difficult to directly reconstruct the original input image information without combining it with an electronic decoding module. Therefore, it has information obfuscation characteristics at the physical level and is suitable for application scenarios with requirements for privacy protection.

[0036] (5) Compared with existing optical neural networks based on diffractive optical structures, the present invention has fundamental differences in system architecture and computation method. Existing ONNs usually use optical modulation structures as trainable weight layers, and their phase distribution needs to be designed for specific tasks through backpropagation or numerical optimization; when the task changes, the optical structure needs to be retrained or redesigned, which limits the system's flexibility. In contrast, the present invention uses a fixed random phase distribution as a general optical front end, and its function does not depend on the optimization results of specific tasks, thereby improving the system's versatility and structural stability.

[0037] In terms of computational complexity, existing ONNs often rely on multi-layer optical modulation structures or large-scale electronic networks for decoding, resulting in a large number of parameters and complex training. This invention achieves high-dimensional random projection in the optical domain, requiring only a lightweight electronic processing module with thousands to tens of thousands of parameters to realize various vision tasks, thereby reducing the dependence on the size of trainable parameters.

[0038] In terms of manufacturing and repeatability, existing ONNs are highly sensitive to precise phase design, while the random projection mechanism of this invention is statistically tolerant of local phase errors, enabling the system to have higher stability and consistency in actual manufacturing and application environments.

[0039] The present invention will be further described below with reference to specific embodiments. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the multi-task integrated optical vision processing system architecture of the present invention.

[0041] Figure 2 This is a schematic diagram of the present invention under the input condition of the first type of image (MNIST dataset), where (a) is the processing flow, (b) is an example of the output result, and (c) is the corresponding output distribution form.

[0042] Figure 3 This is a schematic diagram of the present invention under the input condition of a second type of image (Fashion-MNIST dataset), where (a) is the processing flow, (b) is an example of the output result, and (c) is the corresponding output distribution form.

[0043] Figure 4 This is a schematic diagram of the processing of the present invention using a multilayer diffractive optical metasurface structure, where (a) is the processing flow, (b) is an example of the output result, and (c) is the corresponding output distribution form.

[0044] Figure 5 This is an example of the output result distribution of the present invention under the condition of temporal image input, where (ac) is a human motion video stream image.

[0045] Figure 6 This is an example of the output result of the present invention under biological image input conditions, where (ac) is a flow cytometry cell map.

[0046] Figure 7 This is the representation of the output results of the present invention in regression-type visual tasks, where (a) is the detection based on the original image and (b) is the detection by the optical random projection front end.

[0047] Figure 8This is a schematic diagram illustrating the structure and working principle of the random multifocal two-photon polymerization nanofabrication system for preparing diffractive optical metasurfaces according to the present invention.

[0048] Figure 9 This is a schematic diagram comparing the diffractive optical metasurface preparation method used in this invention with existing related preparation methods. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0051] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0052] Combination Figure 1 A multi-task integrated optical vision processing system is proposed. Its core idea is to construct a fixed random feature mapping front-end in the optical domain, which performs high-dimensional random projection and feature mixing of the input information. Subsequently, a lightweight back-end electronic processing module decodes this fixed feature, thereby enabling the processing of various visual tasks. Under this architecture, the optical structure remains fixed during system operation and does not participate in task-specific training or updates.

[0053] Specifically, this solution includes at least one layer of diffractive optical metasurface, optical propagation components, image sensor, and electronic processing module arranged along the optical path;

[0054] The diffractive optical metasurface is used to perform random phase modulation on the input light field, and the optical propagation component is used to perform linear propagation transformation on the light field modulated by the diffractive optical metasurface.

[0055] The diffractive optical metasurface can be placed individually before the optical propagation component (i.e., a single-layer structure) or in multiple sets (such as a double-layer structure), placed before and after the optical propagation component respectively. Specifically, when using a single-layer diffractive optical metasurface, the system performs a random phase modulation and linear propagation transformation on the input light field. Its advantage lies in its simpler and more direct system structure, enabling high-precision processing of various basic and conventional classification tasks (such as basic classification tasks, human action recognition, and flow cytometry classification) with extremely low electronic computing costs, demonstrating excellent versatility and efficiency. Compared to a single-layer structure, when using a double-layer or other multiple-set structure, the input light field undergoes multiple levels of random mapping during propagation. The double-layer structure forms a cascaded random mapping relationship within the optical domain, thereby changing the distribution of the random feature space. This structural extension method, while maintaining the fixed properties of the optical front end, can effectively enrich the distribution of the random feature space, providing different forms and higher-dimensional random feature representations, thus significantly improving the classification accuracy for complex visual tasks (such as complex backgrounds or multi-class image sets). The subsequent electronic processing module also works in concert to decode this complex feature.

[0056] The image sensor is used to detect the light intensity of the output light field after it has been processed by the diffractive optical metasurface and the optical propagation component, obtain the light intensity distribution characteristics, and convert them into electrical signal output.

[0057] The electronic processing module is used to decode the electrical signal to complete classification or regression visual tasks.

[0058] The above modules work together to form a hybrid optoelectronic computing system, in which the optical part is responsible for feature projection and the electronic part is responsible for task decoding.

[0059] More specifically, in this scheme, the diffractive optical metasurface includes multiple periodically distributed subwavelength-scale phase modulation units, which generate a phase delay for the incident light field.

[0060] The phase modulation unit is a dielectric nanostructure, and its phase modulation is achieved by adjusting at least one geometric parameter of the dielectric nanostructure, including height, lateral dimension, duty cycle, or spatial orientation angle; wherein the height is a discrete value, preferably distributed in the range of 100 nm to 500 nm. The diffractive optical metasurface integrates millions to tens of millions of subwavelength unit structures in a millimeter-scale area, and its unit structure density is not less than 10. 8 To achieve large-scale parallel optical weight mapping and feature mixing, the number of features per square centimeter is increased.

[0061] The geometric parameters of each phase modulation unit are configured in space according to a preset random distribution so that, under the combined action of optical propagation of the optical propagation component and light intensity detection of the image sensor, a fixed random feature mapping is formed on the input light field; the light intensity detection performs amplitude squaring operation on the modulated complex amplitude to form a light intensity signal, thereby introducing a nonlinear feature response.

[0062] Furthermore, the phase values ​​of each phase modulation unit spatially follow a pre-defined random distribution, which can be a uniform distribution, a discrete multilevel distribution, or other distribution forms that satisfy statistical randomness within a predetermined phase interval. This invention does not limit the specific distribution type. The random phase distribution remains fixed after device fabrication and is not optimized or updated based on specific tasks or training data during system operation. By integrating high-density nanounit structures within a millimeter-scale area, the diffractive optical metasurface can achieve high-dimensional, parallel random phase mapping in the optical domain.

[0063] The fixed random phase distribution in this scheme makes the system robust to the following errors: local structural dimensional errors in the manufacturing process of diffractive optical metasurfaces; propagation distance errors generated during the assembly of optical components; and phase disturbances or scattering noise generated in the optical system.

[0064] The aforementioned errors can be statistically equivalent to disturbances in random projections;

[0065] Specifically, when the local nanostructure of a diffractive optical metasurface experiences size or morphological deviations during fabrication, these deviations primarily manifest as local phase perturbations in optics. After propagation in free space or lens transformation, these phase perturbations do not simply superimpose into overall distortion; instead, they appear as minute changes in the spatial frequency components of the output speckle pattern. Since random projections are formed by the combined action of numerous phase modulation units, the overall statistical characteristics of the output speckle intensity distribution are determined by the random mapping structure, rather than depending on the precise phase value of a single modulation unit. Therefore, local structural deviations do not disrupt the statistical stability of the overall mapping relationship. The back-end electronic processing module, through training and decoding of fixed speckle features, can adaptively compensate for the mapping offset introduced by the aforementioned fixed perturbations, thereby ensuring stable operation of the system even under conditions of manufacturing or propagation errors.

[0066] In addition, the output light intensity distribution generated by the diffractive optical metasurface is in the form of random speckle. Without the decoding by the electronic processing module, it is difficult to directly reconstruct the original input image information, thereby achieving information obfuscation or privacy protection at the physical layer.

[0067] The optical propagation component, positioned after the diffractive optical metasurface, is used to propagate and transform the randomly phase-modulated light field. The optical propagation component is a Fourier transform lens or an equivalent optical component. Through free-space propagation and lens transformation, it causes the modulated light field to form a complex amplitude distribution on the output plane that is linearly related to the input light field. This propagation process is physically equivalent to applying a fixed high-dimensional linear mapping to the input information.

[0068] The diffractive optical metasurface and optical propagation components together constitute a universal optical random projection front end. Without optimizing or reconfiguring for a specific task, the optical random projection front end can adapt to various types of machine vision tasks, including image classification tasks and key point regression tasks, by changing the parameter configuration of the electronic processing module.

[0069] The electronic processing module is a decoding structure containing only a single-layer fully connected layer or a single-layer nonlinear mapping layer, thereby forming a photoelectric hybrid computing architecture in which random feature mapping is achieved by linear optical propagation and nonlinear light intensity detection, and in coordination with electronic decoding.

[0070] The electronic processing module is used to perform linear mapping or single-layer nonlinear mapping on the one-dimensional speckle intensity vector output by the image sensor, wherein the speckle intensity vector is a nonlinear characteristic representation formed by the squared amplitude detection of the complex amplitude after random phase modulation and optical propagation.

[0071] After the complex amplitude signal is detected by the image sensor, the complex amplitude is converted into a light intensity signal through amplitude squaring, thus introducing a nonlinear transformation. Therefore, random phase modulation and optical propagation constitute a fixed linear mapping, while the light intensity detection process constitutes a nonlinear transformation; together, these three elements form a random characteristic mapping process for the input signal.

[0072] In practical applications, the optical propagation components do not necessarily need to strictly meet the ideal Fourier transform condition or a strict 2f optical configuration. The diffractive optical metasurface and the image sensor can operate in the Fresnel diffraction region or under certain defocus conditions. In this case, because the electronic processing module decodes and trains fixed random features, the system can still complete classification or regression visual tasks. Therefore, this optical random projection front end has a certain tolerance for propagation distance and imaging conditions, which helps to reduce the system's dependence on precise optical alignment.

[0073] Under the combined effects of the aforementioned random phase modulation, linear optical propagation, and intensity detection, the input light field forms a speckle intensity distribution with statistically random characteristics on the output plane. This speckle intensity distribution is not meaningless noise, but rather a characteristic representation of the input information in a high-dimensional random projection space, preserving discriminative information related to the original input while achieving feature mixing. From a mathematical perspective, this process is equivalent to applying a fixed random feature mapping to the input signal, similar to the kernel function approximation method based on random features: the input signal is first mapped to a high-dimensional feature space, and then linearly or with a light nonlinear decoding is performed by the electronic processing module.

[0074] The image sensor is used to acquire the speckle light intensity distribution after random projection and convert it into an electrical signal. The electronic processing module is used to decode the electrical signal to complete the classification or regression task. Since the front end has completed high-dimensional feature mapping in the optical domain, the back end only needs to perform low-complexity decoding to obtain the target output, thereby reducing the parameter scale and computational complexity of the electronic system.

[0075] Since the random projection is based on a statistically significant overall mapping relationship, its computational function does not depend on the precise parameters of a single phase modulation unit. Therefore, it has a natural tolerance for manufacturing errors, material fluctuations, assembly errors, and propagation disturbances. These errors manifest as disturbances in the mapping as a whole, without altering the statistical structure of the random feature space, thus ensuring the stability of the system in actual manufacturing and deployment environments.

[0076] At the system architecture level, the optical random projection front end of this invention remains fixed during operation. For different visual tasks, only parameter training or replacement of the electronic processing module is required to achieve various visual task processing such as image classification, target recognition, and key point regression on the same optical hardware platform, thereby improving the system's versatility and task scalability.

[0077] Furthermore, since the random projection output is in the form of speckle, it is difficult to directly reconstruct the original input image information without combining it with an electronic decoding module. Therefore, it has information obfuscation characteristics at the physical level and is suitable for application scenarios with requirements for privacy protection.

[0078] This solution can construct a fixed random high-dimensional feature space in the optical domain, improving the system's versatility; it has manufacturing error tolerance in a statistical sense; it reduces the scale of electronic processing parameters and power consumption; it supports multi-task adaptation; and it has scalable manufacturing capabilities.

[0079] In addition, this solution also provides a method for preparing the diffractive optical metasurface in the above system, including the following steps:

[0080] Step 1: Provide a substrate with a surface coated with a photolithography material, and use femtosecond laser two-photon polymerization technology to perform direct writing processing on the photolithography material: solidify multiple phase modulation units that constitute the diffractive optical metasurface point by point or in parallel inside the photolithography material;

[0081] In the direct writing process, the incident laser is modulated using a spatial light modulator or a diffractive optical metasurface to generate a multifocal parallel exposure light field containing multiple focused spots.

[0082] Step 2: Focus the multi-focus parallel exposure light field onto the interior of the photolithography material for exposure; during the exposure process, a randomized scanning sequence is used for the processing area to avoid consecutive exposure of adjacent unit structures in time;

[0083] The randomized scanning sequence divides the processing area into multiple sub-blocks and exposes them in a non-contiguous order between different sub-blocks to suppress proximity effects, cross-aggregation effects, or local heat accumulation effects.

[0084] Step 3: After exposure, the photolithography material is developed to remove uncured areas, thereby forming a nanostructure array with a fixed random phase distribution within a predetermined area.

[0085] Furthermore, the aforementioned diffractive optical metasurface can be mass-produced using soft-mold replication and nanoimprinting processes:

[0086] A flexible mold is obtained by casting an elastic polymer material onto the surface of the nanostructure array, followed by curing and peeling.

[0087] A curable replicable material is introduced between the flexible mold and the substrate and then pressed and cured.

[0088] Separate the flexible mold from the cured replica material.

[0089] For example, in actual implementation, the multi-focus parallel exposure light field generates at least 25 parallel focused light spots simultaneously in each exposure cycle; the repetition frequency of the femtosecond laser is 1 kHz; the feature size of the unit structure is no greater than 500 nm, and each unit structure is formed by the discrete exposure and superposition of multiple pulses of the femtosecond laser along the optical axis.

[0090] Using the above method, millimeter-scale diffractive optical metasurfaces can be fabricated in no more than 15 minutes, with unit structure feature sizes not exceeding 500 nm. They can be replicated by soft lithography or nanoimprinting to achieve mass production.

[0091] The following description, in conjunction with specific embodiments, provides further details.

[0092] Example 1:

[0093] This embodiment provides a multi-task integrated optical vision processing system. The system adopts a modular structure design and includes, in sequence along the optical path, a diffractive optical metasurface, an optical propagation component, an image sensor, and an electronic processing module.

[0094] The effective area of ​​the diffractive optical metasurface is 1 mm², the unit structure period is 500 nm, and the random phase distribution is achieved by discrete height modulation in the range of 100–500 nm.

[0095] The optical propagation component is a Fourier transform lens with a focal length of 150 mm, used to perform Fourier transform on the light field modulated by the diffractive optical metasurface.

[0096] The image sensor is a scientific-grade CMOS camera used to acquire the light intensity distribution of the output plane.

[0097] The electronic processing module is connected to the image sensor via a data interface and is used to decode the acquired electrical signals.

[0098] With an output resolution of 10×10 pixels, the system uses uniform structural parameters for multiple sets of image data. In this configuration, the electronic processing module is a single-layer fully connected network containing 1000 trainable parameters. Depending on the data source, the input image resolution can be set to 28×28 pixels, or pre-processed according to the characteristics of the original data before being input into the system.

[0099] When the output resolution is increased to 50×50 pixels, in order to adapt to higher-dimensional random feature input, the electronic processing module is adjusted to a single-layer fully connected network containing 25,000 trainable parameters, while the parameters of the other optical components remain unchanged.

[0100] Under the aforementioned different output resolution conditions, the optical random projection front end maintains a fixed structure. By adjusting the parameter scale of the electronic processing module, the system can complete the processing flow from image input to classification result output and obtain stable classification results. This demonstrates that, without the optical front end participating in training or structural adjustment, decoding of different feature dimensions can be achieved simply by adapting the scale of the electronic module.

[0101] In this embodiment, the random speckle light intensity distribution acquired by the image sensor is input to the electronic processing module and can be further processed by pixel merging or digital downsampling. This processing is used to reduce feature dimensionality and suppress high-frequency optical noise. The pixel merging or downsampling operation does not change the overall statistical feature structure formed by random projection, but is only an engineering optimization measure to improve system stability and computational efficiency.

[0102] In this embodiment, the diffractive optical metasurface achieves random phase distribution through height modulation, that is, by forming nanostructure units of different heights in the photoresist to generate corresponding phase delays.

[0103] In other alternative embodiments, the diffractive optical metasurface can also employ different types of nanostructure units to achieve random phase modulation. For example, a dielectric nanopillar array structure can be used, and random phase coverage can be achieved by adjusting the geometry, duty cycle, or spatial orientation angle of the nanopillars; or an anisotropic nanostructure based on geometric phase can be used, and a fixed random phase distribution can be achieved by randomizing its orientation angle.

[0104] The aforementioned different physical implementations are all used to construct a fixed optical random projection front end, whose function is to apply random phase mapping to the input light field in a statistical sense, without depending on a specific material system or a single structural form.

[0105] Combination Figures 2 to 7 The specific test results illustrate the processing effect of this embodiment. For example... Figure 2 As shown in (a)–(c), under the input condition of the first type of image, such as the MNIST handwritten digit dataset, the optical system of this embodiment, combined with 10×10 pixel downsampling processing, obtains a highly discriminative output distribution with a classification accuracy of up to 98%. Figure 3 As shown in (a)–(c), the system also demonstrates excellent feature mapping capabilities under the input conditions of the second type of images, such as the Fashion-MNIST dataset.

[0106] In addition, such as Figure 5 The time-series images (such as human motion video streams) shown in (a)–(c) and Figure 6Under the input conditions of biological images (such as flow cytometry images) shown in (a)–(c), the optical vision processing system of this embodiment can output high-precision classification distribution results with low electronic computing costs, which proves the versatility and efficiency of the single-layer fixed optical random projection front end in a variety of classification tasks.

[0107] Example 2:

[0108] Based on Example 1, this embodiment extends the structure of the optical random projection front end by adopting a double-layer diffractive optical metasurface design to form a cascaded random phase modulation structure.

[0109] The optical vision processing system comprises, along the optical path, the following components in sequence:

[0110] The first diffractive optical metasurface has an effective area of ​​1 mm², a unit cell period of 500 nm, and a fixed random phase distribution.

[0111] A Fourier transform lens with a focal length of 150 mm is used to propagate and transform the light field modulated by the first diffractive optical metasurface.

[0112] The second diffractive optical metasurface is maintained at a distance of 3 mm from the first diffractive optical metasurface by a precision support. The second diffractive optical metasurface also has a fixed random phase distribution.

[0113] An image sensor, positioned on the final output plane of the optical system, is used to collect light intensity distribution;

[0114] The electronic processing module is connected to the image sensor via a high-speed data interface and is used to decode and process the acquired electrical signals.

[0115] In this embodiment, the random phase distributions of the two diffractive optical metasurfaces are set independently and remain fixed during system operation. Through the cascaded modulation of the two random phases, the input light field undergoes multiple levels of random mapping during propagation, thereby forming a random feature distribution different from that of a single-layer structure.

[0116] For color image input scenarios, this embodiment configures the input image as follows: the original color image is converted into a grayscale image and uniformly adjusted to a resolution of 32×32 pixels before being input into the system.

[0117] The output light intensity distribution acquired by the image sensor is downsampled to 50×50 pixels before being input into the electronic processing module. The electronic processing module employs a single-layer fully connected network containing 25,000 trainable parameters, used to decode the features after two-layer random projection.

[0118] Under the above configuration conditions, the system completes the overall processing flow of image input, two-layer random phase modulation, optical propagation and electronic decoding, and outputs the corresponding classification results.

[0119] Compared to single-layer diffractive optical structures, double-layer structures form cascaded random mapping relationships within the optical domain, thereby altering the distribution of the random feature space. This structural extension method, while maintaining the fixed properties of the optical front end, provides different forms of random feature representations for subsequent decoding processing by the electronic processing module.

[0120] Combination Figure 4 Sections (a)–(c) describe the processing flow and effects of the double-layer diffractive optical metasurface structure used in this embodiment: For more complex visual datasets such as the CIFAR-10 image set, the input image is subjected to two-layer random phase modulation, and the output light intensity distribution is downsampled to 50×50 pixels. The confusion matrix between the corresponding predicted label and the true label (e.g.) Figure 4 (b) and accuracy distribution (as shown in the figure) Figure 4 (c) shows that cascaded random mapping relationships can effectively enrich the distribution of random feature space and significantly improve the accuracy of complex classification tasks.

[0121] Example 3:

[0122] This embodiment is the same as embodiment 1 in terms of overall system structure configuration, except that this embodiment is used to perform regression-type visual tasks.

[0123] In this embodiment, the input image is a face image with an input resolution of 96×96 pixels, and the system output is the coordinate information of multiple facial key points. The input image is spatially modulated to form an input light field, which is then sequentially subjected to random phase modulation and propagation transformation through a diffractive optical metasurface and an optical propagation component.

[0124] An image sensor is used to acquire the output light intensity distribution after optical random projection. This output light intensity distribution exhibits a random speckle pattern. The electrical signals acquired by the image sensor are input to an electronic processing module for decoding.

[0125] In this embodiment, the electronic processing module is a lightweight neural network structure, employing a regression network composed of convolutional layers and fully connected layers to decode the feature maps in the form of random speckle patterns into facial key point coordinates. The electronic processing module has approximately 20,000 trainable parameters.

[0126] Under the above configuration, the system completes the overall processing flow from input face image to key point coordinate output. The intermediate result output by the image sensor before decoding by the electronic processing module is a random speckle light intensity distribution. This intermediate result does not contain directly identifiable facial structure information, but only serves as a random feature representation for the electronic module to decode.

[0127] Combination Figure 7 Sections (a)–(b) illustrate the application effect of this embodiment in regression-type visual tasks: This embodiment takes facial landmark detection as an example. The input facial image is modulated by an optical processing system to form random speckle patterns for backend decoding. (Comparison) Figure 7 (a) (Pure digit detection based on the original image, with a large average error) and Figure 7 (b) (Detection based on the optical random projection front end in the linear optical processing system of this embodiment) It can be seen that, since the high-dimensional random feature mapping in the optical domain retains sufficient spatial discrimination information, this system performs excellently in the overlap between the output predicted point (red point) and the real point (green point), and the average error is significantly reduced, which proves the high robustness of this architecture in the key point coordinate regression task.

[0128] As can be seen from Examples 1 to 3, under the premise of maintaining a fixed structure at the optical random projection front end, the system can perform classification and regression tasks respectively by configuring electronic processing modules with different structural forms and parameter scales. This architecture does not require task-related training or structural adjustments to the optical layer; it can adapt to different visual tasks simply by changing the configuration of the electronic modules.

[0129] Example 4:

[0130] This embodiment provides a detailed description of the fabrication process of a diffractive optical metasurface. This embodiment also provides a method for fabricating the diffractive optical metasurface applicable to the present invention, used to form a diffractive optical structure with random phase modulation characteristics.

[0131] In this embodiment, the processing system employs a femtosecond laser two-photon polymerization direct-write nanofabrication system. The laser source is a femtosecond pulsed laser with a center wavelength of approximately 800 nm, a repetition rate in the kHz range, a pulse width in the fs range, and an average output power in the watt range.

[0132] The fabrication process of the diffractive optical metasurface includes the following steps:

[0133] (1) Substrate processing steps: A transparent substrate is selected as the substrate. After cleaning the substrate, a photoresist layer is spin-coated on its surface. The thickness of the photoresist layer is on the order of micrometers to meet the requirements of subsequent three-dimensional micro-nano structure processing.

[0134] (2) Parallel light field generation step: The incident laser is modulated by a spatial light modulator to generate a parallel light field distribution containing multiple focused light spots, which is used to simultaneously form multiple exposure positions in the photoresist to improve processing efficiency.

[0135] (3) Randomized scanning exposure step: The parallel light field is focused into the photoresist layer using a high numerical aperture microscope objective, and a randomized scanning strategy is executed in the preset processing area to rearrange the exposure positions in spatial order, thereby forming a three-dimensional micro-nano structure with random spatial distribution characteristics in the photoresist.

[0136] (4) Large-area splicing step: The relative positions between processing areas are controlled by a precision displacement platform to achieve continuous splicing of multiple processing areas in order to obtain a diffractive optical metasurface with a predetermined effective area.

[0137] (5) Post-processing steps: The exposed photoresist is developed and fixed to remove uncured areas and form the final diffractive optical structure.

[0138] In this embodiment, to avoid local stress distortion and proximity exposure effects during the processing of high-density three-dimensional nanostructures, a randomized scanning path is adopted in the femtosecond laser two-photon polymerization process. The randomized scanning sequence satisfies the following constraint: the minimum spatial distance between two adjacent exposure positions is greater than the thermal diffusion characteristic length of the material under the corresponding processing conditions, thereby reducing thermal coupling and cross-polymerization between adjacent nanostructures.

[0139] like Figure 8 As shown, the random multifocal two-photon polymerization nanofabrication system used in this embodiment generates a multifocal parallel exposure light field through a spatial light modulator or a diffractive optical metasurface, rapidly forming a metasurface structure. For example... Figure 9 The schematic diagram comparing the fabrication methods is shown. Compared with existing high-density / low-throughput fabrication methods in the terahertz and visible light bands, the multi-focus parallel direct-write fabrication method adopted in this invention (…) Figure 9 The location of the satellite (marked by a star) is in terms of neuron density (breaking the million level per square centimeter) and preparation speed (reaching 10). 6 It has achieved an order-of-magnitude leap in the number of neurons per minute.

[0140] Under multi-focal parallel exposure conditions, the exposure sequence is not carried out sequentially according to spatially adjacent positions. Instead, the exposure sequence is randomly rearranged in space so that adjacent nanounits are exposed at different times, thereby reducing local heat accumulation and stress concentration in resin polymerization.

[0141] By using the above-mentioned randomized scanning method, the cross-aggregation and local collapse phenomena between adjacent structures can be suppressed, thereby improving the consistency of structural morphology and phase modulation accuracy under large-area splicing conditions.

[0142] Using the above-described fabrication method, a diffractive optical metasurface with fixed random phase modulation characteristics can be formed within a millimeter-scale area. The fabricated device exhibits excellent optical transmittance and can be applied to the optical vision processor system described in this invention.

[0143] Example 5:

[0144] This embodiment illustrates a method for large-scale replication of diffractive optical metasurfaces applicable to the present invention.

[0145] In this embodiment, the diffractive optical metasurface prepared in Example 4 is used as the master template, and the device is mass-produced through soft-mold replication and nanoimprinting. The replication method includes the following steps:

[0146] (1) Flexible mold preparation steps: After mixing the prepolymer of elastic polymer material with the curing agent, the mixture is degassed and poured onto the surface of the master plate. After heating or other curing methods, the material is cured and formed. Then, the flexible mold corresponding to the surface structure of the master plate is obtained by peeling.

[0147] (2) Nanoimprinting replication step: The curable replicating material is introduced between the flexible mold and the substrate, and the replicating material is fully filled into the mold structure under external pressure, and the replicating material is formed by photocuring or thermocuring.

[0148] (3) Demolding and separation step: After the replication material is cured, the flexible mold is separated from the replicated diffractive optical metasurface to obtain the replicated diffractive optical metasurface structure.

[0149] Using the above method, the micro-nano structures in the master template can be transferred to the replication material, enabling the batch replication of this diffractive optical metasurface. This replication process is suitable for large-scale manufacturing needs and helps improve device consistency and production efficiency.

[0150] In combination with the above Figure 9 The present invention demonstrates a significant leap forward in neuron density and fabrication speed. This embodiment uses the diffractive optical metasurface prepared in Example 4 as a template, achieving large-scale batch fabrication through soft-mold replication and nanoimprinting processes. This replication method further breaks through the bottlenecks of high processing costs and difficulty in mass production of traditional optical neural network platforms, perfectly balancing the industrialization requirements of high-resolution nanoscale features and large-area batch manufacturing, which is beneficial to improving device consistency and production efficiency.

[0151] The embodiments described above are merely one implementation method of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A multi-task integrated optical vision processing system, characterized in that, It includes at least one diffractive optical metasurface layer arranged along the optical path, an optical propagation component, an image sensor, and an electronic processing module; The diffractive optical metasurface is used to perform random phase modulation on the input light field, and the optical propagation component is used to perform linear propagation transformation on the light field modulated by the diffractive optical metasurface. The image sensor is used to detect the light intensity of the output light field after it has been processed by the diffractive optical metasurface and the optical propagation component, obtain the light intensity distribution characteristics, and convert them into electrical signal output. The electronic processing module is used to decode the electrical signal to complete classification or regression visual tasks.

2. The multi-task integrated optical vision processing system according to claim 1, characterized in that, The diffractive optical metasurface includes multiple periodically distributed subwavelength-scale phase modulation units, which generate a phase delay for the incident light field. The phase modulation unit is a dielectric nanostructure, and its phase modulation is achieved by adjusting at least one geometric parameter of the dielectric nanostructure, including height, lateral dimension, duty cycle or spatial orientation angle. The geometric parameters of each phase modulation unit are configured in space according to a preset random distribution so that, under the combined action of the optical propagation of the optical propagation component and the light intensity detection of the image sensor, a fixed random feature mapping is formed on the input light field; the light intensity detection of the image sensor performs amplitude squaring operation on the modulated complex amplitude to form a light intensity signal, thereby introducing a nonlinear feature response. The phase values ​​of each phase modulation unit follow a predetermined random distribution in space. The random distribution is a uniform distribution, a discrete multi-level distribution, or other distribution forms that satisfy statistical randomness within a predetermined phase interval.

3. The multi-task integrated optical vision processing system according to claim 2, characterized in that, The height of the phase modulation unit is discrete, preferably distributed in the range of 100 nm to 500 nm.

4. The multi-task integrated optical vision processing system according to claim 2, characterized in that, The structural density of the phase modulation unit is not less than 10. 8 pcs / square centimeter 5. The multi-task integrated optical vision processing system according to claim 1, characterized in that, The optical propagation component is a Fourier transform lens or an equivalent optical component. Through free space propagation and lens transformation, the modulated light field forms a complex amplitude distribution on the output plane that is linearly related to the input light field.

6. The multi-task integrated optical vision processing system according to claim 1, characterized in that, The electronic processing module is a decoding structure containing only a single fully connected layer or a single nonlinear mapping layer, thus forming a photoelectric hybrid computing architecture in which random feature mapping is achieved by linear optical propagation and nonlinear light intensity detection, and in coordination with electronic decoding.

7. The multi-task integrated optical vision processing system according to claim 1, characterized in that, The diffractive optical metasurface and optical propagation components together constitute a universal optical random projection front end. Without optimizing or reconfiguring for a specific task, the optical random projection front end can adapt to various types of machine vision tasks, including image classification tasks and key point regression tasks, by changing the parameter configuration of the electronic processing module.

8. The multi-task integrated optical vision processing system according to claim 1, characterized in that, The diffractive optical metasurface is provided in multiple sets, which are respectively arranged in front of and behind the optical propagation component to form a cascaded random phase modulation structure; The random phase distributions of the multiple sets of diffractive optical metasurfaces are set independently to form a cascaded random mapping relationship during the propagation of the input light field, thereby changing and enriching the distribution form of the random feature space.

9. A method for preparing a diffractive optical metasurface for a multi-task integrated optical vision processing system according to any one of claims 1-8, characterized in that, Includes the following steps: Step 1: Provide a substrate with a surface coated with a photolithography material, and use femtosecond laser two-photon polymerization technology to perform direct writing processing on the photolithography material: solidify multiple phase modulation units that constitute the diffractive optical metasurface point by point or in parallel inside the photolithography material; In the direct writing process, the incident laser is modulated using a spatial light modulator or a diffractive optical metasurface to generate a multifocal parallel exposure light field containing multiple focused spots. Step 2: Focus the multi-focus parallel exposure light field onto the interior of the photolithography material for exposure; During the exposure process, a randomized scanning sequence is adopted for the preset processing area, so that adjacent phase modulation units are exposed discontinuously in time, in order to suppress heat accumulation and cross-aggregation effects during the processing. Step 3: After exposure, the photolithography material is developed to remove uncured areas, thereby forming a nanostructure array with a fixed random phase distribution within a predetermined area.

10. The method for preparing a diffractive optical metasurface according to claim 9, characterized in that, In step 3, the processing area is scanned in a randomized order. The processing area is divided into multiple sub-blocks, and exposure is performed in a non-continuous order between different sub-blocks to suppress proximity effect, cross-aggregation effect or local heat accumulation effect. The diffractive optical metasurface is mass-produced using soft-mold replication and nanoimprinting processes. A flexible mold is obtained by casting an elastic polymer material onto the surface of the nanostructure array, followed by curing and peeling. A curable replicable material is introduced between the flexible mold and the substrate and then pressed and cured. Separate the flexible mold from the cured replica material.

Images