Optical Artificial Neural Network Environmental Monitoring Chip and Preparation Method

Abstract

The present invention provides an optical artificial neural network environmental monitoring chip and a preparation method thereof. The chip provided by the present invention simulates an artificial neural network in a hardware manner and is used for the online identification or analysis of environmental pollutants. The present invention uses an optical filter layer as the input layer of the artificial neural network, an image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer, so that when the environmental monitoring chip is subsequently used for intelligent processing of environmental monitoring, complex signal processing and algorithm processing corresponding to the input layer and the linear layer are no longer required, which can greatly reduce the power consumption and delay during the processing of the artificial neural network.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to an optical artificial neural network environmental monitoring chip and a preparation method thereof. Background Art

[0002] Environmental monitoring is to monitor and measure the indicators reflecting environmental quality to determine the pollution status of the environment and the level of environmental quality. Environmental monitoring includes real-time monitoring of air, water quality, soil, etc., provides scientific, accurate and effective monitoring for environmental management, and formulates reasonable solutions accordingly.

[0003] Traditional environmental pollution monitoring uses environmental monitoring instruments and conducts environmental monitoring based on wet chemical technology and experimental analysis after aspirating samples. However, these instruments are usually limited to single-point measurement, with a small detection range and a single sample, thus affecting the accuracy of environmental monitoring results. Summary of the Invention

[0004] In view of the problems existing in the prior art, an embodiment of the present invention provides an optical artificial neural network environmental monitoring chip and a preparation method thereof.

[0005] Specifically, the embodiment of the present invention provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides an optical artificial neural network environmental monitoring chip, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0007] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light includes the reflected light, transmitted light, and / or radiation light of environmental pollutants;

[0008] The image sensor is used to convert the incident light carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0009] The processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an intelligent processing result for environmental monitoring;

[0010] Among them, the intelligent environmental monitoring processing task includes the identification and / or qualitative analysis of environmental pollutants; the intelligent environmental monitoring processing result includes the intelligent environmental monitoring processing result of environmental pollutants and / or the environmental pollution qualitative analysis result.

[0011] Further, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0012] Further, the optical artificial neural network environmental monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;

[0013] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which is trained using the input training samples and output training samples corresponding to the intelligent environmental monitoring processing task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0014] The input training samples include incident light reflected, transmitted, and / or radiated by samples with different environmental pollutants; the output training samples include the content of environmental pollutants.

[0015] Further, the samples with different environmental pollutants include air samples with different pollutants, water quality samples with different pollutants, and / or soil samples with different pollutants.

[0016] Further, when training the optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0017] Further, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0018] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0019] Further, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0020] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0021] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0022] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0023] Further, the micro-nano unit has polarization-independent characteristics.

[0024] Further, the micro-nano unit has four-fold rotational symmetry.

[0025] Further, the optical filter layer is composed of one or more filter layers;

[0026] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) nanostructures, tunable Fabry - Perot resonators.

[0027] Further, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0028] Further, the thickness of the optical filter layer is 0.1λ - 10λ, where λ represents the central wavelength of the incident light.

[0029] In a second aspect, an environmental monitoring instrument provided by an embodiment of the present invention includes: the optical artificial neural network environmental monitoring chip as described above.

[0030] In a third aspect, a preparation method of an optical artificial neural network environmental monitoring chip provided by an embodiment of the present invention includes:

[0031] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0032] Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals;

[0033] Connecting the image sensor and the processor;

[0034] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0035] The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an intelligent environmental monitoring processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light of environmental pollutants.

[0036] Further, preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor includes:

[0037] Growing one or more layers of preset materials on the surface of the photosensitive area of the image sensor;

[0038] Etching the pattern of the optical modulation structure on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0039] Or performing imprint transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0040] Or obtaining an optical filter layer including an optical modulation structure by applying external dynamic modulation to the one or more layers of preset materials;

[0041] Or performing zone printing on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0042] Or performing zone growth on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0043] Or performing quantum dot transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure.

[0044] Further, it also includes: the training process of the optical artificial neural network environmental monitoring chip, specifically including:

[0045] Using the input training samples and output training samples corresponding to the environmental protection monitoring intelligent processing task, train an optical artificial neural network environmental protection monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0046] The optical artificial neural network environmental monitoring chip and preparation method provided in the embodiment of the present invention simulate artificial neural networks in a hardware manner for online identification or analysis of environmental pollutants. That is, the embodiment of the present invention realizes a new environmental monitoring chip capable of realizing the function of an artificial neural network. In the environmental monitoring chip, an optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure. The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area. Accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions. At the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions to obtain the output signal of the artificial neural network. It can be seen that in this In the environmental monitoring chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the environmental monitoring chip realize the related functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the environmental monitoring chip is used for the subsequent intelligent processing of environmental monitoring of the artificial neural network, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. The processor in the environmental monitoring chip only needs to perform related processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay of the artificial neural network during environmental monitoring. It can be seen that the embodiment of the present invention uses the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor are used to project the information carried by the incident light of environmental pollutants into electrical signals, and then the full connection processing and nonlinear activation processing of the electrical signals are implemented in the processor. It can be seen that the embodiment of the present invention can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present invention can detect environmental pollutants on a large scale, is not limited to the single-point measurement of traditional environmental monitoring instruments, and has richer detection samples, thereby improving the accuracy and diversity of environmental monitoring.

[0047] As can be seen, the embodiments of the present invention provide a novel optoelectronic chip for realizing environmental protection monitoring on a large scale. The chip embeds an artificial neural network part into an image sensor including various optical filter layers to achieve safe, reliable, fast, and accurate environmental protection monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a schematic structural diagram of an optoelectronic artificial neural network environmental protection monitoring chip provided by the first embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of the recognition principle of an optoelectronic artificial neural network environmental protection monitoring chip provided by an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of the disassembly of an optoelectronic artificial neural network environmental protection monitoring chip provided by an embodiment of the present invention;

[0052] Figure 4 It is a schematic diagram of the environmental pollutant environmental protection monitoring process provided by an embodiment of the present invention;

[0053] Figure 5 It is a top view of an optical filter layer provided by an embodiment of the present invention;

[0054] Figure 6 It is a top view of another optical filter layer provided by an embodiment of the present invention;

[0055] Figure 7 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0056] Figure 8 It is a top view of still another optical filter layer provided by an embodiment of the present invention;

[0057] Figure 9 It is a top view of yet still another optical filter layer provided by an embodiment of the present invention;

[0058] Figure 10 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0059] Figure 11 It is a schematic diagram of the broadband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0060] Figure 12 This is a schematic diagram of the narrow-band filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0061] Figure 13 It is a flow chart of a method for preparing an optical artificial neural network environmental monitoring chip provided by the third embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Traditional environmental pollution monitoring uses environmental monitoring instruments to conduct environmental monitoring based on wet chemical technology and experimental analysis after air sampling, but these instruments are usually limited to single-point measurement, with a small detection range and a single sample, which in turn affects the accuracy of environmental monitoring results. Based on this, an embodiment of the present invention provides an optical artificial neural network environmental monitoring chip, in which the optical filter layer corresponds to the input layer of the artificial neural network, the image sensor corresponds to the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. The embodiment of the present invention uses the optical filter layer and the image sensor to project the information carried by the incident light of environmental pollutants into electrical signals, and then implements the full connection processing and nonlinear activation processing of the electrical signals in the processor. It can be seen that the embodiment of the present invention can save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art. The embodiment of the present invention strips the input layer and linear layer of the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and linear layer in the artificial neural network by hardware, so that when the smart chip is used for the subsequent intelligent processing of the artificial neural network, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer. Only the processor in the smart chip needs to perform related processing related to full connection and nonlinear activation of electrical signals, which can greatly reduce the power consumption and delay of the artificial neural network processing. The content provided by the present invention will be explained and illustrated in detail through specific embodiments.

[0064] like Figure 1As shown in the figure, the optical artificial neural network environmental monitoring chip provided by the first embodiment of the present invention is used for intelligent processing tasks of environmental monitoring, and includes: an optical filter layer 1, an image sensor 2, and a processor 3; the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor 2 corresponds to the linear layer of the artificial neural network; the processor 3 corresponds to the non-linear layer and the output layer of the artificial neural network;

[0065] The optical filter layer 1 is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer 1 includes an optical modulation structure. The optical filter layer 1 is used to perform spectral modulation on the incident light entering different position points of the optical modulation structure with intensity modulation varying with wavelength, that is, perform different intensity modulations on incident light of different wavelengths, so as to obtain incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes image information of the target object to be processed by the optical artificial neural network environmental monitoring chip and / or various optical space information. For example, the incident light carrying information includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light; the incident light includes reflected light, transmitted light, and / or radiation light of environmental pollutants;

[0066] The image sensor 2 is used to convert the incident light carrying information corresponding to different position points modulated by the optical filter layer 1 into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor 3; the electrical signals are image signals modulated by the optical filter layer;

[0067] The processor 3 is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an intelligent processing result for environmental monitoring;

[0068] Among them, the intelligent processing tasks for environmental monitoring include the identification and / or qualitative analysis of environmental pollutants; the intelligent processing results for environmental monitoring include the intelligent processing results for environmental monitoring of environmental pollutants and / or the results of qualitative analysis of environmental pollution.

[0069] In this embodiment, the optical filter layer 1 is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer 1 includes an optical modulation structure. The optical filter layer 1 is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the modulated incident light corresponding to different position points on the surface of the photosensitive area. Correspondingly, the image sensor 2 is configured to convert the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points, that is, the image signals modulated by the optical filter layer. At the same time, the processor 3 connected to the image sensor 2 is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0070] In this embodiment, the optical filter layer 1 includes an optical modulation structure, and performs spectral modulations with different intensities on the incident light (such as the reflected light, transmitted light, radiation light, etc. of the environmental pollutants to be identified) entering different position points of the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area of the image sensor 2.

[0071] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the optical modulation structure. For example, different modulation intensities can be achieved by designing different optical modulation structures (such as changing the shape and / or size parameters of the optical modulation structure).

[0072] In this embodiment, it can be understood that the optical modulation structures at different positions on the optical filter layer 1 have different spectral modulation effects on the incident light. The modulation intensity of different wavelength components of the incident light by the optical modulation structure corresponds to the connection strength of the artificial neural network, that is, the connection weights corresponding to the input layer and the connection from the input layer to the linear layer. It should be noted that the optical filter layer 1 is composed of multiple optical filter units. The optical modulation structures at different positions within each optical filter unit are different, so they have different spectral modulation effects on the incident light; the optical modulation structures at different positions between the optical filter units can be the same or different, so they have the same or different spectral modulation effects on the incident light.

[0073] In this embodiment, the image sensor 2 converts the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to the linear layer of the neural network.

[0074] In this embodiment, the processor 3 performs fully connected processing and non-linear activation processing on the electrical signals at different position points, and then obtains the output signal of the artificial neural network.

[0075] It can be understood that the processor 3 corresponds to the non-linear layer and the output layer of the neural network, and can also be understood as corresponding to the remaining layers (all other layers) of the neural network except the input layer and the linear layer.

[0076] In addition, it should be supplemented that the processor 3 can be arranged inside the environmental monitoring chip, that is, the processor 3 can be arranged together with the filter layer 1 and the image sensor 2 inside the environmental monitoring chip, or can be arranged separately outside the environmental monitoring chip and connected to the image sensor 2 inside the environmental monitoring chip through a data line or a connecting device. This embodiment does not make a limitation in this regard.

[0077] In addition, it should be noted that the processor 3 can be implemented by a computer, or can be implemented by an ARM or FPGA circuit board with certain computing capabilities, or can also be implemented by a microprocessor. This embodiment does not make a limitation in this regard. In addition, as described above, the processor 3 can be integrated inside the environmental monitoring chip or can be arranged independently outside the environmental monitoring chip. When the processor 3 is arranged independently outside the environmental monitoring chip, the electrical signal in the image sensor 2 can be read out to the processor 3 through a signal reading circuit, and then the processor 3 performs a fully connected process and a non-linear activation process on the read electrical signal.

[0078] In this embodiment, it can be understood that when the processor 3 performs non-linear activation processing, it can be implemented by using a non-linear activation function. For example, it can use a Sigmoid function, a Tanh function, a ReLU function, etc. This embodiment does not make a limitation in this regard.

[0079] In this embodiment, the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, the image sensor 2 corresponds to the linear layer of the artificial neural network, converts the information carried by the incident light at different spatial positions into an electrical signal, and the processor 3 corresponds to the non-linear layer and the output layer of the artificial neural network, performs a full connection on the electrical signals at different positions, and obtains the output signal of the artificial neural network through a non-linear activation function, realizing the recognition and processing of environmental pollutants.

[0080] As Figure 2 shown on the left, the optical artificial neural network environmental monitoring chip includes an optical filter layer 1, an image sensor 2, and a processor 3. In Figure 2 this, the processor 3 is implemented by using a signal reading circuit and a computer. As Figure 2As shown on the right side, the optical filter layer 1 in the optical artificial neural network environmental monitoring chip corresponds to the input layer of the artificial neural network, the image sensor 2 corresponds to the linear layer of the artificial neural network, and the processor 3 corresponds to the non-linear layer and the output layer of the artificial neural network. The filtering effect of the optical filter layer 1 on the incident light entering the optical filter layer 1 corresponds to the connection weight from the input layer to the linear layer. It can be seen that the optical filter layer and the image sensor in the environmental monitoring chip provided in this embodiment implement the related functions of the input layer and the linear layer in the artificial neural network in a hardware manner, so that subsequent complex signal processing and algorithm processing corresponding to the input layer and the linear layer are no longer required when using this environmental monitoring chip for recognition processing. This can significantly reduce the power consumption and delay during the processing of the artificial neural network. Moreover, the embodiment of the present invention can detect environmental pollutants over a large range, not limited to the single-point measurement of traditional environmental monitoring instruments, with richer detection samples, improving the accuracy and diversity of environmental monitoring.

[0081] As Figure 2 shown on the right side, the incident light spectrum P at different positions of the optical filter layer 1 λ is projected / connected to the photocurrent response I of the image sensor N . The processor 3 includes a signal readout circuit and a computer. The signal readout circuit in the processor 3 reads out the photocurrent response and transmits it to the computer, and the computer performs full connection processing and non-linear activation processing on the electrical signal, and finally outputs the result.

[0082] As Figure 3 shown, the optical modulation structure on the optical filter layer 1 is integrated above the image sensor 2 to modulate the incident light, project / connect the spectral information of the incident light to different pixel points of the image sensor 2, and obtain an electrical signal containing the spectral information and image information of the incident light. That is, after the incident light passes through the optical filter layer 1, it is converted into an electrical signal by the image sensor 2 to form an image containing the spectral information of the incident light, and finally the processor 3 connected to the image sensor 2 processes the electrical signal containing the spectral information and image information of the incident light.

[0083] In this embodiment, the information carried by the incident light may include one or more (including two) of the light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0084] For example, in one implementation, the information carried by the incident light may include the light intensity distribution information. In other implementations, multiple information such as the image information, spectral information, the angle of the incident light, and the phase information of the incident light of environmental pollutants can be used simultaneously to identify environmental pollutants, so that the identification of environmental pollutants can be more accurately achieved.

[0085] It can be seen that the optical artificial neural network environmental monitoring chip provided in this embodiment can simultaneously utilize the image information, spectral information, incident light angle, and incident light phase information of environmental pollutants, that is, the incident light at different points in space carries information, and an artificial neural network is embedded in the hardware. Information such as the composition, image shape, and three-dimensional depth of environmental pollutants can be further extracted from the spatial image, spectrum, angle, and phase information to accurately identify and / or qualitatively analyze environmental pollutants, and then accurately obtain environmental monitoring results. Moreover, the embodiments of the present invention can detect environmental pollutants over a large range, not limited to the single-point measurement of traditional environmental monitoring instruments, with richer detection samples and higher accuracy, and a spectrum optical artificial neural network environmental monitoring chip with low power consumption, low latency, and high accuracy is realized.

[0086] The optical artificial neural network environmental monitoring chip provided in the embodiments of the present invention realizes a brand-new intelligent chip capable of implementing the functions of an artificial neural network. In this intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this intelligent chip realize the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiments of the present invention strip the input layer and the linear layer in the artificial neural network implemented by software in the prior art and implement these two-layer structures in the artificial neural network in a hardware manner. As a result, when using this intelligent chip for artificial neural network intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. Only the processor in the intelligent chip needs to perform relevant processing related to the full connection and nonlinear activation of the electrical signals, which can greatly reduce the power consumption and latency during artificial neural network processing.

[0087] In addition, it should be noted that in the prior art, when analyzing environmental pollutants, it is only limited to single-point measurement, with a small detection range and a single sample, making it difficult to ensure the accuracy of environmental monitoring results. Therefore, based on this, in one implementation, the information carried by the incident light may include light intensity distribution information and spectral information, so that when using the optical artificial neural network environmental monitoring chip provided by the present application to perform an intelligent recognition task, the light intensity distribution information and spectral information of environmental pollutants can be utilized simultaneously. It can be seen that since the information carried by the incident light covers information such as the image, composition, shape, three-dimensional depth, and structure of environmental pollutants, when performing recognition processing based on the information carried by the incident light at different spatial points of environmental pollutants, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of environmental pollutants can be covered, thus enabling more accurate environmental monitoring. In addition, in another implementation, the information carried by the incident light may further include light intensity distribution information, spectral information, and the angle information of the incident light, so as to be able to capture more comprehensive information such as the image, composition, shape, three-dimensional depth, and three-dimensional structure of environmental pollutants, and thus enable more accurate environmental monitoring. In addition, in another implementation, the information carried by the incident light may further include light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light, so as to be able to capture more comprehensive information such as the image, composition, shape, three-dimensional depth, and three-dimensional structure of environmental pollutants, and thus enable more accurate environmental monitoring.

[0088] The optical artificial neural network environmental monitoring chip provided by the embodiment of the present invention includes an optical filter layer, an image sensor, and a processor. The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area. Correspondingly, the image sensor is used to convert the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points. At the same time, the processor connected to the image sensor is used to perform full connection processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in this environmental monitoring chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this environmental monitoring chip implement the related functions of the input layer and the linear layer in the artificial neural network. That is, the embodiment of the present invention separates the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and uses hardware to implement these two layers of structures in the artificial neural network. As a result, when using this environmental monitoring chip for intelligent processing of artificial neural network environmental monitoring subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. It only needs to perform related processing of full connection and non-linear activation with the electrical signals by the processor in the environmental monitoring chip. In this way, the power consumption and delay during artificial neural network environmental monitoring can be significantly reduced.It can be seen that in the embodiment of the present invention, the optical filter layer is used as the input layer of the artificial neural network, and the image sensor is used as the linear layer of the artificial neural network. The filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer. By using the optical filter layer and the image sensor, the information carried by the incident light of environmental pollutants is projected into an electrical signal, and then the full connection processing and non-linear activation processing of the electrical signal are realized in the processor. It can be seen that the embodiment of the present invention can not only omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also actually utilize the image information, spectral information, angle of incident light, and phase information of incident light of environmental pollutants in the embodiment of the present invention, that is, the information carried by the incident light at different points in space of environmental pollutants. It can be seen that since the information carried by the incident light at different points in space of environmental pollutants covers the image, composition, shape, three-dimensional depth, structure, etc. of environmental pollutants, when performing recognition processing based on the information carried by the incident light at different points in space of environmental pollutants, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of environmental pollutants, accurately identify and / or qualitatively analyze environmental pollutants, and then accurately obtain the environmental protection monitoring results. Moreover, the embodiment of the present invention can detect environmental pollutants in a large range, not limited to the single-point measurement of traditional environmental monitoring instruments, with richer detection samples and higher accuracy. It can be seen that the optical artificial neural network environmental protection monitoring chip provided by the embodiment of the present invention can not only achieve the effects of low power consumption and low delay, but also achieve the effect of high accuracy.

[0089] Based on the content of the above embodiment, in this embodiment, the optical artificial neural network environmental protection monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;

[0090] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network environmental protection monitoring chip including different optical modulation structures, image sensors, and processors with different full connection parameters and different non-linear activation parameters, which is trained by using the input training samples and output training samples corresponding to the environmental protection monitoring intelligent processing task to meet the training convergence conditions of the optical modulation structure, image sensor, and processor;

[0091] The input training samples include the incident light reflected or transmitted or radiated by samples with different environmental pollutants; the output training samples include the content of environmental pollutants.

[0092] In this embodiment, the optical artificial neural network environmental monitoring chip can be used for the intelligent processing task of environmental protection monitoring of environmental pollutants. In this embodiment, the reflected light, transmitted light, and / or radiation light of environmental pollutants enter the trained optical artificial neural network environmental monitoring chip to obtain the intelligent processing result of environmental protection monitoring of environmental pollutants. Among them, the intelligent processing result of environmental protection monitoring includes the intelligent processing result of environmental protection monitoring of environmental pollutants and / or the qualitative analysis result of environmental pollution.

[0093] It can be understood that since the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the optical modulation structure in the optical filter layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence condition, the optical modulation structure suitable for the current intelligent processing task of environmental protection monitoring, as well as the fully connected parameters and non-linear activation parameters suitable for the current intelligent processing task of environmental protection monitoring, are determined, thereby completing the training of the optical artificial neural network environmental monitoring chip.

[0094] In this embodiment, taking the intelligent processing task of environmental protection monitoring of environmental pollutants as an example for illustration, it can be understood that when using this environmental monitoring chip for the intelligent processing task of environmental protection monitoring, it is first necessary to train the optical artificial neural network environmental monitoring chip. Here, training the optical artificial neural network environmental monitoring chip means collecting a large number of environmental pollutant samples (including air, water, soil, etc.) as the training set in the early stage to determine the optical modulation structure suitable for the current intelligent processing task of environmental protection monitoring, as well as the fully connected parameters and non-linear activation parameters suitable for the current intelligent processing task of environmental protection monitoring.

[0095] It can be understood that since the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the optical modulation structure in the optical filter layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence condition, the optical modulation structure suitable for the current intelligent processing task of environmental protection monitoring, as well as the fully connected parameters and non-linear activation parameters suitable for the current intelligent processing task of environmental protection monitoring, are determined, thereby completing the training of the environmental monitoring chip.

[0096] It can be understood that the input training samples include incident light reflected, transmitted, and / or radiated by samples with different environmental pollutants. In order to accurately conduct environmental protection monitoring in this embodiment, samples that are typically representative in the ecological environment state are selected for training, such as air samples with different environmental pollutants, water quality samples with different environmental pollutants, and soil samples with different environmental pollutants, so as to reflect the current state of the ecological environment, and then accurately train the environmental protection monitoring chip, so that the environmental protection monitoring intelligent processing results finally obtained by applying the environmental protection monitoring chip can accurately reflect the content of environmental pollutants in the object to be measured (such as air, water, or soil). It can be understood that after training the environmental protection monitoring chip, the environmental protection monitoring chip can be used to execute the environmental protection monitoring intelligent processing task. Specifically, when the incident light carrying environmental pollutant image information and spatial spectral information enters the optical filter layer 1 of the trained environmental protection monitoring chip, the optical modulation structure in the optical filter layer 1 will modulate the incident light, and the intensity of the modulated optical signal is detected by the image sensor 2 and converted into an electrical signal, and then fully connected processing and non-linear activation processing are performed by the processor 3 to obtain the detection result of the content of environmental pollutants.

[0097] As Figure 4 shown, the complete process for environmental pollutant environmental protection monitoring is as follows: The light source 200 under the detection instrument irradiates the environmental pollutant sample 300, and then the reflected light or transmitted light of the environmental pollutant is collected by the optical artificial neural network environmental protection monitoring chip 100, or the light directly radiated outward by the environmental pollutant is collected by the optical artificial neural network environmental protection monitoring chip 100. After being processed by the optical filter layer, image sensor, and processor in the environmental protection monitoring chip, the environmental protection monitoring intelligent processing result can be obtained.

[0098] Among them, the trained optical artificial neural network environmental protection monitoring chip refers to the optical artificial neural network environmental protection monitoring chip including a trained optical modulation structure, image sensor, and processor; the trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network environmental protection monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, which are trained using the input training samples and output training samples corresponding to the recognition processing task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions.

[0099] For example, for the intelligent processing task of environmental monitoring, the input training sample corresponding to the intelligent processing task of environmental monitoring is an environmental pollutant sample, and the output training sample corresponding to the intelligent processing task of environmental monitoring is the intelligent processing result of the environmental pollutant sample for environmental monitoring. It can be understood that for the intelligent processing task of environmental monitoring, since the advantage of the environmental monitoring chip provided in this embodiment also lies in being able to obtain image information, spectral information, incident light angle information, and incident light phase information at different points in the space of environmental pollutants, therefore, to make full use of this advantage, real environmental pollutants are preferentially used as the environmental pollutant samples for the input training samples, rather than two-dimensional images of environmental pollutants. Of course, this does not mean that two-dimensional images cannot be used as environmental pollutant samples.

[0100] In this embodiment, the optical filter layer 1 serves as the input layer of the neural network, and the image sensor 2 serves as the linear layer of the neural network. In order to minimize the loss function of the neural network, the modulation intensity of different wavelength components in the incident light of environmental pollutants by the optical modulation structure in the optical filter layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the filter, the modulation intensity of different wavelength components in the incident light of environmental pollutants can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, and further optimizing the training of the neural network.

[0101] Therefore, in this embodiment, the optical modulation structure is obtained based on neural network training. Through computer optical simulation of environmental pollutant samples, the sample modulation intensity of different wavelength components of the incident light of environmental pollutants by the optical modulation structure in the environmental pollutant samples is obtained. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network, and non-linear activation is performed. The neural network is trained using the training samples corresponding to the intelligent processing task of environmental monitoring until the neural network converges, and the corresponding environmental pollutant sample optical modulation structure is used as the optical filter layer for the corresponding intelligent processing task of environmental monitoring.

[0102] It can be seen that in this embodiment, by implementing the input layer (optical filter layer) and the linear layer (image sensor) of the neural network at the physical layer, not only can the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art be omitted, but in fact, the image information, spectral information, incident light angle, and incident light phase information of environmental pollutants are simultaneously utilized in this embodiment of the present invention, that is, the incident light at different points in space of environmental pollutants carries information. It can be seen that since the incident light at different points in space of environmental pollutants carries information covering the image, composition, shape, three-dimensional depth, structure, etc. of environmental pollutants, when performing recognition processing based on the information carried by the incident light at different points in space of environmental pollutants, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of environmental pollutants can be covered. It can be seen that the optical artificial neural network environmental monitoring chip provided by this embodiment of the present invention can not only achieve the effects of low power consumption and low latency, but also achieve the effect of high accuracy.

[0103] Based on the content of the above embodiment, in this embodiment, when training an optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0104] In this embodiment, the optical modulation structure is designed through computer optical simulation, and the optical modulation structure is adjusted through optical simulation until the neural network converges, and then the corresponding optical modulation structure is determined as the size of the optical modulation structure to be finally fabricated, saving prototype production time and cost, improving product efficiency, and easily solving complex optical problems. Since environmental pollutants contain different component substances, the reflected light, transmitted light, and / or radiation light of different component substances are also different, that is, the incident light entering the optical modulation structure is also different, and thus the optical modulation intensity corresponding to different component substances is also different. In addition, this embodiment can detect environmental pollutants over a large range, not limited to the single-point measurement of environmental monitoring instruments in traditional methods, obtaining richer detection samples and higher accuracy of environmental monitoring results.

[0105] For example, the optical modulation structure can be simulated and designed through FDTD software. By changing the optical modulation structure in optical simulation, the modulation intensity of the optical modulation structure for different incident lights can be accurately predicted, and it can be used as the connection weight between the input layer and the linear layer of the neural network to train the optical artificial neural network environmental monitoring chip and accurately obtain the optical modulation structure.

[0106] It can be seen that in this embodiment, by designing the optical modulation structure by means of computer optical simulation design, the prototype production time and cost of the optical modulation structure are saved, and the product efficiency is improved.

[0107] Based on the content of the above embodiments, in this embodiment, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure.

[0108] In this embodiment, the optical modulation structure in the optical filter layer may only include a regular structure, may only include an irregular structure, or may include both a regular structure and an irregular structure.

[0109] In this embodiment, it can be understood that the optical modulation structure including a regular structure may include: the smallest modulation unit included in the optical modulation structure is a regular structure, the arrangement manner of the smallest modulation units included in the optical modulation structure is regular, and the smallest modulation unit included in the optical modulation structure is a regular structure, and at the same time, the arrangement manner of the smallest modulation units is also regular, etc.

[0110] Among them, the smallest modulation unit included in the optical modulation structure may be a regular figure such as a rectangle, a square, and a circle. The arrangement manner of the smallest modulation units included in the optical modulation structure may be that the smallest modulation units are arranged in a regular array form, in a regular circular form, in a regular trapezoidal form, in a regular polygonal form, etc.

[0111] In this embodiment, the optical modulation structure including an irregular structure here may refer to: the smallest modulation unit included in the optical modulation structure is an irregular structure, such as the smallest modulation unit may be an irregular polygon, a random shape, and other irregular figures. In addition, the optical modulation structure including an irregular structure here may also refer to: the arrangement manner of the smallest modulation units included in the optical modulation structure is irregular, such as the arrangement manner may be an irregular polygonal form, a random arrangement form, etc. In addition, the optical modulation structure including an irregular structure here may also refer to: the smallest modulation unit included in the optical modulation structure is an irregular structure, and at the same time, the arrangement manner of the smallest modulation units is also irregular, etc.

[0112] Based on the content of the above embodiments, in this embodiment, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0113] In this embodiment, the optical modulation structure in the optical filter layer may include a discrete structure, may include a continuous structure, or may include both a discrete structure and a continuous structure.

[0114] In this embodiment, the optical modulation structure including a continuous structure here may refer to: the optical modulation structure is composed of continuous modulation patterns; the optical modulation structure including a discrete structure here may refer to: the optical modulation structure is composed of discrete modulation patterns.

[0115] It can be understood that the continuous modulation patterns here may refer to linear patterns, wavy line patterns, broken line patterns, and so on.

[0116] It is understandable that the discrete modulation pattern here can refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0117] In this embodiment, it should be noted that the optical modulation structure has different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after the light passes through different groups of filter structures.

[0118] Based on the content of the above embodiment, in this embodiment, the optical filter layer is a single-layer structure or a multi-layer structure.

[0119] In this embodiment, it should be noted that the optical filter layer can be a single-layer filter structure or a multi-layer filter structure. For example, it can be a multi-layer structure such as two layers, three layers, four layers, etc.

[0120] In this embodiment, as Figure 1 shown, the optical filter layer 1 is a single-layer structure. The thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths from 400 nm to 10 μm, the thickness of the grating structure can be from 50 nm to 5 μm.

[0121] It is understandable that since the function of the optical filter layer 1 is to perform spectral modulation on the incident light, therefore, it is preferably prepared with materials having a high refractive index and low loss. For example, silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, group III-V materials, etc. can be selected for preparation. Among them, silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, etc.

[0122] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical filter layer 1 can be set as a multi-layer structure, and the corresponding optical modulation structures of each layer can be set as different structures, so as to increase the spectral modulation ability of the optical filter layer on the incident light, thereby more or more complex connection weights can be formed between the input layer and the linear layer, and further improve the accuracy of the environmental monitoring chip when processing environmental monitoring intelligent processing tasks.

[0123] In addition, it should be noted that for the filter layer including a multi-layer structure, the materials of each layer structure can be the same or different. For example, for the two-layer optical filter layer 1, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0124] It should be noted that the thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths from 400 nm to 10 μm, the total thickness of the multi-layer structure can be from 50 nm to 5 μm.

[0125] Based on the content of the above embodiments, in this embodiment, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0126] In this embodiment, in order to obtain connection weights (for the connection weights between the input layer and the linear layer) distributed in an array for the subsequent full connection and non-linear activation processing by the processor, preferably, in this embodiment, the optical modulation structure is in the form of an array structure. Specifically, the optical modulation structure includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor. It should be noted that the structures of the respective micro-nano units can be the same or different. In addition, it should be noted that the structures of the respective micro-nano units can be periodic or non-periodic. In addition, it should be noted that each micro-nano unit can further include multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different, etc. It should be noted that the structure of each micro-nano unit is a structure designed according to the corresponding environmental monitoring intelligent processing task. For example, if the environmental monitoring intelligent processing task is to detect the content of environmental pollutants, then the structure of each micro-nano unit is trained based on the incident light reflected or transmitted or radiated by the environmental pollutant sample as the input and the corresponding environmental pollutant content as the output. Thus, it can be seen that in this embodiment, different micro-nano modulation structures can be designed corresponding to different environmental monitoring intelligent processing tasks, and then environmental monitoring can be accurately and quickly carried out.

[0127] The following is an example in combination with Figures 5 - 9 In this embodiment, as Figure 5 shown, the optical filter layer 1 includes multiple repeating continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, 66. The structure of each micro-nano unit is the same (and each micro-nano unit is a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; as Figure 6 shown, the optical filter layer 1 includes multiple repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. The structure of each micro-nano unit is the same (the difference from Figure 5 is that Figure 6 each micro-nano unit in Figure 7As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (and each micro-nano unit is a periodic structure). Each micro-nano unit corresponds to one or more pixel points on the image sensor 2, and Figure 6 is different from Figure 7 in that the unit shapes of the periodic arrays within each micro-nano unit have four-fold rotational symmetry; as Figure 8 shown, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66. Different from Figure 6 is that each micro-nano unit has a different structure. Each micro-nano unit corresponds to one or more pixel points on the image sensor 2. In this embodiment, the optical filter layer 1 includes a plurality of different micro-nano units, that is, the modulation effects of different regions on the environmental monitoring chip on the incident light are different, thereby improving the design freedom, and further improving the recognition accuracy. As Figure 9 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure. Different from Figure 5 is that each micro-nano unit is composed of a discrete aperiodic array structure. Each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0128] In this embodiment, the micro-nano unit has different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after light passes through different groups of filter structures.

[0129] Based on the content of the above embodiments, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0130] In this embodiment, the micro-nano unit may only include a regular structure, or only include an irregular structure, or may include both a regular structure and an irregular structure.

[0131] In this embodiment, the micro-nano unit including a regular structure may mean that: the smallest modulation unit included in the micro-nano unit is a regular structure, the arrangement manner of the smallest modulation units included in the micro-nano unit is regular, the smallest modulation unit included in the micro-nano unit is a regular structure, and at the same time the arrangement manner of the smallest modulation units is also regular, etc. Among them, the smallest modulation unit may be a regular figure such as a rectangle, a square, and a circle, etc. The arrangement manner of the smallest modulation units included in the micro-nano unit may be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc.

[0132] In this embodiment, the micro-nano unit including an irregular structure may refer to: the smallest modulation unit included in the micro-nano unit is an irregular structure, the arrangement pattern of the smallest modulation units included in the micro-nano unit is irregular, the smallest modulation unit included in the micro-nano unit is an irregular structure, and at the same time, the arrangement pattern of the smallest modulation units is also irregular, etc.

[0133] Among them, the smallest modulation unit included in the micro-nano unit may be an irregular graph such as an irregular polygon, a random shape, etc., and the arrangement pattern of the smallest modulation units included in the micro-nano unit may be an irregular polygon form, a random arrangement form, etc.

[0134] In this embodiment, the micro-nano units in the optical filter layer may include a discrete structure, may also include a continuous structure, or may include both a discrete structure and a continuous structure.

[0135] In this embodiment, the micro-nano unit including a continuous structure may refer to: the micro-nano unit is composed of a continuous modulation pattern; the micro-nano unit including a discrete structure may refer to: the micro-nano unit is composed of discrete modulation patterns.

[0136] It can be understood that the continuous modulation pattern here may refer to a linear pattern, a wavy pattern, a broken line pattern, and so on.

[0137] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphs (such as discrete points, discrete triangles, discrete stars, etc.).

[0138] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano units, the corresponding transmission spectra are different after light passes through different groups of micro-nano units.

[0139] Based on the content of the above embodiment, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0140] In this embodiment, as Figure 5 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 different micro-nano structure arrays 110, 111, 112, and 113, and the filtering unit 44 includes 4 different micro-nano structure arrays 440, 441, 442, and 443. As Figure 10As shown in the figure, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0141] It should be noted that here only a micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of groups of micro-nano structure arrays can also be set according to needs.

[0142] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and the modulation effects of each group of filtering structures on the input light are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after light passes through different groups of micro-nano structure arrays.

[0143] Based on the content of the above embodiment, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0144] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light of environmental pollutants as the connection weight between the input layer and the linear layer of the neural network, broadband filtering and narrowband filtering are realized by adopting different micro-nano structure arrays. Therefore, in this embodiment, the micro-nano structure arrays obtain the modulation intensity of different wavelength components of the incident light of environmental pollutants by performing broadband filtering or narrowband filtering on the incident light of environmental pollutants. As Figure 11 and Figure 12 shown in the figure, each group of micro-nano structure arrays in the optical filter layer has the function of broadband filtering or narrowband filtering.

[0145] It can be understood that for each group of micro-nano structure arrays, they can all have the function of broadband filtering, or they can all have the function of narrowband filtering, or some can have the function of broadband filtering and some can have the function of narrowband filtering. In addition, the broadband filtering ranges and narrowband filtering ranges of each group of micro-nano structure arrays can also be the same or different. For example, by designing the size parameters such as the period, duty cycle, radius, side length, etc. of each group of micro-nano structures in the micro-nano unit, it has the function of narrowband filtering, that is, only light of one (or a few) wavelengths can pass through. Another example is that by designing the size parameters such as the period, duty cycle, radius, side length, etc. of each group of micro-nano structures in the micro-nano unit, it has the function of broadband filtering, that is, light of more wavelengths or all wavelengths is allowed to pass through.

[0146] It can be understood that in specific applications, the filtering states of each group of micro-nano structure arrays can be determined by means of broadband filtering, narrowband filtering, or a combination thereof according to the application scenario.

[0147] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0148] In this embodiment, each group of micro-nano structure arrays can all be periodic structure arrays, or all be aperiodic structure arrays, or some can be periodic structure arrays and some can be aperiodic structure arrays. Among them, periodic structure arrays are easy to perform optical simulation design, and aperiodic structure arrays can achieve more complex modulation effects.

[0149] In this embodiment, as Figure 5 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and the micro-nano structure arrays are aperiodic structures. Among them, the aperiodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in an aperiodic arrangement. As Figure 5 shown, the micro-nano unit 11 includes 4 different aperiodic structure arrays 110, 111, 112, and 113. The micro-nano unit 44 includes 4 different aperiodic structure arrays 440, 441, 442, and 443. The aperiodic structure micro-nano structure arrays are designed by neural network data training for the recognition processing task in the early stage, and are usually structures with irregular shapes. As Figure 6 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other. Different from Figure 5 this, the micro-nano structure arrays are periodic structures. Among them, the periodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. As Figure 6 shown, the micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112, and 113. The micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442, and 443. The filter structures of the periodic structures are designed by neural network data training for the recognition processing task in the early stage, and are usually structures with irregular shapes. As Figure 7As shown, the optical filter layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes on the filter structure are arranged in a periodic arrangement manner, and the size of the period is usually 20 nm to 50 μm. For example, Figure 7 As shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other. The micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112, and 113. The micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442, and 443. The micro-nano structure arrays of the periodic structure are designed by neural network data training for the previous recognition processing task, and are usually structures with irregular shapes.

[0150] It should be noted that Figures 5 - 9 Each micro-nano unit includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are formed by using four different-shaped modulation holes respectively. The four groups of micro-nano structure arrays have different modulation effects on the incident light. It should be noted that here only the micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, and it does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of micro-nano structure arrays can also be set according to needs. In this embodiment, the four different shapes can be circular, cross-shaped, regular polygon, and rectangular (not limited to this).

[0151] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on lights of different wavelengths, and the modulation effects on the input light between each group of micro-nano structure arrays are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after the light passes through different groups of micro-nano structure arrays.

[0152] Based on the content of the above embodiment, in this embodiment, one or more groups of the micro-nano structure arrays included in the micro-nano unit are empty structures.

[0153] Next, combined with Figure 9 the example shown for illustration. In this embodiment, for example, Figure 9As shown, the optical filter layer 1 includes multiple repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiments, for any micro-nano unit, it includes one or more groups of empty structures, and the empty structures are used to directly pass the incident light. It can be understood that when one or more groups of empty structures are included in the multiple groups of micro-nano structure arrays, a richer spectrum modulation effect can be formed, so as to meet the spectrum modulation requirements in specific scenarios (or meet the specific connection weight requirements between the input layer and the linear layer in specific scenarios).

[0154] As Figure 9 As shown, each micro-nano unit includes one group of micro-nano structure arrays and three groups of empty structures. The micro-nano unit 11 includes one aperiodic structure array 111, the micro-nano unit 22 includes one aperiodic structure array 221, the micro-nano unit 33 includes one aperiodic structure array 331, the micro-nano unit 44 includes one aperiodic structure array 441, the micro-nano unit 55 includes one aperiodic structure array 551, and the micro-nano unit 66 includes one aperiodic structure array 661, where the micro-nano structure array is used to perform different modulations on the incident light. It should be noted that here only an example of including one group of micro-nano structure arrays and three groups of empty structures is given, which does not play a restrictive role. In actual applications, micro-nano units including one group of micro-nano structure arrays and five groups of empty structures or other numbers of groups of micro-nano structure arrays can also be set according to needs. In this embodiment, the micro-nano structure array can be made of modulation holes in the shapes of circles, crosses, regular polygons, and rectangles (not limited to this).

[0155] It should be noted that none of the multiple groups of micro-nano structure arrays included in the micro-nano unit may include empty structures, that is, the multiple groups of micro-nano structure arrays can be aperiodic structure arrays or periodic structure arrays.

[0156] Based on the content of the above embodiments, in this embodiment, the micro-nano unit has polarization-independent characteristics.

[0157] In this embodiment, due to the polarization-independent characteristics of the micro-nano unit, the optical filter layer is insensitive to the polarization of the incident light, thus realizing an optical artificial neural network environmental monitoring chip that is insensitive to both the incident angle and polarization. The optical artificial neural network environmental monitoring chip provided by the embodiments of the present invention is insensitive to the incident angle and polarization characteristics of the incident light, that is, the measurement results are not affected by the incident angle and polarization characteristics of the incident light, so as to ensure the stability of the spectral measurement performance, and further ensure the stability of the environmental monitoring intelligent processing. It should be noted that the micro-nano unit can also have polarization-dependent characteristics.

[0158] Based on the content of the above embodiments, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0159] In this embodiment, it should be noted that four-fold rotational symmetry is a specific case of polarization-independent characteristics. By designing the micro-nano unit into a structure with four-fold rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0160] The following is an example in conjunction with Figure 7 the example shown. In this embodiment, as Figure 7 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures corresponding to the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiments, the structure corresponding to each group of micro-nano structure arrays can be a structure with four-fold rotational symmetry, such as a circle, a cross, a regular polygon, a rectangle, etc. That is, after the structure is rotated by 90°, 180°, and 270°, it coincides with the original structure, so that the structure has the characteristic of polarization independence, and the same environmental pollutant detection effect can be obtained when different polarized lights are incident.

[0161] Based on the content of the above embodiments, in this embodiment, the optical filter layer is composed of one or more filter layers;

[0162] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polaritons (SPP) nanostructures, tunable Fabry-Perot cavities (FP cavities).

[0163] The semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0164] Among them, the photonic crystal, as well as the combination of the metasurface and the random structure, can be compatible with the CMOS process, enabling a good modulation effect. Other materials can also be filled in the micropores of the micro-nano modulation structure for surface smoothing; quantum dots and perovskites can minimize the volume of a single modulation structure by utilizing the spectral modulation characteristics of the materials themselves; the SPP has a small volume and can achieve polarization-related optical modulation; liquid crystals can be dynamically regulated by voltage to improve the spatial resolution; the tunable Fabry-Perot resonator can be dynamically regulated to improve the spatial resolution.

[0165] Based on the content of the above embodiments, in this embodiment, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0166] In this embodiment, it should be noted that if the thickness of the optical filter layer is much smaller than the central wavelength of the incident light, it cannot play an effective spectral modulation role; if the thickness of the optical filter layer is much larger than the central wavelength of the incident light, it is difficult to fabricate in terms of technology and will introduce significant optical losses. Therefore, in this embodiment, in order to reduce optical losses and be easy to fabricate, and to ensure an effective spectral modulation effect, the overall size (area) of each micro-nano unit in the optical filter layer 1 is usually λ 2 ~10 5 λ 2 , and the thickness is usually 0.1λ to 10λ (λ represents the central wavelength of the incident light of environmental pollutants). As Figure 5 shown, the overall size of each micro-nano unit is 0.5μm 2 ~40000μm 2 , and the dielectric material in the optical filter layer 1 is polysilicon with a thickness of 50nm to 2μm.

[0167] Based on the content of the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0168] CMOS image sensor (Contact Image Sensor, CIS), charge-coupled device (Charge Coupled Device, CCD), single-photon avalanche diode (Single Photon Avalanche Diode, SPAD) array, and focal plane photodetector array.

[0169] In this embodiment, it should be noted that by using a wafer-level CMOS image sensor CIS and achieving monolithic integration at the wafer level, the distance between the image sensor and the optical filter layer can be minimized to the greatest extent, which is beneficial to reducing the size of the unit, decreasing the device volume and packaging cost. The SPAD can be used for weak light detection, and the CCD can be used for strong light detection.

[0170] In this embodiment, the optical filter layer and the image sensor can be fabricated by a complementary metal oxide semiconductor (CMOS) integrated process, which is beneficial to reducing the device failure rate, improving the device yield, and reducing costs. For example, one or more dielectric materials can be directly grown on the image sensor and then etched. Before removing the sacrificial layer used for etching, a metal material is deposited, and finally the sacrificial layer is removed to prepare the optical filter layer.

[0171] Based on the content of the above embodiment, in this embodiment, the types of the artificial neural network include: feedforward neural network.

[0172] In this embodiment, a feedforward neural network (FNN), also known as a deep feedforward network (DFN) or a multi-layer perceptron (MLP), is the simplest neural network, and its neurons are arranged in layers. Each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer, and there is no feedback between layers. The feedforward neural network has a simple structure, is easy to implement on hardware, has a wide range of applications, can approximate any continuous function and square-integrable function with arbitrary precision, and can accurately implement any finite training sample set. The feedforward network is a static non-linear mapping. Through the composite mapping of simple non-linear processing units, complex non-linear processing capabilities can be obtained.

[0173] Based on the content of the above embodiment, in this embodiment, a light-transmitting dielectric layer is provided between the optical filter layer and the image sensor.

[0174] In this embodiment, it should be noted that setting a light-transmitting dielectric layer between the optical filter layer and the image sensor can effectively separate the optical filter layer and the image sensor layer and avoid interference between the two.

[0175] Based on the content of the above embodiment, in this embodiment, the image sensor is a front-illuminated type and includes: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on the side of the metal wire layer away from the light detection layer; or,

[0176] The image sensor is a back-illuminated type and includes: a light detection layer and a metal wire layer arranged from top to bottom, and the optical filter layer is integrated on the side of the light detection layer away from the metal wire layer.

[0177] In this embodiment, for the front-illuminated image sensor, the silicon detection layer is below the metal wire layer, and the optical filter layer 1 is directly integrated onto the metal wire layer.

[0178] In this embodiment, the difference between the back-illuminated image sensor and the front-illuminated image sensor is that the silicon detection layer is above the metal wire layer, and the optical filter layer 1 is directly integrated onto the silicon detection layer.

[0179] It should be noted that for the back-illuminated image sensor, the silicon detection layer above the metal wire layer can reduce the influence of the metal wire layer on the incident light, thereby improving the quantum efficiency of the device.

[0180] According to the above content, in this embodiment, the optical filter layer is used as the input layer of the artificial neural network, the image sensor is used as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor in the optical artificial neural network environmental monitoring chip provided in this embodiment implement the related functions of the input layer and the linear layer in the artificial neural network through hardware, so that subsequent complex signal processing and algorithm processing corresponding to the input layer and the linear layer are no longer required when using this environmental monitoring chip for recognition processing, which can significantly reduce the power consumption and delay during the processing of the artificial neural network. In addition, since this embodiment simultaneously utilizes the image information, spectral information, incident light angle information, and incident light phase information at different points in space of environmental pollutants, the recognition processing of environmental pollutants can be more accurately achieved.

[0181] Based on the same inventive concept, another embodiment of the present invention provides an environmental monitoring instrument, including: the optical artificial neural network environmental monitoring chip as described in the above embodiment.

[0182] Since the environmental monitoring instrument provided in this embodiment includes the optical artificial neural network environmental monitoring chip described in the above embodiment, therefore, the environmental monitoring instrument provided in this embodiment has all the beneficial effects of the optical artificial neural network environmental monitoring chip described in the above embodiment. Since the above embodiment has described this in detail, this embodiment will not be repeated here.

[0183] Based on the same inventive concept, another embodiment of the present invention provides a preparation method of the optical artificial neural network environmental monitoring chip as described in the above embodiment, as Figure 13 shown, which specifically includes the following steps:

[0184] Step 1310: Prepare an optical filter layer containing an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0185] Step 1320: Generate a processor with the functions of fully connecting and non-linearly activating the signal for processing;

[0186] Step 1330: Connect the image sensor and the processor;

[0187] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light;

[0188] The image sensor is used to convert the information carried by the incident light corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an intelligent environmental monitoring processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light of environmental pollutants.

[0189] In this embodiment, the training process of the optical artificial neural network environmental monitoring chip is further included, specifically including:

[0190] Using the input training samples and output training samples corresponding to the intelligent environmental monitoring processing task, training the optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0191] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor, including:

[0192] Growing one or more layers of preset materials on the surface of the photosensitive area of the image sensor;

[0193] Performing dry etching on the one or more layers of preset materials with an optical modulation structure pattern to obtain an optical filter layer including an optical modulation structure;

[0194] Or performing imprint transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0195] Or obtaining an optical filter layer including an optical modulation structure by applying external dynamic regulation to the one or more layers of preset materials;

[0196] Or perform zone printing on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0197] Or perform zone material growth on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0198] Or perform quantum dot transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure.

[0199] When the optical artificial neural network environmental monitoring chip is used for the recognition and processing task of environmental pollutants, the optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters is trained by using the input training samples and output training samples corresponding to the recognition and processing task to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions.

[0200] In this embodiment, it should be noted that as Figure 1 shown, the optical filter layer 1 can be prepared by directly growing one or more layers of dielectric materials on the image sensor 2, then performing etching, depositing metal materials before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the optical modulation structure, each unit can have different modulation effects on lights with different wavelengths within the target range, and this modulation effect is insensitive to the incident angle and polarization. Each unit in the optical filter layer 1 corresponds to one or more pixels on the image sensor 2. 1 is directly prepared on 2.

[0201] In this embodiment, it should be noted that assuming the image sensor 2 is a back-illuminated structure, the optical filter layer 1 can be directly etched on the silicon detector layer of the back-illuminated image sensor, and then metal is deposited for preparation.

[0202] In addition, it should be noted that the optical modulation structure on the optical filter layer can be obtained by dry etching the optical modulation structure pattern on one or more preset materials. Dry etching is to directly remove the unnecessary parts of one or more preset materials on the surface of the photosensitive area of the image sensor, so as to obtain an optical filter layer containing the optical modulation structure; or by imprint transfer of one or more preset materials. Imprint transfer is to prepare the required structure by etching on other substrates, and then transfer the structure to the photosensitive area of the image sensor through materials such as PDMS, so as to obtain an optical filter layer containing the optical modulation structure; or by applying external dynamic regulation to one or more preset materials. External dynamic regulation is to use active materials, and then apply electrodes to regulate the optical modulation characteristics of the corresponding area by changing the voltage, so as to obtain an optical filter layer containing the optical modulation structure; or by zone printing of one or more preset materials. Zone printing is to use the printing technology in zones to obtain an optical filter layer containing the optical modulation structure; or by zone material growth of one or more preset materials to obtain an optical filter layer containing the optical modulation structure; or by quantum dot transfer of one or more preset materials to obtain an optical filter layer containing the optical modulation structure.

[0203] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the optical artificial neural network environmental monitoring chip in the above embodiment, for the detailed content of some principles and structures, etc., reference can be made to the introduction of the above embodiment, and this embodiment will not elaborate on it here.

[0204] Based on this, the optical artificial neural network environmental monitoring chip provided by the embodiments of the present invention realizes a brand-new intelligent chip capable of implementing the functions of an artificial neural network. In this intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this intelligent chip realize the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiments of the present invention separate the input layer and the linear layer in the artificial neural network implemented by software in the prior art and implement these two-layer structures in the artificial neural network by means of hardware. As a result, when using this intelligent chip for artificial neural network intelligent processing subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. It only needs the processor in the intelligent chip to perform relevant processing related to the full connection and non-linear activation of electrical signals. In this way, the power consumption and delay during artificial neural network processing can be significantly reduced. Moreover, the embodiments of the present invention can also utilize the image information, spectral information, angle of incident light, and phase information of environmental pollutants at the same time, that is, the incident light at different points in space of environmental pollutants carries information. It can be seen that since the incident light at different points in space of environmental pollutants carries information covering the image, composition, shape, three-dimensional depth, structure, etc. of environmental pollutants, when performing recognition processing based on the information carried by the incident light at different points in space of environmental pollutants, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of environmental pollutants can be covered, thereby improving the accuracy of environmental monitoring results. It can be seen that the optical artificial neural network environmental monitoring chip provided by the embodiments of the present invention can not only achieve the effects of low power consumption and low delay, but also improve the accuracy rate of environmental monitoring results, and thus can be preferably applied in the field of environmental monitoring and processing.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An environmental monitoring chip for an optical artificial neural network, characterized in that, For environmental monitoring intelligent processing tasks, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network; The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the incident light includes the reflected light, transmitted light, and / or radiation light of environmental pollutants; The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signal is an image signal modulated by the optical filter layer; The processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the environmental monitoring intelligent processing result; Wherein, the environmental monitoring intelligent processing task includes the identification and / or qualitative analysis of environmental pollutants; the environmental monitoring intelligent processing result includes the environmental monitoring intelligent processing result of environmental pollutants and / or the environmental pollution qualitative analysis result.

2. The optoelectronic artificial neural network environmental monitoring chip according to claim 1, characterized in that The information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

3. The optoelectronic artificial neural network environmental monitoring chip according to claim 1, characterized in that The optical artificial neural network environmental monitoring chip includes a trained optical modulation structure, an image sensor, and a processor; The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which are trained using the input training samples and output training samples corresponding to the environmental monitoring intelligent processing tasks to obtain the optical modulation structure, image sensor, and processor that meet the training convergence conditions; The input training samples include the incident light reflected, transmitted, and / or radiated by samples with different environmental pollutants; the output training samples include the content of environmental pollutants.

4. The optoelectronic artificial neural network environmental monitoring chip according to claim 3, wherein, The samples with different environmental pollutants include air samples with different pollutants, water quality samples with different pollutants, and / or soil samples with different pollutants.

5. The optoelectronic artificial neural network environmental monitoring chip according to claim 3, characterized in that, When training the optical artificial neural network environmental monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

6. The optical artificial neural network environmental monitoring chip according to any one of claims 1 to 4, characterized in that, The optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

7. The optoelectronic artificial neural network environmental monitoring chip according to any one of claims 1 to 4, characterized in that The optical modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

8. The optical artificial neural network environmental monitoring chip according to claim 7, characterized in that, The micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

9. The optoelectronic artificial neural network environmental monitoring chip according to claim 7, characterized in that The micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

10. The optical artificial neural network environmental monitoring chip according to claim 9, wherein, Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

11. The optical artificial neural network environmental monitoring chip according to claim 9, characterized in that, Each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

12. The optical artificial neural network environmental monitoring chip according to claim 9, wherein, One or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

13. The optoelectronic artificial neural network environmental monitoring chip according to claim 9, wherein The micro-nano unit has polarization-independent characteristics.

14. The optical artificial neural network environmental monitoring chip according to claim 13, characterized in that, The micro-nano unit has four-fold rotational symmetry.

15. The optical artificial neural network environmental monitoring chip according to claim 1, characterized in that, The optical filter layer is composed of one or more filter layers; The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP nanostructures, tunable Fabry-Perot resonators.

16. The optoelectronic artificial neural network environmental monitoring chip according to claim 15, wherein, The semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

17. The optical artificial neural network environmental monitoring chip according to claim 1, characterized in that, The thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

18. An environmental monitoring instrument, characterized in that, Including: The optical artificial neural network environmental monitoring chip according to any one of claims 1 to 17.

19. A method for preparing an optical artificial neural network environmental monitoring chip according to any one of claims 1 to 17, characterized in that, Including: Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor; Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals; Connecting the image sensor and the processor; Wherein, the optical filter layer is used to respectively perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light; The image sensor is used to convert the information carried by the incident light corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an intelligent environmental monitoring processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light of environmental pollutants.

20. The preparation method of the optical artificial neural network environmental monitoring chip according to claim 19, wherein, Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor, including: Grow one or more layers of preset materials on the surface of the photosensitive area of the image sensor; Etch a light modulation structure pattern on the one or more layers of preset materials to obtain an optical filter layer including a light modulation structure; Or perform imprint transfer on the one or more layers of preset materials to obtain an optical filter layer including a light modulation structure; Or obtain an optical filter layer including a light modulation structure by applying external dynamic modulation to the one or more layers of preset materials; Or perform partition printing on the one or more layers of preset materials to obtain an optical filter layer including a light modulation structure; Or perform partition growth on the one or more layers of preset materials to obtain an optical filter layer including a light modulation structure; Or perform quantum dot transfer on the one or more layers of preset materials to obtain an optical filter layer including a light modulation structure.

21. The preparation method of the optical artificial neural network environmental monitoring chip according to claim 19, characterized in that, Further included are: The training process of the optical artificial neural network environmental monitoring chip specifically includes: Using input training samples and output training samples corresponding to the environmental monitoring intelligent processing task, training an optical artificial neural network environmental monitoring chip including different light modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain a light modulation structure, image sensor, and processor that meet the training convergence conditions, and using the light modulation structure, image sensor, and processor that meet the training convergence conditions as the trained light modulation structure, image sensor, and processor.

Images