Optical Artificial Neural Network Intelligent Chip, Intelligent Processing Device and Preparation Method

Abstract

The present invention provides an optical artificial neural network intelligent chip, an intelligent processing device and a preparation method. The optical filter layer is used as the input layer of the artificial neural network, the image sensor is used as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer. Thus, the optical filter layer and the image sensor in the intelligent chip implement the relevant functions of the input layer and the linear layer in the artificial neural network in a hardware manner, so that when using the intelligent chip for intelligent processing subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, thereby greatly reducing the power consumption and delay during the processing of the artificial neural network.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to an optical artificial neural network intelligent chip, an intelligent processing device and a preparation method thereof. Background Art

[0002] In the existing intelligent recognition technology, it is usually necessary to first image a person or an object, and then input the image into a neural network recognition model for processing, so as to realize the recognition of the person or the object.

[0003] It can be seen that the current intelligent recognition tasks generally rely on the neural network recognition model. That is to say, in the current intelligent recognition tasks, imaging needs to be carried out first and then transmitted to a computer for subsequent neural network recognition model algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay. Summary of the Invention

[0004] In view of the problems existing in the prior art, embodiments of the present invention provide an optical artificial neural network intelligent chip, an intelligent processing device and a preparation method thereof.

[0005] Specifically, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, an optical artificial neural network intelligent chip provided by an embodiment of the present invention includes: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0007] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor, and the optical filter layer includes an optical modulation structure. The optical filter layer is configured to perform different spectral modulations on incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain incident light carrying information corresponding to different position points on the surface of the photosensitive area;

[0008] The image sensor is configured to convert the incident light carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0009] The processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an output signal of the artificial neural network.

[0010] Furthermore, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0011] Furthermore, the optical artificial neural network intelligent chip is used for intelligent processing tasks of a target object; the intelligent processing tasks include at least one or more of intelligent perception, intelligent recognition, and intelligent decision-making tasks;

[0012] The reflected light, transmitted light, and / or radiated light of the target object enter the trained optical artificial neural network intelligent chip to obtain the intelligent processing result of the target object; the intelligent processing result includes at least one or more of intelligent perception results, intelligent recognition results, and / or intelligent decision-making results;

[0013] Among them, the trained optical artificial neural network intelligent chip refers to an optical artificial neural network intelligent chip including a trained optical modulation structure, an image sensor, and a processor; the trained optical modulation structure, image sensor, and processor refer to an optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, which is trained using input training samples and output training samples corresponding to the intelligent processing tasks to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions.

[0014] Furthermore, when training an optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0015] Furthermore, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0016] Furthermore, the optical filter layer is a single-layer structure or a multi-layer structure.

[0017] Furthermore, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0018] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0019] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0020] Furthermore, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0021] Furthermore, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0022] Furthermore, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0023] Furthermore, the micro-nano unit has polarization-independent characteristics.

[0024] Furthermore, the micro-nano unit has four-fold rotational symmetry.

[0025] Furthermore, the optical filter layer is composed of one or more filter layers;

[0026] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, tunable Fabry-Perot resonators.

[0027] Furthermore, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0028] Furthermore, the thickness of the optical filter layer is 0.1λ - 10λ, where λ represents the central wavelength of the incident light

[0029] Furthermore, the image sensor is any one or more of the following:

[0030] CMOS image sensor CIS, charge-coupled device CCD, single-photon avalanche diode SPAD array, and focal plane photodetector array.

[0031] Furthermore, the type of the artificial neural network includes: feedforward neural network.

[0032] Furthermore, a light-transmitting medium layer is provided between the optical filter layer and the image sensor.

[0033] Furthermore, the image sensor is front-illuminated and includes: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on a side of the metal wire layer away from the light detection layer; or,

[0034] The image sensor is a back-illuminated type, including: a light detection layer and a metal wire layer arranged from top to bottom, and the optical filter layer is integrated on a side of the light detection layer away from the metal wire layer.

[0035] In a second aspect, an intelligent processing device provided by an embodiment of the present invention includes: the optical artificial neural network intelligent chip as described in the first aspect.

[0036] Further, the intelligent processing device includes one or more of a smart phone, a smart computer, a smart identification device, a smart sensing device, and a smart decision-making device.

[0037] In a third aspect, a preparation method of an optical artificial neural network intelligent chip provided by an embodiment of the present invention includes:

[0038] Preparing an optical filter layer including an optical modulation structure on a surface of a photosensitive area of the image sensor;

[0039] Generating a processor with functions of performing fully connected processing and non-linear activation processing on signals;

[0040] Connecting the image sensor and the processor;

[0041] Wherein, the optical filter layer is used to respectively perform different spectral modulations on incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0042] The image sensor is used to convert the incident light carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an output signal of the artificial neural network; the electrical signal is an image signal modulated by the optical filter layer.

[0043] Further, preparing an optical filter layer including an optical modulation structure on a surface of a photosensitive area of the image sensor includes:

[0044] Growing one or more layers of preset materials on a surface of the photosensitive area of the image sensor;

[0045] Etching a pattern of the optical modulation structure on the one or more layers of preset materials to obtain an optical filter layer including the optical modulation structure;

[0046] Or perform imprint transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0047] Or obtain an optical filter layer including an optical modulation structure by applying external dynamic modulation to the one or more layers of preset materials;

[0048] Or perform zone printing on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0049] Or perform zone growth on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0050] Or perform quantum dot transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure.

[0051] Further, when the optical artificial neural network intelligent chip is used for the intelligent processing task of a target object, the optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters is trained by using the input training samples and output training samples corresponding to the intelligent processing task, so as to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions.

[0052] An embodiment of the present invention further provides an optical artificial neural network face recognition chip for face recognition processing tasks, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0053] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor, the optical filter layer includes an optical modulation structure, and the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the incident light is the reflected light of the user's face;

[0054] The image sensor is used to convert the information carried by the incident light corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0055] The processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a face recognition processing result.

[0056] Furthermore, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0057] Furthermore, the optical artificial neural network face recognition chip includes a trained optical modulation structure, an image sensor, and a processor;

[0058] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which is trained using input training samples and output training samples corresponding to the face recognition task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0059] The input training samples are incident lights reflected by different faces under different lighting environments; the output training samples include corresponding face recognition results.

[0060] Furthermore, when training the optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0061] Furthermore, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0062] Furthermore, the optical filter layer is a single-layer structure or a multi-layer structure.

[0063] Furthermore, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0064] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0065] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0066] Furthermore, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0067] Furthermore, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0068] Further, one or more groups of the micro-nano structure arrays included in the micro-nano unit are empty structures.

[0069] Further, the micro-nano unit has polarization-independent characteristics.

[0070] Further, the polarization-independent micro-nano unit has four-fold rotational symmetry.

[0071] Further, the optical filter layer is composed of one or more filter layers;

[0072] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) micro-nano structures, tunable Fabry–Pérot resonators.

[0073] Further, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0074] Further, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0075] An embodiment of the present invention further provides a face recognition device, including the optical artificial neural network face recognition chip as described above.

[0076] An embodiment of the present invention further provides a method for manufacturing the optical artificial neural network face recognition chip as described above, including:

[0077] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0078] Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals;

[0079] Connecting the image sensor and the processor;

[0080] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain incident light-carrying information corresponding to different position points on the surface of the photosensitive area; the incident light-carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0081] The image sensor is configured to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a face recognition processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light is the reflected light of a human face under different lighting environments.

[0082] Further, the method for manufacturing the optical artificial neural network face recognition chip further includes: a training process for the optical artificial neural network face recognition chip, specifically including:

[0083] Using input training samples and output training samples corresponding to the face recognition task, training an optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0084] Further, when training an optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0085] An embodiment of the present invention further provides an optical artificial neural network blood glucose detection chip for blood glucose detection tasks, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0086] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor, the optical filter layer includes an optical modulation structure, and the optical filter layer is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; wherein, the incident light includes the reflected light and / or transmitted light of the part of the human body to be measured.

[0087] The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0088] The processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a blood glucose detection result.

[0089] Further, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0090] Further, the optical artificial neural network blood glucose detection chip includes a trained optical modulation structure, an image sensor, and a processor;

[0091] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which are trained using input training samples and output training samples corresponding to the blood glucose detection task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0092] The input training samples include incident light reflected and transmitted by a human body part to be measured with different blood glucose values; the output training samples include corresponding blood glucose values.

[0093] Further, when training the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0094] Further, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0095] Further, the optical filter layer is a single-layer structure or a multi-layer structure.

[0096] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0097] Furthermore, the micro-nano unit includes regular structures and / or irregular structures; and / or, the micro-nano unit includes discrete structures and / or continuous structures.

[0098] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0099] Furthermore, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0100] Furthermore, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0101] Furthermore, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0102] Furthermore, the micro-nano unit has polarization-independent characteristics.

[0103] Furthermore, the polarization-independent micro-nano unit has four-fold rotational symmetry.

[0104] Furthermore, the optical filter layer is composed of one or more filter layers;

[0105] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) micro-nano structures, tunable Fabry-Perot resonators.

[0106] Furthermore, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0107] Furthermore, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0108] An embodiment of the present invention also provides an intelligent blood glucose detector, including the optical artificial neural network blood glucose detection chip as described above.

[0109] An embodiment of the present invention also provides a preparation method of the optical artificial neural network blood glucose detection chip as described above, including:

[0110] Preparing an optical filter layer containing an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0111] Generate a processor with the functions of fully connecting and non-linearly activating the signals;

[0112] Connect the image sensor and the processor;

[0113] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0114] The image sensor is used to convert the information carried by the incident light corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a blood glucose detection result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light and / or transmitted light of the human body part to be measured.

[0115] Furthermore, the preparation method of the optical artificial neural network blood glucose detection chip further includes: the training process of the optical artificial neural network blood glucose detection chip, specifically including:

[0116] Using input training samples and output training samples corresponding to the blood glucose detection task, train the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0117] Furthermore, when training the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0118] The embodiment of the present invention also provides an optical artificial neural network intelligent agriculture precise control chip for agricultural precise control intelligent processing tasks, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0119] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is configured to perform different spectral modulations on incident light entering different position points of the optical modulation structure respectively through the optical modulation structure, so as to obtain information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the incident light includes reflected light, transmitted light, and / or radiation light of agricultural objects; the agricultural objects include crops and / or soil;

[0120] The image sensor is configured to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0121] The processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an agricultural precise control processing result;

[0122] Wherein, the agricultural precise control intelligent processing tasks include one or more of soil fertility detection, pesticide spraying detection, trace element content detection, drug resistance detection, and crop growth condition detection; the agricultural precise control processing results include one or more of soil fertility detection results, pesticide spraying detection results, trace element content detection results, drug resistance detection results, and crop growth condition detection results.

[0123] Further, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0124] Further, the optical artificial neural network intelligent agricultural precise control chip includes a trained optical modulation structure, an image sensor, and a processor;

[0125] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network intelligent agricultural precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which are trained by using input training samples and output training samples corresponding to the agricultural precise control intelligent processing tasks to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0126] Wherein, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different fertilities; the output training samples include corresponding soil fertilities;

[0127] and / or,

[0128] The input training samples include incident light reflected, transmitted, and / or radiated by soils with different pesticide spreading conditions; the output training samples include the corresponding pesticide spreading conditions;

[0129] and / or,

[0130] The input training samples include incident light reflected, transmitted, and / or radiated by soils with different trace element content conditions; the output training samples include the corresponding trace element content conditions;

[0131] and / or,

[0132] The input training samples include incident light reflected, transmitted, and / or radiated by soils with different drug resistances; the output training samples include the corresponding drug resistances;

[0133] and / or,

[0134] The input training samples include incident light reflected, transmitted, and / or radiated by crops with different growth conditions; the output training samples include the corresponding crop growth conditions.

[0135] Further, when training an optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0136] Further, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0137] Further, the optical filter layer is a single-layer structure or a multi-layer structure.

[0138] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0139] Further, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0140] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0141] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0142] Furthermore, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0143] Furthermore, one or more groups of the multi-group micro-nano structure arrays included in the micro-nano unit are empty structures.

[0144] Furthermore, the micro-nano unit has polarization-independent characteristics.

[0145] Furthermore, the micro-nano unit has four-fold rotational symmetry.

[0146] Furthermore, the optical filter layer is composed of one or more filter layers;

[0147] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) micro-nano structures, tunable Fabry–Perot resonators.

[0148] Furthermore, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0149] Furthermore, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0150] An embodiment of the present invention further provides an intelligent agricultural control device, including the optical artificial neural network intelligent agricultural precise control chip as described above.

[0151] An embodiment of the present invention further provides a preparation method of the optical artificial neural network intelligent agricultural precise control chip as described above, including:

[0152] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0153] Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals;

[0154] Connecting the image sensor and the processor;

[0155] Wherein, the optical filter layer is configured to perform different spectral modulations on incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0156] The image sensor is configured to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an agricultural precision control processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light of an agricultural object; the agricultural object includes crops and / or soil.

[0157] Further, the method for manufacturing the optical artificial neural network intelligent agricultural precision control chip further includes: a training process for the optical artificial neural network intelligent agricultural precision control chip, specifically including:

[0158] Using input training samples and output training samples corresponding to the intelligent processing task of agricultural precision control to train an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0159] Further, when training an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0160] An embodiment of the present invention further provides an optical artificial neural network end-point monitoring chip for end-point monitoring tasks in smelting, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0161] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the incident light includes reflected light, transmitted light, and / or radiation light from the steelmaking furnace mouth;

[0162] The image sensor is configured to convert the information carried by the incident light corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signal is an image signal modulated by the optical filter layer;

[0163] The processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a smelting end monitoring result;

[0164] Wherein, the smelting end monitoring task includes identifying the smelting end, and the smelting end monitoring result includes the smelting end identification result.

[0165] Further, the smelting end monitoring task further includes identifying the carbon content and / or molten steel temperature during the smelting process, and the smelting end monitoring result includes the carbon content and / or molten steel temperature identification result during the smelting process.

[0166] Further, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0167] Further, the optical artificial neural network smelting end monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;

[0168] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network smelting end monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which are trained using input training samples and output training samples corresponding to the smelting end monitoring task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0169] The input training samples include incident light reflected, transmitted, and / or radiated from the steelmaking furnace mouth during smelting to the end and not smelting to the end; the output training samples include the determination result of whether smelting reaches the end.

[0170] Further, when the smelting end-point monitoring task further includes identifying the carbon content and / or molten steel temperature during the smelting process, correspondingly, the input training samples further include the incident light reflected, transmitted, and / or radiated from the steelmaking furnace mouth at different carbon contents and / or molten steel temperatures, and the output training samples further include the corresponding carbon content and / or molten steel temperature.

[0171] Further, when training the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0172] Further, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0173] Further, the optical filter layer is a single-layer structure or a multi-layer structure.

[0174] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0175] Further, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0176] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0177] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0178] Further, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0179] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0180] Further, the micro-nano unit has polarization-independent characteristics.

[0181] Further, the micro-nano unit has four-fold rotational symmetry.

[0182] Further, the optical filter layer is composed of one or more filter layers;

[0183] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, and tunable Fabry-Perot resonators.

[0184] Further, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0185] Further, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0186] An embodiment of the present invention further provides an intelligent smelting control device, which is characterized by including the optical artificial neural network smelting end point monitoring chip as described above.

[0187] An embodiment of the present invention further provides a preparation method for optical artificial neural network smelting end point monitoring as described above, including:

[0188] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0189] Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals;

[0190] Connecting the image sensor and the processor;

[0191] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0192] The image sensor is used to convert the incident light carrying information corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the smelting end point monitoring result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light from the steelmaking furnace mouth.

[0193] Further, the method for preparing the optical artificial neural network smelting end-point monitoring chip further includes: the training process of the optical artificial neural network smelting end-point monitoring chip, specifically including:

[0194] Using the input training samples and output training samples corresponding to the smelting end-point monitoring task, training the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0195] Further, when training the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0196] The optical artificial neural network intelligent chip, intelligent processing device and preparation method provided by the embodiments of the present invention realize a brand-new intelligent chip capable of implementing the functions of an artificial neural network. In this intelligent chip, an optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area. Correspondingly, the image sensor is used to convert the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points. At the same time, the processor connected to the image sensor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in this intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weights from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this intelligent chip realize the related functions of the input layer and the linear layer in the artificial neural network. That is, the embodiments of the present invention separate the input layer and the linear layer in the artificial neural network implemented by software in the prior art and implement these two layers of structures in the artificial neural network in a hardware manner. As a result, when using this intelligent chip for artificial neural network intelligent processing subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the relevant processing of full connection and non-linear activation of electrical signals needs to be performed by the processor in the intelligent chip. In this way, the power consumption and delay during artificial neural network processing can be significantly reduced. Description of the Drawings

[0197] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0198] Figure 1 It is a schematic structural diagram of the optical artificial neural network intelligent chip provided by the first embodiment of the present invention;

[0199] Figure 2 It is a schematic diagram of the recognition principle of the optical artificial neural network intelligent chip provided by an embodiment of the present invention;

[0200] Figure 3 It is a schematic disassembling diagram of the optical artificial neural network intelligent chip provided by an embodiment of the present invention;

[0201] Figure 4 It is a schematic diagram of the target object recognition process provided by an embodiment of the present invention;

[0202] Figure 5 It is a top view of an optical filter layer provided by an embodiment of the present invention;

[0203] Figure 6 It is a top view of another optical filter layer provided by an embodiment of the present invention;

[0204] Figure 7 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0205] Figure 8 It is a top view of yet another different optical filter layer provided by an embodiment of the present invention;

[0206] Figure 9 It is a top view of still another optical filter layer provided by an embodiment of the present invention;

[0207] Figure 10 It is a top view of yet another further optical filter layer provided by an embodiment of the present invention;

[0208] Figure 11 It is a schematic diagram of the broadband filtering effect of the micro-nano structure provided by an embodiment of the present invention;

[0209] Figure 12 It is a schematic diagram of the narrowband filtering effect of the micro-nano structure provided by an embodiment of the present invention;

[0210] Figure 13 It is a schematic diagram of the structure of a front-illuminated image sensor provided by an embodiment of the present invention;

[0211] Figure 14 It is a schematic diagram of the structure of a back-illuminated image sensor provided by an embodiment of the present invention;

[0212] Figure 15 It is a schematic flow diagram of the preparation method of an optical artificial neural network intelligent chip provided by the third embodiment of the present invention;

[0213] Figure 16 It is a schematic diagram of the face recognition process provided by an embodiment of the present invention;

[0214] Figure 17 It is a schematic diagram of the finger blood glucose detection process provided by an embodiment of the present invention;

[0215] Figure 18 It is a schematic diagram of the wrist blood glucose detection process provided by an embodiment of the present invention;

[0216] Figure 19It is a schematic diagram of the process for identifying agricultural objects provided by an embodiment of the present invention;

[0217] Figure 20 It is a three-dimensional schematic diagram of identifying and / or qualitatively analyzing crops and / or soil provided by an embodiment of the present invention;

[0218] Figure 21 It is a schematic diagram of identifying the furnace mouth during the smelting process to determine the smelting end point provided by an embodiment of the present invention. Detailed implementation manners

[0219] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0220] In the existing intelligent recognition technology, it is usually necessary to first image a person or an object, and then input the image into a neural network recognition model for processing, so as to realize the recognition of the person or the object. It can be seen that the current intelligent recognition tasks generally rely on the neural network recognition model. That is to say, in the current intelligent recognition tasks, imaging needs to be performed first and then transmitted to a computer for subsequent neural network recognition model algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay. Based on this, an embodiment of the present invention provides an optical artificial neural network intelligent chip. The optical filter layer in the intelligent chip corresponds to the input layer of the artificial neural network, the image sensor corresponds to the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. In the embodiment of the present invention, the spatial spectral information of the target object is projected into an electrical signal by using the optical filter layer and the image sensor, and then the full connection processing and nonlinear activation processing of the electrical signal are realized in the processor. It can be seen that the embodiment of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art. The embodiment of the present invention separates the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and realizes these two layers of structures in the artificial neural network by using hardware. Therefore, when using this intelligent chip for artificial neural network intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the relevant processing of full connection and nonlinear activation of the electrical signal needs to be performed by the processor in the intelligent chip. In this way, the power consumption and delay during artificial neural network processing can be significantly reduced. The content provided by the present invention will be explained and described in detail below through specific embodiments.

[0221] As Figure 1 shown, the optical artificial neural network intelligent chip provided by the first embodiment of the present invention includes: an optical filter layer 1, an image sensor 2, and a processor 3; the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor 2 corresponds to the linear layer of the artificial neural network; the processor 3 corresponds to the non-linear layer and the output layer of the artificial neural network;

[0222] The optical filter layer 1 is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer 1 includes an optical modulation structure. The optical filter layer 1 is configured to perform spectral modulation on the incident light entering different position points of the optical modulation structure by varying the intensity with wavelength, that is, perform different intensity modulations on incident light of different wavelengths, so as to obtain incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes image information of a target object to be processed by the optical artificial neural network intelligent chip and / or various optical space information. For example, the incident light carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0223] The image sensor 2 is configured to convert the incident light carrying information corresponding to different position points after being modulated by the optical filter layer 1 into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor 3; the electrical signals are image signals after being modulated by the optical filter layer;

[0224] The processor 3 is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0225] In this embodiment, the optical filter layer 1 is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer 1 includes an optical modulation structure. The optical filter layer 1 is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure, so as to obtain the modulated incident light carrying information corresponding to different position points on the surface of the photosensitive area. Correspondingly, the image sensor 2 is configured to convert the incident light carrying information corresponding to different position points into electrical signals corresponding to different position points, that is, the image signals after being modulated by the optical filter layer. At the same time, the processor 3 is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0226] In this embodiment, the optical filter layer 1 includes an optical modulation structure. The incident light (such as the reflected light, transmitted light, radiation light, etc. of the target to be recognized) entering different position points of the optical modulation structure is subjected to spectral modulation with different intensities, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area of the image sensor 2.

[0227] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the optical modulation structure. For example, different modulation intensities can be achieved by designing different optical modulation structures (such as changing the shape and / or size parameters of the optical modulation structure).

[0228] In this embodiment, it can be understood that the optical modulation structures at different positions on the optical filter layer 1 have different spectral modulation effects on the incident light. The modulation intensity of different wavelength components of the incident light by the optical modulation structure corresponds to the connection strength of the artificial neural network, that is, the connection weights corresponding to the input layer and the connection from the input layer to the linear layer. It should be noted that the optical filter layer 1 is composed of multiple optical filter units. The optical modulation structures at different positions within each optical filter unit are different, so they have different spectral modulation effects on the incident light; the optical modulation structures at different positions between the optical filter units can be the same or different, so they have the same or different spectral modulation effects on the incident light.

[0229] In this embodiment, the image sensor 2 converts the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to the linear layer of the neural network.

[0230] In this embodiment, the processor 3 performs a fully connected process and a non-linear activation process on the electrical signals at different position points, and then obtains the output signal of the artificial neural network.

[0231] It can be understood that the processor 3 corresponds to the non-linear layer and the output layer of the neural network, and can also be understood as corresponding to the remaining layers (all other layers) of the neural network except the input layer and the linear layer.

[0232] In addition, it should be supplemented that the processor 3 can be arranged in the intelligent chip, that is, the processor 3 can be arranged in the intelligent chip together with the filter layer 1 and the image sensor 2, or can be arranged separately outside the intelligent chip and connected to the image sensor 2 in the intelligent chip through a data line or a connecting device. This embodiment does not limit this.

[0233] In addition, it should be noted that the processor 3 can be implemented by a computer, or by an ARM or FPGA circuit board with certain computing capabilities, or by a microprocessor. This embodiment does not limit this. In addition, as mentioned above, the processor 3 can be integrated in the intelligent chip or set independently outside the intelligent chip. When the processor 3 is set independently outside the intelligent chip, the electrical signal in the image sensor 2 can be read out to the processor 3 through a signal reading circuit, and then the processor 3 performs a fully connected process and a non-linear activation process on the read electrical signal.

[0234] In this embodiment, it can be understood that when the processor 3 performs non-linear activation processing, it can be implemented by a non-linear activation function. For example, it can use a Sigmoid function, a Tanh function, a ReLU function, etc. This embodiment does not limit this.

[0235] In this embodiment, the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer. The image sensor 2 corresponds to the linear layer of the artificial neural network, converting the information carried by the incident light at different spatial positions into electrical signals. The processor 3 corresponds to the non-linear layer and the output layer of the artificial neural network, fully connecting the electrical signals at different positions, and obtaining the output signal of the artificial neural network through a non-linear activation function, realizing intelligent perception, recognition, and / or decision-making of specific targets.

[0236] As Figure 2 shown on the left, the optical artificial neural network intelligent chip includes an optical filter layer 1, an image sensor 2, and a processor 3. In Figure 2 this, the processor 3 is implemented by a signal reading circuit and a computer. As Figure 2 shown on the right, the optical filter layer 1 in the optical artificial neural network intelligent chip corresponds to the input layer of the artificial neural network, the image sensor 2 corresponds to the linear layer of the artificial neural network, and the processor 3 corresponds to the non-linear layer and the output layer of the artificial neural network. The filtering effect of the incident light on the optical filter layer 1 corresponds to the connection weights from the input layer to the linear layer. It can be seen that the optical filter layer and the image sensor in the intelligent chip provided in this embodiment implement the related functions of the input layer and the linear layer in the artificial neural network through hardware, so that subsequent complex signal processing and algorithm processing corresponding to the input layer and the linear layer are not required when using this intelligent chip for intelligent processing, which can greatly reduce the power consumption and delay during the processing of the artificial neural network.

[0237] As Figure 2 shown on the right, project / connect the incident light spectrum P at different positions of the optical filter layer 1 λ to the photocurrent response I of the image sensor λAbove, the processor 3 includes a signal reading circuit and a computer. The signal reading circuit in the processor 3 reads out the photocurrent response and transmits it to the computer, where the computer performs full connection processing and non-linear activation processing on the electrical signal, and finally outputs the result.

[0238] As Figure 3 shown, the optical modulation structure on the optical filter layer 1 is integrated above the image sensor 2 to modulate the incident light and project / connect the spectral information of the incident light to different pixel points of the image sensor 2, obtaining an electrical signal containing the spectral information and image information of the incident light. That is, after the incident light passes through the optical filter layer 1, it is converted into an electrical signal by the image sensor 2, forming an image containing the spectral information of the incident light, and finally the processor 3 connected to the image sensor 2 processes the electrical signal containing the spectral information and image information of the incident light.

[0239] In this embodiment, the information carried by the incident light may include one or more (including two) of the light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0240] For example, in one implementation, the information carried by the incident light may include the light intensity distribution information. In other implementations, multiple information such as the image information, spectral information, the angle of the incident light, and the phase information of the incident light of the target object can be used simultaneously to identify the target object, so that the intelligent identification of the target object can be achieved more accurately.

[0241] It can be seen that the optical artificial neural network chip provided in this embodiment can simultaneously utilize the image information, spectral information, the angle of the incident light, and the phase information of the target object, that is, the information carried by the light at different points in the space of the target object. Since the information carried by the incident light at different points in the space of the target object covers the image, composition, shape, three-dimensional depth, structure, etc. of the target object, when performing identification processing based on the information carried by the incident light at different points in the space of the target object, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, structure, etc. of the target object, thus solving the problem mentioned in the background art that it is difficult to ensure the accuracy of identification using the two-dimensional image information of the target object, such as it is difficult to distinguish between a real person and a picture, realizing intelligent perception, identification, and / or decision-making functions for different application fields, and realizing a spectrum optical artificial neural network intelligent chip with low power consumption, low latency, and high accuracy.

[0242] The optical artificial neural network intelligent chip provided by the embodiments of the present invention realizes a brand-new intelligent chip capable of implementing the functions of an artificial neural network. In this intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this intelligent chip implement the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiments of the present invention separate the input layer and the linear layer in the artificial neural network implemented by software in the prior art and implement these two layers of structures in the artificial neural network in a hardware manner. As a result, when using this intelligent chip for artificial neural network intelligent processing subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the processor in the intelligent chip needs to perform relevant processing related to the full connection of electrical signals and nonlinear activation. In this way, the power consumption and delay during artificial neural network processing can be significantly reduced.

[0243] In addition, it should be noted that in the prior art, when identifying a person or an object, only the two-dimensional image information of the person or the object is used. However, it is difficult to ensure the accuracy of identification with two-dimensional image information. For example, it is difficult to distinguish a real face from a photo of a face. Therefore, based on this, in one implementation manner, the information carried by the incident light may include light intensity distribution information and spectral information. As a result, when using the optical artificial neural network intelligent chip provided in this application to perform an intelligent recognition task, the light intensity distribution information and spectral information of the object to be recognized can be used simultaneously. It can be seen that since the information carried by the incident light covers information such as the image, composition, shape, three-dimensional depth, and structure of the target object, when performing recognition processing based on the information carried by the incident light at different points in space of the target object, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the target object can be covered, thereby solving the problem that it is difficult to ensure the accuracy of recognition using the two-dimensional image information of the target object, such as it is difficult to distinguish whether it is a real person or a picture, and thus enabling more accurate recognition of the object to be recognized. In addition, in another implementation manner, the information carried by the incident light may further include light intensity distribution information, spectral information, and the angle information of the incident light, so as to be able to capture more comprehensive information such as the image, composition, shape, three-dimensional depth, and three-dimensional structure of the target object, and thus enable more accurate recognition of the object to be recognized. In addition, in another implementation manner, the information carried by the incident light may further include light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light, so as to be able to capture more comprehensive information such as the image, composition, shape, three-dimensional depth, and three-dimensional structure of the target object, and thus enable more accurate recognition of the object to be recognized.

[0244] The optical artificial neural network intelligent chip provided by the embodiment of the present invention includes an optical filter layer, an image sensor, and a processor. The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area. Correspondingly, the image sensor is used to convert the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points. At the same time, the processor connected to the image sensor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in this intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. That is to say, the optical filter layer and the image sensor in this intelligent chip realize the related functions of the input layer and the linear layer in the artificial neural network. That is, the embodiment of the present invention separates the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and uses hardware to implement these two layers of structures in the artificial neural network. As a result, when using this intelligent chip for artificial neural network intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. It only needs the processor in the intelligent chip to perform related processing of full connection and non-linear activation of electrical signals. In this way, the power consumption and delay during artificial neural network processing can be greatly reduced.As can be seen, in the embodiment of the present invention, the optical filter layer is used as the input layer of the artificial neural network, and the image sensor is used as the linear layer of the artificial neural network. The filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor are used to project the information carried by the incident light of the target object into an electrical signal, and then the fully connected processing and non-linear activation processing of the electrical signal are implemented in the processor. As can be seen, the embodiment of the present invention can not only omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also actually utilize the image information, spectral information, incident light angle information, and incident light phase information of the target object at the same time, that is, the information carried by the incident light at different points in the space of the target object. As can be seen, since the information carried by the incident light at different points in the space of the target object covers the image, composition, shape, three-dimensional depth, structure, etc. of the target object, when performing recognition processing based on the information carried by the incident light at different points in the space of the target object, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the target object can be covered, so that the problem mentioned in the background art that it is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the target object, such as it is difficult to distinguish between a real person and a picture, can be solved. As can be seen, the optical artificial neural network chip provided by the embodiment of the present invention can not only achieve the effects of low power consumption and low latency, but also achieve the effect of high accuracy, thus preparing for intelligent processing tasks such as intelligent perception, recognition, and / or decision-making.

[0245] Based on the content of the above embodiment, in this embodiment, the optical artificial neural network intelligent chip is used for the intelligent processing task of the target object; the intelligent processing task includes at least one or more of intelligent perception, intelligent recognition, and intelligent decision-making tasks;

[0246] The reflected light, transmitted light, and / or radiation light of the target object enter the trained optical artificial neural network intelligent chip to obtain the intelligent processing result of the target object; the intelligent processing result includes at least one or more of an intelligent perception result, an intelligent recognition result, and / or an intelligent decision result;

[0247] The trained optical artificial neural network intelligent chip refers to an optical artificial neural network intelligent chip including a trained optical modulation structure, an image sensor, and a processor; the trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, which is trained by using the input training samples and output training samples corresponding to the intelligent processing task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions.

[0248] In this embodiment, the optical artificial neural network intelligent chip can be used for intelligent processing tasks of a target object. For example, it includes one or more tasks among intelligent perception, intelligent recognition, and intelligent decision-making tasks.

[0249] In this embodiment, it can be understood that intelligent perception refers to mapping signals in the physical world through hardware devices such as cameras, microphones, or other sensors, with the help of cutting-edge technologies such as speech recognition and image recognition, into the digital world, and then further enhancing these digital information to a recognizable level, such as memory, understanding, planning, decision-making, and so on. Intelligent recognition refers to the technology of using a computer to process, analyze, and understand images to identify various different patterns of targets and objects. At present, intelligent recognition technologies are generally divided into face recognition and commodity recognition. Face recognition is mainly used in security checks, identity verification, and mobile payments. Commodity recognition is mainly used in the process of commodity circulation, especially in unmanned retail fields such as unmanned shelves and intelligent retail cabinets. Intelligent decision-making refers to solving the automatic organization and coordination of multiple models by a computer, accessing and processing data in a large database, and performing corresponding data processing and numerical calculations.

[0250] In this embodiment, the reflected light, transmitted light, and / or radiation light of the target object enter the trained optical artificial neural network intelligent chip to obtain the intelligent processing result of the target object.

[0251] In this embodiment, taking the recognition task of the target object as an example for illustration, it can be understood that when using this intelligent chip for the recognition task, first, the optical artificial neural network intelligent chip needs to be trained. Here, training the optical artificial neural network intelligent chip means determining the optical modulation structure applicable to the current recognition task, as well as the fully connected parameters and non-linear activation parameters applicable to the current recognition task.

[0252] It can be understood that since the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the optical modulation structure in the optical filter layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence condition, the optical modulation structure applicable to the current recognition task, as well as the fully connected parameters and non-linear activation parameters applicable to the current recognition task, are determined, thereby completing the training of the intelligent chip.

[0253] It can be understood that after the intelligent chip is trained, the trained intelligent chip can be used to perform recognition tasks. Specifically, when the incident light carrying the image information and spatial spectral information of the target object enters the optical filter layer 1 of the trained intelligent chip, the optical modulation structure in the optical filter layer 1 modulates the incident light. The intensity of the modulated optical signal is detected by the image sensor 2 and converted into an electrical signal, and then fully connected processing and non-linear activation processing are performed by the processor 3 to obtain the recognition result of the target object.

[0254] As Figure 4 shown, the complete process for target object recognition is as follows: The broadband light source 100 irradiates the target object 200, and then the reflected light or transmitted light of the target object is collected by the optical artificial neural network intelligent chip 300, or the light directly radiated by the target object is collected by the optical artificial neural network intelligent chip 300. After being processed by the optical filter layer, image sensor, and processor in the intelligent chip, the recognition result can be obtained.

[0255] Among them, the trained optical artificial neural network intelligent chip refers to an optical artificial neural network intelligent chip including a trained optical modulation structure, an image sensor, and a processor; the trained optical modulation structure, image sensor, and processor refer to the optical modulation structure, image sensor, and processor that meet the training convergence conditions obtained by training an optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters using the input training samples and output training samples corresponding to the intelligent processing task.

[0256] For example, for the intelligent recognition task, the input training sample corresponding to the intelligent recognition task is the recognition object sample, and the output training sample corresponding to the intelligent recognition task is the recognition result of the recognition object sample. It can be understood that for the recognition task, since the advantage of the intelligent chip provided in this embodiment is also that it can obtain the spatial spectral information of the recognition object, therefore, to make full use of this advantage, the recognition object sample used as the input training sample preferably uses the real recognition object instead of the two-dimensional image of the recognition object. Of course, this does not mean that the two-dimensional image cannot be used as the recognition object sample.

[0257] In addition, the optical artificial neural network intelligent chip provided in this embodiment can also be used for other intelligent processing tasks of the target object, such as intelligent perception, intelligent decision-making, etc.

[0258] In this embodiment, the optical filter layer 1 serves as the input layer of the neural network, and the image sensor 2 serves as the linear layer of the neural network. To minimize the loss function of the neural network, the modulation intensity of different wavelength components in the incident light of the target object by the optical modulation structure in the optical filter layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the filter, the modulation intensity of different wavelength components in the incident light of the target object can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, and further optimizing the training of the neural network.

[0259] Therefore, in this embodiment, the optical modulation structure is obtained based on the training of the neural network. Through computer optical simulation of the training samples, the sample modulation intensity of different wavelength components of the incident light of the target object by the optical modulation structure in the training samples is obtained. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network, and non-linear activation is performed. The neural network is trained using the training samples corresponding to the intelligent processing task until the neural network converges, and the corresponding training sample optical modulation structure is used as the optical filter layer for the corresponding intelligent processing task.

[0260] It can be seen that in this embodiment, by implementing the input layer (optical filter layer) and the linear layer (image sensor) of the neural network at the physical layer, not only can the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art be omitted, but in fact, the image information, spectral information, incident light angle information, and incident light phase information of the target object are simultaneously utilized in this embodiment of the present invention, that is, the incident light at different points in space of the target object carries information. It can be seen that since the incident light at different points in space of the target object carries information covering the image, composition, shape, three-dimensional depth, structure, etc. of the target object, when performing recognition processing based on the information carried by the incident light at different points in space of the target object, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the target object can be covered, thereby solving the problem that it is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the target object in the background art, such as it is difficult to distinguish between a real person and a picture. It can be seen that the optical artificial neural network chip provided in this embodiment of the present invention can not only achieve the effects of low power consumption and low latency, but also achieve the effect of high accuracy, thus preparing for intelligent processing tasks such as intelligent perception, recognition, and / or decision-making.

[0261] Based on the content of the above embodiment, in this embodiment, when training an optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0262] In this embodiment, optical simulation enables users to experience the product in a digital environment before making a physical prototype. For example, in the case of an automobile, since light and reflected light can interfere with the driver's attention, especially when driving at night, a suitable optical simulation solution can not only effectively help users improve design efficiency, but also simulate the interaction between light and materials to understand the display effect of the product under real conditions. Therefore, in this embodiment, the light modulation structure is designed through computer optical simulation, and the light modulation structure is adjusted through optical simulation until the corresponding light modulation structure is determined as the size of the light modulation structure to be finally fabricated when the neural network converges, saving prototype production time and cost, improving product efficiency, and easily solving complex optical problems.

[0263] For example, the light modulation structure can be simulated and designed by FDTD software. By changing the light modulation structure in optical simulation, the modulation intensity of the light modulation structure for different incident lights can be accurately predicted and used as the connection weight between the input layer and the linear layer of the neural network to train the optical artificial neural network intelligent chip and accurately obtain the light modulation structure.

[0264] It can be seen that in this embodiment, by adopting the method of computer optical simulation design to design the light modulation structure, the prototype production time and cost of the light modulation structure are saved, and the product efficiency is improved.

[0265] Based on the content of the above embodiment, in this embodiment, the light modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the light modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0266] In this embodiment, the light modulation structure in the optical filter layer can include only a regular structure, only an irregular structure, or both a regular structure and an irregular structure.

[0267] In this embodiment, the fact that the light modulation structure includes a regular structure can mean that: the smallest modulation unit included in the light modulation structure is a regular structure, such as the smallest modulation unit can be a regular shape such as a rectangle, a square, and a circle. In addition, the fact that the light modulation structure includes a regular structure can also mean that: the arrangement mode of the smallest modulation units included in the light modulation structure is regular, such as the arrangement mode can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc. In addition, the fact that the light modulation structure includes a regular structure can also mean that: the smallest modulation unit included in the light modulation structure is a regular structure, and at the same time, the arrangement mode of the smallest modulation units is also regular, etc.

[0268] In this embodiment, the optical modulation structure including an irregular structure may mean that the smallest modulation unit included in the optical modulation structure is an irregular structure, such as an irregular polygon, a random shape, or other irregular graphics. In addition, the optical modulation structure including an irregular structure may also mean that the arrangement of the smallest modulation units included in the optical modulation structure is irregular, such as an irregular polygon form, a random arrangement form, etc. In addition, the optical modulation structure including an irregular structure may also mean that the smallest modulation unit included in the optical modulation structure is an irregular structure, and at the same time, the arrangement of the smallest modulation units is also irregular, etc.

[0269] In this embodiment, the optical modulation structure in the optical filter layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0270] In this embodiment, the optical modulation structure including a continuous structure may mean that the optical modulation structure is composed of continuous modulation patterns; the optical modulation structure including a discrete structure may mean that the optical modulation structure is composed of discrete modulation patterns.

[0271] It can be understood that the continuous modulation patterns here may refer to linear patterns, wavy patterns, broken line patterns, etc.

[0272] It can be understood that the discrete modulation patterns here may refer to modulation patterns formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0273] In this embodiment, it should be noted that the optical modulation structure has different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after light passes through different groups of filter structures.

[0274] Based on the content of the above embodiment, in this embodiment, the optical filter layer is a single-layer structure or a multi-layer structure.

[0275] In this embodiment, it should be noted that the optical filter layer may be a single-layer filter structure or a multi-layer filter structure, such as a multi-layer structure of two layers, three layers, four layers, etc.

[0276] In this embodiment, as Figure 1 shown, the optical filter layer 1 is a single-layer structure, and the thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths of 400 nm to 10 μm, the thickness of the grating structure can be 50 nm to 5 μm.

[0277] It can be understood that since the function of the optical filter layer 1 is to perform spectral modulation on the incident light, it is preferably prepared from materials with high refractive index and low loss. For example, silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, group III-V materials, etc. can be selected for preparation. Among them, silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, etc.

[0278] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical filter layer 1 can be set as a multi-layer structure, and the corresponding optical modulation structures of each layer can be set as different structures, so as to increase the spectral modulation ability of the optical filter layer for the incident light, thereby forming more or more complex connection weights between the input layer and the linear layer, and further improving the accuracy of the intelligent chip when processing intelligent tasks.

[0279] In addition, it should be noted that for the filter layer including a multi-layer structure, the materials of each layer structure can be the same or different. For example, for the two-layer optical filter layer 1, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0280] It should be noted that the thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths of 400 nm to 10 μm, the total thickness of the multi-layer structure can be 50 nm to 5 μm.

[0281] Based on the content of the above embodiments, in this embodiment, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0282] In this embodiment, in order to obtain connection weights (connection weights for connecting the input layer and the linear layer) distributed in an array for the convenience of the subsequent fully connected and non-linear activation processing by the processor, preferably, in this embodiment, the optical modulation structure is in the form of an array structure. Specifically, the optical modulation structure includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor. It should be noted that the structures of the respective micro-nano units can be the same or different. In addition, it should be noted that the structures of the respective micro-nano units can be periodic or non-periodic. In addition, it should be noted that each micro-nano unit can further include multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different, etc.

[0283] The following combines Figures 5 - 9 to give an example. In this embodiment, as Figure 5As shown, the optical filter layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (and each micro-nano unit is a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; such as Figure 6 As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure ( Figure 5 different from Figure 6 in that each micro-nano unit in Figure 7 is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; such as Figure 6 different from Figure 7 in that the unit shape of the periodic array in each micro-nano unit in Figure 8 has four-fold rotational symmetry; as shown, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66, Figure 6 different from Figure 9 in that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2. In this embodiment, the optical filter layer 1 includes a plurality of different micro-nano units, that is, the modulation effects of different regions on the intelligent chip on the incident light are different, thereby improving the design freedom, and further improving the recognition accuracy. Such as Figure 5 different from

[0284] in that each micro-nano unit is composed of a discrete non-periodic array structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0284] In this embodiment, the micro-nano unit has different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after light passes through different groups of filter structures.

[0285] Based on the content of the above embodiments, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0286] In this embodiment, the micro-nano unit may only include regular structures, or only include irregular structures, or may include both regular structures and irregular structures.

[0287] In this embodiment, when it is said that the micro-nano unit includes regular structures, it may mean that: the smallest modulation unit included in the micro-nano unit is a regular structure, such as the smallest modulation unit can be a regular shape like a rectangle, a square, a circle, etc. In addition, when it is said that the micro-nano unit includes regular structures, it may also mean that: the arrangement pattern of the smallest modulation units included in the micro-nano unit is regular, such as the arrangement pattern can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc. In addition, when it is said that the micro-nano unit includes regular structures, it may also mean that: the smallest modulation unit included in the micro-nano unit is a regular structure, and at the same time the arrangement pattern of the smallest modulation units is also regular, etc.

[0288] In this embodiment, when it is said that the micro-nano unit includes irregular structures, it may mean that: the smallest modulation unit included in the micro-nano unit is an irregular structure, such as the smallest modulation unit can be an irregular polygon, a random shape, etc. In addition, when it is said that the micro-nano unit includes irregular structures, it may also mean that: the arrangement pattern of the smallest modulation units included in the micro-nano unit is irregular, such as the arrangement pattern can be an irregular polygonal form, a random arrangement form, etc. In addition, when it is said that the micro-nano unit includes irregular structures, it may also mean that: the smallest modulation unit included in the micro-nano unit is an irregular structure, and at the same time the arrangement pattern of the smallest modulation units is also irregular, etc.

[0289] In this embodiment, the micro-nano units in the optical filter layer may include discrete structures, or may include continuous structures, or may include both discrete structures and continuous structures.

[0290] In this embodiment, when it is said that the micro-nano unit includes a continuous structure, it may mean that: the micro-nano unit is composed of continuous modulation patterns; when it is said that the micro-nano unit includes a discrete structure, it may mean that: the micro-nano unit is composed of discrete modulation patterns.

[0291] It can be understood that the continuous modulation patterns here may refer to linear patterns, wavy patterns, broken-line patterns, etc.

[0292] It can be understood that the discrete modulation patterns here may refer to modulation patterns formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0293] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano units, the corresponding transmission spectra are different after light passes through different groups of micro-nano units.

[0294] Based on the content of the above embodiments, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0295] In this embodiment, as Figure 5 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 different micro-nano structure arrays 110, 111, 112, and 113, and the filtering unit 44 includes 4 different micro-nano structure arrays 440, 441, 442, and 443. As Figure 10 shown, the optical filter layer 1 includes multiple micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0296] It should be noted that here only a micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of groups of micro-nano structure arrays can also be set according to needs.

[0297] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and the modulation effects of each group of filtering structures on the incident light are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after light passes through different groups of micro-nano structure arrays.

[0298] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0299] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light of the target object as the connection weight between the input layer and the linear layer of the neural network, broadband filtering and narrowband filtering are realized by adopting different micro-nano structure arrays. Therefore, in this embodiment, the micro-nano structure array performs broadband filtering or narrowband filtering on the incident light of the target object to obtain the modulation intensity of different wavelength components of the incident light of the target object. As Figure 11 and Figure 12 shown, each group of micro-nano structure arrays in the optical filter layer has the function of broadband filtering or narrowband filtering.

[0300] It can be understood that for each group of micro-nano structure arrays, they can all have broadband filtering effect, or all have narrowband filtering effect, or some have broadband filtering effect and some have narrowband filtering effect. In addition, the broadband filtering range and narrowband filtering range of each group of micro-nano structure arrays can be the same or different. For example, by designing the size parameters such as the period, duty cycle, radius, and side length of each group of micro-nano structures in the micro-nano unit, it can have narrowband filtering effect, that is, only light of one (or a few) wavelengths can pass through. Another example is that by designing the size parameters such as the period, duty cycle, radius, and side length of each group of micro-nano structures in the micro-nano unit, it can have broadband filtering effect, that is, allowing light of more wavelengths or all wavelengths to pass through.

[0301] It can be understood that in specific use, the filtering state of each group of micro-nano structure arrays can be determined by means of broadband filtering, narrowband filtering or their combination according to the application scenario.

[0302] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0303] In this embodiment, each group of micro-nano structure arrays can all be periodic structure arrays, or all be aperiodic structure arrays, or some be periodic structure arrays and some be aperiodic structure arrays. Among them, periodic structure arrays are easy to carry out optical simulation design, and aperiodic structure arrays can achieve more complex modulation effects.

[0304] In this embodiment, as Figure 5 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and the micro-nano structure arrays are aperiodic structures. Among them, the aperiodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in an aperiodic arrangement. As Figure 5 shown, the micro-nano unit 11 includes 4 different aperiodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes 4 different aperiodic structure arrays 440, 441, 442 and 443. The aperiodic structure micro-nano structure arrays are designed by neural network data training for intelligent processing tasks in the early stage, and are usually structures with irregular shapes. As Figure 6 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and Figure 5The difference is that the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. For example, Figure 6 As shown, the micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112, and 113, and the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442, and 443. The filter structure of the periodic structure is designed by neural network data training for intelligent processing tasks in the early stage, and is usually a structure with an irregular shape. For example, Figure 7 As shown, the optical filter layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of the micro-nano structure arrays are different from each other, and the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes on the filter structure are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. For example, Figure 7 As shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other. The micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112, and 113, and the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442, and 443. The micro-nano structure array of the periodic structure is designed by neural network data training for intelligent processing tasks in the early stage, and is usually a structure with an irregular shape.

[0305] It should be noted that, Figures 5 - 9 Each micro-nano unit includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are formed by using four different shapes of modulation holes respectively. The four groups of micro-nano structure arrays have different modulation effects on the incident light. It should be noted that here only the micro-nano unit including four groups of micro-nano structure arrays is taken as an example for illustration, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of micro-nano structure arrays can also be set according to needs. In this embodiment, the four different shapes can be circular, cross-shaped, regular polygon, and rectangular (not limited to this).

[0306] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on lights of different wavelengths, and the modulation effects of each group of micro-nano structure arrays on the input light are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after the light passes through different groups of micro-nano structure arrays.

[0307] Based on the content of the above embodiment, in this embodiment, one or more groups of the micro-nano structure arrays included in the micro-nano unit are empty structures.

[0308] The following will be described by way of example with reference to Figure 9 the example shown. In this embodiment, as Figure 9 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiment, for any micro-nano unit, it includes one or more groups of empty structures, and the empty structures are used to directly transmit the incident light. It can be understood that when one or more groups of empty structures are included in the multiple groups of micro-nano structure arrays, a richer spectrum modulation effect can be formed, so as to meet the spectrum modulation requirements in specific scenarios (or meet the specific connection weight requirements between the input layer and the linear layer in specific scenarios).

[0309] As Figure 9 shown, each micro-nano unit includes one group of micro-nano structure arrays and three groups of empty structures. The micro-nano unit 11 includes 1 aperiodic structure array 111, the micro-nano unit 22 includes 1 aperiodic structure array 221, the micro-nano unit 33 includes 1 aperiodic structure array 331, the micro-nano unit 44 includes 1 aperiodic structure array 441, the micro-nano unit 55 includes 1 aperiodic structure array 551, and the micro-nano unit 66 includes 1 aperiodic structure array 661, where the micro-nano structure array is used to perform different modulations on the incident light. It should be noted that here only an example of including one group of micro-nano structure arrays and three groups of empty structures is given, which does not play a limiting role. In actual applications, micro-nano units including one group of micro-nano structure arrays and five groups of empty structures or other numbers of groups of micro-nano structure arrays can also be set according to needs. In this embodiment, the micro-nano structure array can be made of modulation holes in the shape of a circle, a cross, a regular polygon, and a rectangle (not limited to this).

[0310] It should be noted that none of the multiple groups of micro-nano structure arrays included in the micro-nano unit may include empty structures, that is, the multiple groups of micro-nano structure arrays may be aperiodic structure arrays or periodic structure arrays.

[0311] Based on the content of the above embodiment, in this embodiment, the micro-nano unit has polarization-independent characteristics.

[0312] In this embodiment, since the micro-nano unit has polarization-independent characteristics, the optical filter layer is insensitive to the polarization of the incident light, thus realizing an optical artificial neural network intelligent chip that is insensitive to both the incident angle and polarization. The optical artificial neural network intelligent chip provided by the embodiment of the present invention is insensitive to the incident angle and polarization characteristics of the incident light, that is, the measurement result will not be affected by the incident angle and polarization characteristics of the incident light, so as to ensure the stability of the spectral measurement performance, and further ensure the stability of intelligent processing, such as the stability of intelligent perception, intelligent recognition, intelligent decision-making, etc. It should be noted that the micro-nano unit can also have polarization-dependent characteristics.

[0313] Based on the content of the above embodiment, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0314] In this embodiment, it should be noted that four-fold rotational symmetry belongs to a specific case of polarization-independent characteristics. By designing the micro-nano unit into a structure with four-fold rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0315] The following Figure 7 is illustrated by the following examples. In this embodiment, as Figure 7 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiment, the corresponding structure of each group of micro-nano structure arrays can be a structure with four-fold rotational symmetry, such as a circle, a cross, a regular polygon, a rectangle, etc., that is, after the structure rotates 90°, 180°, 270°, it coincides with the original structure, so that the structure has polarization-independent characteristics, and the same intelligent recognition effect can be obtained when different polarized lights are incident.

[0316] Based on the content of the above embodiment, in this embodiment, the optical filter layer is composed of one or more filter layers;

[0317] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polaritons (SPP) micro-nano structures, tunable Fabry-Perot cavities (FP cavities).

[0318] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0319] Among them, the photonic crystal, and the combination of the metasurface and the random structure can be compatible with the CMOS process, and can have a good modulation effect. Other materials can also be filled in the micropores of the micro-nano modulation structure for surface smoothing; the quantum dots and perovskites can minimize the volume of a single modulation structure by utilizing the spectral modulation characteristics of the materials themselves; the SPP has a small volume and can achieve polarization-related optical modulation; the liquid crystal can be dynamically regulated by voltage to improve the spatial resolution; the tunable Fabry-Perot resonator can be dynamically regulated to improve the spatial resolution.

[0320] Based on the content of the above embodiments, in this embodiment, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0321] In this embodiment, it should be noted that if the thickness of the optical filter layer is much smaller than the central wavelength of the incident light, it cannot play an effective spectral modulation role; if the thickness of the optical filter layer is much larger than the central wavelength of the incident light, it is difficult to fabricate in terms of technology and will introduce large optical losses. Therefore, in this embodiment, in order to reduce optical losses and be easy to fabricate, and to ensure an effective spectral modulation effect, the overall size (area) of each micro-nano unit in the optical filter layer 1 is usually λ 2 ~10 5 λ 2 , and the thickness is usually 0.1λ to 10λ (λ represents the central wavelength of the incident light of the target object). As Figure 5 shown, the overall size of each micro-nano unit is 0.5μm 2 ~40000μm 2 , and the dielectric material in the optical filter layer 1 is polysilicon with a thickness of 50nm to 2μm.

[0322] Based on the content of the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0323] CMOS image sensor (Contact Image Sensor, CIS), charge coupled device (Charge Coupled Device, CCD), single photon avalanche diode (Single Photon Avalanche Diode, SPAD) array, and focal plane photodetector array.

[0324] In this embodiment, it should be noted that a wafer-level CMOS image sensor (CIS) is adopted, and monolithic integration is achieved at the wafer level, which can minimize the distance between the image sensor and the optical filter layer, facilitate reducing the size of the unit, and reduce the device volume and packaging cost. The SPAD can be used for low-light detection, and the CCD can be used for high-light detection.

[0325] In this embodiment, the optical filter layer and the image sensor can be manufactured by complementary metal oxide semiconductor (CMOS) integration process, which is beneficial to reducing the device failure rate, improving the device yield and reducing the cost. For example, a layer or multiple layers of dielectric materials can be directly grown on the image sensor and then etched. Before removing the sacrificial layer used for etching, metal materials are deposited, and finally the sacrificial layer is removed to prepare the optical filter layer.

[0326] Based on the content of the above embodiment, in this embodiment, the types of the artificial neural network include: feedforward neural network.

[0327] In this embodiment, a feedforward neural network (FNN), also known as a deep feedforward network (DFN) and a multi-layer perceptron (MLP), is the simplest neural network, and the neurons are arranged in layers. Each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer, and there is no feedback between layers. The feedforward neural network has a simple structure, is easy to implement on hardware, has a wide range of applications, can approximate any continuous function and square-integrable function with arbitrary precision, and can accurately implement any finite training sample set. The feedforward network is a static non-linear mapping. Through the composite mapping of simple non-linear processing units, complex non-linear processing capabilities can be obtained.

[0328] Based on the content of the above embodiment, in this embodiment, a light-transmitting dielectric layer is provided between the optical filter layer and the image sensor.

[0329] In this embodiment, it should be noted that setting a light-transmitting dielectric layer between the optical filter layer and the image sensor can effectively separate the optical filter layer and the image sensor layer and avoid mutual interference between the two.

[0330] Based on the content of the above embodiment, in this embodiment, the image sensor is a front-illuminated type, including: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on the side of the metal wire layer away from the light detection layer; or,

[0331] The image sensor is a back-illuminated type, including: a light detection layer and a metal wire layer arranged from top to bottom, and the optical filter layer is integrated on a side of the light detection layer away from the metal wire layer.

[0332] In this embodiment, as Figure 13 shown is a front-illuminated image sensor, where the silicon detection layer 21 is below the metal wire layer 22, and the optical filter layer 1 is directly integrated onto the metal wire layer 22.

[0333] In this embodiment, different from Figure 13 that, Figure 14 shown is a back-illuminated image sensor, where the silicon detection layer 21 is above the metal wire layer 22, and the optical filter layer 1 is directly integrated onto the silicon detection layer 21.

[0334] It should be noted that for the back-illuminated image sensor, with the silicon detection layer 21 above the metal wire layer 22, the influence of the metal wire layer on the incident light can be reduced, thereby improving the quantum efficiency of the device.

[0335] According to the above content, in this embodiment, the optical filter layer is used as the input layer of the artificial neural network, the image sensor is used as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor in the optical artificial neural network intelligent chip provided in this embodiment implement the relevant functions of the input layer and the linear layer in the artificial neural network through hardware, so that subsequent complex signal processing and algorithm processing corresponding to the input layer and the linear layer are no longer required when using this intelligent chip for intelligent processing, which can significantly reduce the power consumption and delay during the processing of the artificial neural network. In addition, in this embodiment, since both the image information of the target object and the spectral information at different spatial points are utilized, the intelligent processing of the target object can be more accurately achieved.

[0336] Based on the same inventive concept, another embodiment of the present invention provides an intelligent processing device, including: the optical artificial neural network intelligent chip as described in the above embodiment. The intelligent processing device includes one or more of a smart phone, a smart computer, a smart recognition device, a smart sensing device, and a smart decision-making device.

[0337] Since the intelligent processing device provided in this embodiment includes the optical artificial neural network intelligent chip described in the above embodiment, therefore, the intelligent processing device provided in this embodiment has all the beneficial effects of the optical artificial neural network intelligent chip described in the above embodiment. Since the above embodiment has described this in detail, this embodiment will not be elaborated herein.

[0338] Based on the same inventive concept, another embodiment of the present invention provides a method for preparing an optical artificial neural network intelligent chip as described in the above embodiment, as Figure 15 shown, which specifically includes the following steps:

[0339] Step 1510: Prepare an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0340] Step 1520: Generate a processor with the functions of fully connecting and non-linearly activating the signal;

[0341] Step 1530: Connect the image sensor and the processor;

[0342] Among them, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0343] The image sensor is used to convert the incident light carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linearly activating processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0344] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor includes:

[0345] Grow one or more layers of preset materials on the surface of the photosensitive area of the image sensor;

[0346] Perform dry etching on the one or more layers of preset materials with the pattern of the optical modulation structure to obtain an optical filter layer including the optical modulation structure;

[0347] Or perform imprint transfer on the one or more layers of preset materials to obtain an optical filter layer including the optical modulation structure;

[0348] Or obtain an optical filter layer including the optical modulation structure by applying external dynamic regulation to the one or more layers of preset materials;

[0349] Or perform zone printing on the one or more layers of preset materials to obtain an optical filter layer including the optical modulation structure;

[0350] Or performing zoned material growth on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0351] Or performing quantum dot transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure.

[0352] When the optical artificial neural network intelligent chip is used for the intelligent processing task of a target object, using the input training samples and output training samples corresponding to the intelligent processing task, training the optical artificial neural network intelligent chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions.

[0353] In this embodiment, it should be noted that as Figure 1 shown, the optical filter layer 1 can be prepared by directly growing one or more layers of dielectric materials on the image sensor 2, then performing etching, depositing metal materials before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the optical modulation structure, each unit can have different modulation effects on lights with different wavelengths within the target range, and this modulation effect is insensitive to the incident angle and polarization. Each unit in the optical filter layer 1 corresponds to one or more pixels on the image sensor 2. 1 is directly prepared on 2.

[0354] In this embodiment, it should be noted that as Figure 14 shown, assuming that the image sensor 2 is a back-illuminated structure, the optical filter layer 1 can be directly etched on the silicon detector layer 21 of the back-illuminated image sensor, and then metal is deposited for preparation.

[0355] In addition, it should be noted that the optical modulation structure on the optical filter layer can be obtained by dry etching the optical modulation structure pattern on one or more preset materials. Dry etching is to directly remove the unnecessary parts of one or more preset materials on the surface of the photosensitive area of the image sensor, so as to obtain an optical filter layer containing the optical modulation structure; or by imprint transfer of one or more preset materials. Imprint transfer is to prepare the required structure by etching on other substrates, and then transfer the structure to the photosensitive area of the image sensor through materials such as PDMS, so as to obtain an optical filter layer containing the optical modulation structure; or by applying external dynamic regulation to one or more preset materials. External dynamic regulation is to use active materials, and then apply electrodes to regulate the optical modulation characteristics of the corresponding area by changing the voltage, so as to obtain an optical filter layer containing the optical modulation structure; or by zone printing of one or more preset materials. Zone printing is to use printing technology in zones to obtain an optical filter layer containing the optical modulation structure; or by zone material growth of one or more preset materials to obtain an optical filter layer containing the optical modulation structure; or by quantum dot transfer of one or more preset materials to obtain an optical filter layer containing the optical modulation structure.

[0356] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the optical artificial neural network intelligent chip in the above embodiment, for the detailed content of some principles and structures, etc., reference can be made to the introduction of the above embodiment, and this embodiment will not repeat it here.

[0357] Based on this, the optical artificial neural network intelligent chip provided by the embodiment of the present invention realizes a new intelligent chip that can realize the function of an artificial neural network. In the intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the intelligent chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and realizes the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the intelligent chip is used for subsequent intelligent processing of the artificial neural network, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the processor in the intelligent chip needs to be fully connected with the electrical signal The related processing with nonlinear activation is sufficient, which can greatly reduce the power consumption and delay during artificial neural network processing. Moreover, the embodiment of the present invention can also simultaneously utilize the image information, spectral information, angle of incident light and phase information of incident light of the target object, that is, the incident light carrying information at different points in the target object space. It can be seen that since the incident light carrying information at different points in the target object space covers the image, composition, shape, three-dimensional depth, structure and other information of the target object, when the identification processing is performed based on the incident light carrying information at different points in the target object space, the image, composition, shape, three-dimensional depth, structure and other multi-dimensional information of the target object can be covered, thereby improving the accuracy of intelligent processing (such as intelligent recognition). It can be seen that the optical artificial neural network chip provided by the embodiment of the present invention can not only achieve the effects of low power consumption and low delay, but also improve the accuracy of intelligent processing, so that it can be better applied in intelligent processing fields such as intelligent perception, recognition and / or decision-making.

[0358] It is understandable that face recognition technology is a biometric recognition technology that is widely used in access control and attendance systems, criminal investigation systems, e-commerce and other fields. Existing face recognition technology usually requires first imaging the face, and then inputting the face image into the neural network recognition model for processing, thereby realizing face recognition. It can be seen that the current face recognition task generally relies on the neural network recognition model, that is, the current face recognition task needs to be imaged first and then transmitted to the computer for subsequent neural network recognition model algorithm processing. The transmission and processing of large amounts of data causes greater power consumption and delay. Therefore, it is of great significance to mine more face information and achieve faster face recognition with higher accuracy, safety and reliability.

[0359] Therefore, based on the optical artificial neural network intelligent chip introduced in the foregoing embodiments, this embodiment provides a new optoelectronic chip for accurate face recognition. This chip consists of an optical filter layer that constitutes the input layer of the optical artificial neural network and the connection weights from the input layer to the linear layer, and an image sensor that constitutes the linear layer of the optical artificial neural network. By collecting the image information and spectral information of the face to be recognized, fast, accurate, safe, and reliable face recognition can be achieved. The content of this embodiment will be specifically explained and described in detail below.

[0360] An embodiment of the present invention also provides an optical artificial neural network face recognition chip for face recognition processing tasks, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0361] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes the image information of the target object to be processed by the optical artificial neural network intelligent chip and / or various optical space information. For example, the incident light carrying information includes at least one of the light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light; the incident light includes the reflected light of the user's face;

[0362] The image sensor is used to convert the incident light carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signal is an image signal modulated by the optical filter layer;

[0363] The processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a face recognition processing result.

[0364] As can be seen, this embodiment realizes a brand-new optical artificial neural network face recognition chip capable of realizing the functions of an artificial neural network for face recognition tasks. In this embodiment of the present invention, an artificial neural network is embedded on a hardware chip. The optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network. In this embodiment of the present invention, the spatial spectral information of a face is incident on a pre-trained hardware chip, and the hardware chip performs artificial neural network analysis on the spatial spectral information of the face to obtain a face recognition result. It should be noted that this embodiment of the present invention realizes fast and accurate face recognition with low power consumption, safety and reliability.

[0365] It can be understood that in the optical artificial neural network face recognition chip, the hardware structure thereon - the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the hardware structure thereon - the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and output layer of the artificial neural network. Specifically, the optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure, and the optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area, and accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions, and at the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions to obtain the output signal of the artificial neural network. It can be seen that in the face recognition chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. The filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the face recognition chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention separates the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the face recognition chip is used for artificial neural network face recognition processing in the subsequent process, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and the processor in the face recognition chip only needs to perform relevant processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay of the artificial neural network face recognition. It can be seen that the embodiment of the present invention uses the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor are used to project the spatial spectral information of the face into an electrical signal, and then the full connection processing and nonlinear activation processing of the electrical signal are implemented in the processor. It can be seen that the embodiment of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art.

[0366] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0367] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, multiple types of information such as the image information, spectral information, angle of the incident light, and phase information of the incident light of a human face can be utilized simultaneously to perform face recognition, thereby enabling more accurate face recognition.

[0368] It can be seen that the embodiments of the present invention can simultaneously utilize the image information, spectral information, angle information of the incident light, and phase information of the incident light of a human face, that is, the information carried by the incident light at different points in the face space. It can be seen that since the information carried by the incident light at different points in the face space covers information such as the image, composition, shape, three-dimensional depth, and structure of the human face, when performing recognition processing based on the information carried by the incident light at different points in the face space, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the human face can be covered, thereby enabling accurate face recognition.

[0369] Furthermore, the optical artificial neural network face recognition chip includes a trained optical modulation structure, an image sensor, and a processor;

[0370] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which are trained using input training samples corresponding to the face recognition task and output training samples to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0371] The input training samples include incident light reflected by different human faces; the output training samples include corresponding face recognition results.

[0372] Furthermore, when training the optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0373] In this embodiment, due to the addition of optical spectrum modulation, an artificial neural network optoelectronic chip with an object image and its spectrum as the input can be realized, enabling fast, accurate, safe, and reliable live face recognition.

[0374] In this embodiment, for face recognition, the faces of a large number of people can be collected first, and the weights from the input layer to the linear layer, that is, the system function of the optical filter layer, can be obtained through data training. Then, the required optical filter layer can be reversely designed and integrated above the image sensor. During actual training, using the face samples to be recognized and the output of the fabricated optical filter layer, the weights of the fully connected layer of the electrical signal are further trained and optimized, and thus a high-accuracy optical artificial neural network can be realized to complete the fast and accurate recognition of the user's face.

[0375] It can be understood that the specific modulation pattern of the modulation structure on the optical filter layer is designed through data training of the artificial neural network by collecting the faces of a large number of people in the early stage. It is usually a structure with an irregular shape, and of course, it may also be a structure with a regular shape.

[0376] As Figure 16 shown, the complete process for face recognition is as follows: Ambient light or other light sources irradiate on the user's face, and then the reflected light is collected by the chip and the recognition result is obtained after internal processing.

[0377] It can be understood that this chip actually utilizes the image information, spectral information, incident light angle information, and incident light phase information of the face at the same time, improving the accuracy and security of face recognition. In particular, it can accurately exclude non-living face models. At the same time, this chip partially implements the artificial neural network in hardware, improving the speed of face recognition. In addition, this chip solution can be mass-produced using existing CMOS processes, reducing the volume, power consumption, and cost of the device.

[0378] Furthermore, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0379] Furthermore, the optical filter layer is a single-layer structure or a multi-layer structure.

[0380] Furthermore, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0381] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0382] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0383] Furthermore, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0384] Furthermore, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0385] Furthermore, one or more groups of the multi-group micro-nano structure arrays included in the micro-nano unit are empty structures.

[0386] Furthermore, the micro-nano unit has polarization-independent characteristics.

[0387] Furthermore, the micro-nano unit has four-fold rotational symmetry.

[0388] Furthermore, the optical filter layer is composed of one or more filter layers;

[0389] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) micro-nano structures, tunable Fabry-Perot resonators.

[0390] Furthermore, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0391] Furthermore, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0392] An embodiment of the present invention further provides a face recognition device, including the optical artificial neural network face recognition chip described in the above embodiment. This face recognition device can be a portable face recognition device or a face recognition device installed at a fixed position. Since this face recognition device has similar beneficial effects to the above optical artificial neural network face recognition chip, it will not be elaborated here.

[0393] An embodiment of the present invention further provides a preparation method of the optical artificial neural network face recognition chip as described above, including:

[0394] Preparing an optical filter layer containing an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0395] Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals;

[0396] Connecting the image sensor and the processor;

[0397] Wherein, the optical filter layer is configured to perform different spectral modulations on incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain information carried by the incident light corresponding to different position points on the surface of the photosensitive region; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0398] The image sensor is configured to convert the information carried by the incident light corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a face recognition processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light of a human face.

[0399] Further, the method for manufacturing the optical artificial neural network face recognition chip further includes: a training process of the optical artificial neural network face recognition chip, specifically including:

[0400] Using input training samples and output training samples corresponding to the face recognition task, training an optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, the image sensor, and the processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0401] Further, when training an optical artificial neural network face recognition chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0402] It should be noted that the optical artificial neural network face recognition chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A. Embedding the artificial neural network part into an image sensor including various optical filter layers to achieve safe, reliable, fast, and accurate face recognition. B. The chip can be fabricated through a single CMOS process flow, which is beneficial to reducing the device failure rate, improving the device yield, and reducing costs. C. Monolithic integration can be achieved at the wafer level, which can minimize the distance between the sensor and the optical filter layer, is beneficial to reducing the size of the unit, and reducing the device volume and packaging cost.

[0403] It should be noted that for the detailed structural description of the optical artificial neural network face recognition chip provided in this embodiment, reference can be made to the introduction of the optical artificial neural network chip in the foregoing embodiment. To avoid redundancy, it will not be introduced here. In addition, for the detailed introduction of the manufacturing method of the optical artificial neural network face recognition chip, reference can also be made to the introduction of the manufacturing method of the optical artificial neural network chip in the foregoing embodiment, and it will not be elaborated here.

[0404] Currently, the methods for non-invasive blood glucose detection mainly include optical and radiation methods, reverse iontophoresis analysis, electromagnetic wave method, ultrasonic method, and interstitial fluid extraction method, etc. For example, the near-infrared spectroscopy detection method mainly utilizes the relationship between blood glucose concentration and its near-infrared spectrum absorption. The skin is irradiated with near-infrared light, and the blood glucose concentration is reflected from the change in the intensity of the reflected light. This method has the advantages of fast measurement, no need for chemical reagents and consumables, etc. However, due to the large individual differences of the measured objects and the very weak signals obtained, there are still key technologies to be further solved in aspects such as the selection of the measurement site, the selection of measurement conditions, and the method of extracting weak chemical information from the overlapping spectra. Moreover, the signal processing system is large in size and cannot be carried around. There is also the subcutaneous interstitial fluid detection method, which reflects the blood glucose concentration by measuring the glucose concentration of the subcutaneous exuded interstitial fluid. According to this principle, a watch for detecting glucose can be made, and the blood glucose concentration can be continuously monitored in real time. However, this method has poor accuracy and slow response speed, so it is very difficult to replace the existing invasive blood glucose meters. Therefore, non-invasive blood glucose detection has become an urgent need for the treatment of diabetes.

[0405] Therefore, based on the optical artificial neural network intelligent chip introduced in the foregoing embodiment, this embodiment provides a novel optoelectronic chip for blood glucose detection. The chip consists of an optical filter layer that constitutes the input layer of the optical artificial neural network and the connection weights from the input layer to the linear layer, and an image sensor that constitutes the linear layer of the optical artificial neural network. By collecting the image information and spectral information of the human body part to be measured, fast, accurate, safe and reliable non-invasive blood glucose detection can be achieved. The content of this embodiment will be specifically explained and described below.

[0406] The embodiment of the present invention also provides an optical artificial neural network blood glucose detection chip for blood glucose detection tasks, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0407] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is configured to perform different spectral modulations on incident light entering different position points of the optical modulation structure respectively, so as to obtain information carried by the incident light corresponding to different position points on the surface of the photosensitive area. The information carried by the incident light includes image information of a target object to be processed by the optical artificial neural network intelligent chip and / or various optical space information. For example, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light. The incident light includes reflected light and / or transmitted light of a human body part to be measured.

[0408] The image sensor is configured to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor. The electrical signal is an image signal modulated by the optical filter layer.

[0409] The processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a blood glucose detection result.

[0410] It can be seen that this embodiment realizes a brand-new optical artificial neural network blood glucose detection chip capable of realizing the function of an artificial neural network for blood glucose detection tasks. In the embodiment of the present invention, an artificial neural network is embedded on a hardware chip. The optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight from the input layer to the linear layer. The image sensor on the hardware chip is used as the linear layer of the artificial neural network. In the embodiment of the present invention, the spatial spectral information of the human body part to be measured is incident on the pre-trained hardware chip, and the hardware chip performs artificial neural network analysis on the spatial spectral information of the human body part to be measured to obtain a blood glucose detection result. It should be noted that the embodiment of the present invention realizes fast, accurate, non-invasive blood glucose detection with low power consumption, safety and reliability.

[0411] It can be understood that, in the optical artificial neural network blood glucose detection chip, the hardware structure thereon - the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, and the hardware structure thereon - the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and output layer of the artificial neural network. Specifically, the optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure, and the optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to different positions on the surface of the photosensitive area, and accordingly, the image sensor is used to convert the incident light carrying information corresponding to different positions into electrical signals corresponding to different positions, and at the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network. It can be seen that in the blood glucose detection chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. The filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the blood glucose detection chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention separates the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the blood glucose detection chip is used for artificial neural network blood glucose detection processing in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. The processor in the face recognition chip only needs to perform relevant processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay of the artificial neural network face recognition. It can be seen that the embodiment of the present invention uses the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor are used to project the spatial spectrum information of the part to be tested of the human body into an electrical signal, and then the full connection processing and nonlinear activation processing of the electrical signal are implemented in the processor. It can be seen that the embodiment of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art.

[0412] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0413] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, multiple types of information such as the image information, spectral information, incident light angle, and incident light phase information of the blood in the human body part to be measured can be utilized simultaneously, so that blood glucose detection can be performed more accurately, achieving non-invasive blood glucose detection.

[0414] In the embodiments of the present invention, the image information, spectral information, incident light angle information, and incident light phase information of the blood in the human body part to be measured can be utilized simultaneously, that is, the information carried by the incident light at different points in the blood space of the human body part to be measured. Thus, it can be seen that since the information carried by the incident light at different points in the blood space of the human body part to be measured covers information such as the image, composition, shape, three-dimensional depth, and structure of the blood in the human body part to be measured, when performing recognition processing based on the information carried by the incident light at different points in the blood space of the human body part to be measured, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the blood in the human body part to be measured can be covered, so that blood glucose detection can be accurately performed.

[0415] Further, the optical artificial neural network blood glucose detection chip includes a trained optical modulation structure, an image sensor, and a processor;

[0416] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which is trained using input training samples and output training samples corresponding to the blood glucose detection task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0417] The input training samples include incident light reflected, transmitted, and / or radiated by the human body part to be measured with different blood glucose values; the output training samples include the corresponding blood glucose values.

[0418] Further, when training the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0419] In this embodiment, due to the addition of optical spectrum modulation, an artificial neural network optoelectronic chip with the input of the image and its spectrum of the human body part to be measured can be realized, and rapid, accurate, safe, and reliable non-invasive blood glucose detection can be achieved.

[0420] In this embodiment, for non-invasive blood glucose detection, spectral signal data corresponding to a large number of human body parts with blood glucose information can be collected first, and the response of incident light passing through the micro-nano modulation structure can be simulated by a computer. Through data training, the required micro-nano modulation structure can be designed reversely and integrated above the image sensor. During the process of a user detecting blood glucose, by algorithmically restoring the electrical signals obtained by modulating incident light of different wavelengths, rapid and accurate detection of the user's blood glucose value can be achieved.

[0421] It can be understood that the specific modulation pattern of the modulation structure on the optical filter layer is designed through data training of an artificial neural network by collecting a large amount of spectral signal data corresponding to human body parts with blood glucose information in the early stage. It is usually a structure with an irregular shape, and of course, it may also be a structure with a regular shape.

[0422] As Figure 17 shown, the complete process for blood glucose detection is as follows: Ambient light or other light sources irradiate the part of the human body to be measured (finger), and then the reflected light is collected by the chip and the blood glucose detection result is obtained after internal processing. As Figure 18 shown, the complete process for blood glucose detection is as follows: Ambient light or other light sources irradiate the part of the human body to be measured (wrist), and then the reflected light is collected by the chip and the blood glucose detection result is obtained after internal processing.

[0423] It can be understood that this chip actually utilizes both the image information and spectral information of the part of the human body to be measured, improving the accuracy and safety of blood glucose detection. At the same time, this chip partially implements an artificial neural network in hardware, improving the speed of blood glucose detection. In addition, this chip solution can be mass-produced using existing CMOS processes, reducing the volume, power consumption, and cost of the device.

[0424] Furthermore, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0425] Furthermore, the optical filter layer is a single-layer structure or a multi-layer structure.

[0426] Furthermore, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0427] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0428] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0429] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0430] Further, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0431] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0432] Further, the micro-nano unit has polarization-independent characteristics.

[0433] Further, the micro-nano unit has four-fold rotational symmetry.

[0434] Further, the optical filter layer is composed of one or more filter layers;

[0435] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) nanostructures, tunable Fabry-Perot resonators.

[0436] Further, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

[0437] Further, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0438] An embodiment of the present invention also provides an intelligent blood glucose detector, including the optical artificial neural network blood glucose detection chip described in the above embodiment, and this blood glucose detector can be a wearable blood glucose detector. Since this intelligent blood glucose detector has beneficial effects similar to those of the above optical artificial neural network blood glucose detection chip, it will not be elaborated here.

[0439] An embodiment of the present invention also provides a preparation method of an optical artificial neural network blood glucose detection chip as described above, including:

[0440] Preparing an optical filter layer containing an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0441] Generate a processor with the functions of fully connecting and non-linearly activating the signals;

[0442] Connect the image sensor and the processor;

[0443] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0444] The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform full connection processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a blood glucose detection result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light and / or transmitted light of the human body part to be measured.

[0445] Further, the preparation method of the optical artificial neural network blood glucose detection chip further includes: the training process of the optical artificial neural network blood glucose detection chip, specifically including:

[0446] Using the input training samples and output training samples corresponding to the blood glucose detection task, train the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different full connection parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0447] Further, when training the optical artificial neural network blood glucose detection chip including different optical modulation structures, image sensors, and processors with different full connection parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0448] It should be noted that the optical artificial neural network blood glucose detection chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A. Embed the artificial neural network part into the image sensor including various optical filter layers to achieve safe, reliable, fast and accurate non-invasive blood glucose detection. B. The chip can be fabricated by a single CMOS process flow, which is beneficial to reducing the device failure rate, improving the device yield, and reducing costs. C. Monolithic integration is achieved at the wafer level, which can minimize the distance between the sensor and the optical filter layer, is beneficial to reducing the size of the unit, and reducing the device volume and packaging cost.

[0449] It should be noted that for the detailed structural description of the optical artificial neural network blood glucose detection chip provided in this embodiment, reference can be made to the introduction of the optical artificial neural network chip in the foregoing embodiment. To avoid repetition, it will not be introduced here. In addition, for the detailed introduction of the preparation method of the optical artificial neural network blood glucose detection chip, reference can also be made to the introduction of the preparation method of the optical artificial neural network intelligent chip in the foregoing embodiment, which will not be elaborated here.

[0450] Precision agriculture is a system that is supported by information technology and implements a complete set of modern farming operation technologies and management according to spatial variability, positioning, timing, and quantification. Its basic meaning is to adjust the input for crops according to the soil properties of crop growth, judge the spatial variability of soil properties and productivity within the farmland. On the other hand, determine the production goals of crops and conduct targeted "system diagnosis, optimized formula, technology assembly, and scientific management", mobilize soil productivity, achieve the same income or higher income with the least or most economical input, improve the environment, and efficiently utilize various agricultural resources to obtain economic and environmental benefits.

[0451] Currently, when performing precision agriculture control, it is usually necessary to first image the soil, and then input the image into a neural network recognition model for processing, so as to realize the recognition of soil fertility. It can be seen that the current soil fertility recognition task generally relies on the neural network recognition model, that is, the current soil fertility recognition task needs to first perform imaging and then transmit it to a computer for subsequent neural network recognition model algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay. In addition, there are similar problems for the recognition of crop growth conditions.

[0452] The technical principle of precision agriculture is to adjust the input for crops according to the spatial differences in soil fertility and crop growth conditions. When the recognition of soil fertility and crop growth conditions is inaccurate and there are relatively large power consumption and delays, it will seriously affect the development of precision agriculture.

[0453] Therefore, based on the optical artificial neural network intelligent chip introduced in the foregoing embodiments, this embodiment provides a novel optoelectronic chip for intelligent processing tasks in precise agricultural control. This chip consists of an optical filter layer that forms the input layer of the optical artificial neural network and the connection weights from the input layer to the linear layer, and an image sensor that forms the linear layer of the optical artificial neural network. By collecting the image information and spectral information of farmland soil and crops, it can achieve rapid, accurate, safe, and reliable identification and qualitative analysis of soil fertility, pesticide spraying conditions, trace element content, and crop growth status, etc. The following specifically explains and elaborates on the content of this embodiment.

[0454] This embodiment of the present invention also provides an optical artificial neural network intelligent agricultural precise control chip for intelligent processing tasks in precise agricultural control, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network;

[0455] The optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the incident light-carrying information corresponding to different position points on the surface of the photosensitive area; the incident light-carrying information includes the image information and / or various optical space information of the target object to be processed by the optical artificial neural network intelligent chip. For example, the incident light-carrying information includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light; the incident light includes the reflected light, transmitted light, and / or radiation light of agricultural objects; the agricultural objects include crops and / or soil;

[0456] The image sensor is used to convert the incident light-carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signal is an image signal modulated by the optical filter layer;

[0457] The processor is used to perform a fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain a precise agricultural control processing result;

[0458] Among them, the intelligent processing tasks for agricultural precise control include one or more of soil fertility detection, pesticide spraying detection, trace element content detection, drug resistance detection, and crop growth condition detection; the processing results of agricultural precise control include one or more of soil fertility detection results, pesticide spraying detection results, trace element content detection results, drug resistance detection results, and crop growth condition detection results.

[0459] It can be seen that this embodiment realizes a brand-new optical artificial neural network intelligent agricultural precise control chip capable of realizing the functions of an artificial neural network, which is used for intelligent processing tasks of agricultural precise control. In this embodiment of the present invention, an artificial neural network is embedded on a hardware chip. The optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network. In this embodiment of the present invention, the spatial spectral information of farmland soil and crops is incident into the pre-trained hardware chip, and the hardware chip performs artificial neural network analysis on the spatial spectral information of farmland soil and crops to obtain the processing results of agricultural precise control. It should be noted that this embodiment of the present invention realizes fast and accurate identification and qualitative analysis of soil fertility, pesticide spraying situation, trace element content, and crop growth condition with low power consumption, safety, and reliability.

[0460] It can be understood that in the optical artificial neural network intelligent agriculture precise control chip, the hardware structure - optical filter layer thereon corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the hardware structure - image sensor thereon corresponds to the linear layer of the artificial neural network; the processor corresponds to the non - linear layer and the output layer of the artificial neural network. Specifically, the optical filter layer is disposed on the surface of the photosensitive area of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the incident light - carried information corresponding to different position points on the surface of the photosensitive area. Correspondingly, the image sensor is used to convert the incident light - carried information corresponding to different position points into electrical signals corresponding to different position points. At the same time, the processor connected to the image sensor is used to perform a fully - connected processing and a non - linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. Thus, it can be seen that in this intelligent agriculture precise control chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weights from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this intelligent agriculture precise control chip implement the related functions of the input layer and the linear layer in the artificial neural network. That is, the embodiments of the present invention strip the input layer and the linear layer in the artificial neural network implemented by software in the prior art and implement these two layers of structures in the artificial neural network in a hardware manner. As a result, when subsequently using this intelligent agriculture precise control chip for artificial neural network - based intelligent agriculture precise control processing, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. Only the processor in the intelligent agriculture precise control chip needs to perform related processing of full - connection with the electrical signals and non - linear activation. In this way, the power consumption and latency during artificial neural network - based intelligent agriculture precise control can be significantly reduced. Thus, it can be seen that the embodiments of the present invention use the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weights from the input layer to the linear layer, project the spatial spectral information of the human face into electrical signals by using the optical filter layer and the image sensor, and then implement the fully - connected processing and non - linear activation processing of the electrical signals in the processor. Thus, it can be seen that the embodiments of the present invention can eliminate the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art.

[0461] In this embodiment, the incident light - carried information may include one or more (including two) of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0462] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, multiple types of information such as the image information, spectral information, incident light angle, and incident light phase information of farmland soil and crops can be utilized simultaneously to identify the farmland soil and crops, so as to more accurately identify and qualitatively analyze soil fertility, pesticide spraying conditions, trace element content, and crop growth conditions, etc.

[0463] Embodiments of the present invention can simultaneously utilize the image information, spectral information, incident light angle information, and incident light phase information of farmland soil and crops, that is, the information carried by the incident light at different points in the space of farmland soil and crops. It can be seen that since the information carried by the incident light at different points in the space of farmland soil and crops covers information such as the images, components, shapes, three-dimensional depths, and structures of farmland soil and crops, when performing identification processing based on the information carried by the incident light at different points in the space of farmland soil and crops, it can cover multi-dimensional information such as the images, components, shapes, three-dimensional depths, and structures of farmland soil and crops, thereby enabling accurate identification and qualitative analysis of soil fertility, pesticide spraying conditions, trace element content, and crop growth conditions, etc.

[0464] Furthermore, the optical artificial neural network intelligent agriculture precise control chip includes a trained optical modulation structure, an image sensor, and a processor;

[0465] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which is trained using the input training samples and output training samples corresponding to the intelligent processing task of agricultural precise control to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0466] Among them, the input training samples include the incident light reflected, transmitted, and / or radiated by soils with different fertilities; the output training samples include the corresponding soil fertilities;

[0467] and / or,

[0468] The input training samples include the incident light reflected, transmitted, and / or radiated by soils with different pesticide spraying conditions; the output training samples include the corresponding pesticide spraying conditions;

[0469] and / or,

[0470] The input training samples include the incident light reflected, transmitted, and / or radiated by soils with different trace element content conditions; the output training samples include the corresponding trace element content conditions;

[0471] and / or,

[0472] The input training samples include incident light reflected, transmitted, and / or radiated by soil with different drug resistances; the output training samples include corresponding drug resistances.

[0473] and / or,

[0474] The input training samples include incident light reflected, transmitted, and / or radiated by crops with different growth conditions; the output training samples include corresponding crop growth conditions.

[0475] Further, when training an optical artificial neural network intelligent agriculture precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0476] In this embodiment, for the intelligent processing task of agricultural precision control, a large number of soil samples and crop samples in different states and positions can be collected first. By data training, the weights from the input layer to the linear layer, that is, the system function of the optical filter layer, can be obtained, and then the required optical filter layer can be reversely designed and integrated above the image sensor. During actual training, using the soil samples and crop samples, and using the output of the fabricated optical filter layer, the weights of the electrical signal fully connected layer are further trained and optimized, and then a high-accuracy optical artificial neural network can be realized to complete the recognition and qualitative analysis of soil fertility, pesticide spraying situation, trace element content, and crop growth conditions.

[0477] It can be understood that the specific modulation pattern of the modulation structure on the optical filter layer is designed by artificial neural network data training through collecting a large number of soil samples and crop samples in different states and positions in the early stage. It is usually a structure with an irregular shape, and of course, it may also be a structure with a regular shape.

[0478] Such as Figure 19 shown, the complete process for agricultural object recognition is: ambient light or other light sources irradiate on the agricultural object, and then the reflected light is collected by the chip and the recognition result is obtained after internal processing. As Figure 20 shown, the spectral data and images of farmland soil and crops are collected by a spectral and image collector, and then the reflected light is collected by the precision agriculture control chip and processed by the internal algorithm of the processor to obtain the recognition result.

[0479] It is understandable that the chip actually utilizes both the image information and spectral information of farmland soil and crops simultaneously to achieve safe, reliable, fast, and accurate precision agriculture control. In addition, this chip solution can be mass-produced using existing CMOS processes, reducing the volume, power consumption, and cost of the device.

[0480] Further, the optical modulation structure in the optical filter layer includes regular structures and / or irregular structures; and / or, the optical modulation structure in the optical filter layer includes discrete structures and / or continuous structures.

[0481] Further, the optical filter layer is a single-layer structure or a multi-layer structure.

[0482] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0483] Further, the micro-nano unit includes regular structures and / or irregular structures; and / or, the micro-nano unit includes discrete structures and / or continuous structures.

[0484] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0485] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0486] Further, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0487] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0488] Further, the micro-nano unit has polarization-independent characteristics.

[0489] Further, the micro-nano unit has four-fold rotational symmetry.

[0490] Further, the optical filter layer is composed of one or more filter layers;

[0491] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP nanostructures, tunable Fabry-Perot resonators.

[0492] Further, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0493] Further, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0494] An embodiment of the present invention further provides an intelligent agricultural control device, including: an optical artificial neural network intelligent agricultural precise control chip as described in the above embodiment. The intelligent agricultural control device may include a fertilizer applicator device, a pesticide spraying device, a drug resistance analysis device, a drone device, an agricultural intelligent robot device, a crop health analysis device, a crop growth status monitoring device, etc.

[0495] Another embodiment of the present invention provides a method for preparing an optical artificial neural network intelligent agricultural precise control chip as described in the above embodiment, including:

[0496] Preparing an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0497] Generating a processor with the function of performing fully connected processing and non-linear activation processing on signals;

[0498] Connecting the image sensor and the processor;

[0499] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive area; the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0500] The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an agricultural precise control processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light of an agricultural object; the agricultural object includes crops and / or soil.

[0501] Further, the method for preparing the optical artificial neural network intelligent agriculture precise control chip further includes: the training process of the optical artificial neural network intelligent agriculture precise control chip, specifically including:

[0502] Using the input training samples and output training samples corresponding to the intelligent processing tasks of agricultural precise control, training the optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0503] Further, when training the optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0504] It should be noted that the optical artificial neural network intelligent agriculture precise control chip provided in this embodiment has the following effects: A. Embedding the artificial neural network part into the image sensor including various optical filter layers to achieve safe, reliable, fast, and accurate precise agriculture control. B. It is possible to introduce artificial neural network training and recognition for soil, crops, etc., which is convenient for subsequent integration into industrial intelligent control systems such as unmanned aerial vehicles and intelligent robots to achieve large-area precise agriculture control, and has high recognition accuracy and precise qualitative analysis. C. The preparation of this chip can be completed by a single CMOS process flow, which is beneficial to reducing the device failure rate, improving the device yield, and reducing costs. D. Monolithic integration is achieved at the wafer level, which can minimize the distance between the sensor and the optical filter layer, is beneficial to reducing the size of the unit, and reducing the device volume and packaging cost.

[0505] It should be noted that for the detailed structural description of the optical artificial neural network intelligent agriculture precise control chip provided in this embodiment, reference can be made to the introduction of the optical artificial neural network intelligent chip in the foregoing embodiment. To avoid repetition, it will not be introduced here. In addition, for the detailed introduction of the method for preparing the optical artificial neural network intelligent agriculture precise control chip, reference can also be made to the introduction of the method for preparing the optical artificial neural network intelligent chip in the foregoing embodiment, which will not be elaborated here.

[0506] Converter steelmaking is the most widely used and most efficient steelmaking method in the world. End point control is one of the key technologies in converter production. Accurate determination of the end point is of great significance in improving the quality of molten steel and shortening the smelting cycle. However, due to the instability of the raw materials entering the furnace, complex chemical reactions and strict requirements for the type of steel being produced, accurate control of the end point of smelting remains difficult. Accurate online detection of the end point carbon content and molten steel temperature has always been a problem that needs to be solved urgently in the metallurgical industry around the world. At present, the industry mainly relies on manual experience or complex large-scale instruments and equipment to measure the furnace mouth temperature and the qualitative measurement of slag residues for end point control, which has low accuracy and high cost.

[0507] To this end, based on the optical artificial neural network intelligent chip introduced in the previous embodiment, this embodiment provides a new optoelectronic chip for monitoring the smelting endpoint, the chip is composed of an optical filter layer to form the input layer of the optical artificial neural network and the connection weight from the input layer to the linear layer, and an image sensor to form the linear layer of the optical artificial neural network, by collecting the image information and spectral information of the steelmaking furnace mouth, can achieve fast, accurate, safe and reliable identification of the smelting endpoint. The content of this embodiment is explained and described in detail below.

[0508] The embodiment of the present invention also provides an optical artificial neural network smelting endpoint monitoring chip, which is used for smelting endpoint monitoring tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;

[0509] The optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure. The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area; the incident light carrying information includes image information and / or various optical spatial information of the target object to be processed by the optical artificial neural network intelligent chip, for example, the incident light carrying information includes at least one of light intensity distribution information, spectrum information, angle information of the incident light and phase information of the incident light; the incident light includes reflected light, transmitted light and / or radiated light from the steelmaking furnace mouth;

[0510] The image sensor is used to convert the information carried by the incident light corresponding to different positions after being modulated by the optical filter layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0511] The processor is used to perform a fully connected process and a non-linear activation process on the electrical signals corresponding to different position points to obtain a smelting end-point monitoring result;

[0512] Among them, the smelting end-point monitoring task includes identifying the smelting end-point, and the smelting end-point monitoring result includes a smelting end-point identification result.

[0513] It can be seen that this embodiment realizes a brand-new optical artificial neural network smelting end-point monitoring chip capable of realizing the functions of an artificial neural network, which is used for the smelting end-point monitoring task. In the embodiment of the present invention, an artificial neural network is embedded in the hardware chip. The optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network. In the embodiment of the present invention, the spatial spectral information of the steelmaking furnace mouth is incident into the pre-trained hardware chip, and the hardware chip performs an artificial neural network analysis on the spatial spectral information of the steelmaking furnace mouth to obtain a smelting end-point identification result. It should be noted that the embodiment of the present invention realizes fast and accurate smelting end-point identification with low power consumption, safety and reliability.

[0514] It can be understood that in this optical artificial neural network end-point monitoring chip, the hardware structure - optical filter layer thereon corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the hardware structure - image sensor thereon corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network. Specifically, the optical filter layer is disposed on the surface of the photosensitive region of the image sensor. The optical filter layer includes an optical modulation structure. The optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the photosensitive region. Correspondingly, the image sensor is used to convert the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points. At the same time, the processor connected to the image sensor is used to perform full connection processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. Thus, it can be seen that in this end-point monitoring chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weights from the input layer to the linear layer. That is, the optical filter layer and the image sensor in this end-point monitoring chip implement the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiments of the present invention strip the input layer and the linear layer in the artificial neural network implemented by software in the prior art and implement these two layers of structures in the artificial neural network in a hardware manner. As a result, when using this end-point monitoring chip to perform artificial neural network end-point recognition processing subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the processor in the end-point monitoring chip needs to perform relevant processing of full connection and non-linear activation of the electrical signals. This can significantly reduce the power consumption and delay during artificial neural network end-point monitoring. Thus, it can be seen that the embodiments of the present invention use the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weights from the input layer to the linear layer, project the spatial spectral information of the steelmaking furnace mouth into electrical signals by using the optical filter layer and the image sensor, and then implement full connection processing and non-linear activation processing of the electrical signals in the processor. Thus, it can be seen that the embodiments of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art.

[0515] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0516] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, multiple types of information such as the image information, spectral information, angle of the incident light, and phase information of the incident light at the steelmaking furnace mouth can be used simultaneously to identify the steelmaking furnace mouth, so that the identification of the tapping end point can be achieved more accurately.

[0517] In the embodiments of the present invention, the image information, spectral information, angle information of the incident light, and phase information of the incident light at the steelmaking furnace mouth can be used simultaneously, that is, the information carried by the incident light at different points in the space of the steelmaking furnace mouth. It can be seen that since the information carried by the incident light at different points in the space of the steelmaking furnace mouth covers information such as the image, composition, shape, three-dimensional depth, and structure of the steelmaking furnace mouth, when performing identification processing based on the information carried by the incident light at different points in the space of the steelmaking furnace mouth, multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the steelmaking furnace mouth can be covered, so that the identification of the tapping end point can be accurately performed.

[0518] Furthermore, the tapping end point monitoring task further includes identifying the carbon content and / or molten steel temperature during the smelting process, and the tapping end point monitoring result includes the identification result of the carbon content and / or molten steel temperature during the smelting process.

[0519] Furthermore, the optical artificial neural network tapping end point monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;

[0520] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network tapping end point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters, which is trained using the input training samples and output training samples corresponding to the tapping end point monitoring task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0521] The input training samples include the incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth that has been smelted to the end point and not smelted to the end point; the output training samples include the determination result of whether it has been smelted to the end point.

[0522] Furthermore, when the tapping end point monitoring task further includes identifying the carbon content and / or molten steel temperature during the smelting process, correspondingly, the input training samples further include the incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth with different carbon contents and / or molten steel temperatures, and the output training samples further include the corresponding carbon content and / or molten steel temperature.

[0523] Further, when training an optical artificial neural network endpoint monitoring chip for smelting that includes different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0524] In this embodiment, for smelting endpoint recognition, a large number of converter mouth images and spectral information at the endpoint moment can be collected first. By data training, the weights from the input layer to the linear layer, that is, the system function of the optical filter layer, can be obtained, and then the required optical filter layer can be reversely designed and integrated above the image sensor. During actual training, using the sample of the steelmaking furnace mouth to be recognized and the output of the fabricated optical filter layer, the weights of the fully connected layer of the electrical signal are further trained and optimized, and then a high-accuracy optical artificial neural network can be realized to complete the rapid and accurate recognition of the smelting endpoint.

[0525] It can be understood that the specific modulation pattern of the modulation structure on the optical filter layer is designed by artificial neural network data training through collecting a large number of converter mouth images and spectral information at the endpoint moment in the early stage. It is usually a structure with an irregular shape, and of course, it may also be a structure with a regular shape.

[0526] Such as Figure 21 As shown, the complete process for recognizing the steelmaking furnace mouth to determine whether it is the smelting endpoint is as follows: ambient light or other light sources irradiate the steelmaking furnace mouth, and then the reflected light is collected by the chip and the recognition result is obtained after internal processing.

[0527] It can be understood that this chip actually utilizes both the image information and spectral information of the steelmaking furnace mouth, improving the accuracy of smelting endpoint recognition. At the same time, this chip partially implements an artificial neural network in hardware, improving the speed of smelting endpoint recognition. In addition, this chip solution can be mass-produced using existing CMOS processes, reducing the volume, power consumption, and cost of the device.

[0528] Further, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0529] Further, the optical filter layer is a single-layer structure or a multi-layer structure.

[0530] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0531] Further, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0532] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0533] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0534] Further, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0535] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0536] Further, the micro-nano unit has polarization-independent characteristics.

[0537] Further, the micro-nano unit has four-fold rotational symmetry.

[0538] Further, the optical filter layer is composed of one or more filter layers;

[0539] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) nanostructures, tunable Fabry-Perot resonators.

[0540] Further, the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0541] Further, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0542] An embodiment of the present invention also provides an intelligent smelting control device, including: an optical artificial neural network smelting endpoint monitoring chip as described in the above embodiment. The intelligent smelting control device may include various devices related to smelting process control, and this embodiment does not limit this. The intelligent smelting control device provided in this embodiment has all the beneficial effects of the optical artificial neural network smelting endpoint monitoring chip described in the above embodiment. Since the above embodiment has described this in detail, this embodiment will not be elaborated here.

[0543] An embodiment of the present invention also provides a preparation method for the smelting end-point monitoring of the optical artificial neural network as described above, including:

[0544] Prepare an optical filter layer including an optical modulation structure on the surface of the photosensitive area of the image sensor;

[0545] Generate a processor with the functions of performing fully connected processing and non-linear activation processing on signals;

[0546] Connect the image sensor and the processor;

[0547] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area; the incident light carrying information includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light;

[0548] The image sensor is used to convert the incident light carrying information corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the smelting end-point monitoring result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light, and / or radiation light from the steelmaking furnace mouth.

[0549] Furthermore, the preparation method of the optical artificial neural network smelting end-point monitoring chip further includes: the training process of the optical artificial neural network smelting end-point monitoring chip, specifically including:

[0550] Use the input training samples and output training samples corresponding to the smelting end-point monitoring task to train the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0551] Furthermore, when training the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0552] It should be noted that the optical artificial neural network smelting endpoint monitoring chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A. Embed the artificial neural network part into the image sensor containing various optical filter layers to achieve safe, reliable, fast and accurate smelting endpoint control. B. The detectable samples include but are not limited to the endpoint control of converter steelmaking. Introduce artificial neural network training to detect the temperature and material elements at the furnace mouth of the smelting furnace, which is extremely easy to integrate with the backend industrial control system, has high recognition accuracy and precise qualitative analysis. C. The preparation of the chip can be completed by a single CMOS process flow, which is beneficial to reducing the device failure rate, improving the finished product yield of the device, and reducing costs. D. Monolithic integration is achieved at the wafer level, which can minimize the distance between the sensor and the optical filter layer, is beneficial to reducing the size of the unit, and reducing the device volume and packaging cost.

[0553] It should be noted that for the detailed structural description of the optical artificial neural network smelting endpoint monitoring chip provided in this embodiment, reference can be made to the introduction of the optical artificial neural network intelligent chip in the foregoing embodiment. To avoid redundancy, it will not be introduced here. In addition, for the detailed introduction of the preparation method of the optical artificial neural network smelting endpoint monitoring chip, reference can also be made to the introduction of the preparation method of the optical artificial neural network intelligent chip in the foregoing embodiment, and it will not be elaborated here.

[0554] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A photonic artificial neural network intelligent chip, characterized in that, it includes: a photonic filter layer, an image sensor, and a processor; the photonic filter layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the non-linear layer and the output layer of the artificial neural network; the photonic filter layer is disposed on the surface of the photosensitive area of the image sensor, the photonic filter layer includes a photonic modulation structure, and the photonic filter layer is configured to perform different spectral modulations on the incident light entering different position points of the photonic modulation structure respectively through the photonic modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area; the image sensor is configured to convert the incident light carrying information corresponding to different position points after being modulated by the photonic filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signal is an image signal modulated by the photonic filter layer; the processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

2. The photonic artificial neural network intelligent chip according to claim 1, characterized in that, the incident light carrying information includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

3. The photonic artificial neural network intelligent chip according to claim 1, characterized in that, the photonic artificial neural network intelligent chip is used for intelligent processing tasks of a target object; the intelligent processing tasks include at least one or more of intelligent perception, intelligent recognition, and intelligent decision-making tasks; the reflected light, transmitted light, and / or radiation light of the target object enter into the trained photonic artificial neural network intelligent chip to obtain the intelligent processing result of the target object; the intelligent processing result includes at least one or more of intelligent perception results, intelligent recognition results, and / or intelligent decision-making results; wherein, the trained photonic artificial neural network intelligent chip refers to a photonic artificial neural network intelligent chip including a trained photonic modulation structure, an image sensor, and a processor; the trained photonic modulation structure, image sensor, and processor refer to a photonic modulation structure, an image sensor, and a processor that are obtained by training a photonic artificial neural network intelligent chip including different photonic modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters using input training samples and output training samples corresponding to the intelligent processing tasks and that satisfy the training convergence conditions.

4. The photonic artificial neural network intelligent chip according to claim 3, characterized in that, when training a photonic artificial neural network intelligent chip including different photonic modulation structures, image sensors, and processors with different fully connected parameters and non-linear activation parameters, the different photonic modulation structures are designed and implemented by means of computer optical simulation design.

5. The photonic artificial neural network intelligent chip according to any one of claims 1 to 4, It is characterized in that the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

6. The optical artificial neural network intelligent chip according to any one of claims 1 to 4, It is characterized in that the optical filter layer is a single-layer structure or a multi-layer structure.

7. The optical artificial neural network intelligent chip according to any one of claims 1 to 4, It is characterized in that the optical modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

8. The optical artificial neural network intelligent chip according to claim 7, It is characterized in that the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

9. The optical artificial neural network intelligent chip according to claim 7, It is characterized in that the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

10. The optical artificial neural network intelligent chip according to claim 9, It is characterized in that each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

11. The optical artificial neural network intelligent chip according to claim 9, It is characterized in that each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

12. The optical artificial neural network intelligent chip according to claim 9, It is characterized in that one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

13. The optical artificial neural network intelligent chip according to claim 9, It is characterized in that the micro-nano unit has polarization-independent characteristics.

14. The optical artificial neural network intelligent chip according to claim 13, It is characterized in that the micro-nano unit has four-fold rotational symmetry.

15. The optical artificial neural network intelligent chip according to claim 1, It is characterized in that the optical filter layer is composed of one or more filter layers; the filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton SPP nanostructures, tunable Fabry-Perot resonators.

16. The optical artificial neural network intelligent chip according to claim 15, It is characterized in that the semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

17. The optical artificial neural network intelligent chip according to claim 1, It is characterized in that the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

18. The optical artificial neural network intelligent chip according to claim 1, characterized in that, the image sensor is any one or more of the following: CMOS image sensor CIS, charge coupled device CCD, single photon avalanche diode SPAD array, and focal plane photodetector array.

19. The optical artificial neural network intelligent chip according to claim 1, characterized in that, the type of the artificial neural network includes: feedforward neural network.

20. The optical artificial neural network intelligent chip according to claim 1, characterized in that, a light-transmitting medium layer is provided between the optical filter layer and the image sensor.

21. The optical artificial neural network intelligent chip according to claim 1, characterized in that, the image sensor is front-illuminated, including: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on a surface of the metal wire layer away from the light detection layer; or, the image sensor is back-illuminated, including: a light detection layer and a metal wire layer arranged from top to bottom, and the optical filter layer is integrated on a surface of the light detection layer away from the metal wire layer.

22. An intelligent processing device, characterized in that, including: the optical artificial neural network intelligent chip according to any one of claims 1 to 21.

23. The intelligent processing device according to claim 22, characterized in that, the intelligent processing device includes one or more of a smart phone, a smart computer, a smart identification device, a smart sensing device, and a smart decision-making device.

24. A method for preparing an optical artificial neural network intelligent chip according to any one of claims 1 to 21, characterized in that, including: preparing an optical filter layer including a light modulation structure on a surface of a photosensitive area of the image sensor; generating a processor having functions of performing fully connected processing and non-linear activation processing on signals; connecting the image sensor and the processor; wherein, the optical filter layer is configured to perform different spectral modulations on incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain incident light-carrying information corresponding to different position points on the surface of the photosensitive area; the incident light-carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light; the image sensor is configured to convert the incident light-carrying information corresponding to different position points modulated by the optical filter layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is configured to perform fully connected processing and non-linear activation processing on the electrical signals corresponding to different position points to obtain an output signal of the artificial neural network; the electrical signal is an image signal modulated by the optical filter layer.

25. The method for preparing an optical artificial neural network intelligent chip according to claim 24, characterized in that, preparing an optical filter layer including a light modulation structure on a surface of a photosensitive area of the image sensor includes: growing one or more layers of a preset material on a surface of the photosensitive area of the image sensor; Etch the light modulation structure pattern on the one or more preset materials to obtain an optical filter layer containing a light modulation structure; Or perform imprint transfer on the one or more preset materials to obtain an optical filter layer containing a light modulation structure; Or obtain an optical filter layer containing a light modulation structure by applying external dynamic modulation to the one or more preset materials; Or perform zone printing on the one or more preset materials to obtain an optical filter layer containing a light modulation structure; Or perform zone growth on the one or more preset materials to obtain an optical filter layer containing a light modulation structure; Or perform quantum dot transfer on the one or more preset materials to obtain an optical filter layer containing a light modulation structure.

26. The method for preparing an optical artificial neural network intelligent chip according to claim 24, characterized in that, When the optical artificial neural network intelligent chip is used for the intelligent processing task of a target object, use the input training sample and output training sample corresponding to the intelligent processing task to train the optical artificial neural network intelligent chip including different light modulation structures, image sensors and processors with different fully connected parameters and non-linear activation parameters, so as to obtain a light modulation structure, an image sensor and a processor that meet the training convergence conditions.

Images