Optical Artificial Neural Network Intelligent Agriculture Precision Control Chip and Preparation Method

Abstract

The present invention provides an optical artificial neural network intelligent agriculture precise control chip and a preparation method thereof, which are used for intelligent processing tasks of agriculture precise control. The optical modulation layer is used as the input layer and the linear layer of the artificial neural network, the filtering effect of the optical modulation layer on the incident light is used as the connection weight from the input layer to the linear layer, the square detection response of the image sensor is used as the first non-linear activation function in the non-linear layer of the artificial neural network, and the processor is used as the second non-linear activation function and the output layer in the fully connected and non-linear layer of the artificial neural network. Thus, the optical modulation layer and the image sensor implement the related functions of the input layer, the linear layer and the non-linear activation function in the artificial neural network in a hardware manner, thereby greatly reducing the power consumption and delay during the processing of the artificial neural network.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to an optical artificial neural network intelligent agriculture precise control chip and a preparation method thereof. Background Art

[0002] Precision agriculture is a system supported by information technology that implements a complete set of modern farming operation technologies and management according to spatial variation, positioning, timing, and quantification. Its basic meaning is to adjust the input for crops according to the soil properties of crop growth, judge the spatial variation of soil properties and productivity within the farmland. On the other hand, determine the production goals of crops, and carry out "system diagnosis, optimized formula, technology assembly, and scientific management" for positioning, mobilize soil productivity, achieve the same income or higher income with the least or most economical input, improve the environment, and efficiently utilize various agricultural resources to obtain economic and environmental benefits.

[0003] Currently, when performing precision agriculture control, there is a problem of slow processing speed. For example, for the identification of soil fertility, steps such as image acquisition, image preprocessing, feature extraction, and feature matching are required to achieve the identification of soil fertility. Currently, the intelligent identification tasks of agricultural objects generally rely on neural network identification models, that is, currently, for the intelligent identification tasks of agricultural objects, it is necessary to first image the agricultural objects and then transmit them to a computer for subsequent neural network identification model algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay.

[0004] In addition, currently, when performing precision agriculture control, there is also a problem of inaccurate identification. For example, it is difficult to ensure the accuracy of identification only by using the two-dimensional image information of the soil. For example, there are distortion factors in the two-dimensional image, and moreover, during the imaging process, optical information needs to be converted into digital electronic signals and then transmitted to a computer for subsequent algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay. In addition, there are similar problems for the identification of the growth status of crops.

[0005] The technical principle of precision agriculture is to adjust the input for crops according to the spatial differences in soil fertility and the growth status of crops. When the identification of soil fertility and the growth status of crops is inaccurate and there are relatively large power consumption and delays, it will seriously affect the development of precision agriculture. Summary of the Invention

[0006] In view of the problems existing in the prior art, an embodiment of the present invention provides an optical artificial neural network intelligent agriculture precise control chip and a preparation method thereof to solve at least one of the above technical problems.

[0007] Specifically, the embodiment of the present invention provides the following technical solutions:

[0008] In a first aspect, an embodiment of the present invention provides an optical artificial neural network intelligent agriculture precise control chip for intelligent processing tasks of agricultural precise control, including: an optical modulation layer, an image sensor, and a processor; the optical modulation layer corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer, and the square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor corresponds to the fully connected layer and the output layer of the artificial neural network, or, the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network;

[0009] The optical modulation layer is disposed on the surface of the image sensor, and the optical modulation layer includes an optical modulation structure for respectively performing different spectral modulations on incident light entering different position points of the optical modulation structure, so as to obtain incident light-carrying information corresponding to different position points on the surface of the image sensor; the incident light includes reflected light, transmitted light, and / or radiation light of an agricultural object; the agricultural object includes crops and / or soil;

[0010] The image sensor converts the incident light-carrying information corresponding to different position points after being modulated by the optical modulation layer into electrical signals corresponding to different position points through a square-law detection response after performing a first non-linear activation process, and sends the electrical signals corresponding to different position points to the processor;

[0011] The processor performs a fully connected process on the electrical signals corresponding to different position points, or, the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain an agricultural precise control processing result;

[0012] Wherein, the intelligent processing tasks of agricultural precise control include: one or more of soil fertility detection, pesticide spraying detection, trace element content detection, drug resistance detection, and crop growth condition detection; the agricultural precise control processing results include: one or more of soil fertility detection results, pesticide spraying detection results, trace element content detection results, drug resistance detection results, and crop growth condition detection results.

[0013] Further, the incident light-carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0014] Further, the optical artificial neural network intelligent agriculture precise control chip includes a trained optical modulation structure, an image sensor, and a processor;

[0015] The trained optical modulation structure, image sensor, and processor refer to those obtained by training an optical artificial neural network intelligent agricultural precision control chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters using input training samples and output training samples corresponding to the agricultural precision control intelligent processing task, and meeting the training convergence conditions; or, the trained optical modulation structure, image sensor, and processor refer to those obtained by training an optical artificial neural network intelligent agricultural precision control chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters using input training samples and output training samples corresponding to the agricultural precision control intelligent processing task, and meeting the training convergence conditions.

[0016] Among them, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different fertilities; the output training samples include the corresponding soil fertilities.

[0017] and / or,

[0018] The input training samples include incident light reflected, transmitted, and / or radiated by soils with different pesticide spraying conditions; the output training samples include the corresponding pesticide spraying conditions.

[0019] and / or,

[0020] The input training samples include incident light reflected, transmitted, and / or radiated by soils with different trace element content conditions; the output training samples include the corresponding trace element content conditions.

[0021] and / or,

[0022] The input training samples include incident light reflected, transmitted, and / or radiated by soils with different drug resistances; the output training samples include the corresponding drug resistances.

[0023] and / or,

[0024] The input training samples include incident light reflected, transmitted, and / or radiated by crops with different growth conditions; the output training samples include the corresponding crop growth conditions.

[0025] Further, when training an optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0026] Further, the optical modulation structures in the optical modulation layer include regular structures and / or irregular structures; and / or, the optical modulation structures in the optical modulation layer include discrete structures and / or continuous structures.

[0027] Further, the optical modulation layer is a single-layer structure or a multi-layer structure.

[0028] Further, the optical modulation structures in the optical modulation layer include a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0029] Further, the micro-nano units include regular structures and / or irregular structures; and / or, the micro-nano units include discrete structures and / or continuous structures.

[0030] Further, the micro-nano units include multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0031] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0032] Further, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0033] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano units are empty structures.

[0034] Further, the micro-nano units have four-fold rotational symmetry.

[0035] Further, the optical modulation layer is composed of one or more filter layers; the filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polariton (SPP) micro-nano structures, tunable Fabry-Perot resonators.

[0036] Further, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0037] In a second aspect, an embodiment of the present invention further provides an intelligent agricultural control device, including the optical artificial neural network intelligent agricultural precise control chip as described in the first aspect.

[0038] In a third aspect, an embodiment of the present invention further provides a method for preparing the optical artificial neural network intelligent agricultural precise control chip as described in the first aspect, including:

[0039] Preparing a light modulation layer containing a light modulation structure on the surface of the image sensor;

[0040] Generating a processor with the function of fully connecting and processing signals or generating a processor with the functions of fully connecting and processing signals and performing a second non-linear activation processing on the signals;

[0041] Connecting the image sensor and the processor;

[0042] Wherein, the light modulation layer is used to respectively perform different spectral modulations on the incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor;

[0043] The image sensor converts the information carried by the incident light corresponding to different position points after being modulated by the light modulation layer into electrical signals corresponding to different position points through square-law detection response, and sends the electrical signals corresponding to different position points to the processor;

[0044] The processor performs a full connection process on the electrical signals corresponding to different position points, or the processor performs a full connection process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain an agricultural precise control processing result.

[0045] Further, the method for preparing the optical artificial neural network intelligent agricultural precise control chip further includes: a training process for the optical artificial neural network intelligent agricultural precise control chip, specifically including:

[0046] Using the input training samples and output training samples corresponding to the agricultural precision control intelligent processing task, train an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor;

[0047] Alternatively, using the input training samples and output training samples corresponding to the agricultural precision control intelligent processing task, train an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0048] Further, prepare an optical modulation layer including an optical modulation structure on the surface of the image sensor, including:

[0049] Grow one or more layers of a preset material on the surface of the image sensor;

[0050] Etch the optical modulation structure pattern on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure;

[0051] Or perform imprint transfer on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure;

[0052] Or obtain an optical modulation layer including an optical modulation structure by applying external dynamic modulation to the one or more layers of the preset material;

[0053] Or perform zone printing on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure;

[0054] Or perform zone growth on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure;

[0055] Or perform quantum dot transfer on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure.

[0056] The optical artificial neural network intelligent agriculture precise control chip and its preparation method provided by the embodiments of the present invention realize a brand-new optical artificial neural network intelligent agriculture precise control chip capable of realizing the functions of an artificial neural network, which is used for intelligent processing tasks of agricultural precise control. In this optical artificial neural network intelligent agriculture precise control chip, the optical modulation layer corresponds to the input layer and the linear layer of the artificial neural network, and the image sensor corresponds to a part of the non-linear layer of the artificial neural network; the processor corresponds to another part of the non-linear layer of the artificial neural network and the output layer. Specifically, the optical modulation layer is arranged on the surface of the image sensor, and the optical modulation layer includes an optical modulation structure, which is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor. In the embodiments of the present invention, the modulation effect of the optical modulation structure on the incident light on the optical modulation layer is equivalent to the connection weight from the input layer to the linear layer.Meanwhile, in the embodiments of the present invention, the image sensor converts the information carried by the incident light corresponding to different position points after being modulated by the optical modulation layer into electrical signals corresponding to different position points through square-law detection response for the first non-linear activation process, and sends the electrical signals corresponding to different position points to the processor. The processor performs a fully-connected process on the electrical signals corresponding to different position points, or the processor performs a fully-connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in this optical artificial neural network intelligent agriculture precision control chip, the optical modulation layer corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network. The processor corresponds to the fully-connected and output layers of the artificial neural network, or the processor corresponds to the fully-connected, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. That is, the optical modulation layer and the image sensor in this optical artificial neural network intelligent agriculture precision control chip implement the related functions of the input layer, the linear layer, and part of the non-linear activation function in the artificial neural network. That is, the embodiments of the present invention strip the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network implemented by software in the prior art, and use hardware to implement the structures of the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network. As a result, when using this optical artificial neural network intelligent agriculture precision control chip for artificial neural network intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part or all of the non-linear activation functions. It only needs the processor in the optical artificial neural network intelligent agriculture precision control chip to perform a fully-connected process on the electrical signals or a fully-connected and second non-linear activation process. This can greatly reduce the power consumption and delay during artificial neural network processing.

[0057] It can be seen that the embodiments of the present invention provide a novel optoelectronic chip for precision agriculture control. The chip consists of an optical modulation layer and an image sensor, which constitute the input layer and the linear layer of the optical artificial neural network, and collect the information carried by the incident light at different points in the farmland soil and crop space to achieve rapid, accurate, safe, and reliable identification and qualitative analysis of soil fertility, pesticide spraying situation, trace element content, and crop growth status, etc. Description of the Drawings

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0059] Figure 1 It is a schematic structural diagram of an optical artificial neural network intelligent agriculture precision control chip provided by the first embodiment of the present invention;

[0060] Figure 2 It is a schematic diagram of the recognition principle of an optical artificial neural network intelligent agriculture precision control chip provided by an embodiment of the present invention;

[0061] Figure 3 It is a schematic disassembly diagram of an optical artificial neural network intelligent agriculture precision control chip provided by an embodiment of the present invention;

[0062] Figure 4a It is a schematic diagram of the process of recognizing agricultural objects provided by an embodiment of the present invention;

[0063] Figure 4b It is a three-dimensional schematic diagram of recognizing or qualitatively analyzing crops and / or soil provided by an embodiment of the present invention;

[0064] Figure 5 It is a top view of an optical modulation layer provided by an embodiment of the present invention;

[0065] Figure 6 It is a top view of another optical modulation layer provided by an embodiment of the present invention;

[0066] Figure 7 It is a top view of yet another optical modulation layer provided by an embodiment of the present invention;

[0067] Figure 8 It is a top view of still another optical modulation layer provided by an embodiment of the present invention;

[0068] Figure 9 It is a top view of yet still another optical modulation layer provided by an embodiment of the present invention;

[0069] Figure 10 It is a top view of still yet another optical modulation layer provided by an embodiment of the present invention;

[0070] Figure 11 It is a schematic diagram of the broadband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0071] Figure 12 It is a schematic diagram of the narrowband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0072] Figure 13 It is a schematic structural diagram of a front-illuminated image sensor provided by an embodiment of the present invention;

[0073] Figure 14 It is a schematic structural diagram of a back-illuminated image sensor provided by an embodiment of the present invention;

[0074] Figure 15 It is a schematic flowchart of a preparation method of an optical artificial neural network intelligent agriculture precision control chip provided by the third embodiment of the present invention. Detailed implementation manners

[0075] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0076] Precision agriculture is a system that is supported by information technology and implements a set of modern farming operation technologies and management in a location-based, timed, and quantitative manner according to spatial variations. Its basic meaning is to adjust the input for crops based on the soil properties of crop growth, judge the spatial variations of soil properties and productivity within the farmland. On the other hand, determine the production goals of crops and conduct location-based "system diagnosis, optimized formulation, technology assembly, and scientific management" to mobilize soil productivity, achieve the same or higher income with the least or most economical input, improve the environment, and efficiently utilize various agricultural resources to obtain economic and environmental benefits. Currently, when performing precision agriculture control, there is a problem of slow processing speed. For example, for the identification of soil fertility, steps such as image acquisition, image preprocessing, feature extraction, and feature matching are required to achieve the identification of soil fertility. Currently, intelligent recognition tasks of agricultural objects generally rely on neural network recognition models, that is, for current intelligent recognition tasks of agricultural objects, it is necessary to first image the agricultural objects and then transmit them to a computer for subsequent neural network recognition model algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay. In addition, when performing precision agriculture control currently, there is also a problem of inaccurate recognition. For example, it is difficult to ensure the accuracy of recognition only by using the two-dimensional image information of the soil. For example, there are distortion factors in the two-dimensional image, and during the imaging process, optical information needs to be converted into digital electronic signals and then transmitted to a computer for subsequent algorithm processing. The transmission and processing of a large amount of data cause relatively large power consumption and delay. In addition, there are similar problems for the recognition of crop growth conditions. Based on this, the embodiments of the present invention provide an optical artificial neural network intelligent agricultural precision control chip. The optical modulation layer in the optical artificial neural network intelligent agricultural precision control chip corresponds to the input layer, linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor corresponds to the fully connected layer and output layer of the artificial neural network, or, the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. The embodiments of the present invention use the optical modulation layer and the image sensor to project the information carried by the incident light of the agricultural object into an electrical signal, and then perform fully connected processing or fully connected processing and second non-linear activation processing on the electrical signal in the processor. It can be seen that the embodiments of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and part or all of the non-linear activation functions in the prior art, thereby greatly reducing the power consumption and delay during the processing of the artificial neural network.

[0077] In addition, embodiments of the present invention can also utilize one or more of the image information, spectral information, incident light angle, and incident light phase information of agricultural objects, that is, the information carried by the incident light at different points in the space of agricultural objects. It can be seen that since the information carried by the incident light at different points in the space of agricultural objects covers information such as the image, composition, shape, three-dimensional depth, and structure of agricultural objects, when performing recognition processing based on the information carried by the incident light at different points in the space of agricultural objects, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of agricultural objects, thereby improving the accuracy of intelligent processing (such as intelligent recognition). It can be seen that the optical artificial neural network intelligent agricultural precision control chip provided by the embodiments of the present invention can not only achieve the effects of low power consumption and low latency, but also improve the accuracy rate of intelligent processing, thereby enabling fast, accurate, safe, and reliable identification and qualitative analysis of soil fertility, pesticide spraying conditions, trace element content, and crop growth conditions. Therefore, it can be preferably applied in the field of precision agricultural control. The content provided by the present invention will be explained and described in detail through specific embodiments below.

[0078] As Figure 1 shown, the optical artificial neural network intelligent agricultural precision control chip provided by the first embodiment of the present invention is used for intelligent processing tasks of agricultural precision control, and includes: an optical modulation layer 1, an image sensor 2, and a processor 3; the optical modulation layer 1 corresponds to the input layer, linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer, and the square-law detection response of the image sensor 2 corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor 3 corresponds to the fully connected layer and output layer of the artificial neural network, or the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network;

[0079] The optical modulation layer 1 is disposed on the surface of the image sensor or the surface of the photosensitive area of the image sensor. The optical modulation layer 1 includes an optical modulation structure. The optical modulation layer 1 is used to perform spectral modulation on the incident light entering different position points of the optical modulation structure by modulating the intensity with wavelength, that is, performing different intensity modulations on incident light of different wavelengths, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor;

[0080] In this embodiment, the square-law detection response of the image sensor 2 means that the image sensor detects the intensity information of the incident light field, and the intensity information of the incident light field is the square of the modulus of the optical field signal. That is, the image sensor 2 performs the first non-linear activation process on the information carried by the incident light corresponding to different position points after being modulated by the optical modulation layer 1 through the square-law detection response and converts it into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3; the electrical signals are image signals modulated by the optical modulation layer; wherein, the incident light includes the reflected light, transmitted light, and / or radiation light of the agricultural object; the agricultural object includes crops and / or soil;

[0081] The processor 3 is configured to perform a fully-connected process on the electrical signals corresponding to different position points, or the processor performs a fully-connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network, and further obtain the agricultural precision control processing result.

[0082] In this embodiment, the agricultural precision control intelligent processing tasks include one or more of soil fertility detection, pesticide spraying detection, trace element content detection, drug resistance detection, and crop growth condition detection; the agricultural precision control processing results include one or more of soil fertility detection results, pesticide spraying detection results, trace element content detection results, drug resistance detection results, and crop growth condition detection results.

[0083] The information carried by the incident light includes the image information and / or various optical space information of the target object to be processed by the optical artificial neural network intelligent chip. For example, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light. In this embodiment, the optical modulation layer 1 is disposed on the surface of the image sensor, the optical modulation layer 1 includes an optical modulation structure, and the optical modulation layer 1 is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively to obtain the modulated incident light carrying information corresponding to different position points on the surface of the image sensor. Thus, it can be seen that in the embodiment, the modulation effect of the optical modulation structure on the incident light on the optical modulation layer can be regarded as the connection weight from the input layer to the linear layer;

[0084] In this embodiment, when the image sensor 2 performs photoelectric conversion on the information carried by the modulated incident light, since the image sensor 2 can only detect the intensity information of the light, the electrical signal obtained by processing the light field distribution signal is proportional to the square of the modulus of the light field distribution signal. Therefore, the image sensor 2 has a square-law detection response, and thus the image sensor 2 can be regarded as part of the non-linear layer of the artificial neural network. That is, the square-law detection response of the image sensor 2 can be regarded as the first non-linear activation function of the artificial neural network.

[0085] In this embodiment, the image sensor 2 performs the first non-linear activation process on the information carried by the incident light corresponding to different position points after being modulated by the optical modulation layer 1 through the square-law detection response and converts it into an electrical signal corresponding to different position points, that is, the image signal after being modulated by the optical modulation layer. At the same time, the processor 3 connected to the image sensor 2 is used to perform a fully connected process or a fully connected and second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0086] In this embodiment, the optical modulation layer 1 includes an optical modulation structure, and the incident light entering different position points of the optical modulation structure (such as the reflected light, transmitted light, radiation light, and other related acting lights of the target to be recognized) is subjected to spectral modulation of different intensities, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor 2.

[0087] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the optical modulation structure. For example, different modulation intensities can be achieved by designing different optical modulation structures (such as changing the shape and / or size parameters of the optical modulation structure).

[0088] In this embodiment, it can be understood that the optical modulation structures at different positions on the optical modulation layer 1 have different spectral modulation effects on the incident light. The modulation intensity of the optical modulation structure on different wavelength components of the incident light corresponds to the connection strength of the linear layer of the artificial neural network, that is, the connection weights corresponding to the input layer and the connection from the input layer to the linear layer. It should be noted that the optical modulation layer 1 is composed of multiple optical filter units, and the optical modulation structures at different positions within each optical filter unit are different, so they have different spectral modulation effects on the incident light; the optical modulation structures at different positions between the optical filter units can be the same or different, so they have the same or different spectral modulation effects on the incident light.

[0089] In this embodiment, the image sensor 2 performs a first non-linear activation process on the information carried by the incident light corresponding to different position points after being modulated by the optical modulation layer 1 through square-law detection response, and converts it into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to a part of the non-linear layer of the neural network.

[0090] In this embodiment, the processor 3 performs a fully connected process on the electrical signals of different position points, or the processor 3 performs a fully connected process on the electrical signals of different position points and a second non-linear activation process, so as to obtain the output signal of the artificial neural network.

[0091] It can be understood that in this embodiment, the image sensor 2 corresponds to a part of the non-linear layer of the neural network, and the processor 3 corresponds to another part of the non-linear layer of the neural network and the output layer. It can also be understood as corresponding to the remaining layers (all other layers) in the neural network except the input layer, the linear layer, and the first non-linear activation function in the non-linear layer.

[0092] In this embodiment, it should be noted that the square-law detection response of the image sensor 2 corresponds to the first non-linear activation function in the non-linear layer of the neural network. In this case, the processor may only perform a fully connected process without performing a second non-linear activation process, or the processor may perform both a fully connected process and a second non-linear activation process. It can be specifically determined according to the actual application scenario of the chip, and this embodiment does not limit this.

[0093] In addition, it should be supplemented and explained that the processor 3 may be set inside the optical artificial neural network intelligent agriculture precision control chip, that is, the processor 3 may be set together with the filter layer 1 and the image sensor 2 inside the optical artificial neural network intelligent agriculture precision control chip, or may be set separately outside the optical artificial neural network intelligent agriculture precision control chip and connected to the image sensor 2 inside the optical artificial neural network intelligent agriculture precision control chip through a data line or a connecting device. This embodiment does not limit this.

[0094] In addition, it should be noted that the processor 3 can be implemented by a computer, or by an ARM or FPGA circuit board with certain computing capabilities, or by a microprocessor. This embodiment does not make any limitations in this regard. In addition, as mentioned above, the processor 3 can be integrated within the optical artificial neural network intelligent agriculture precise control chip, or can be arranged independently outside the optical artificial neural network intelligent agriculture precise control chip. When the processor 3 is arranged independently outside the optical artificial neural network intelligent agriculture precise control chip, the electrical signals in the image sensor 2 can be read out to the processor 3 through a signal reading circuit, and then the processor 3 performs a fully connected process and a non-linear activation process on the read electrical signals.

[0095] In this embodiment, it can be understood that when the processor 3 performs the second non-linear activation process, it can be implemented by using a non-linear activation function. For example, it can use a Sigmoid function, a Tanh function, a ReLU function, etc. This embodiment does not make any limitations in this regard.

[0096] In this embodiment, the optical modulation layer 1 corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The image sensor 2 corresponds to a part of the non-linear layer of the artificial neural network. That is, the square-law detection response of the image sensor 2 corresponds to the first non-linear activation function of the artificial neural network. The image sensor 2 is used to perform a non-linear activation process on the information carried by the incident light at different spatial positions through the square-law detection response and then convert it into electrical signals. The processor 3 corresponds to the remaining layers of the artificial neural network, fully connects the electrical signals at different positions, and can further obtain the output signal of the artificial neural network through the second non-linear activation function, realizing the intelligent perception, recognition, and / or decision-making of specific targets.

[0097] As Figure 2 shown on the left, the optical artificial neural network intelligent agriculture precise control chip includes an optical modulation layer 1, an image sensor 2, and a processor 3. In Figure 2 it, the processor 3 is implemented by using a signal reading circuit and a computer. As Figure 2As shown on the right side, the optical modulation layer 1 in the optical artificial neural network intelligent agriculture precise control chip corresponds to the input layer and the linear layer of the artificial neural network. The image sensor 2 corresponds to a part of the non-linear layer of the artificial neural network, and the processor 3 corresponds to another part of the non-linear layer and the output layer of the artificial neural network. The filtering effect of the optical modulation layer 1 on the incident light entering the optical modulation layer 1 corresponds to the connection weights from the input layer to the linear layer. The square-law detection response of the image sensor 2 corresponds to the first non-linear activation function of the artificial neural network. It can be seen that the optical modulation layer and the image sensor in the optical artificial neural network intelligent agriculture precise control chip provided in this embodiment implement the related functions of the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network through hardware. As a result, when using this optical artificial neural network intelligent agriculture precise control chip for intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer (for example, omitting calculations such as the connection weights from the input layer to the linear layer). This can significantly reduce the power consumption and delay during the processing of the artificial neural network. In addition, since this embodiment simultaneously utilizes the image information, spectral information, incident light angle information, and incident light phase information at different points in the space of the agricultural object, it can more accurately achieve the intelligent processing of the agricultural object.

[0098] As Figure 2 shown on the right side, the optical modulation layer 1 has different broadband spectral modulation effects on the incident light, and projects / connects the incident light spectrum P λ at the corresponding unit position to the outgoing optical field E N ; the square-law detection response of the image sensor 2 corresponds to a part of the non-linear activation function of the optical artificial neural network, and converts the outgoing optical field E N of the optical modulation layer 1 into the photocurrent response I N of the image sensor. The processor 3 includes a signal readout circuit and a computer. The signal readout circuit in the processor 3 reads out the photocurrent response I N and transmits it to the computer, and the computer performs full-connection processing of the electrical signal or performs non-linear activation processing again, and finally outputs the result.

[0099] As Figure 3As shown, the optical modulation structure on the optical modulation layer 1 is integrated on the surface of the image sensor 2 to modulate the incident light, project / connect the spectral information of the incident light to different pixel points of the image sensor 2, and obtain an electrical signal containing the spectral information and image information of the incident light. That is, after the incident light passes through the optical modulation layer 1, it is non-linearly activated by the square-law detection response of the image sensor 2 and then converted into an electrical signal, forming image information, spectral information, angle information of the incident light, and phase information of the incident light at different points in space. Finally, the processor 3 connected to the image sensor 2 processes the electrical signal containing the spectral information and image information of the incident light to obtain an output result.

[0100] In this embodiment, the information carried by the incident light may include one or more (including two) of the light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0101] For example, in one implementation, the information carried by the incident light may include the light intensity distribution information. In other implementations, multiple types of information such as the image information, spectral information, angle of the incident light, and phase information of the incident light of the target object can be used simultaneously to identify the target object, so that the intelligent identification of the target object can be achieved more accurately.

[0102] It can be seen that the optical artificial neural network intelligent agricultural precise control chip provided in this embodiment can simultaneously utilize one or more of the image information, spectral information, angle of the incident light, and phase information of the agricultural object, that is, the information carried by the incident light at different points in space, and embeds an artificial neural network in the hardware. Information such as material composition, image shape, and three-dimensional depth can be further extracted from the spatial image, spectrum, angle, and phase information, thereby solving the problem that it is difficult to ensure the accuracy of identification by using the two-dimensional image information of the agricultural object mentioned in the background art. At the same time, the embodiment of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and part of the non-linear activation function in the prior art, so as to achieve low power consumption and low latency. It can be seen that the optical artificial neural network intelligent agricultural precise control chip provided in the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency, and high recognition rate, so as to quickly and accurately identify the agricultural production status (such as soil fertility, pesticide spraying situation, soil trace element content, crop growth status, etc.), thereby preparing for the precise control of intelligent agriculture.

[0103] The optical artificial neural network intelligent agriculture precise control chip and preparation method provided by the embodiments of the present invention realize a brand-new optical artificial neural network intelligent agriculture precise control chip capable of realizing the functions of an artificial neural network, which is used for intelligent processing tasks of agricultural precise control. In this optical artificial neural network intelligent agriculture precise control chip, the optical modulation layer corresponds to the input layer and the linear layer of the artificial neural network, and the image sensor corresponds to a part of the non-linear layer of the artificial neural network; the processor corresponds to the other part of the non-linear layer of the artificial neural network and the output layer. Specifically, the optical modulation layer is disposed on the surface of the image sensor, and the optical modulation layer includes an optical modulation structure, which is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor. In the embodiments of the present invention, the modulation effect of the optical modulation structure on the incident light on the optical modulation layer is equivalent to the connection weight from the input layer to the linear layer.Meanwhile, in the embodiments of the present invention, the image sensor performs a first non-linear activation process on the information carried by the incident light corresponding to different position points after being modulated by the optical modulation layer through square-law detection response, and converts it into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor. The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. Thus, in this optical artificial neural network intelligent agricultural precision control chip, the optical modulation layer corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor corresponds to the fully connected and output layers of the artificial neural network, or the processor corresponds to the fully connected, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. That is, the optical modulation layer and the image sensor in this optical artificial neural network intelligent agricultural precision control chip implement the related functions of the input layer, the linear layer, and part of the non-linear activation function in the artificial neural network. That is, the embodiments of the present invention strip the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network implemented by software in the prior art, and use hardware to implement the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network. As a result, when using this optical artificial neural network intelligent agricultural precision control chip for artificial neural network intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part or all of the non-linear activation functions. It only needs the processor in the optical artificial neural network intelligent agricultural precision control chip to perform a fully connected process on the electrical signals or a fully connected and second non-linear activation process. This can greatly reduce the power consumption and delay during artificial neural network processing.As can be seen, in the embodiment of the present invention, the optical modulation layer serves as the input layer, linear layer of the artificial neural network, and the connection weight from the input layer to the linear layer, and the square detection response of the image sensor serves as the first non-linear activation function in the non-linear layer of the artificial neural network; the processor serves as the fully connected layer and output layer of the artificial neural network, or the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. As can be seen, the embodiment of the present invention can not only eliminate the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and part of the non-linear activation function in the prior art, but also actually utilize the image information, spectral information, incident light angle, and incident light phase information of the agricultural object at the same time, that is, the incident light at different points in space of the agricultural object carries information. As can be seen, since the incident light at different points in space of the agricultural object carries information covering the image, composition, shape, three-dimensional depth, structure, etc. of the agricultural object, when performing identification processing based on the incident light carrying information at different points in space of the agricultural object, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, structure, etc. of the agricultural object, thereby solving the problem that it is difficult to ensure the accuracy of identification by using the two-dimensional image information of the agricultural object mentioned in the background art. As can be seen, the optical artificial neural network intelligent agriculture precise control chip provided by the embodiment of the present invention can simultaneously achieve the effects of low power consumption, low latency, and high recognition rate, so as to quickly and accurately identify the agricultural production status (such as soil fertility, pesticide spraying situation, soil trace element content, crop growth status, etc.), thus preparing for the precise control of intelligent agriculture.

[0104] As can be seen, the embodiment of the present invention provides a new type of optoelectronic chip for precise agricultural control. The chip consists of an optical modulation layer and an image sensor to form the input layer and linear layer of the optical artificial neural network, and simultaneously collects the image information, spectral information, incident light angle information, and incident light phase information at different points in space of the farmland soil and crops, so as to realize the rapid, accurate, safe, and reliable identification and qualitative analysis of soil fertility, pesticide spraying situation, trace element content, and crop growth status, etc.

[0105] Based on the content of the above embodiment, in this embodiment, the optical artificial neural network intelligent agriculture precise control chip includes a trained optical modulation structure, an image sensor, and a processor;

[0106] The trained optical modulation structure, image sensor, and processor refer to the optical modulation structure, image sensor, and processor that satisfy the training convergence condition obtained by training an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters using input training samples and output training samples corresponding to the agricultural precision control intelligent processing task; or, the trained optical modulation structure, image sensor, and processor refer to the optical modulation structure, image sensor, and processor that satisfy the training convergence condition obtained by training an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters using input training samples and output training samples corresponding to the agricultural precision control intelligent processing task.

[0107] Wherein, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different fertilities; the output training samples include the corresponding soil fertilities; and / or, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different pesticide spraying conditions; the output training samples include the corresponding pesticide spraying conditions; and / or, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different trace element content conditions; the output training samples include the corresponding trace element content conditions; and / or, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different drug resistances; the output training samples include the corresponding drug resistances; and / or, the input training samples include incident light reflected, transmitted, and / or radiated by crops with different growth conditions; the output training samples include the corresponding crop growth conditions.

[0108] In this embodiment, it can be understood that the input training samples include incident light reflected, transmitted, and / or radiated by agricultural objects in the corresponding intelligent processing task; the output training samples include the intelligent processing results of agricultural objects (such as recognition results, perception results, decision results, or qualitative analysis results, etc.).

[0109] In this embodiment, the reflected light, transmitted light, and / or radiated light of the agricultural object enter the trained optical artificial neural network intelligent agricultural precision control chip to obtain the intelligent processing result of the agricultural object.

[0110] In this embodiment, taking the recognition task of agricultural objects as an example for illustration, it can be understood that when using the optical artificial neural network intelligent agricultural precise control chip for the recognition task, first, the optical artificial neural network intelligent agricultural precise control chip needs to be trained. Here, training the optical artificial neural network intelligent agricultural precise control chip means determining the optical modulation structure applicable to the current recognition task, as well as the fully connected parameters and non-linear activation parameters applicable to the current recognition task through training.

[0111] It can be understood that since the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer corresponds to the connection weights from the input layer to the linear layer of the artificial neural network, therefore, during training, changing the optical modulation structure in the optical modulation layer is equivalent to changing the connection weights from the input layer to the linear layer of the artificial neural network. Through the training convergence condition, the optical modulation structure applicable to the current recognition task, as well as the fully connected parameters and non-linear activation parameters applicable to the current recognition task, are determined, thereby completing the training of the optical artificial neural network intelligent agricultural precise control chip.

[0112] It can be understood that after training the optical artificial neural network intelligent agricultural precise control chip, the optical artificial neural network intelligent agricultural precise control chip can be used to execute the recognition task. Specifically, the incident light carrying the image information, spectral information, incident light angle information, and incident light phase information at different points in the space of the agricultural object enters the optical modulation layer 1 of the trained optical artificial neural network intelligent agricultural precise control chip. The optical modulation structure in the optical modulation layer 1 will modulate the incident light. The intensity of the modulated optical signal is detected by the image sensor 2 and converted into an electrical signal, and then the processor 3 performs a fully connected process or simultaneously performs a fully connected and second non-linear activation process to obtain the recognition result of the agricultural object.

[0113] As Figure 4a shown, the complete process for agricultural object recognition is as follows: The broadband light source 100 irradiates the agricultural object 200, and then the reflected light or transmitted light of the agricultural object is collected by the optical artificial neural network intelligent agricultural precise control chip 300, or the light directly radiated by the agricultural object is collected by the optical artificial neural network intelligent agricultural precise control chip 300. After being processed by the optical modulation layer, image sensor, and processor in the optical artificial neural network intelligent agricultural precise control chip, the recognition result can be obtained.

[0114] Among them, the trained optical artificial neural network intelligent agriculture precise control chip refers to the optical artificial neural network intelligent agriculture precise control chip including a trained optical modulation structure, an image sensor, and a processor; the trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network intelligent agriculture precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters, which is trained using the input training samples and output training samples corresponding to the intelligent processing task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions.

[0115] In this embodiment, since the intelligent processing task of agricultural precise control includes one or more of soil fertility detection, pesticide spraying detection, trace element content detection, drug resistance detection, and crop growth condition detection, the corresponding input training samples and output training samples are also different.

[0116] For example, when the intelligent processing task of agricultural precise control includes soil fertility detection, the corresponding input training samples include the incident light reflected, transmitted, and / or radiated by soils with different fertilities, and the corresponding output training samples include the corresponding soil fertilities.

[0117] When the intelligent processing task of agricultural precise control includes pesticide spraying detection, the corresponding input training samples include the incident light reflected, transmitted, and / or radiated by soils with different pesticide spraying conditions, and the corresponding output training samples include the corresponding pesticide spraying conditions.

[0118] When the intelligent processing task of agricultural precise control includes trace element content detection, the corresponding input training samples include the incident light reflected, transmitted, and / or radiated by soils with different trace element content conditions, and the corresponding output training samples include the corresponding trace element content conditions.

[0119] When the intelligent processing task of agricultural precise control includes drug resistance detection, the corresponding input training samples include the incident light reflected, transmitted, and / or radiated by soils with different drug resistances, and the corresponding output training samples include the corresponding drug resistances.

[0120] When the intelligent processing task of agricultural precise control includes crop growth condition detection, the corresponding input training samples include the incident light reflected, transmitted, and / or radiated by crops with different growth conditions, and the corresponding output training samples include the corresponding crop growth conditions.

[0121] It can be understood that a large number of soil samples and crop samples in different states and positions can be collected in advance and data training can be carried out, and then the required optical modulation structure (i.e., the micro-nano modulation structure in the optical modulation layer) can be designed and prepared, so as to implement the input layer, linear layer and the first non-linear activation function in the non-linear layer of the artificial neural network in a hardware manner on the chip.

[0122] It can be understood that for the recognition task, since the advantage of the optical artificial neural network intelligent agriculture precise control chip provided by this embodiment also lies in being able to obtain the image information, spectral information, incident light angle information and incident light phase information at different points in the space of the recognition object, therefore, in order to make full use of this advantage, the real recognition object is preferentially used as the recognition object sample for the input training sample, rather than the two-dimensional image of the recognition object. Of course, this does not mean that the two-dimensional image cannot be used as the recognition object sample.

[0123] In addition, the optical artificial neural network intelligent agriculture precise control chip provided by this embodiment can also be used for other intelligent processing tasks of agricultural objects, such as intelligent agriculture perception, intelligent agriculture decision-making and other tasks.

[0124] In this embodiment, the optical modulation layer 1 is used as the input layer and linear layer of the neural network, and the image sensor 2 is used as a part of the non-linear layer of the neural network (i.e., the square-law detection response of the image sensor 2 is used as the first non-linear activation function of the neural network). In order to minimize the loss function of the neural network, the modulation intensity of different wavelength components in the incident light of the agricultural object by the optical modulation structure in the optical modulation layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the optical modulation layer, the modulation intensity of different wavelength components in the incident light of the agricultural object can be adjusted, so as to realize the adjustment of the connection weight from the input layer to the linear layer, and further optimize the training of the neural network.

[0125] Therefore, the optical modulation structure in this embodiment is obtained based on neural network training. Through computer optical simulation of the training samples, the sample modulation intensity of different wavelength components of the incident light of the agricultural object by the optical modulation structure in the training samples is obtained. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network, and non-linear activation is carried out. And the neural network is trained using the training samples corresponding to the intelligent processing tasks until the neural network converges, and the corresponding training sample optical modulation structure is used as the optical modulation layer for the corresponding intelligent processing tasks.

[0126] It can be seen that in this embodiment, by implementing the input layer and the linear layer (optical modulation layer) of the neural network and a part of the non-linear layer (the square-law detection response of the image sensor 2 as the first non-linear activation function of the neural network) at the physical layer, it is possible to eliminate the complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part or all of the non-linear activation functions in the prior art, thereby improving the processing speed and reducing the time delay. At the same time, the embodiment of the present invention actually utilizes the image information, spectral information, incident light angle information, and incident light phase information at different points in the space of the agricultural object, that is, the spectral information at different points in the space of the agricultural object, so that information such as the composition, shape, and three-dimensional depth of the agricultural object can be further extracted from the image information, spectral information, incident light angle information, and incident light phase information at different points in the space, thereby solving the problem that it is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the agricultural object mentioned in the background art part.

[0127] Based on the content of the above embodiment, in this embodiment, when training an optical artificial neural network intelligent agriculture precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network intelligent agriculture precision control chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0128] In this embodiment, the optical modulation structure is designed through computer optical simulation, and the optical modulation structure is adjusted through optical simulation until the corresponding optical modulation structure is determined as the size of the optical modulation structure to be finally fabricated when the neural network converges, saving the prototype production time and cost, improving the product efficiency, and easily solving complex optical problems. For example, the optical modulation structure can be simulated and designed by using FDTD software, and the optical modulation structure is changed in the optical simulation, so that the modulation intensity of the optical modulation structure for different incident lights can be accurately predicted and used as the connection weight between the input layer and the linear layer of the neural network to train the optical artificial neural network intelligent agriculture precision control chip and accurately obtain the optical modulation structure.

[0129] It can be understood that a large number of soil samples and crop samples in different states and positions can be collected and data-trained in advance, and then the required optical modulation structure (i.e., the micro-nano modulation structure in the optical modulation layer) can be designed and prepared, so as to implement the input layer, the linear layer, and the first non-linear activation function in the non-linear layer of the artificial neural network in hardware on the chip.

[0130] It can be seen that in this embodiment, by adopting the method of computer optical simulation design for the optical modulation structure, the time and cost for fabricating the prototype of the optical modulation structure are saved, and the product efficiency is improved.

[0131] It can be understood that the technical principle of precision agriculture is to adjust the input to crops according to the spatial differences in soil fertility and crop growth conditions. Based on the quantitative real-time diagnosis of cultivated land and crop growth trends and a full understanding of the spatial variability of field productivity, with the goal of balancing soil fertility and increasing yields, precise field management of positioning and quantification is implemented to achieve the sustainable development goal of efficiently utilizing various agricultural resources and improving the environment. Implementing precision agriculture can not only maximize agricultural productivity but also achieve the goal of sustainable agricultural development with high quality, high yields, low consumption, and environmental protection. The characteristics of the soil and crops in the farmland are not uniform and change over time and space. However, in traditional and still widely used farmland management, it is considered that the above characteristics are uniform, and the same fertilization time and amount of fertilizer are used. However, excessive pesticides and fertilizers will flow into surface water and groundwater, causing environmental pollution. Therefore, it is of great significance to agricultural modernization to collect spectral data and images of farmland soil and crops and implement a precision agriculture control chip that can be integrated with drones, intelligent robots, etc., is small in size, low in cost, and can be used over a large area. The purpose of this embodiment is to provide a new optoelectronic chip for precision agriculture control. The chip consists of a light modulation layer and an image sensor, which form a part of the input layer, linear layer, and non-linear layer of the optical artificial neural network, specifically a non-linear activation function. At the same time, it collects the image information and spectral information of the farmland soil and crops to achieve qualitative analysis of soil fertility, pesticide spraying conditions, trace element content, and crop growth conditions quickly, accurately, safely, and reliably. The micro-nano modulation structure is directly prepared on the photosensitive area surface of the image sensor. Several discrete or continuous micro-nano structures form a unit. The micro-nano modulation structures at different positions have different spectral modulation effects on the incident light, jointly constituting the light modulation layer. The modulation intensity of different wavelength components of the incident light by these micro-nano modulation structures corresponds to the connection strength (linear layer weight) of the artificial neural network. At the same time, the square detection response of the image sensor performs the first non-linear activation process on the optical field distribution signals corresponding to different position points after being modulated by the light modulation layer and converts them into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor. The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. For precision agriculture control, a large number of soil samples and crop samples in different states and positions can be collected first. By training the data to obtain the weights of the linear layer, that is, the system function of the light modulation layer, the required light modulation layer can be reversely designed and integrated above the image sensor.In addition, during the application of the chip, the output of the fabricated optical modulation layer can be utilized to further train and optimize the weights of the fully connected layer of the electrical signals and the second non-linear activation function, enabling qualitative analysis of soil fertility, pesticide spraying conditions, trace element content, and crop growth status.

[0132] It can be understood that the chip actually simultaneously utilizes the image information, spectral information, incident light angle information, and incident light phase information at different points in the space of the soil and crops, improving the accuracy and diversity of precision agriculture control; and partially implements an artificial neural network in hardware, enhancing the analysis speed of precision agriculture control. Moreover, the chip can be integrated into drones and intelligent robot systems to accurately analyze the soil and crop conditions in large-scale farmland areas. In addition, the chip solution can be mass-produced using existing CMOS processes, reducing the volume, power consumption, and cost of the device.

[0133] The precision agriculture control chip based on the micro-nano modulation structure and image sensor provided in this embodiment has a structural schematic diagram as Figure 2 shown, including an optical modulation layer 1, an image sensor 2, and a processor 3. The optical modulation layer 1 corresponds to the input layer and the linear layer of the optical artificial neural network. Among them, several discrete or continuous micro-nano modulation structures form a unit, and these modulation structures have different broadband spectral modulation effects on the incident light. The micro-nano modulation structures included in different units can be the same or different, or can be spatially reconstructed according to the image. Each unit corresponds to multiple photosensitive pixels of the image sensor in the vertical direction. The square detection response of the image sensor 2 corresponds to a part of the non-linear activation function of the optical artificial neural network, converting the outgoing light field E N of the optical modulation layer 1 into the photocurrent response I N of the image sensor. The processor 3 includes a signal readout circuit and a computer. The signal readout circuit in the processor 3 reads out the photocurrent response I N and transmits it to the computer, where the computer performs full connection processing of the electrical signals or non-linear activation processing again, and finally outputs the result.

[0134] It can be understood that for precision agriculture control, by performing optical simulation of the micro-nano modulation structure on a computer, the modulation intensity (transmittance) of the modulation structure for different wavelength components of the incident light can be obtained, which is used as the connection weight from the input layer to the linear layer of the artificial neural network. At the same time, the square detection response of the image sensor is used to perform the first non-linear activation processing on the optical field distribution signal corresponding to different position points after being modulated by the optical modulation layer, and then convert it into an electrical signal corresponding to different position points, and send the electrical signal corresponding to different position points to the processor. The processor performs a fully connected processing on the electrical signal corresponding to different position points, or the processor performs a fully connected processing and a second non-linear activation processing on the electrical signal corresponding to different position points to obtain the output signal of the artificial neural network. By collecting and training data on a large number of soil samples and crop samples in different states and positions in advance, the required micro-nano modulation structure can be designed and prepared, and the input layer, linear layer, and the first non-linear activation function in the non-linear layer of the artificial neural network can be implemented on the chip.

[0135] Vertically, as Figure 2 shown, each micro-nano modulation structure in the optical modulation layer is designed through pre-training of the artificial neural network and can be prepared by directly growing one or more layers of dielectric or metal materials on the image sensor and then etching. The overall size of each modulation unit in the optical modulation layer is usually λ 2 ~10 5 λ 2 , and the thickness is usually 0.1λ - 10λ, where λ is the central wavelength of the target band. Each modulation unit structure in the optical modulation layer corresponds to multiple pixels on the image sensor. The optical modulation layer is directly prepared on the image sensor, and the image sensor and the processor are connected through electrical contacts.

[0136] In this embodiment, the complete process of analyzing samples by the precision agriculture control chip is as Figure 4b shown. The spectral data and images of farmland soil and crops are collected by the spectrum and image collector, and then the reflected light is collected by the precision agriculture control chip and processed by the internal algorithm of the processor to obtain the recognition result.

[0137] It can be understood that both the optical modulation layer and the CIS wafer (the CIS wafer is used as the image sensor) can be manufactured by semiconductor CMOS integration technology. The optical modulation layer is monolithically integrated on the image sensor directly at the wafer level, and the preparation of the chip can be completed by a single CMOS process flow. Thus, monolithic integration can be achieved at the wafer level, which is beneficial to reducing the distance between the sensor and the optical modulation layer, shrinking the volume of the device, and reducing the packaging cost.

[0138] It can be seen that in this embodiment, the optical modulation layer corresponds to the input layer and the linear layer of the artificial neural network, as well as the connection weights from the input layer to the linear layer. The square detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network. Furthermore, the spatial spectral information of soil samples and crop samples in different states and positions is projected into the photocurrent response of the image sensor, and the full connection of the electrical signal and the second non-linear activation are realized in the processor, thus achieving a fast and accurate agricultural control chip with low power consumption, large detection range, and high reliability.

[0139] The optical artificial neural network precision agricultural control chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A. Embed the artificial neural network part into the image sensor containing various optical modulation layers to achieve safe, reliable, fast, and accurate precision agricultural control. B. It is possible to introduce artificial neural network training and recognition for soil, crops, etc., which is convenient for subsequent integration into industrial intelligent control systems such as drones and intelligent robots to achieve large-area precision agricultural control, and the recognition accuracy is high and the qualitative analysis is precise. C. The preparation of this chip can be completed by a single CMOS process flow, which is beneficial to reducing the device failure rate, improving the finished product yield of the device, and reducing costs. D. Monolithic integration is achieved at the wafer level, which can minimize the distance between the sensor and the optical modulation layer to the greatest extent, is beneficial to reducing the size of the unit, and reducing the device volume and packaging cost.

[0140] Based on the content of the above embodiment, in this embodiment, the optical modulation structure in the optical modulation layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical modulation layer includes a discrete structure and / or a continuous structure.

[0141] In this embodiment, the optical modulation structure in the optical modulation layer can include only a regular structure, only an irregular structure, or both a regular structure and an irregular structure.

[0142] In this embodiment, the fact that the optical modulation structure includes a regular structure can mean: the smallest modulation unit included in the optical modulation structure is a regular structure, such as the smallest modulation unit can be regular shapes such as rectangles, squares, and circles. In addition, the fact that the optical modulation structure includes a regular structure can also mean: the arrangement method of the smallest modulation units included in the optical modulation structure is regular, such as the arrangement method can be a regular array form, circular form, trapezoidal form, polygonal form, etc. In addition, the fact that the optical modulation structure includes a regular structure can also mean: the smallest modulation unit included in the optical modulation structure is a regular structure, and at the same time, the arrangement method of the smallest modulation units is also regular, etc.

[0143] In this embodiment, the light modulation structure including an irregular structure may refer to: the smallest modulation unit included in the light modulation structure is an irregular structure, such as the smallest modulation unit may be an irregular polygon, a random shape, or other irregular graphics. In addition, the light modulation structure including an irregular structure may also refer to: the arrangement of the smallest modulation units included in the light modulation structure is irregular, such as the arrangement may be in the form of an irregular polygon, a random arrangement, etc. In addition, the light modulation structure including an irregular structure may also refer to: the smallest modulation unit included in the light modulation structure is an irregular structure, and at the same time, the arrangement of the smallest modulation units is also irregular, etc.

[0144] In this embodiment, the light modulation structure in the light modulation layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0145] In this embodiment, the light modulation structure including a continuous structure may mean that the light modulation structure is composed of continuous modulation patterns; the light modulation structure including a discrete structure may mean that the light modulation structure is composed of discrete modulation patterns.

[0146] It can be understood that the continuous modulation patterns here may refer to linear patterns, wavy line patterns, broken line patterns, and so on.

[0147] It can be understood that the discrete modulation patterns here may refer to modulation patterns formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0148] In this embodiment, it should be noted that the light modulation structure has different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after the light passes through different groups of filter structures.

[0149] Based on the content of the above embodiments, in this embodiment, the light modulation layer is a single-layer structure or a multi-layer structure.

[0150] In this embodiment, it should be noted that the light modulation layer may be a single-layer filter structure or a multi-layer filter structure, such as a multi-layer structure of two layers, three layers, four layers, etc.

[0151] In this embodiment, as Figure 1 shown, the light modulation layer 1 is a single-layer structure, and the thickness of the light modulation layer 1 is related to the target wavelength range. For wavelengths from 400 nm to 10 μm, the thickness of the grating structure can be from 50 nm to 5 μm.

[0152] It can be understood that since the function of the optical modulation layer 1 is to perform spectral modulation on the incident light, preferably, materials with high refractive index and low loss are selected for preparation. For example, silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, group III-V materials, etc. can be selected for preparation. Among them, silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, etc.

[0153] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical modulation layer 1 can be set as a multi-layer structure, and the corresponding optical modulation structures of each layer can be set as different structures, so as to increase the spectral modulation ability of the optical modulation layer for the incident light, so that more or more complex connection weights can be formed between the input layer and the linear layer, and further improve the accuracy of the optical artificial neural network intelligent agriculture precision control chip when processing intelligent tasks.

[0154] In addition, it should be noted that for the filter layer including a multi-layer structure, the materials of each layer structure can be the same or different. For example, for the two-layer optical modulation layer 1, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0155] It should be noted that the thickness of the optical modulation layer 1 is related to the target wavelength range. For wavelengths of 400 nm to 10 μm, the total thickness of the multi-layer structure can be 50 nm to 5 μm.

[0156] Based on the content of the above embodiments, in this embodiment, the optical modulation structure in the optical modulation layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0157] In this embodiment, in order to obtain connection weights (used to connect the input layer and the linear layer) distributed in an array for subsequent fully connected and non-linear activation processing by the processor, preferably, in this embodiment, the optical modulation structure is in the form of an array structure. Specifically, the optical modulation structure includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor. It should be noted that the structures of the respective micro-nano units can be the same or different. In addition, it should be noted that the structures of the respective micro-nano units can be periodic or non-periodic. In addition, it should be noted that each micro-nano unit can further include multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different, etc.

[0158] The following is combined with Figures 5 - 9 for example. In this embodiment, as Figure 5As shown, the optical modulation layer 1 includes a plurality of repeating continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (and each micro-nano unit is an aperiodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; such as Figure 6 As shown, the optical modulation layer 1 includes a plurality of repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (different from Figure 5 in that Figure 6 each micro-nano unit in is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; such as Figure 7 As shown, the optical modulation layer 1 includes a plurality of repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (and each micro-nano unit is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2, different from Figure 6 in that Figure 7 the unit shape of the periodic array in each micro-nano unit has four-fold rotational symmetry; such as Figure 8 As shown, the optical modulation layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66, different from Figure 6 in that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2. In this embodiment, the optical modulation layer 1 includes a plurality of different micro-nano units, that is, the modulation effects of different regions on the light incident on the optical artificial neural network intelligent agriculture precision control chip are different, thereby improving the design freedom, and further improving the recognition accuracy. Such as Figure 9 As shown, the optical modulation layer 1 includes a plurality of repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure, different from Figure 5 in that each micro-nano unit is composed of discrete aperiodic array structures, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0159] In this embodiment, the micro-nano units have different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after the light passes through different groups of filter structures.

[0160] Based on the content of the above embodiments, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0161] In this embodiment, the micro-nano unit may include only regular structures, only irregular structures, or both regular and irregular structures.

[0162] In this embodiment, the fact that the micro-nano unit includes regular structures may refer to: the smallest modulation unit included in the micro-nano unit is a regular structure, such as regular shapes like rectangles, squares, and circles. In addition, the fact that the micro-nano unit includes regular structures may also refer to: the arrangement pattern of the smallest modulation units included in the micro-nano unit is regular, such as regular array forms, circular forms, trapezoidal forms, polygonal forms, etc. In addition, the fact that the micro-nano unit includes regular structures may also refer to: the smallest modulation unit included in the micro-nano unit is a regular structure, and at the same time, the arrangement pattern of the smallest modulation units is also regular, etc.

[0163] In this embodiment, the fact that the micro-nano unit includes irregular structures may refer to: the smallest modulation unit included in the micro-nano unit is an irregular structure, such as irregular polygons, random shapes, and other irregular figures. In addition, the fact that the micro-nano unit includes irregular structures may also refer to: the arrangement pattern of the smallest modulation units included in the micro-nano unit is irregular, such as irregular polygonal forms, random arrangement forms, etc. In addition, the fact that the micro-nano unit includes irregular structures may also refer to: the smallest modulation unit included in the micro-nano unit is an irregular structure, and at the same time, the arrangement pattern of the smallest modulation units is also irregular, etc.

[0164] In this embodiment, the micro-nano units in the optical modulation layer may include discrete structures, continuous structures, or both discrete and continuous structures.

[0165] In this embodiment, the fact that the micro-nano unit includes continuous structures may refer to: the micro-nano unit is composed of continuous modulation patterns; the fact that the micro-nano unit includes discrete structures may refer to: the micro-nano unit is composed of discrete modulation patterns.

[0166] It can be understood that the continuous modulation patterns here may refer to linear patterns, wavy patterns, zigzag patterns, and so on.

[0167] It can be understood that the discrete modulation patterns here may refer to modulation patterns formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0168] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano units, the corresponding transmission spectra are different after light passes through different groups of micro-nano units.

[0169] Based on the content of the above embodiments, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0170] In this embodiment, as Figure 5 shown, the optical modulation layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 different micro-nano structure arrays 110, 111, 112, and 113, and the filtering unit 44 includes 4 different micro-nano structure arrays 440, 441, 442, and 443. As Figure 10 shown, the optical modulation layer 1 includes multiple micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0171] It should be noted that here only a micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of groups of micro-nano structure arrays can also be set according to needs.

[0172] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and the modulation effects of each group of filtering structures on the incident light are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after the light passes through different groups of micro-nano structure arrays.

[0173] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0174] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light of the agricultural object as the connection weight between the input layer and the linear layer of the neural network, broadband filtering and narrowband filtering are realized by adopting different micro-nano structure arrays. Therefore, in this embodiment, the micro-nano structure array obtains the modulation intensity of different wavelength components of the incident light of the agricultural object by performing broadband filtering or narrowband filtering on the incident light of the agricultural object. As Figure 11 and Figure 12 shown, each group of micro-nano structure arrays in the optical modulation layer has the function of broadband filtering or narrowband filtering.

[0175] It can be understood that for each group of micro-nano structure arrays, they can all have broadband filtering effects, or all have narrowband filtering effects, or some can have broadband filtering effects while some have narrowband filtering effects. In addition, the broadband filtering ranges and narrowband filtering ranges of each group of micro-nano structure arrays can be the same or different. For example, by designing the size parameters such as the period, duty cycle, radius, and side length of each group of micro-nano structures in the micro-nano unit, it can have narrowband filtering effects, that is, only light of one (or a few) wavelengths can pass through. Another example is that by designing the size parameters such as the period, duty cycle, radius, and side length of each group of micro-nano structures in the micro-nano unit, it can have broadband filtering effects, that is, it allows light of more wavelengths or all wavelengths to pass through.

[0176] It can be understood that in specific applications, the filtering states of each group of micro-nano structure arrays can be determined by means of broadband filtering, narrowband filtering, or their combination according to the application scenarios.

[0177] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0178] In this embodiment, each group of micro-nano structure arrays can all be periodic structure arrays, or all be aperiodic structure arrays, or some be periodic structure arrays while some be aperiodic structure arrays. Among them, periodic structure arrays are easy to perform optical simulation design, and aperiodic structure arrays can achieve more complex modulation effects.

[0179] In this embodiment, as Figure 5 shown, the optical modulation layer 1 includes a plurality of repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and the micro-nano structure arrays are aperiodic structures. Among them, the aperiodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in an aperiodic arrangement. As Figure 5 shown, the micro-nano unit 11 includes 4 different aperiodic structure arrays 110, 111, 112, and 113. The micro-nano unit 44 includes 4 different aperiodic structure arrays 440, 441, 442, and 443. The aperiodic structure micro-nano structure arrays are designed through neural network data training for intelligent processing tasks in the early stage and are usually structures with irregular shapes. As Figure 6 shown, the optical modulation layer 1 includes a plurality of repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and Figure 5The difference is that the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. For example, Figure 6 As shown, the micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442 and 443. The filter structure of the periodic structure is designed by neural network data training for intelligent processing tasks in the early stage, and is usually a structure with an irregular shape. For example, Figure 7 As shown, the optical modulation layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures of the micro-nano structure arrays are different from each other, and the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes on the filter structure are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. For example, Figure 7 As shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other. The micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442 and 443. The micro-nano structure array of the periodic structure is designed by neural network data training for intelligent processing tasks in the early stage, and is usually a structure with an irregular shape.

[0180] It should be noted that, Figures 5 - 9 Each micro-nano unit includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are formed by using four different shapes of modulation holes respectively. The four groups of micro-nano structure arrays have different modulation effects on the incident light. It should be noted that only the micro-nano unit including four groups of micro-nano structure arrays is taken as an example here, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of micro-nano structure arrays can also be set according to needs. In this embodiment, the four different shapes can be circular, cross-shaped, regular polygon and rectangular (not limited to this).

[0181] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on lights of different wavelengths, and the modulation effects of each group of micro-nano structure arrays on the input light are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after the light passes through different groups of micro-nano structure arrays.

[0182] Based on the content of the above embodiment, in this embodiment, one or more groups of the multi-group micro-nano structure arrays included in the micro-nano unit are empty structures.

[0183] The following will be described by way of example with reference to Figure 9 the example shown. In this embodiment, as Figure 9 shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures corresponding to the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiment, for any micro-nano unit, it includes one or more groups of empty structures, and the empty structures are used to directly transmit the incident light. It can be understood that when one or more groups of empty structures are included in the multiple groups of micro-nano structure arrays, a richer spectrum modulation effect can be formed, so as to meet the spectrum modulation requirements in specific scenarios (or meet the specific connection weight requirements between the input layer and the linear layer in specific scenarios).

[0184] As Figure 9 shown, each micro-nano unit includes one group of micro-nano structure arrays and three groups of empty structures. The micro-nano unit 11 includes 1 aperiodic structure array 111, the micro-nano unit 22 includes 1 aperiodic structure array 221, the micro-nano unit 33 includes 1 aperiodic structure array 331, the micro-nano unit 44 includes 1 aperiodic structure array 441, the micro-nano unit 55 includes 1 aperiodic structure array 551, and the micro-nano unit 66 includes 1 aperiodic structure array 661, where the micro-nano structure array is used to perform different modulations on the incident light. It should be noted that here only an example of including one group of micro-nano structure arrays and three groups of empty structures is given, which does not play a limiting role. In actual applications, micro-nano units including one group of micro-nano structure arrays and five groups of empty structures or other numbers of groups of micro-nano structure arrays can also be set as needed. In this embodiment, the micro-nano structure array can be made of modulation holes in the shape of a circle, a cross, a regular polygon, and a rectangle (not limited to this).

[0185] It should be noted that none of the multiple groups of micro-nano structure arrays included in the micro-nano unit may include empty structures, that is, the multiple groups of micro-nano structure arrays may be aperiodic structure arrays or periodic structure arrays.

[0186] Based on the content of the above embodiment, in this embodiment, the micro-nano unit has polarization-independent characteristics.

[0187] In this embodiment, since the micro-nano unit has polarization-independent characteristics, the optical modulation layer is insensitive to the polarization of the incident light, thus realizing an optical artificial neural network intelligent agriculture precise control chip that is insensitive to both the incident angle and polarization. The optical artificial neural network intelligent agriculture precise control chip provided by the embodiment of the present invention is insensitive to the incident angle and polarization characteristics of the incident light, that is, the measurement result will not be affected by the incident angle and polarization characteristics of the incident light, so as to ensure the stability of the spectral measurement performance, and further ensure the stability of intelligent processing, such as the stability of intelligent perception, intelligent recognition, intelligent decision-making, etc. It should be noted that the micro-nano unit can also have polarization-dependent characteristics.

[0188] Based on the content of the above embodiment, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0189] In this embodiment, it should be noted that four-fold rotational symmetry belongs to a specific case of polarization-independent characteristics. By designing the micro-nano unit into a structure with four-fold rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0190] The following Figure 7 is illustrated by the following example. In this embodiment, as Figure 7 shown, the optical modulation layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiment, the corresponding structure of each group of micro-nano structure arrays can be a structure with four-fold rotational symmetry, such as a circle, a cross, a regular polygon, a rectangle, etc., that is, after the structure rotates 90°, 180°, 270°, it coincides with the original structure, so that the structure has polarization-independent characteristics, enabling the same intelligent recognition effect to be obtained when different polarized lights are incident.

[0191] Based on the content of the above embodiment, in this embodiment, the optical modulation layer is composed of one or more filter layers;

[0192] The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polaritons (SPP) micro-nano structures, tunable Fabry-perot cavities (FP cavities).

[0193] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0194] Among them, the photonic crystal, as well as the combination of the metasurface and the random structure, can be compatible with the CMOS process and have a good modulation effect. Other materials can also be filled in the micropores of the micro-nano modulation structure for surface smoothing; quantum dots and perovskites can minimize the volume of a single modulation structure by utilizing the spectral modulation characteristics of the materials themselves; the SPP has a small volume and can achieve polarization-related optical modulation; liquid crystals can be dynamically regulated by voltage to improve the spatial resolution; the tunable Fabry-Perot resonator can be dynamically regulated to improve the spatial resolution.

[0195] Based on the content of the above embodiments, in this embodiment, the thickness of the optical modulation layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0196] In this embodiment, it should be noted that if the thickness of the optical modulation layer is much smaller than the central wavelength of the incident light, it cannot play an effective spectral modulation role; if the thickness of the optical modulation layer is much larger than the central wavelength of the incident light, it is difficult to fabricate in terms of technology and will introduce large optical losses. Therefore, in this embodiment, in order to reduce optical losses and be easy to fabricate, and to ensure an effective spectral modulation effect, the overall size (area) of each micro-nano unit in the optical modulation layer 1 is usually λ 2 ~10 5 λ 2 , and the thickness is usually 0.1λ to 10λ (λ represents the central wavelength of the incident light of the agricultural object). As Figure 5 shown, the overall size of each micro-nano unit is 0.5μm 2 ~40000μm 2 , and the dielectric material in the optical modulation layer 1 is polysilicon with a thickness of 50nm to 2μm.

[0197] Based on the content of the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0198] CMOS image sensor (Contact Image Sensor, CIS), charge coupled device (Charge Coupled Device, CCD), single photon avalanche diode (Single Photon Avalanche Diode, SPAD) array, and focal plane optoelectronic image sensor array.

[0199] In this embodiment, it should be noted that a wafer-level CMOS image sensor (CIS) is adopted to achieve monolithic integration at the wafer level, which can minimize the distance between the image sensor and the light modulation layer, facilitating the reduction of the unit size, the device volume, and the packaging cost. The SPAD can be used for low-light detection, and the CCD can be used for high-light detection.

[0200] In this embodiment, the light modulation layer and the image sensor can be fabricated by complementary metal oxide semiconductor (CMOS) integrated process, which is beneficial to reducing the device failure rate, improving the device yield, and reducing the cost. For example, one or more dielectric materials can be directly grown on the image sensor and then etched. Before removing the sacrificial layer used for etching, a metal material is deposited, and finally the sacrificial layer is removed to prepare the light modulation layer.

[0201] Based on the content of the above embodiment, in this embodiment, the types of the artificial neural network include: feedforward neural network.

[0202] In this embodiment, a feedforward neural network (FNN), also known as a deep feedforward network (DFN) or a multi-layer perceptron (MLP), is the simplest neural network, and its neurons are arranged in layers. Each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer, without feedback between layers. The feedforward neural network has a simple structure, is easy to implement on hardware, has a wide range of applications, and can approximate any continuous function and square-integrable function with arbitrary precision. Moreover, it can accurately implement any finite training sample set. The feedforward network is a static non-linear mapping. Through the composite mapping of simple non-linear processing units, complex non-linear processing capabilities can be obtained.

[0203] Based on the content of the above embodiment, in this embodiment, a light-transmitting dielectric layer is provided between the light modulation layer and the image sensor.

[0204] In this embodiment, it should be noted that setting a light-transmitting dielectric layer between the light modulation layer and the image sensor can effectively separate the light modulation layer and the image sensor layer and avoid mutual interference between the two.

[0205] Based on the content of the above embodiment, in this embodiment, the image sensor is front-illuminated and includes: a metal wire layer and a light detection layer arranged from top to bottom, and the light modulation layer is integrated on the side of the metal wire layer away from the light detection layer; or,

[0206] The image sensor is a back-illuminated type, including: a light detection layer and a metal wire layer arranged from top to bottom, and the light modulation layer is integrated on the side of the light detection layer away from the metal wire layer.

[0207] In this embodiment, as Figure 13 shown is a front-illuminated image sensor, where the silicon detection layer 21 is below the metal wire layer 22, and the light modulation layer 1 is directly integrated onto the metal wire layer 22.

[0208] In this embodiment, different from Figure 13 that, Figure 14 shown is a back-illuminated image sensor, where the silicon detection layer 21 is above the metal wire layer 22, and the light modulation layer 1 is directly integrated onto the silicon detection layer 21.

[0209] It should be noted that for the back-illuminated image sensor, with the silicon detection layer 21 above the metal wire layer 22, the influence of the metal wire layer on the incident light can be reduced, thereby improving the quantum efficiency of the device.

[0210] According to the above content, in this embodiment, the light modulation layer is used as the input layer and the linear layer of the artificial neural network, the image sensor is used as a part of the non-linear layer of the artificial neural network (the square detection response of the image sensor is used as the first non-linear activation function of the artificial neural network), and the filtering effect of the light modulation layer on the incident light entering the light modulation layer is used as the connection weight from the input layer to the linear layer. The light modulation layer and the image sensor in the optical artificial neural network intelligent agriculture precise control chip provided in this embodiment implement the related functions of the input layer, the linear layer, and part of the non-linear activation function in the artificial neural network through hardware, so that when using this optical artificial neural network intelligent agriculture precise control chip for intelligent processing later, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part of the non-linear activation function. This can significantly reduce the power consumption and delay during the processing of the artificial neural network. In addition, in this embodiment, since the image information, spectral information, incident light angle information, and incident light phase information at different points in the space of the agricultural object are utilized simultaneously, the intelligent processing of the agricultural object can be achieved more accurately.

[0211] It can be seen that in the embodiment of the present invention, the optical modulation layer is used as the input layer and the linear layer of the artificial neural network, and the image sensor is used as a part of the non-linear layer of the artificial neural network. The spatial spectral information of the object is projected into the photocurrent response of the image sensor, and the full connection and second-order non-linear activation of the electrical signal are realized in the processor, realizing functions such as intelligent perception, recognition, and / or decision-making with low power consumption, low latency, and high accuracy. The optical artificial neural network intelligent agricultural precision control chip based on the optical filter and the image sensor in the embodiment of the present invention has the following effects: embedding the artificial neural network part into the image sensor including various optical modulation layers to realize fast and accurate intelligent perception, recognition, and / or decision-making functions. In addition, the embodiment of the present invention can also achieve monolithic integration at the wafer level, thereby minimizing the distance between the sensor and the optical modulation layer to the greatest extent, which is beneficial to reducing the size of the unit and the volume and packaging cost of the device.

[0212] Based on the same inventive concept, another embodiment of the present invention provides an intelligent agricultural control device, including: the optical artificial neural network intelligent agricultural precision control chip as described in the above embodiment. The intelligent agricultural control device may include a fertilizer applicator device, a pesticide spraying device, a drug resistance analysis device, a drone device, an agricultural intelligent robot device, a crop health analysis device, a crop growth status monitoring device, etc.

[0213] Since the intelligent agricultural control device provided in this embodiment includes the optical artificial neural network intelligent agricultural precision control chip described in the above embodiment, therefore, the intelligent agricultural control device provided in this embodiment has all the beneficial effects of the optical artificial neural network intelligent agricultural precision control chip described in the above embodiment. Since the above embodiment has described this in detail, this embodiment will not be elaborated here.

[0214] Based on the same inventive concept, another embodiment of the present invention provides a preparation method of the optical artificial neural network intelligent agricultural precision control chip as described in the above embodiment, as Figure 15 shown, which specifically includes the following steps:

[0215] Step 1510: Prepare an optical modulation layer containing an optical modulation structure on the surface of the image sensor;

[0216] Step 1520: Generate a processor with the function of fully connecting signals or generate a processor with the functions of fully connecting signals and second-order non-linear activation processing;

[0217] Step 1530: Connect the image sensor and the processor;

[0218] Among them, the light modulation layer is used to perform different spectral modulations on the incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor; the information carried by the incident light includes light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light;

[0219] The image sensor converts the information carried by the incident light corresponding to different position points after being modulated by the light modulation layer into electrical signals corresponding to different position points through square-law detection response, and sends the electrical signals corresponding to different position points to the processor;

[0220] The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0221] In this embodiment, it also includes the training process of the optical artificial neural network intelligent agricultural precision control chip, which specifically includes:

[0222] Using the input training samples and output training samples corresponding to the agricultural precision control intelligent processing task, training the optical artificial neural network intelligent agricultural precision control chip including different light modulation structures, image sensors, and processors with different fully connected parameters to obtain a light modulation structure, image sensor, and processor that meet the training convergence conditions, and using the light modulation structure, image sensor, and processor that meet the training convergence conditions as the trained light modulation structure, image sensor, and processor;

[0223] Or, using the input training samples and output training samples corresponding to the agricultural precision control intelligent processing task, training the optical artificial neural network intelligent agricultural precision control chip including different light modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters to obtain a light modulation structure, image sensor, and processor that meet the training convergence conditions, and using the light modulation structure, image sensor, and processor that meet the training convergence conditions as the trained light modulation structure, image sensor, and processor.

[0224] It can be understood that when training an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0225] It can be understood that a large number of soil samples and crop samples in different states and positions can be collected in advance for data training, and then the required optical modulation structure (i.e., the micro-nano modulation structure in the optical modulation layer) can be designed and prepared, so as to implement the input layer, linear layer, and the first non-linear activation function in the non-linear layer of the artificial neural network in a hardware manner on the chip.

[0226] In this embodiment, preparing an optical modulation layer including an optical modulation structure on the surface of the photosensitive area of the image sensor includes:

[0227] Growing one or more layers of preset materials on the surface of the image sensor;

[0228] Performing dry etching on the one or more layers of preset materials with an optical modulation structure pattern to obtain an optical modulation layer including an optical modulation structure;

[0229] Or performing imprint transfer on the one or more layers of preset materials to obtain an optical modulation layer including an optical modulation structure;

[0230] Or obtaining an optical modulation layer including an optical modulation structure by applying external dynamic regulation to the one or more layers of preset materials;

[0231] Or performing partition printing on the one or more layers of preset materials to obtain an optical modulation layer including an optical modulation structure;

[0232] Or performing partition material growth on the one or more layers of preset materials to obtain an optical modulation layer including an optical modulation structure;

[0233] Or performing quantum dot transfer on the one or more layers of preset materials to obtain an optical modulation layer including an optical modulation structure.

[0234] When the optical artificial neural network intelligent agricultural precise control chip is used for the intelligent processing task of agricultural objects, the optical artificial neural network intelligent agricultural precise control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters is trained by using the input training samples and output training samples corresponding to the intelligent processing task to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions; or, the optical artificial neural network intelligent agricultural precise control chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters is trained to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions.

[0235] In this embodiment, it should be noted that, as Figure 1 shown, the optical modulation layer 1 can be prepared by directly growing one or more layers of dielectric materials on the image sensor 2, then etching, depositing metal materials before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the optical modulation structure, each unit can have different modulation effects on light with different wavelengths within the target range, and this modulation effect is insensitive to the incident angle and polarization. Each unit in the optical modulation layer 1 corresponds to one or more pixels on the image sensor 2. 1 is directly prepared on 2.

[0236] In this embodiment, it should be noted that, as Figure 14 shown, assuming that the image sensor 2 is a back-illuminated structure, the optical modulation layer 1 can be directly etched on the silicon image sensor layer 21 of the back-illuminated image sensor, and then metal is deposited for preparation.

[0237] In addition, it should be noted that the optical modulation structure on the optical modulation layer can be obtained by dry etching the optical modulation structure pattern on one or more preset materials. Dry etching is to directly remove the unnecessary parts of one or more preset materials on the surface of the photosensitive area of the image sensor, so as to obtain an optical modulation layer containing the optical modulation structure; or by imprint transfer of one or more preset materials. Imprint transfer is to prepare the required structure by etching on other substrates, and then transfer the structure to the photosensitive area of the image sensor through materials such as PDMS, so as to obtain an optical modulation layer containing the optical modulation structure; or by applying external dynamic regulation to one or more preset materials. External dynamic regulation is to use active materials, and then apply electrodes to regulate the optical modulation characteristics of the corresponding area by changing the voltage, so as to obtain an optical modulation layer containing the optical modulation structure; or by zone printing of one or more preset materials. Zone printing is to use the printing technology in zones to obtain an optical modulation layer containing the optical modulation structure; or by zone material growth of one or more preset materials to obtain an optical modulation layer containing the optical modulation structure; or by quantum dot transfer of one or more preset materials to obtain an optical modulation layer containing the optical modulation structure.

[0238] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the optical artificial neural network intelligent agriculture precision control chip in the above-mentioned embodiment, for the detailed content in terms of some principles and structures, etc., reference can be made to the introduction in the above-mentioned embodiment, and this embodiment will not be elaborated herein.

[0239] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A light artificial neural network intelligent agriculture precise control chip, characterized in that, For agricultural precision control intelligent processing tasks, including: a light modulation layer, an image sensor, and a processor; the light modulation layer corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer, and the square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor corresponds to the fully connected layer and the output layer of the artificial neural network, or, the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network; The light modulation layer is disposed on the surface of the image sensor, and the light modulation layer includes a light modulation structure, and the light modulation structure is configured to perform different spectral modulations on the incident light entering different position points of the light modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor; the incident light includes the reflected light, transmitted light, and / or radiation light of the agricultural object; the agricultural object includes crops and / or soil; The image sensor converts the information carried by the incident light corresponding to different position points modulated by the light modulation layer into electrical signals corresponding to different position points after the first non-linear activation process through the square-law detection response, and sends the electrical signals corresponding to different position points to the processor; The processor performs a fully connected process on the electrical signals corresponding to different position points, or, the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain an agricultural precision control processing result; Wherein, the agricultural precision control intelligent processing tasks include one or more of soil fertility detection, pesticide spraying detection, trace element content detection, drug resistance detection, and crop growth condition detection; the agricultural precision control processing results include one or more of soil fertility detection results, pesticide spraying detection results, trace element content detection results, drug resistance detection results, and crop growth condition detection results.

2. The optical artificial neural network intelligent agriculture precise control chip according to claim 1, wherein The information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

3. The optical artificial neural network intelligent agriculture precise control chip according to claim 1, characterized in that The optical artificial neural network intelligent agricultural precision control chip includes a trained light modulation structure, an image sensor, and a processor; The trained optical modulation structure, image sensor, and processor refer to those obtained by training an optical artificial neural network intelligent agricultural precision control chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters using input training samples and output training samples corresponding to the agricultural precision control intelligent processing task and meeting the training convergence conditions; or, the trained optical modulation structure, image sensor, and processor refer to those obtained by training an optical artificial neural network intelligent agricultural precision control chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters using input training samples and output training samples corresponding to the agricultural precision control intelligent processing task and meeting the training convergence conditions. Among them, the input training samples include incident light reflected, transmitted, and / or radiated by soils with different fertilities; the output training samples include the corresponding soil fertilities. and / or, The input training samples include incident light reflected, transmitted, and / or radiated by soils with different pesticide application conditions; the output training samples include the corresponding pesticide application conditions. and / or, The input training samples include incident light reflected, transmitted, and / or radiated by soils with different trace element content conditions; the output training samples include the corresponding trace element content conditions. and / or, The input training samples include incident light reflected, transmitted, and / or radiated by soils with different drug resistances; the output training samples include the corresponding drug resistances. and / or, The input training samples include incident light reflected, transmitted, and / or radiated by crops with different growth conditions; the output training samples include the corresponding crop growth conditions.

4. The optical artificial neural network intelligent agriculture precise control chip according to claim 3, wherein When training an optical artificial neural network intelligent agricultural precision control chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters, or training an optical artificial neural network intelligent agricultural precision control chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

5. The optical artificial neural network intelligent agriculture precise control chip according to any one of claims 1 to 4, characterized in that, The optical modulation structure in the optical modulation layer includes regular structures and / or irregular structures; and / or, the optical modulation structure in the optical modulation layer includes discrete structures and / or continuous structures.

6. The optical artificial neural network intelligent agriculture precise control chip according to any one of claims 1 to 4, characterized in that, The optical modulation layer is a single-layer structure or a multi-layer structure.

7. The optical artificial neural network intelligent agriculture precise control chip according to any one of claims 1 to 4, characterized in that The optical modulation structure in the optical modulation layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

8. The optical artificial neural network intelligent agriculture precise control chip according to claim 7, characterized in that, The micro-nano unit includes regular structures and / or irregular structures; and / or, the micro-nano unit includes discrete structures and / or continuous structures.

9. The optical artificial neural network intelligent agriculture precise control chip according to claim 7, characterized in that, The micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

10. The optical artificial neural network intelligent agriculture precise control chip according to claim 9, characterized in that, Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

11. The optical artificial neural network intelligent agriculture precise control chip according to claim 9, characterized in that, Each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

12. The optical artificial neural network intelligent agriculture precise control chip according to claim 9, characterized in that, One or more groups of the multi-group micro-nano structure arrays included in the micro-nano unit are empty structures.

13. The optical artificial neural network intelligent agriculture precise control chip according to claim 9, characterized in that The micro-nano unit has four-fold rotational symmetry.

14. The optical artificial neural network intelligent agriculture precise control chip according to claim 1, wherein, The light modulation layer is composed of one or more filter layers; The filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, tunable Fabry-Perot resonators.

15. The optical artificial neural network intelligent agriculture precise control chip according to claim 14, wherein, The semiconductor materials include one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructures include one or more of two-dimensional nanodot materials, two-dimensional nanorod materials, and two-dimensional nanowire materials.

16. The optical artificial neural network intelligent agriculture precise control chip according to claim 1, wherein The thickness of the light modulation layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

17. An intelligent agricultural control device, characterized in that, It includes the optical artificial neural network intelligent agriculture precise control chip as described in any one of claims 1 to 16.

18. A method for preparing an optical artificial neural network intelligent agriculture precise control chip according to any one of claims 1 to 16, characterized in that, It includes: Preparing a light modulation layer containing a light modulation structure on the surface of the image sensor; Generating a processor with the function of fully connecting and processing signals or generating a processor with the functions of fully connecting and processing signals and performing a second non-linear activation process; Connecting the image sensor and the processor; Wherein, the light modulation layer is used to respectively perform different spectral modulations on the incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor; The image sensor converts the information carried by the incident light corresponding to different position points after being modulated by the light modulation layer into electrical signals corresponding to different position points through square-law detection response, and sends the electrical signals corresponding to different position points to the processor; The processor performs a fully connected process on the electrical signals corresponding to different position points, or, the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain an agricultural precise control processing result.

19. The method for preparing the optical artificial neural network intelligent agriculture precise control chip according to claim 18, wherein It further includes: The training process of the optical artificial neural network intelligent agriculture precise control chip specifically includes: Using the input training samples and output training samples corresponding to the agricultural precise control intelligent processing task to train the optical artificial neural network intelligent agriculture precise control chip including different light modulation structures, image sensors, and processors with different fully connected parameters to obtain a light modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the light modulation structure, image sensor, and processor that meet the training convergence conditions as the trained light modulation structure, image sensor, and processor; Alternatively, use the input training samples and output training samples corresponding to the agricultural precision control intelligent processing task to train an optical artificial neural network intelligent agricultural precision control chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions, and use the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

20. The preparation method of the optical artificial neural network intelligent agriculture precise control chip according to claim 18, characterized in that, Prepare an optical modulation layer including an optical modulation structure on the surface of the image sensor, including: Grow one or more layers of a preset material on the surface of the image sensor; Etch an optical modulation structure pattern on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure; Or perform imprint transfer on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure; Or obtain an optical modulation layer including an optical modulation structure by applying external dynamic modulation to the one or more layers of the preset material; Or perform zone printing on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure; Or perform zone growth on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure; Or perform quantum dot transfer on the one or more layers of the preset material to obtain an optical modulation layer including an optical modulation structure.

Images