Agricultural control chip, intelligent agricultural control equipment and preparation method

Abstract

The present invention provides an agricultural control chip, intelligent agricultural control equipment and a preparation method. The present invention embeds an artificial neural network on a hardware chip, uses a light modulation layer on the hardware chip as an input layer of the artificial neural network, uses an image sensor as a linear layer of the artificial neural network, uses the filtering effect of the light modulation layer on incident light as a connection weight from the input layer to the linear layer, and uses a processor to implement a nonlinear layer and an output layer. The light modulation layer and the image sensor in the present invention implement the related functions of the input layer and the linear layer in the artificial neural network in a hardware manner, thereby greatly reducing power consumption and delay during artificial neural network processing.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an agricultural control chip, intelligent agricultural control equipment and a preparation method thereof. Background Art

[0002] Precision agriculture is a system supported by information technology that implements a set of modern agricultural operation technologies and management in a positioning, timing and quantitative manner according to spatial variation. Its basic meaning is to adjust the input to crops according to the soil properties of crop growth, judge the spatial variation of soil properties and productivity within the farmland, and on the other hand determine the production goals of crops, and carry out "system diagnosis, optimized formula, technical assembly, scientific management" to mobilize soil productivity, achieve the same or higher income with the least or most economical input, improve the environment, and efficiently utilize various agricultural resources to achieve economic and environmental benefits.

[0003] At present, there is a problem of slow processing speed when performing precision agricultural control. For example, for the identification of soil fertility, image acquisition, image preprocessing, feature extraction, feature matching and other steps are required to realize the identification of soil fertility. The current intelligent recognition task of agricultural objects generally relies on the neural network recognition model, that is, the current intelligent recognition task of agricultural objects requires imaging of agricultural objects first and then transmitting them to the computer for subsequent neural network recognition model algorithm processing. The transmission and processing of large amounts of data causes large power consumption and delay. Summary of the invention

[0004] In view of the problems existing in the prior art, the embodiments of the present invention provide an agricultural control chip, an intelligent agricultural control device and a preparation method.

[0005] Specifically, the embodiment of the present invention provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides an agricultural control chip for agricultural control intelligent processing tasks, comprising: a light modulation layer and an image sensor, wherein the image sensor is connected to a processor; the light modulation layer corresponds to an input layer of an artificial neural network and a connection weight from the input layer to a linear layer, and the image sensor corresponds to a linear layer of the artificial neural network; the processor corresponds to a nonlinear layer and an output layer of the artificial neural network;

[0007] The light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure, and the light modulation structure is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area; the image sensor is used to convert the incident light carrying information corresponding to the different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signal is an image signal modulated by the light modulation layer; the incident light includes reflected light, transmitted light and / or radiated light of an agricultural object; the agricultural object includes crops and / or soil;

[0008] The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain agricultural control processing results;

[0009] Among them, the agricultural control intelligent processing tasks include: one or more of soil fertility detection, pesticide spreading detection, trace element content detection, drug resistance detection and crop growth condition detection; the agricultural control processing results include: one or more of soil fertility detection results, pesticide spreading detection results, trace element content detection results, drug resistance detection results and crop growth condition detection results.

[0010] Furthermore, the information carried by the incident light includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0011] Further, the agricultural control chip includes a trained light modulation structure and an image sensor, and the processor is a trained processor;

[0012] The trained light modulation structure, image sensor and processor refer to the light modulation structure, image sensor and processor that meet the training convergence conditions by training agricultural control chips including different light modulation structures and image sensors and processors with different full connection parameters and nonlinear activation parameters using input training samples and output training samples corresponding to the agricultural control intelligent processing task;

[0013] Wherein, the input training samples include incident light reflected, transmitted and / or radiated by soils with different fertility; the output training samples include the corresponding soil fertility;

[0014] and / or,

[0015] The input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide spreading conditions; the output training samples include corresponding pesticide spreading conditions;

[0016] and / or,

[0017] The input training samples include incident light reflected, transmitted and / or radiated by soils having different trace element content conditions; the output training samples include corresponding trace element content conditions;

[0018] and / or,

[0019] The input training samples include incident light reflected, transmitted and / or radiated by soils with different pesticide resistances; the output training samples include corresponding pesticide resistances;

[0020] and / or,

[0021] The input training samples include incident light reflected, transmitted and / or radiated by crops with different growth conditions; the output training samples include corresponding crop growth conditions.

[0022] Furthermore, when training an agricultural control chip including different light modulation structures and image sensors and a processor with different fully connected parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0023] Furthermore, the light modulation structure in the light modulation layer includes a regular structure and / or an irregular structure; and / or the light modulation structure in the light modulation layer includes a discrete structure and / or a continuous structure.

[0024] Furthermore, the light modulation layer is a single-layer structure or a multi-layer structure.

[0025] Furthermore, the light modulation structure in the light modulation layer includes a unit array composed of a plurality of micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; and the structures of the various micro-nano units are the same or different.

[0026] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or the micro-nano unit includes a discrete structure and / or a continuous structure.

[0027] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0028] Furthermore, each group of micro-nano structure arrays has a broadband filtering or narrowband filtering function.

[0029] Furthermore, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0030] Furthermore, the micro-nano unit includes one or more groups of empty structures in the multiple groups of micro-nano structure arrays.

[0031] Furthermore, the micro-nano unit has four-fold rotational symmetry.

[0032] Further, the light modulation layer is composed of one or more filter layers;

[0033] The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, and adjustable Fabry-Perot resonant cavities.

[0034] Furthermore, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

[0035] Furthermore, the thickness of the light modulation layer is 0.1λ to 10λ, wherein λ represents the central wavelength of the incident light.

[0036] In a second aspect, an embodiment of the present invention further provides an intelligent agricultural control device, comprising the agricultural control chip as described in the first aspect.

[0037] In a third aspect, an embodiment of the present invention further provides a method for preparing the agricultural control chip as described in the first aspect, comprising:

[0038] Preparing a light modulation layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor;

[0039] Generate a processor capable of fully connected signal processing and nonlinear activation processing;

[0040] connecting the image sensor and the processor;

[0041] The light modulation layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area;

[0042] The image sensor is used to convert the information carried by the incident light corresponding to different position points after modulation by the light modulation layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different position points to obtain agricultural control processing results.

[0043] Furthermore, the method for preparing the agricultural control chip further includes: a training process for the agricultural control chip, specifically including:

[0044] Using input training samples and output training samples corresponding to the agricultural control intelligent processing task, agricultural control chips containing different light modulation structures and image sensors and processors with different fully connected parameters and nonlinear activation parameters are trained to obtain light modulation structures, image sensors and processors that meet the training convergence conditions, and the light modulation structures, image sensors and processors that meet the training convergence conditions are used as trained light modulation structures, image sensors and processors.

[0045] Furthermore, a light modulation layer including a light modulation structure is prepared on the surface of the image sensor, including:

[0046] Growing one or more layers of a preset material on the surface of the image sensor;

[0047] Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light modulation layer including the light modulation structure;

[0048] or performing embossing transfer on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0049] or obtaining a light modulation layer including a light modulation structure by externally dynamically modulating the one or more layers of the preset material;

[0050] or performing zone printing on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0051] or performing partitioned growth on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0052] Or quantum dots are transferred to the one or more layers of the preset materials to obtain a light modulation layer including a light modulation structure.

[0053] The agricultural control chip, intelligent agricultural control equipment and preparation method provided by the embodiments of the present invention, in which an artificial neural network is embedded in a hardware manner, wherein a light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure, and the light modulation layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area, and accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions, and at the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions, so as to obtain the output signal of the artificial neural network. It can be seen that in the agricultural control chip, the light modulation layer is used as an artificial The input layer of the neural network, the image sensor serves as the linear layer of the artificial neural network, and at the same time, the filtering effect of the light modulation layer on the incident light entering the light modulation layer corresponds to the connection weight from the input layer to the linear layer, that is, the light modulation layer and the image sensor in the agricultural control chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention separates the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the agricultural control chip is used for agricultural control processing in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and the processor in the agricultural control chip only needs to perform relevant processing with full connection and nonlinear activation of electrical signals, which can greatly reduce power consumption and delay during agricultural control processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 is a schematic diagram of the structure of an agricultural control chip provided by an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the agricultural control chip identification principle provided by an embodiment of the present invention;

[0057] Figure 3 This is a disassembly diagram of an agricultural control chip provided by an embodiment of the present invention;

[0058] Figure 4ais a schematic diagram of a process for identifying agricultural objects provided by an embodiment of the present invention;

[0059] Figure 4b It is a three-dimensional schematic diagram of identifying or qualitatively analyzing crops and / or soil provided by an embodiment of the present invention;

[0060] Figure 5 is a top view of a light modulation layer provided by one embodiment of the present invention;

[0061] Figure 6 is a top view of another light modulation layer provided by an embodiment of the present invention;

[0062] Figure 7 is a top view of another light modulation layer provided by an embodiment of the present invention;

[0063] Figure 8 is a top view of another light modulation layer provided by an embodiment of the present invention;

[0064] Fig. 9 is a top view of yet another light modulation layer provided by an embodiment of the present invention;

[0065] Fig.10 is a top view of yet another light modulation layer provided by an embodiment of the present invention;

[0066] Fig.11 This is a schematic diagram of the broadband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0067] Fig.12 This is a schematic diagram of the narrow-band filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0068] Fig.13 is a schematic structural diagram of a front-illuminated image sensor provided by an embodiment of the present invention;

[0069] Fig.14 is a schematic structural diagram of a back-illuminated image sensor provided by an embodiment of the present invention;

[0070] Fig.15 It is a schematic flow chart of a method for preparing an agricultural control chip provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Precision agriculture is a system that implements a set of modern agricultural operation technologies and management in a positioning, timing and quantitative manner based on spatial variation supported by information technology. Its basic meaning is to adjust the input of crops according to the soil properties of crop growth, judge the soil properties and spatial variation of productivity within the farmland, and determine the production goals of crops on the other hand, and carry out positioning "system diagnosis, optimization formula, technical assembly, scientific management", mobilize soil productivity, achieve the same or higher income with the least or most economical input, and improve the environment, efficiently utilize various agricultural resources, and achieve economic and environmental benefits. At present, there is a problem of slow processing speed when carrying out precision agricultural control. For example, for the identification of soil fertility, image acquisition, image preprocessing, feature extraction, feature matching and other steps are required to realize the identification of soil fertility. The current intelligent recognition task of agricultural objects generally relies on the neural network recognition model, that is, the current intelligent recognition task of agricultural objects requires imaging of agricultural objects first and then transmitting them to the computer for subsequent neural network recognition model algorithm processing. The transmission and processing of large amounts of data causes large power consumption and delay. In addition, there is still the problem of inaccurate recognition when carrying out precision agricultural control. For example, it is difficult to ensure the accuracy of recognition by only using two-dimensional image information of soil. For example, there are distortion factors in two-dimensional images. In addition, the imaging process needs to convert optical information into digital electronic signals and then transmit them to the computer for subsequent algorithm processing. The transmission and processing of large amounts of data causes large power consumption and delays. In addition, there are similar problems in the recognition of crop growth conditions. Based on this, an embodiment of the present invention provides an agricultural control chip, in which an optical modulation layer serves as an input layer of an artificial neural network, and an image sensor serves as a linear layer of the artificial neural network. At the same time, the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical modulation layer and the image sensor in the agricultural control chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the agricultural control chip is used for agricultural control processing in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and the processor in the agricultural control chip only needs to perform relevant processing with full connection and nonlinear activation of electrical signals, which can greatly reduce power consumption and delay during agricultural control processing.It can be seen that the embodiment of the present invention uses the light modulation layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the light modulation layer on the incident light entering the light modulation layer as the connection weight from the input layer to the linear layer. The light modulation layer and the image sensor are used to project the spatial spectral information of the agricultural object to be agriculturally controlled into an electrical signal, and then the full connection processing and nonlinear activation processing of the electrical signal are implemented in the processor. The embodiment of the present invention can eliminate the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, thereby greatly reducing the power consumption and delay during artificial neural network processing.

[0073] In addition, the embodiment of the present invention can also use one or more information of the image information, spectral information, angle of incident light and phase information of incident light of agricultural objects, that is, the incident light carrying information at different points in the space of agricultural objects. Since the incident light carrying information at different points in the space of agricultural objects covers the image, composition, shape, three-dimensional depth, structure, color and other information of each spatial position point of the agricultural object, when agricultural control is performed based on the incident light carrying information at different spatial points of the agricultural object, the image, composition, shape, three-dimensional depth, three-dimensional color and other information of the agricultural object can be covered, thereby effectively improving the recognition accuracy, and then quickly and accurately identifying the agricultural production status (such as soil fertility, pesticide spreading, soil trace element content, crop growth status, etc.), so as to prepare for the precise control of intelligent agriculture. It can be seen that the agricultural control chip provided by the embodiment of the present invention can not only achieve the effect of low power consumption and low latency, but also improve the accuracy of intelligent processing, so as to achieve fast, accurate, safe and reliable recognition and qualitative analysis of soil fertility, pesticide spreading, trace element content and crop growth status, so it can be well applied in the field of precision agricultural control. The contents provided by the present invention will be explained and illustrated in detail below through specific embodiments.

[0074] like Figure 1 As shown, the agricultural control chip provided by the first embodiment of the present invention is used for agricultural control intelligent processing tasks, including: a light modulation layer 1 and an image sensor 2, wherein the image sensor 2 and a processor 3 are connected and arranged, and the processor 3 can be arranged in the chip or independently arranged from the chip; the light modulation layer 1 corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, and the image sensor 2 corresponds to the linear layer of the artificial neural network; the processor 3 corresponds to the nonlinear layer and the output layer of the artificial neural network;

[0075] The light modulation layer 1 is arranged on the surface of the photosensitive area of ​​the image sensor (the surface of the image sensor), and the light modulation layer 1 includes a light modulation structure. The light modulation layer 1 is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area; the image sensor is used to convert the incident light carrying information corresponding to the different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signal is an image signal modulated by the light modulation layer; the incident light includes reflected light, transmitted light and / or radiated light of an agricultural object; the agricultural object includes crops and / or soil;

[0076] The processor 3 is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain agricultural control processing results;

[0077] In this embodiment, the agricultural control intelligent processing tasks include: one or more of soil fertility detection, pesticide spreading detection, trace element content detection, drug resistance detection and crop growth condition detection; the agricultural control processing results include: one or more of soil fertility detection results, pesticide spreading detection results, trace element content detection results, drug resistance detection results and crop growth condition detection results.

[0078] The information carried by the incident light includes image information and / or various optical spatial information of the target object to be processed by the optical artificial neural network intelligent chip. For example, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0079] In this embodiment, the light modulation layer 1 is disposed on the surface of the image sensor, and the light modulation layer 1 includes a light modulation structure. The optical filter 1 layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the information carried by the modulated incident light corresponding to different positions on the surface of the image sensor. It can be seen that in the embodiment, the modulation effect of the light modulation structure on the light modulation layer on the incident light can be regarded as the connection weight from the input layer to the linear layer.

[0080] In this embodiment, the light modulation layer 1 includes a light modulation structure, which performs spectral modulation of different intensities on incident light (such as reflected light, transmitted light, radiated light and other related action light of the target to be identified) entering different positions of the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the image sensor 2.

[0081] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the light modulation structure. For example, different modulation intensities can be achieved by designing different light modulation structures (such as changing the shape and / or size parameters of the light modulation structure).

[0082] It can be seen that in this embodiment, it is necessary to use a light source to illuminate the agricultural object, and then allow the transmitted light or reflected light to enter the light modulation layer of the chip, and the light modulation structure in the light modulation layer adjusts the light incident on the light modulation structure. Since the light modulation structure in the light modulation layer is a pre-designed modulation pattern that can be used to identify agricultural objects, when the incident light reflected, transmitted or radiated by the agricultural object enters the light modulation layer of the chip, the light modulation layer, image sensor and processor on the chip together as a whole of the artificial neural network identify the reflected light, transmitted light or radiated light, and then obtain relevant identification results of the agricultural object.

[0083] Optionally, the modulation pattern in the light modulation layer that can be identified by the agricultural object can be set in the following manner:

[0084] For agricultural object recognition, the modulation pattern can be optically simulated on a computer to obtain the modulation intensity (transmittance) of the modulation pattern for different wavelength components of the incident light, which is used as the connection weight from the input layer to the linear layer of the artificial neural network, and a nonlinear activation function is implemented in the computer. The spectral signal data corresponding to a large number of agricultural objects are collected in advance and data training is performed. The required modulation pattern can be designed and prepared, and the input layer and linear layer of the artificial neural network are implemented on the chip by hardware (optical modulation layer and image sensor), so that in the process of agricultural object recognition, the electrical signal obtained by modulating the incident light of different wavelengths is restored by algorithm, so that the agricultural object can be detected quickly and accurately. It can be understood that the agricultural object here can refer to crops and / or soil, and the agricultural object recognition result here (which can also be understood as the agricultural control treatment result) can refer to one or more of the soil fertility test results, pesticide spreading test results, trace element content test results, drug resistance test results, and crop growth condition test results. In this embodiment, the light modulation layer 1 is disposed on the surface of the image sensor, and the light modulation layer 1 includes a light modulation structure. The optical filter 1 layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain information carried by the modulated incident light corresponding to different positions on the surface of the image sensor. It can be seen that in the embodiment, the light modulation layer corresponds to the input layer of the artificial neural network, and the modulation effect of the light modulation structure on the light modulation layer on the incident light can be regarded as the connection weight from the input layer to the linear layer;

[0085] In this embodiment, the light modulation layer 1 includes a light modulation structure, through which the incident light (such as reflected light, transmitted light, radiated light and other related action light of the target to be identified) entering different positions of the light modulation structure is spectrally modulated with different intensities, so as to obtain light field distribution signals corresponding to different positions on the surface of the image sensor 2.

[0086] In this embodiment, it can be understood that the light modulation structures at different positions on the light modulation layer 1 have different spectral modulation effects on the incident light, and the modulation intensity of the light modulation structure on different wavelength components of the incident light corresponds to the connection strength of the linear layer of the artificial neural network, that is, the connection weight from the input layer to the linear layer. It should be noted that the light modulation layer 1 is composed of a plurality of optical filter units, and the light modulation structures at different positions in each optical filter unit are different, so they have different spectral modulation effects on the incident light; the light modulation structures at different positions between the optical filter units can be the same or different, so they have the same or different spectral modulation effects on the incident light.

[0087] In this embodiment, the image sensor 2 converts the light intensity distribution signals corresponding to different position points into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to the linear layer of the neural network.

[0088] In this embodiment, the processor 3 performs full connection processing and nonlinear activation processing on the electrical signals at different positions, thereby obtaining the output signal of the artificial neural network.

[0089] It can be understood that processor 3 corresponds to the nonlinear layer and output layer of the neural network, and can also be understood as corresponding to the remaining layers (all other layers) in the neural network except the input layer and the linear layer.

[0090] In addition, it should be noted that the processor 3 can be set in the agricultural control chip, that is, the processor 3 can be set in the agricultural control chip together with the filter layer 1 and the image sensor 2, or it can be set separately outside the agricultural control chip and connected to the image sensor 2 in the agricultural control chip through a data line or a connecting device. This embodiment does not limit this.

[0091] In addition, it should be noted that the processor 3 can be implemented by a computer, or by an ARM or FPGA circuit board with certain computing power, or by a microprocessor, which is not limited in this embodiment. In addition, as mentioned above, the processor 3 can be integrated in the agricultural control chip, or can be independently set outside the agricultural control chip. When the processor 3 is independently set outside the agricultural control chip, the electrical signal in the image sensor 2 can be read into the processor 3 through the signal readout circuit, and then the processor 3 performs full connection processing and nonlinear activation processing on the read electrical signal.

[0092] In this embodiment, it can be understood that when the processor 3 performs nonlinear activation processing, it can be implemented by using a nonlinear activation function, for example, a Sigmoid function, a Tanh function, a ReLU function, etc., and this embodiment does not limit this.

[0093] like Figure 2 As shown on the left, the agricultural control chip includes a light modulation layer 1 and an image sensor 2. The processor 3 is connected to the image sensor 2 in the agricultural control chip. Figure 2 In the embodiment, the processor 3 is implemented by a signal readout circuit and a computer. Figure 2 As shown on the right, the light modulation layer 1 in the agricultural control chip corresponds to the input layer of the artificial neural network, the image sensor 2 corresponds to the linear layer of the artificial neural network, the processor 3 corresponds to the nonlinear layer and output layer of the artificial neural network, and the filtering effect of the light modulation layer 1 on the incident light entering the light modulation layer 1 corresponds to the connection weight from the input layer to the linear layer. It can be seen that the light modulation layer and the image sensor in the agricultural control chip provided in this embodiment realize the relevant functions of the input layer and the linear layer in the artificial neural network by hardware, so that when the agricultural control chip is used for subsequent recognition processing, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer (for example, calculations such as the connection weight from the input layer to the linear layer are omitted), which can greatly reduce the power consumption and delay during artificial neural network processing.

[0094] like Figure 2 As shown on the right, the incident light spectrum P at different positions of the light modulation layer 1 λ Projected / connected to the image sensor photocurrent response I N In the figure, processor 3 includes a signal readout circuit and a computer. The signal readout circuit in processor 3 reads out the photocurrent response and transmits it to the computer, which performs full connection processing and nonlinear activation processing on the electrical signal and finally outputs the result.

[0095] like Figure 3As shown, the light modulation structure on the light modulation layer 1 is integrated above the image sensor 2, modulates the incident light, and projects / connects the spectral information of the incident light to different pixel points of the image sensor 2 to obtain an electrical signal containing the spectral information of the incident light and the image information. That is, after the incident light passes through the light modulation layer 1, it is converted into an electrical signal by the image sensor 2 to form an image containing the spectral information of the incident light. Finally, the processor 3 connected to the image sensor 2 processes the electrical signal containing the spectral information of the incident light and the image information.

[0096] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0097] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, the target object can be identified by simultaneously utilizing multiple information including the target object's image information, spectral information, angle of the incident light, and phase information of the incident light, thereby more accurately realizing intelligent identification of the target object.

[0098] It can be seen that the present embodiment can utilize one or more of the image information, spectral information, angle of incident light and phase information of incident light at different points in the space of the agricultural object, that is, utilize the information carried by the incident light at different points in the space of the agricultural object. Since the information carried by the incident light at different points of the agricultural object covers the image, composition, shape, three-dimensional depth, structure, color and other information at different points of the agricultural object, when agricultural control is performed based on the information carried by the incident light at different spatial points of the agricultural object, the image, composition, shape, three-dimensional depth, three-dimensional color and other information of the agricultural object can be covered, thereby solving the problem mentioned in the background technology part that it is difficult to ensure the accuracy of recognition by using two-dimensional image information of agricultural objects. It can be seen that the agricultural control chip provided by the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, so that the agricultural production conditions (such as soil fertility, pesticide spreading conditions, soil trace element content, crop growth conditions, etc.) can be quickly and accurately identified, thereby preparing for the precise control of smart agriculture.

[0099] It can be seen that this embodiment projects one or more of the image information, spectral information, angle of incident light and phase information of incident light at different positions of the agricultural object into the photocurrent response of the hardware chip, making full use of the light-related information of various dimensions of the agricultural object, thereby enabling fast, efficient and precise agricultural control.

[0100] It can be seen that the agricultural control chip provided in the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, so that agricultural objects can be identified quickly and accurately. In addition, the chip provided in this embodiment is small in size and can be integrated into a smaller device, making it convenient to carry.

[0101] The embodiment of the present invention embeds an artificial neural network on a hardware chip, uses the light modulation layer on the hardware chip as the input layer of the artificial neural network, uses the image sensor as the linear layer of the artificial neural network, uses the filtering effect of the light modulation layer on the incident light as the connection weight from the input layer to the linear layer, and uses a processor to implement the nonlinear layer and output layer of the artificial neural network. It can be seen that the light modulation layer and the image sensor in the present invention implement the relevant functions of the input layer and the linear layer in the artificial neural network in a hardware manner, thereby greatly reducing the power consumption and delay during the processing of the artificial neural network, thereby quickly and efficiently solving the problem of agricultural precision control.

[0102] The agricultural control chip, intelligent agricultural control equipment and preparation method provided by the embodiments of the present invention realize a new agricultural control chip capable of realizing the function of an artificial neural network, which is used for agricultural control intelligent processing tasks. In the agricultural control chip, an artificial neural network is embedded in a hardware manner, wherein a light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure. The light modulation layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area. Accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions. At the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions to obtain the output signal of the artificial neural network, thereby It can be seen that in the agricultural control chip, the light modulation layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the light modulation layer on the incident light entering the light modulation layer corresponds to the connection weight from the input layer to the linear layer, that is, the light modulation layer and the image sensor in the agricultural control chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the agricultural control chip is used for agricultural control processing in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and the processor in the agricultural control chip only needs to perform relevant processing with full connection and nonlinear activation of electrical signals, which can greatly reduce power consumption and delay during agricultural control processing.It can be seen that the embodiment of the present invention uses the light modulation layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the light modulation layer on the incident light entering the light modulation layer as the connection weight from the input layer to the linear layer. The light modulation layer and the image sensor are used to project the spatial spectral information of the agricultural object to be agriculturally controlled into an electrical signal, and then the full connection processing and nonlinear activation processing of the electrical signal are implemented in the processor. It can be seen that the embodiment of the present invention can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present invention actually simultaneously utilizes the image information, spectral information, angle of the incident light and phase information of the incident light at different spatial positions of the agricultural object, that is, the incident light at different points in the space of the agricultural object carries information. Since the agricultural The information carried by the incident light at different points in the space of the agricultural object covers the image, composition, shape, three-dimensional depth, structure, color and other information of different spatial positions of the agricultural object. Therefore, when agricultural control is performed based on the information carried by the incident light at different spatial points of the agricultural object, the image, composition, shape, three-dimensional depth, three-dimensional color and other information of the agricultural object can be covered, thereby solving the problem mentioned in the background technology part that it is difficult to ensure the accuracy of recognition using two-dimensional image information of agricultural objects. It can be seen that the agricultural control chip provided by the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, so that agricultural production conditions (such as soil fertility, pesticide spreading conditions, soil trace element content, crop growth conditions, etc.) can be quickly and accurately identified, thereby preparing for the precise control of smart agriculture.

[0103] It can be seen that the embodiment of the present invention provides a new optoelectronic chip for precision agriculture control. The chip consists of an optical modulation layer and an image sensor to form an input layer and a linear layer of an optical artificial neural network. It collects and uses image information, spectral information, angle of incident light and phase information of incident light at different spatial positions of agricultural objects to achieve fast, accurate, safe and reliable identification and qualitative analysis of soil fertility, pesticide spreading conditions, trace element content and crop growth conditions.

[0104] Based on the content of the above embodiment, in this embodiment, the agricultural control chip includes a trained light modulation structure and an image sensor, and the processor is a trained processor;

[0105] The trained light modulation structure, image sensor and processor refer to the input training samples and output training samples corresponding to the agricultural control intelligent processing task.

[0106] Training agricultural control chips including different light modulation structures and image sensors and processors with different fully connected parameters and nonlinear activation parameters to obtain light modulation structures, image sensors and processors that meet training convergence conditions;

[0107] Wherein, the input training samples include incident light reflected, transmitted and / or radiated by soil with different fertility; the output training samples include the corresponding soil fertility; and / or, the input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide spreading conditions; the output training samples include the corresponding pesticide spreading conditions; and / or, the input training samples include incident light reflected, transmitted and / or radiated by soil with different trace element content conditions; the output training samples include the corresponding trace element content conditions; and / or, the input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide resistance; the output training samples include the corresponding pesticide resistance; and / or, the input training samples include incident light reflected, transmitted and / or radiated by crops with different growth conditions; the output training samples include the corresponding crop growth conditions.

[0108] It can be understood that, for the soil fertility detection task, the input training sample includes incident light reflected, transmitted and / or radiated by soils with different fertility; and the output training sample includes the corresponding soil fertility.

[0109] It can be understood that, for the task of detecting pesticide spreading conditions, the input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide spreading conditions; and the output training samples include corresponding pesticide spreading conditions.

[0110] For other tasks, you can follow the above ideas and make analogies, so I will not give examples one by one here.

[0111] In this embodiment, it can be understood that the input training samples include incident light reflected, transmitted and / or radiated by agricultural objects in the corresponding intelligent processing tasks; the output training samples include intelligent processing results of agricultural objects (such as recognition results, perception results, decision results or qualitative analysis results, etc.).

[0112] In this embodiment, the reflected light, transmitted light and / or radiated light of the agricultural object enters the trained agricultural control chip to obtain the intelligent processing result of the agricultural object.

[0113] In this embodiment, the recognition task of agricultural objects is taken as an example. It can be understood that when using the agricultural control chip to perform recognition tasks, the agricultural control chip needs to be trained first. Training the agricultural control chip here means determining the optical modulation structure suitable for the current recognition task through training, as well as the fully connected parameters and nonlinear activation parameters suitable for the current recognition task.

[0114] It can be understood that since the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the optical modulation structure in the optical modulation layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence conditions, the optical modulation structure suitable for the current recognition task is determined, as well as the fully connected parameters and nonlinear activation parameters suitable for the current recognition task, thereby completing the training of the agricultural control chip.

[0115] It is understandable that after the agricultural control chip is trained, the agricultural control chip can be used to perform recognition tasks. Specifically, after the incident light carrying the image information and spatial spectrum information of the agricultural object enters the light modulation layer 1 of the trained agricultural control chip, the light modulation structure in the light modulation layer 1 will modulate the incident light, and the intensity of the modulated light signal is detected by the image sensor 2 and converted into an electrical signal, and then the processor 3 performs full connection processing and nonlinear activation processing to obtain the recognition result of the agricultural object.

[0116] like Figure 4a As shown, the complete process for agricultural object recognition is: the wide-spectrum light source 100 is irradiated onto the agricultural object 200, and then the reflected light or transmitted light of the agricultural object is collected by the agricultural control chip 300, or the light directly radiated outward by the agricultural object is collected by the agricultural control chip 300, and the recognition result can be obtained after being processed by the light modulation layer, image sensor and processor in the agricultural control chip.

[0117] Among them, the trained agricultural control chip refers to an agricultural control chip including a trained light modulation structure, an image sensor and a processor; the trained light modulation structure, image sensor and processor refer to the light modulation structure, image sensor and processor that meet the training convergence conditions by training the agricultural control chip including different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters using input training samples and output training samples corresponding to the intelligent processing task.

[0118] In this embodiment, since the agricultural control intelligent processing tasks include one or more of soil fertility detection, pesticide spreading detection, trace element content detection, pesticide resistance detection, and crop growth condition detection, the corresponding input training samples and output training samples are also different.

[0119] For example, when the agricultural control intelligent processing task includes soil fertility detection, the corresponding input training samples include incident light reflected, transmitted and / or radiated by soils with different fertility, and the corresponding output training samples include the corresponding soil fertility.

[0120] When the agricultural control intelligent processing task includes pesticide spreading detection, the corresponding input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide spreading conditions, and the corresponding output training samples include corresponding pesticide spreading conditions.

[0121] When the agricultural control intelligent processing task includes trace element content detection, the corresponding input training samples include incident light reflected, transmitted and / or radiated by soil with different trace element content conditions, and the corresponding output training samples include the corresponding trace element content conditions.

[0122] When the agricultural control intelligent processing task includes pesticide resistance detection, the corresponding input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide resistance, and the corresponding output training samples include the corresponding pesticide resistance.

[0123] When the agricultural control intelligent processing task includes crop growth condition detection, the corresponding input training samples include incident light reflected, transmitted and / or radiated by crops with different growth conditions, and the corresponding output training samples include corresponding crop growth conditions.

[0124] It is understandable that a large number of soil samples and crop samples in different states and positions can be collected in advance and data training can be carried out, and then the required optical modulation structure (that is, the micro-nano modulation structure in the optical modulation layer) can be designed and prepared, so as to realize the input layer and linear layer of the artificial neural network in hardware on the chip.

[0125] It is understandable that, for the recognition task, since the advantage of the agricultural control chip provided in this embodiment is that it can obtain the spatial spectrum information of the recognition object, in order to make full use of this advantage, the recognition object sample used as the input training sample is preferably a real recognition object, rather than a two-dimensional image of the recognition object. Of course, this does not mean that a two-dimensional image cannot be used as a recognition object sample.

[0126] In addition, the agricultural control chip provided in this embodiment can also be used for other intelligent processing tasks of agricultural objects, such as intelligent agricultural perception, intelligent agricultural decision-making and other tasks.

[0127] In this embodiment, the light modulation layer is used as the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, and the image sensor is used as the linear layer of the artificial neural network; the processor is used as the nonlinear layer and the output layer of the artificial neural network. In order to minimize the loss function of the neural network, the modulation intensity of different wavelength components in the incident light of the agricultural object by the light modulation structure in the light modulation layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the light modulation layer, the modulation intensity of different wavelength components in the incident light of the agricultural object can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, thereby optimizing the training of the neural network.

[0128] In this embodiment, the optical modulation structure is obtained based on neural network training. The training samples are optically simulated by a computer to obtain the sample modulation intensity of the optical modulation structure in the training samples for different wavelength components of the incident light of the agricultural object in the intelligent processing task. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network to perform nonlinear activation. The neural network is trained using the training samples corresponding to the intelligent processing task until the neural network converges and the corresponding training sample optical modulation structure is used as the optical modulation layer of the corresponding intelligent processing task.

[0129] It can be seen that this embodiment can save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art by realizing the input layer and the linear layer of the neural network at the physical layer, thereby improving the processing speed and reducing the delay. At the same time, the embodiment of the present invention actually simultaneously utilizes the image information, spectral information, angle of incident light and phase information of incident light at different spatial positions of agricultural objects, that is, the incident light carrying information at different points in the space of agricultural objects. Since the incident light carrying information at different points in the space of agricultural objects covers the image, composition, shape, three-dimensional depth, structure, color and other information of each spatial position of the agricultural object, when agricultural control is performed based on the incident light carrying information at different spatial points of the agricultural object, the image, composition, shape, three-dimensional depth, three-dimensional color and other information of the agricultural object can be covered, thereby solving the problem mentioned in the background technology part that it is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the agricultural object.

[0130] Based on the contents of the above embodiments, in this embodiment, when training an agricultural control chip including different light modulation structures, image sensors and processors with different fully connected parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0131] This embodiment designs the light modulation structure through computer optical simulation, adjusts the light modulation structure through optical simulation, and determines the corresponding light modulation structure as the size of the light modulation structure that needs to be produced in the end when the neural network converges, thereby saving prototype production time and cost, improving product efficiency, and easily solving complex optical problems. For example, the light modulation structure can be simulated and designed through FDTD software, and the light modulation structure can be changed in the optical simulation, so that the modulation intensity of the light modulation structure for different incident lights can be accurately predicted, and it can be used as the connection weight between the input layer and the linear layer of the neural network to train the agricultural control chip and accurately obtain the light modulation structure.

[0132] It is understandable that a large number of soil samples and crop samples in different states and positions can be collected in advance and data training can be carried out, and then the required optical modulation structure (that is, the micro-nano modulation structure in the optical modulation layer) can be designed and prepared, so as to realize the input layer and linear layer of the artificial neural network in hardware on the chip.

[0133] It can be seen that this embodiment designs the light modulation structure by adopting the computer optical simulation design method, which saves the time and cost of prototyping the light modulation structure and improves product efficiency.

[0134] It is understandable that the technical principle of precision agriculture is to adjust the input of crops according to the spatial differences in soil fertility and crop growth conditions, and to implement positioning and quantitative precision field management based on quantitative real-time diagnosis of cultivated land and crop growth and full understanding of the spatial variation of field productivity, with the goal of balancing soil fertility and increasing yield, so as to achieve the sustainable development goal of efficient use of various agricultural resources and improvement of the environment. The implementation of precision agriculture can not only maximize agricultural productivity, but also achieve the goal of sustainable agricultural development with high quality, high yield, low consumption and environmental protection. The characteristics of field soil and crops in farmland are not uniform, but change with time and space. However, in traditional farmland management, which is still in use, the above characteristics are considered to be uniform, and uniform fertilization time and amount are adopted. However, excessive pesticides and fertilizers will flow into surface water and groundwater, causing environmental pollution. Therefore, it is of great significance to agricultural modernization to realize precision agricultural control chips that can be integrated with drones, intelligent robots, etc., with small size, low cost and wide range of use through the acquisition of spectral data and images of farmland soil and crops. The purpose of this embodiment is to provide a new type of optoelectronic chip for precision agricultural control. The chip consists of an optical modulation layer and an image sensor to form an input layer, a linear layer, and a part of the nonlinear activation function in the nonlinear layer of an optical artificial neural network. The chip collects and uses the image information, spectral information, angle of incident light, and phase information of incident light at different spatial positions of agricultural objects to achieve fast, accurate, safe and reliable qualitative analysis of soil fertility, pesticide spreading, trace element content, and crop growth conditions. The chip directly prepares a micro-nano modulation structure on the surface of the photosensitive area of ​​the image sensor. Several discrete or continuous micro-nano structures constitute a unit. The micro-nano modulation structures at different positions have different spectrum modulation effects on the incident light, and together constitute the optical modulation layer. The modulation intensity of these micro-nano modulation structures on different wavelength components of the incident light corresponds to the connection strength (linear layer weight) of the artificial neural network. At the same time, the image sensor converts the light field distribution signal corresponding to the different position points after being modulated by the optical modulation layer into an electrical signal corresponding to the different position points, and sends the electrical signal corresponding to the different position points to the processor. The processor performs full connection processing and nonlinear activation processing on the electrical signal corresponding to the different position points to obtain the output signal of the artificial neural network. For precision agriculture control, a large number of soil samples and crop samples in different states and positions can be collected first, and the weights of the linear layer, that is, the system function of the light modulation layer, can be obtained through data training. Then, the required light modulation layer can be reversely designed and integrated on top of the image sensor. In addition, during the application of the chip, the output of the completed light modulation layer can be used to further train and optimize the weights of the fully connected layer of the electrical signal and the nonlinear activation function, which can realize the qualitative analysis of soil fertility, pesticide spreading, trace element content and crop growth status.

[0135] It is understandable that the chip actually uses the image information and spectral information of soil and crops at the same time, improving the accuracy and diversity of precision agricultural control; and partially realizes artificial neural networks in hardware, improving the analysis speed of precision agricultural control, and the chip can be integrated into drones and intelligent robot systems to accurately analyze the soil and crop status of large areas of farmland. In addition, the chip solution can be mass-produced using existing CMOS processes, reducing the size, power consumption and cost of the device.

[0136] The optical artificial neural network precision agriculture control chip based on micro-nano modulation structure and image sensor provided in this embodiment has a structural schematic diagram as shown in FIG. Figure 2 As shown, it includes a light modulation layer 1, an image sensor 2 and a processor 3. The light modulation layer 1 corresponds to the input layer and the linear layer of the optical artificial neural network, wherein a plurality of discrete or continuous micro-nano modulation structures constitute a unit, and these modulation structures have different broadband spectrum modulation effects on the incident light. The micro-nano modulation structures contained in different units can be the same or different, and can also be spatially reconstructed according to the image. Each unit corresponds to a plurality of image sensor photosensitive pixels in the vertical direction. The image sensor 2 acts as a linear layer, which converts the outgoing light field E of the light modulation layer 1 into a linear layer. N Converted to the photocurrent response I of the image sensor N The processor 3 includes a signal readout circuit and a computer. The signal readout circuit in the processor 3 reads out the photocurrent response I N The signal is then transmitted to the computer, which performs full connection processing on the electrical signal or performs nonlinear activation processing again, and finally outputs the result.

[0137] It can be understood that for precision agricultural control, by performing optical simulation of the micro-nano modulation structure on a computer, the modulation intensity (transmittance) of the modulation structure for different wavelength components of the incident light can be obtained, and it is used as the connection weight from the input layer to the linear layer of the artificial neural network. At the same time, the image sensor is used as the linear layer to convert the light field distribution signal corresponding to different position points after modulation by the optical modulation layer into electrical signals corresponding to different position points, and the electrical signals corresponding to different position points are sent to the processor. The processor performs full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. A large number of soil samples and crop samples in different states and positions are collected in advance and data training is performed. The required micro-nano modulation structure can be designed and prepared, and the input layer and linear layer of the artificial neural network can be realized on the chip.

[0138] From a vertical perspective, Figure 2As shown, each micro-nano modulation structure in the optical modulation layer is obtained by pre-training and designing an artificial neural network, and can be prepared by directly growing one or more layers of dielectric or metal materials on the image sensor and then etching. The overall size of each modulation unit in the optical modulation layer is usually λ 2 ~10 5 λ 2 , the thickness is usually 0.1λ~10λ, λ is the central wavelength of the target band. Each modulation unit structure in the optical modulation layer corresponds to multiple pixels on the image sensor. The optical modulation layer is directly prepared on the image sensor, and the image sensor and the processor are connected through electrical contacts.

[0139] In this embodiment, the complete process of analyzing samples for the precision agriculture control chip is as follows: Figure 4b As shown, the spectral data and images of farmland soil and crops are collected by the spectrum and image collector, and then the reflected light is collected by the precision agriculture control chip, and then processed by the internal algorithm of the processor to obtain the recognition result.

[0140] It can be understood that both the light modulation layer and the CIS wafer (CIS wafer as an image sensor) can be manufactured by the semiconductor CMOS integration process. The light modulation layer can be monolithically integrated on the image sensor directly from the wafer level, and the chip can be prepared by a one-time tape-out using the CMOS process, thereby achieving monolithic integration at the wafer level, which is beneficial to reducing the distance between the sensor and the light modulation layer, reducing the size of the device, and reducing packaging costs.

[0141] It can be seen that in this embodiment, the light modulation layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer, and the image sensor corresponds to the linear layer of the artificial neural network, and then the spatial spectral information of soil samples and crop samples in different states and positions are projected into the photocurrent response of the image sensor, and the full connection and nonlinear activation of the electrical signal are realized in the processor, thereby realizing a fast and precise agricultural control chip with low power consumption, large detection range and high reliability.

[0142] The optical artificial neural network precision agriculture control chip based on micro-nano modulation structure and image sensor provided in this embodiment has the following effects: A. The artificial neural network part is embedded in the image sensor containing various optical modulation layers to realize safe, reliable, fast and accurate precision agriculture control. B. Artificial neural network training and recognition can be introduced for soil, crops, etc., which is convenient for subsequent integration into industrial intelligent control systems such as drones and intelligent robots to realize precision agriculture control of large areas, and the recognition accuracy is high and the qualitative analysis is accurate. C. The preparation of the chip can be completed in one-time tape-out through CMOS process, which is conducive to reducing the device failure rate, improving the finished product yield of the device, and reducing costs. D. Realizing monolithic integration at the wafer level can minimize the distance between the sensor and the optical modulation layer, which is conducive to reducing the size of the unit, reducing the device volume and packaging cost.

[0143] Based on the contents of the above embodiments, in the present embodiment, the light modulation structure in the light modulation layer includes a regular structure and / or an irregular structure; and / or, the light modulation structure in the light modulation layer includes a discrete structure and / or a continuous structure.

[0144] In this embodiment, the light modulation structure in the light modulation layer may include only regular structures, or only irregular structures, or both regular structures and irregular structures.

[0145] In this embodiment, the optical modulation structure includes a regular structure, which may refer to: the minimum modulation unit included in the optical modulation structure is a regular structure, such as the minimum modulation unit can be a regular shape such as a rectangle, a square, and a circle. In addition, the optical modulation structure includes a regular structure, which may refer to: the arrangement of the minimum modulation unit included in the optical modulation structure is regular, such as the arrangement can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc. In addition, the optical modulation structure includes a regular structure, which may refer to: the minimum modulation unit included in the optical modulation structure is a regular structure, and the arrangement of the minimum modulation unit is also regular, etc.

[0146] In this embodiment, the light modulation structure includes an irregular structure, which may refer to: the smallest modulation unit included in the light modulation structure is an irregular structure, such as the smallest modulation unit may be an irregular polygon, a random shape, or other irregular graphics. In addition, the light modulation structure includes an irregular structure, which may refer to: the arrangement of the smallest modulation unit included in the light modulation structure is irregular, such as the arrangement may be an irregular polygon, a random arrangement, etc. In addition, the light modulation structure includes an irregular structure, which may refer to: the smallest modulation unit included in the light modulation structure is an irregular structure, and the arrangement of the smallest modulation unit is also irregular, etc.

[0147] In this embodiment, the light modulation structure in the light modulation layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0148] In this embodiment, the light modulation structure includes a continuous structure, which may mean that the light modulation structure is composed of a continuous modulation pattern; the light modulation structure includes a discrete structure, which may mean that the light modulation structure is composed of a discrete modulation pattern.

[0149] It can be understood that the continuous modulation pattern here may refer to a straight line pattern, a wavy line pattern, a broken line pattern, etc.

[0150] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0151] In this embodiment, it should be noted that the light modulation structure has different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra of light passing through different groups of filter structures are different.

[0152] Based on the contents of the above embodiments, in this embodiment, the light modulation layer is a single-layer structure or a multi-layer structure.

[0153] In this embodiment, it should be noted that the optical modulation layer may be a single-layer filter structure or a multi-layer filter structure, for example, a two-layer, three-layer, four-layer or other multi-layer structure.

[0154] In this embodiment, if Figure 1 As shown, the light modulation layer 1 is a single-layer structure, and the thickness of the light modulation layer 1 is related to the target wavelength range. For a wavelength of 400 nm to 10 μm, the thickness of the grating structure can be 50 nm to 5 μm.

[0155] It can be understood that since the function of the optical modulation layer 1 is to perform spectral modulation on the incident light, it is preferred to use a material with a high refractive index and low loss. For example, silicon, germanium, germanium silicon materials, silicon compounds, germanium compounds, III-V materials, etc. may be selected for preparation, wherein silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, etc.

[0156] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical modulation layer 1 can be set to a multi-layer structure, and the optical modulation structure corresponding to each layer can be set to a different structure, thereby increasing the optical modulation layer's ability to modulate the spectrum of incident light, so that more or more complex connection weights can be formed between the input layer and the linear layer, thereby improving the accuracy of the agricultural control chip in processing intelligent tasks.

[0157] In addition, it should be noted that, for a filter layer comprising a multi-layer structure, the material of each layer of the structure may be the same or different. For example, for a two-layer optical modulation layer 1, the first layer may be a silicon layer, and the second layer may be a silicon nitride layer.

[0158] It should be noted that the thickness of the light modulation layer 1 is related to the target wavelength range. For a wavelength of 400 nm to 10 μm, the total thickness of the multilayer structure can be 50 nm to 5 μm.

[0159] Based on the contents of the above embodiments, in this embodiment, the light modulation structure in the light modulation layer includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of each micro-nano unit are the same or different.

[0160] In this embodiment, in order to obtain connection weights distributed in an array (connection weights for connecting the input layer and the linear layer) so that the processor can perform subsequent full connection and nonlinear activation processing, preferably, in this embodiment, the light modulation structure is in the form of an array structure. Specifically, the light modulation structure includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixels on the image sensor. It should be noted that the structures of each micro-nano unit may be the same or different. In addition, it should be noted that the structure of each micro-nano unit may be periodic or non-periodic. In addition, it should be noted that each micro-nano unit may further include a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays may be the same or different.

[0161] Combine the following Figures 5 to 9 For example, in this embodiment, Figure 5 As shown, the light modulation layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, 66, each micro-nano unit has the same structure (and each micro-nano unit is a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; Figure 6 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure (similar to Figure 5 The difference is Figure 6Each micro-nano unit is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; Figure 7 As shown, the light modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure (and each of which is a periodic structure), and each of which corresponds to one or more pixels on the image sensor 2. Figure 6 The difference is Figure 7 The unit shape of the periodic array in each micro-nano unit has four-fold rotational symmetry; Figure 8 As shown, the optical modulation layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, and 66. Figure 6 The difference is that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixels on the image sensor 2. In this embodiment, the light modulation layer 1 includes a plurality of different micro-nano units, that is, different areas on the agricultural control chip have different modulation effects on the incident light, thereby increasing the degree of freedom of design and further improving the accuracy of recognition. Fig. 9 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure. Figure 5 The difference is that each micro-nano unit is composed of a discrete non-periodic array structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0162] In this embodiment, the micro-nano unit has different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra of light passing through different groups of filter structures are different.

[0163] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0164] In this embodiment, the micro-nano unit may include only regular structures, only irregular structures, or both regular structures and irregular structures.

[0165] In this embodiment, the micro-nano unit includes a regular structure, which may refer to: the minimum modulation unit included in the micro-nano unit is a regular structure, such as the minimum modulation unit can be a regular shape such as a rectangle, a square, and a circle. In addition, the micro-nano unit includes a regular structure, which may refer to: the arrangement of the minimum modulation unit included in the micro-nano unit is regular, such as the arrangement can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc. In addition, the micro-nano unit includes a regular structure, which may refer to: the minimum modulation unit included in the micro-nano unit is a regular structure, and the arrangement of the minimum modulation unit is also regular, etc.

[0166] In this embodiment, the micro-nano unit includes an irregular structure, which may refer to: the minimum modulation unit included in the micro-nano unit is an irregular structure, such as the minimum modulation unit may be an irregular polygon, a random shape, or other irregular graphics. In addition, the micro-nano unit includes an irregular structure, which may refer to: the arrangement of the minimum modulation unit included in the micro-nano unit is irregular, such as the arrangement may be an irregular polygon, a random arrangement, etc. In addition, the micro-nano unit includes an irregular structure, which may refer to: the minimum modulation unit included in the micro-nano unit is an irregular structure, and the arrangement of the minimum modulation unit is also irregular, etc.

[0167] In this embodiment, the micro-nano units in the light modulation layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0168] In this embodiment, the micro-nano unit including a continuous structure may refer to that the micro-nano unit is composed of a continuous modulation pattern; the micro-nano unit including a discrete structure may refer to that the micro-nano unit is composed of a discrete modulation pattern.

[0169] It can be understood that the continuous modulation pattern here may refer to a straight line pattern, a wavy line pattern, a broken line pattern, etc.

[0170] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0171] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano units, the corresponding transmission spectra of light passing through different groups of micro-nano units are different.

[0172] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0173] In this embodiment, if Figure 5 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, and each micro-nano unit includes a plurality of groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes four different micro-nano structure arrays 110, 111, 112, and 113, and the filter unit 44 includes four different micro-nano structure arrays 440, 441, 442, and 443. Fig.10 As shown, the optical modulation layer 1 includes multiple micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0174] It should be noted that the micro-nano unit including four groups of micro-nano structure arrays is only used as an example here, which does not serve as a limitation. In practical applications, the micro-nano unit can also be provided with six groups, eight groups or other number of groups of micro-nano structure arrays as needed.

[0175] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and each group of filter structures has different modulation effects on input light, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra of light passing through different groups of micro-nano structure arrays are different.

[0176] Based on the contents of the above embodiments, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0177] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light of the agricultural object as the connection weight of the neural network input layer and the linear layer, different micro-nano structure arrays are used to achieve broadband filtering and narrowband filtering. Therefore, in this embodiment, the micro-nano structure array performs broadband filtering or narrowband filtering on the incident light of the agricultural object to obtain the modulation intensity of different wavelength components of the incident light of the agricultural object. Fig.11 and Fig.12 As shown, each group of micro-nanostructure arrays in the light modulation layer has the function of broadband filtering or narrowband filtering.

[0178] It is understandable that, for each group of micro-nano structure arrays, they may all have broadband filtering effects, or they may all have narrowband filtering effects, or they may partially have broadband filtering effects and partially have narrowband filtering effects. In addition, the broadband filtering range and narrowband filtering range of each group of micro-nano structure arrays may also be the same or different. For example, by designing the period, duty cycle, radius, side length and other dimensional parameters of each group of micro-nano structures in the micro-nano unit, it has a narrowband filtering effect, that is, only one (or fewer) wavelengths of light can pass through. For another example, by designing the period, duty cycle, radius, side length and other dimensional parameters of each group of micro-nano structures in the micro-nano unit, it has a broadband filtering effect, that is, allowing more wavelengths or all wavelengths of light to pass through.

[0179] It is understandable that, in specific use, the filtering state of each group of micro-nano structure arrays can be determined by performing broadband filtering, narrowband filtering or a combination thereof according to the application scenario.

[0180] Based on the contents of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0181] In this embodiment, each group of micro-nano structure arrays can be periodic structure arrays, non-periodic structure arrays, or partly periodic structure arrays and partly non-periodic structure arrays. Among them, the periodic structure array is easy to perform optical simulation design, and the non-periodic structure array can achieve more complex modulation effects.

[0182] In this embodiment, if Figure 5 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The structures of the micro-nano structure arrays are different from each other, and the micro-nano structure arrays are non-periodic structures. The non-periodic structure refers to that the modulation hole shapes on the micro-nano structure array are arranged in a non-periodic arrangement. Figure 5 As shown, the micro-nano unit 11 includes four different non-periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different non-periodic structure arrays 440, 441, 442 and 443. The micro-nano structure arrays with non-periodic structures are designed by neural network data training for intelligent processing tasks in the early stage, and are usually irregularly shaped structures. Figure 6 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The structures of the micro-nano structure arrays are different from each other. Figure 5The difference is that the micro-nano structure array is a periodic structure. The periodic structure refers to the arrangement of the modulated holes on the micro-nano structure array in a periodic arrangement, and the period size is usually 20nm to 50μm. Figure 6 As shown, the micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442 and 443. The filter structure of the periodic structure is obtained by training and designing the neural network data for intelligent processing tasks in the early stage, and is usually an irregular structure. Figure 7 As shown, the optical modulation layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures. The periodic structure refers to the arrangement of the shapes on the filter structure in a periodic arrangement, and the size of the period is usually 20nm to 50μm. Figure 7 As shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different. The micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442 and 443. The micro-nano structure arrays with periodic structures are obtained through early training and design of neural network data for intelligent processing tasks, and are usually irregularly shaped structures.

[0183] It should be noted that Figures 5 to 9 Each micro-nano unit includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are formed by modulation holes of four different shapes, and the four groups of micro-nano structure arrays are used to have different modulation effects on the incident light. It should be noted that here, only the micro-nano unit including four groups of micro-nano structure arrays is used as an example, which does not play a limiting role. In practical applications, micro-nano units including six groups, eight groups or other number groups of micro-nano structure arrays can also be set as needed. In this embodiment, the four different shapes can be circular, cross, regular polygon and rectangular (not limited to this).

[0184] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and each group of micro-nano structure arrays has different modulation effects on input light, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra of light passing through different groups of micro-nano structure arrays are different.

[0185] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes a plurality of micro-nano structure arrays, and there is one or more groups of empty structures.

[0186] Combine the following Fig. 9 The example shown is used for illustration. In this embodiment, Fig. 9 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which is composed of a plurality of micro-nano structure arrays, the structures corresponding to the plurality of micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures. Different from the above-mentioned embodiments, any micro-nano unit includes one or more groups of empty structures, and the empty structures are used to pass incident light directly. It can be understood that when a plurality of groups of micro-nano structure arrays include one or more groups of empty structures, a richer spectrum modulation effect can be formed, thereby meeting the spectrum modulation requirements in a specific scenario (or meeting the specific connection weight requirements between the input layer and the linear layer in a specific scenario).

[0187] like Fig. 9 As shown, each micro-nano unit includes a group of micro-nano structure arrays and three groups of empty structures, micro-nano unit 11 includes a non-periodic structure array 111, micro-nano unit 22 includes a non-periodic structure array 221, micro-nano unit 33 includes a non-periodic structure array 331, micro-nano unit 44 includes a non-periodic structure array 441, micro-nano unit 55 includes a non-periodic structure array 551, and micro-nano unit 66 includes a non-periodic structure array 661, wherein the micro-nano structure array is used to modulate the incident light in different ways. It should be noted that here, only one group of micro-nano structure arrays and three groups of empty structures are used as an example, which does not play a limiting role. In practical applications, a micro-nano unit including a group of micro-nano structure arrays and five groups of empty structures or other number groups of micro-nano structure arrays can also be set according to needs. In this embodiment, the micro-nano structure array can be made of circular, cross-shaped, regular polygonal and rectangular modulation holes (not limited to this).

[0188] It should be noted that the multiple groups of micro-nano structure arrays included in the micro-nano unit may not include any empty structures, that is, the multiple groups of micro-nano structure arrays may be non-periodic structure arrays or periodic structure arrays.

[0189] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit has polarization-independent characteristics.

[0190] In this embodiment, since the micro-nano unit has polarization-independent characteristics, the optical modulation layer is insensitive to the polarization of the incident light, thereby realizing an agricultural control chip that is insensitive to both the incident angle and polarization. The agricultural control chip provided by the embodiment of the present invention is insensitive to the incident angle and polarization characteristics of the incident light, that is, the measurement result will not be affected by the incident angle and polarization characteristics of the incident light, thereby ensuring the stability of the spectral measurement performance, and further ensuring the stability of intelligent processing, such as the stability of intelligent perception, the stability of intelligent identification, the stability of intelligent decision-making, etc. It should be noted that the micro-nano unit may also have polarization-related characteristics.

[0191] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0192] In this embodiment, it should be noted that four-fold rotational symmetry is a specific case of polarization-independent characteristics. By designing the micro-nano unit to have a structure with four-fold rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0193] Combine the following Figure 7 The example shown is used for illustration. In this embodiment, Figure 7 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which is composed of a plurality of groups of micro-nano structure arrays. The structures corresponding to the plurality of groups of micro-nano structure arrays are different from each other. The micro-nano structure arrays are periodic structures. Different from the above-mentioned embodiments, the structure corresponding to each group of micro-nano structure arrays can be a circle, a cross, a regular polygon, a rectangle, and the like having four-fold rotational symmetry, that is, after the structure is rotated 90°, 180°, and 270°, it overlaps with the original structure, so that the structure has the characteristic of being independent of polarization, so that the same intelligent recognition effect can be achieved when different polarized light is incident.

[0194] Based on the contents of the above embodiments, in this embodiment, the optical modulation layer is composed of one or more filter layers;

[0195] The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is a filter layer made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP (Surface Plasmon Polaritons, SPP) micro-nano structures, and adjustable Fabry-Perot cavity (Fabry-perot Cavity, FP cavity).

[0196] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

[0197] Among them, photonic crystals, as well as the combination of metasurfaces and random structures are compatible with CMOS technology and can have better modulation effects. Other materials can also be filled in the micropores of the micro-nano modulation structure to smooth the surface. Quantum dots and perovskites can utilize the spectral modulation characteristics of the material itself to minimize the volume of a single modulation structure. SPP has a small volume and can achieve polarization-dependent light modulation. Liquid crystals can be dynamically controlled by voltage to improve spatial resolution. Adjustable Fabry-Perot cavities can be dynamically controlled to improve spatial resolution.

[0198] Based on the content of the above embodiment, in this embodiment, the thickness of the light modulation layer is 0.1λ-10λ, where λ represents the central wavelength of the incident light.

[0199] In this embodiment, it should be noted that if the thickness of the optical modulation layer is much smaller than the central wavelength of the incident light, it cannot play an effective spectrum modulation role; if the thickness of the optical modulation layer is much larger than the central wavelength of the incident light, it is difficult to prepare in terms of process, and will introduce large optical losses. Therefore, in this embodiment, in order to reduce optical losses, facilitate preparation, and ensure effective spectrum modulation, the overall size (area) of each micro-nano unit in the optical modulation layer 1 is usually λ 2 ~10 5 λ 2 , the thickness is usually 0.1λ~10λ (λ represents the central wavelength of the incident light of the agricultural object). Figure 5 As shown, the overall size of each micro-nano unit is 0.5 μm 2 ~40000μm 2 The dielectric material in the light modulation layer 1 is polysilicon with a thickness of 50 nm to 2 μm.

[0200] Based on the contents of the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0201] CMOS image sensor (Contact Image Sensor, CIS), Charge Coupled Device (Charge Coupled Device, CCD), Single Photon Avalanche Diode (Single Photon Avalanche Diode, SPAD) array and focal plane photoelectric image sensor array.

[0202] In this embodiment, it should be noted that the use of a wafer-level CMOS image sensor CIS and monolithic integration at the wafer level can minimize the distance between the image sensor and the light modulation layer, which is beneficial for reducing the size of the unit, reducing the device volume and packaging cost. SPAD can be used for weak light detection, and CCD can be used for strong light detection.

[0203] In this embodiment, the light modulation layer and the image sensor can be manufactured by a complementary metal oxide semiconductor (CMOS) integrated process, which is conducive to reducing device failure rate, improving device yield and reducing cost. For example, the light modulation layer can be prepared by directly growing one or more layers of dielectric material on the image sensor, then etching, depositing metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer.

[0204] Based on the contents of the above embodiments, in this embodiment, the types of the artificial neural network include: feedforward neural network.

[0205] In this embodiment, the feedforward neural network (FNN), also known as the deep feedforward network (DFN) and the multi-layer perceptron (MLP), is the simplest neural network, in which neurons are arranged in layers. Each neuron is only connected to the neurons in the previous layer. It receives the output of the previous layer and outputs it to the next layer, and there is no feedback between the layers. The feedforward neural network has a simple structure, is easy to implement on hardware, and is widely used. It can approximate any continuous function and square integrable function with arbitrary precision, and can accurately implement any finite training sample set. The feedforward network is a static nonlinear mapping. Complex nonlinear processing capabilities can be obtained through the composite mapping of simple nonlinear processing units.

[0206] Based on the contents of the above embodiments, in this embodiment, a light-transmitting medium layer is disposed between the light modulation layer and the image sensor.

[0207] In this embodiment, it should be noted that a light-transmitting medium layer is provided between the light modulation layer and the image sensor, so as to effectively separate the light modulation layer from the image sensor layer and avoid mutual interference between the two.

[0208] Based on the contents of the above embodiments, in this embodiment, the image sensor is front-illuminated, comprising: a metal wire layer and a light detection layer arranged from top to bottom, and the light modulation layer is integrated on a side of the metal wire layer away from the light detection layer; or,

[0209] The image sensor is back-illuminated, and comprises: a light detection layer and a metal wire layer arranged from top to bottom, and the light modulation layer is integrated on a side of the light detection layer away from the metal wire layer.

[0210] In this embodiment, if Fig.13 The image sensor shown is a front-illuminated image sensor, in which the silicon detection layer 21 is below the metal wire layer 22 , and the light modulation layer 1 is directly integrated onto the metal wire layer 22 .

[0211] In this embodiment, Fig.13 The difference is, Fig.14 The image sensor shown is a back-illuminated one, in which the silicon detection layer 21 is above the metal wire layer 22 , and the light modulation layer 1 is directly integrated onto the silicon detection layer 21 .

[0212] It should be noted that, for a back-illuminated image sensor, the silicon detection layer 21 is above the metal wire layer 22, which can reduce the influence of the metal wire layer on the incident light, thereby improving the quantum efficiency of the device.

[0213] According to the above content, it can be seen that the light modulation layer and image sensor in the agricultural control chip provided in this embodiment realize the relevant functions of the input layer and the linear layer in the artificial neural network through hardware, so that when the agricultural control chip is used for intelligent processing in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, which can greatly reduce the power consumption and delay during artificial neural network processing.

[0214] The embodiments of the present invention can also realize monolithic integration at the wafer level, thereby minimizing the distance between the sensor and the light modulation layer, which is beneficial to reducing the size of the unit, the device volume and the packaging cost.

[0215] Based on the same inventive concept, another embodiment of the present invention provides an intelligent agricultural control device, including: the agricultural control chip as described in the above embodiment. The intelligent agricultural control device may include fertilizer applicator equipment, pesticide spreader equipment, pesticide resistance analysis equipment, drone equipment, agricultural intelligent robot equipment, crop health analysis equipment, crop growth status monitoring equipment, etc.

[0216] Since the intelligent agricultural control device provided in this embodiment includes the agricultural control chip described in the above embodiment, the intelligent agricultural control device provided in this embodiment has all the beneficial effects of the agricultural control chip described in the above embodiment. Since the above embodiment has already described this in detail, this embodiment will not repeat it.

[0217] Based on the same inventive concept, another embodiment of the present invention provides a method for preparing the agricultural control chip as described in the above embodiment, such as Fig.15As shown, the specific steps include:

[0218] Step 1510: preparing a light modulation layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor;

[0219] Step 1520, generating a processor capable of performing full connection processing and nonlinear activation processing on the signal;

[0220] Step 1530: connecting the image sensor and the processor;

[0221] The light modulation layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light carrying information includes light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light;

[0222] The image sensor is used to convert the information carried by the incident light corresponding to different position points after modulation by the light modulation layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different position points to obtain agricultural control processing results.

[0223] In this embodiment, the training process of the optical artificial neural agricultural control chip is also included, which specifically includes:

[0224] Using input training samples and output training samples corresponding to the agricultural control intelligent processing task, an agricultural control chip including different light modulation structures, image sensors, and processors with different fully connected parameters and nonlinear activation parameters is trained to obtain light modulation structures, image sensors, and processors that meet the training convergence conditions, and the light modulation structures, image sensors, and processors that meet the training convergence conditions are used as the trained light modulation structures, image sensors, and processors;

[0225] It can be understood that when training an optical artificial neural agricultural control chip comprising different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation functions, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0226] It is understandable that a large number of soil samples and crop samples in different states and positions can be collected in advance and data training can be carried out, and then the required optical modulation structure (that is, the micro-nano modulation structure in the optical modulation layer) can be designed and prepared, so as to realize the input layer and linear layer of the artificial neural network in hardware on the chip.

[0227] In this embodiment, a light modulation layer including a light modulation structure is prepared on the surface of the photosensitive area of ​​the image sensor, including:

[0228] Growing one or more layers of a preset material on the surface of the image sensor;

[0229] Dry-etching the one or more layers of the preset material to form a light modulation structure pattern to obtain a light modulation layer including the light modulation structure;

[0230] or performing embossing transfer on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0231] Or by externally dynamically regulating the one or more layers of the preset material, a light modulation layer including a light modulation structure is obtained;

[0232] or performing zone printing on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0233] or performing partitioned material growth on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0234] Or quantum dots are transferred to the one or more layers of the preset materials to obtain a light modulation layer including a light modulation structure.

[0235] When the agricultural control chip is used for intelligent processing tasks of agricultural objects, the agricultural control chip including different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters is trained using input training samples and output training samples corresponding to the intelligent processing tasks to obtain light modulation structures, image sensors and processors that meet the training convergence conditions.

[0236] In this embodiment, it should be noted that Figure 1 As shown, the light modulation layer 1 can be prepared by directly growing one or more layers of dielectric material on the image sensor 2, then etching, depositing metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the light modulation structure, each unit can have different modulation effects on light of different wavelengths within the target range, and the modulation effect is insensitive to the incident angle and polarization. Each unit in the light modulation layer 1 corresponds to one or more pixels on the image sensor 2. 1 is prepared directly on 2.

[0237] In this embodiment, it should be noted that Fig.14As shown, assuming that the image sensor 2 is a back-illuminated structure, the light modulation layer 1 can be prepared by directly etching the silicon image sensor layer 21 of the back-illuminated image sensor and then depositing metal.

[0238] In addition, it should be noted that the light modulation structure on the light modulation layer can be obtained by dry etching the light modulation structure pattern of one or more layers of preset materials. Dry etching is to directly remove the unnecessary parts of one or more layers of preset materials on the surface of the photosensitive area of ​​the image sensor to obtain a light modulation layer containing a light modulation structure; or one or more layers of preset materials are subjected to imprint transfer. Imprint transfer is to prepare the required structure on other substrates by etching, and then transfer the structure to the photosensitive area of ​​the image sensor through materials such as PDMS to obtain a light modulation layer containing a light modulation structure; or by externally dynamically regulating one or more layers of preset materials. The external dynamic regulation is to use active materials, and then the external electrodes regulate the light modulation characteristics of the corresponding areas by changing the voltage to obtain a light modulation layer containing a light modulation structure; or one or more layers of preset materials are subjected to zone printing. Zone printing is to use printing technology in zones to obtain a light modulation layer containing a light modulation structure; or one or more layers of preset materials are subjected to zone material growth to obtain a light modulation layer containing a light modulation structure; or one or more layers of preset materials are subjected to quantum dot transfer to obtain a light modulation layer containing a light modulation structure.

[0239] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the agricultural control chip in the above embodiment, for detailed contents on some principles and structures, etc., please refer to the introduction of the above embodiment, and this embodiment will not go into details.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An agricultural control chip, characterized in that: Used for agricultural control intelligent processing tasks, including: a light modulation layer and an image sensor, the image sensor is connected to a processor; the light modulation layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network; The light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure, and the light modulation structure is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area; the image sensor is used to convert the incident light carrying information corresponding to the different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signal is an image signal modulated by the light modulation layer; the incident light includes reflected light, transmitted light and / or radiated light of an agricultural object; the agricultural object includes crops and / or soil; The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain agricultural control processing results; Among them, the agricultural control intelligent processing tasks include: one or more of soil fertility detection, pesticide spreading detection, trace element content detection, drug resistance detection and crop growth condition detection; the agricultural control processing results include: one or more of soil fertility detection results, pesticide spreading detection results, trace element content detection results, drug resistance detection results and crop growth condition detection results.

2. The agricultural control chip according to claim 1, characterized in that: The information carried by the incident light includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

3. The agricultural control chip according to claim 1, characterized in that: The agricultural control chip includes a trained light modulation structure and an image sensor, and the processor is a trained processor; The trained light modulation structure, image sensor and processor refer to the light modulation structure, image sensor and processor that meet the training convergence conditions by training agricultural control chips including different light modulation structures and image sensors and processors with different full connection parameters and nonlinear activation parameters using input training samples and output training samples corresponding to the agricultural control intelligent processing task; Wherein, the input training samples include incident light reflected, transmitted and / or radiated by soils with different fertility; the output training samples include the corresponding soil fertility; and / or, The input training samples include incident light reflected, transmitted and / or radiated by soil with different pesticide spreading conditions; the output training samples include corresponding pesticide spreading conditions; and / or, The input training samples include incident light reflected, transmitted and / or radiated by soils having different trace element content conditions; the output training samples include corresponding trace element content conditions; and / or, The input training samples include incident light reflected, transmitted and / or radiated by soils with different pesticide resistances; the output training samples include corresponding pesticide resistances; and / or, The input training samples include incident light reflected, transmitted and / or radiated by crops with different growth conditions; the output training samples include corresponding crop growth conditions.

4. The agricultural control chip according to claim 3, characterized in that: When training an agricultural control chip including different light modulation structures and image sensors and a processor with different fully connected parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

5. The agricultural control chip according to any one of claims 1 to 4, characterized in that: The light modulation structure in the light modulation layer includes a regular structure and / or an irregular structure; and / or, the light modulation structure in the light modulation layer includes a discrete structure and / or a continuous structure.

6. The agricultural control chip according to any one of claims 1 to 4, characterized in that: The light modulation layer is a single-layer structure or a multi-layer structure.

7. The agricultural control chip according to any one of claims 1 to 4, characterized in that: The light modulation structure in the light modulation layer includes a unit array composed of a plurality of micro-nano units, each of which corresponds to one or more pixel points on the image sensor; and the structures of the various micro-nano units are the same or different.

8. The agricultural control chip according to claim 7, characterized in that: The micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

9. The agricultural control chip according to claim 7, characterized in that: The micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

10. The agricultural control chip according to claim 9, characterized in that: Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

11. The agricultural control chip according to claim 9, characterized in that: Each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

12. The agricultural control chip according to claim 9, characterized in that: The micro-nano unit includes a plurality of micro-nano structure arrays, and there is one or more groups of empty structures.

13. The agricultural control chip according to claim 9, characterized in that: The micro-nano unit has four-fold rotational symmetry.

14. The agricultural control chip according to claim 1, characterized in that: The light modulation layer is composed of one or more filter layers; The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, and adjustable Fabry-Perot resonant cavities.

15. The agricultural control chip according to claim 14, characterized in that: The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

16. The agricultural control chip according to claim 1, characterized in that: The thickness of the light modulation layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

17. An intelligent agricultural control device, characterized in that: It comprises the agricultural control chip as described in any one of claims 1 to 16.

18. A method for preparing an agricultural control chip according to any one of claims 1 to 16, characterized in that: include: Preparing a light modulation layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor; Generate a processor capable of fully connected signal processing and nonlinear activation processing; connecting the image sensor and the processor; The light modulation layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area; The image sensor is used to convert the information carried by the incident light corresponding to different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain agricultural control processing results.

19. The method for preparing the agricultural control chip according to claim 18, characterized in that: Also includes: The training process of the agricultural control chip specifically includes: Using input training samples and output training samples corresponding to the agricultural control intelligent processing task, agricultural control chips containing different light modulation structures and image sensors and processors with different fully connected parameters and nonlinear activation parameters are trained to obtain light modulation structures, image sensors and processors that meet the training convergence conditions, and the light modulation structures, image sensors and processors that meet the training convergence conditions are used as trained light modulation structures, image sensors and processors.

20. The method for preparing the agricultural control chip according to claim 18, characterized in that: A light modulation layer including a light modulation structure is prepared on the surface of the image sensor, comprising: Growing one or more layers of a preset material on the surface of the image sensor; Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light modulation layer including the light modulation structure; or performing embossing transfer on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure; or obtaining a light modulation layer including a light modulation structure by externally dynamically modulating the one or more layers of the preset material; or performing zone printing on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure; or performing partitioned growth on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure; Alternatively, quantum dots are transferred to the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure.

Images