Optical artificial neural network blood glucose detection chip and preparation method

Abstract

This invention provides an optical artificial neural network blood glucose detection chip and its fabrication method for blood glucose detection tasks. The invention embeds an artificial neural network on a hardware chip, using the optical modulation layer on the hardware chip as the input and linear layers of the artificial neural network. The filtering effect of the optical modulation layer on the incident light is used as the connection weight from the input layer to the linear layer. The square detection response of the image sensor on the hardware chip is used as the first nonlinear activation function in the nonlinear layer of the artificial neural network. This invention projects image information, spectral information, incident light angle, and incident light phase information at different points in the blood space onto the photocurrent response of the hardware chip. The processor implements full connectivity of the electrical signal, the second nonlinear activation function in the nonlinear layer, and the output layer, thereby achieving low-power, safe, reliable, fast, accurate, and non-invasive blood glucose detection.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an optical artificial neural network blood glucose detection chip and its preparation method. Background Technology

[0002] Diabetes mellitus (DM) is a multifactorial metabolic disease caused by insufficient insulin secretion or impaired insulin utilization. It is characterized by chronic hyperglycemia, accompanied by disturbances in carbohydrate, fat, and protein metabolism, and can lead to a series of serious complications. As one of the major diseases threatening human health, there is currently no cure for diabetes. According to relevant data, there were 425 million people with diabetes worldwide at the end of 2017, and this number is projected to rise to 629 million by 2045. Diabetes not only significantly reduces the quality of life for patients but also increases the global economic burden; therefore, research on the prevention and treatment of diabetes is urgently needed.

[0003] With the continuous improvement of living standards, diabetes has become the third leading cause of disease worldwide, seriously affecting human health. To effectively monitor blood glucose levels in diabetic patients, portable home blood glucose meters and test strips have become widely used. Currently, blood glucose testing on the market mainly involves invasive or minimally invasive methods requiring blood sample collection, causing significant pain to patients and posing a risk of viral infection through blood. Therefore, non-invasive blood glucose testing has become an urgent need for the treatment of diabetes.

[0004] Currently, non-invasive blood glucose testing methods mainly include optical and radiation methods, reverse iontophoresis analysis, electromagnetic wave methods, ultrasound methods, and tissue fluid extraction methods. For example, near-infrared spectroscopy mainly utilizes the relationship between blood glucose concentration and its absorption in the near-infrared spectrum. Near-infrared light is irradiated onto the skin, and changes in the intensity of reflected light reflect the blood glucose concentration. This method has advantages such as rapid measurement and no need for chemical reagents or consumables. However, due to significant individual differences among the subjects and the very weak signal obtained, key technologies such as the selection of measurement sites, measurement conditions, and methods for extracting weak chemical information from overlapping spectra still need further development. Furthermore, the signal processing system is bulky and not portable. Another method is subcutaneous tissue fluid detection, which reflects blood glucose concentration by measuring the glucose concentration of subcutaneous exudate. Based on this principle, a glucose-detecting watch can be made to continuously monitor blood glucose concentration in real time. However, this method has poor accuracy and slow response speed, making it difficult to replace existing invasive blood glucose meters. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an optical artificial neural network blood glucose detection chip and its fabrication method.

[0006] Specifically, the embodiments of the present invention provide the following technical solutions:

[0007] In a first aspect, embodiments of the present invention provide an optical artificial neural network blood glucose detection chip for blood glucose detection tasks, comprising: an optical modulation layer, an image sensor, and a processor; the optical modulation layer corresponds to the input layer, the linear layer, and the connection weights from the input layer to the linear layer of the artificial neural network; the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor corresponds to the fully connected layer and the output layer of the artificial neural network, or the processor corresponds to the fully connected layer, the second nonlinear activation function in the nonlinear layer, and the output layer of the artificial neural network;

[0008] The light modulation layer is disposed on the surface of the image sensor. The light modulation layer includes a light modulation structure. The light modulation structure is used to perform different spectral modulations on the incident light reflected and / or transmitted through the human body part to different positions of the light modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the image sensor. The human body part to be tested is a part that has blood glucose information.

[0009] The image sensor converts the incident light carrying information corresponding to different position points after modulation by the optical modulation layer into electrical signals corresponding to different position points through a square detection response. The electrical signals corresponding to different position points are then sent to the processor. The processor performs fully connected processing on the electrical signals corresponding to different position points to obtain the blood glucose detection result. Alternatively, the processor performs fully connected processing and a second nonlinear activation processing on the electrical signals corresponding to different position points to obtain the blood glucose detection result.

[0010] Furthermore, the incident light carries information including at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0011] Furthermore, the optical artificial neural network blood glucose detection chip includes a trained optical modulation structure, an image sensor, and a processor;

[0012] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip that satisfies the training convergence condition by training it with input training samples and output training samples corresponding to the blood glucose detection task, including different optical modulation structures, image sensors, and processors with different fully connected parameters; or, the trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip that satisfies the training convergence condition by training it with input training samples and output training samples corresponding to the blood glucose detection task, including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters.

[0013] The input training samples include incident light reflected or transmitted from the human body test sites with different blood glucose values; the output training samples include the corresponding blood glucose values.

[0014] Furthermore, when training an optical artificial neural network blood glucose detection chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network blood glucose detection chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters, the different optical modulation structures are designed and implemented using computer optical simulation design.

[0015] Furthermore, the light source used to irradiate the part of the human body to be tested is a near-infrared light source.

[0016] Furthermore, the optical modulation structure in the optical modulation layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical modulation layer includes a discrete structure and / or a continuous structure.

[0017] Furthermore, the optical modulation structure in the optical modulation layer comprises a unit array composed of multiple micro-nano units, each micro-nano unit corresponding to one or more pixels on the image sensor; the structures of each micro-nano unit may be the same or different.

[0018] Furthermore, the micro / nano unit comprises a regular structure and / or an irregular structure; and / or, the micro / nano unit comprises a discrete structure and / or a continuous structure.

[0019] Furthermore, the micro-nano unit comprises multiple sets of micro-nano structure arrays, and the structures of each set of micro-nano structure arrays may be the same or different.

[0020] Furthermore, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0021] Furthermore, each group of micro / nano structure arrays is either a periodic structure array or an aperiodic structure array.

[0022] Furthermore, the micro-nano unit contains one or more sets of empty structures among the multiple micro-nano structure arrays.

[0023] Furthermore, the micro / nano unit has fourfold rotational symmetry.

[0024] Furthermore, the optical modulation layer is composed of one or more filter layers;

[0025] The filter layer is prepared from one or more of semiconductor materials, metallic materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, surface plasmon polariton (SPP) micro / nano structures, and tunable Fabry-Perot resonators.

[0026] Furthermore, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanopillar two-dimensional materials, and nanowire two-dimensional materials.

[0027] Furthermore, the thickness of the optical modulation layer is 0.1λ to 10λ, where λ represents the center wavelength of the incident light.

[0028] Secondly, embodiments of the present invention provide an intelligent blood glucose meter, including the optical artificial neural network blood glucose detection chip as described above.

[0029] Secondly, embodiments of the present invention provide a method for fabricating a blood glucose detection chip using an optical artificial neural network as described above, comprising:

[0030] An optical modulation layer containing an optical modulation structure is fabricated on the surface of the image sensor;

[0031] Generate a processor capable of performing fully connected signal processing or a processor capable of performing fully connected signal processing and a second nonlinear activation processing.

[0032] Connect the image sensor and the processor;

[0033] The light modulation layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the image sensor; the incident light carrying information includes light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0034] The image sensor converts the information carried by the incident light, which is modulated by the optical modulation layer at different locations, into electrical signals corresponding to different locations after the first nonlinear activation processing through the square detection response, and sends the electrical signals corresponding to different locations to the processor.

[0035] The processor performs fully connected processing on the electrical signals corresponding to different location points to obtain blood glucose detection results; or, the processor performs fully connected processing on the electrical signals corresponding to different location points and a second nonlinear activation processing to obtain blood glucose detection results.

[0036] Furthermore, it also includes: the training process of the optical artificial neural network blood glucose detection chip, specifically including:

[0037] Using the input training samples and output training samples corresponding to the blood glucose detection task, an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors and processors with different fully connected parameters is trained to obtain optical modulation structures, image sensors and processors that meet the training convergence conditions, and the optical modulation structures, image sensors and processors that meet the training convergence conditions are used as trained optical modulation structures, image sensors and processors.

[0038] Alternatively, using the input and output training samples corresponding to the blood glucose detection task, an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters can be trained to obtain optical modulation structures, image sensors, and processors that meet the training convergence conditions, and the optical modulation structures, image sensors, and processors that meet the training convergence conditions can be used as trained optical modulation structures, image sensors, and processors.

[0039] Further, an optical modulation layer comprising an optical modulation structure is fabricated on the surface of the image sensor, including:

[0040] One or more layers of a predetermined material are grown on the surface of the image sensor;

[0041] Etching the optical modulation structure pattern on one or more layers of the preset material yields an optical modulation layer containing the optical modulation structure.

[0042] Alternatively, one or more layers of the preset material can be imprinted and transferred to obtain an optical modulation layer containing an optical modulation structure;

[0043] Alternatively, by applying external dynamic modulation to one or more preset materials, an optical modulation layer containing an optical modulation structure can be obtained;

[0044] Alternatively, one or more layers of the preset material can be printed in sections to obtain a light modulation layer containing a light modulation structure;

[0045] Alternatively, one or more layers of the preset material can be grown in sections to obtain a light modulation layer containing a light modulation structure;

[0046] Alternatively, quantum dot transfer can be performed on one or more layers of the preset material to obtain an optical modulation layer containing an optical modulation structure.

[0047] This invention implements a novel optical artificial neural network (ALN) blood glucose detection chip capable of performing artificial neural network functions for blood glucose detection tasks. This invention embeds an ANN on a hardware chip, using the optical modulation layer on the hardware chip as the input and linear layers of the ANN. The filtering effect of the optical modulation layer on the incident light is used as the connection weight from the input layer to the linear layer. The squared detection response of the image sensor on the hardware chip is used as the first nonlinear activation function in the nonlinear layer of the ANN. This invention transmits incident light carrying blood glucose information from different points in the blood sample area of ​​the human body to the pre-trained hardware chip. The hardware chip analyzes the information carried by the incident light from the blood sample area to the pre-trained hardware chip using the ANN to obtain the blood glucose detection result. It should be noted that this invention achieves low-power, safe, reliable, fast, accurate, and non-invasive blood glucose detection, thus well meeting users' needs for accurate non-invasive blood glucose detection.

[0048] It is understood that in this optical artificial neural network blood glucose detection chip, the optical modulation layer corresponds to the input layer and linear layer of the artificial neural network, and the image sensor corresponds to a part of the nonlinear layer of the artificial neural network; the processor corresponds to another part of the nonlinear layer and the output layer of the artificial neural network. Specifically, the optical modulation layer is disposed on the surface of the image sensor, and the optical modulation layer includes an optical modulation structure. The optical modulation structure is used to perform different spectral modulations on the incident light entering different positions of the optical modulation structure, so as to obtain the incident light carrying information corresponding to different positions on the surface of the image sensor. In this embodiment of the invention, the modulation effect of the optical modulation structure on the incident light is equivalent to the connection weight from the input layer to the linear layer. Meanwhile, in this embodiment of the invention, the image sensor converts the incident light carrying information corresponding to different position points after modulation by the optical modulation layer into electrical signals corresponding to different position points after a first nonlinear activation process via the square detection response. These electrical signals are then sent to the processor. The processor performs fully connected processing on the electrical signals corresponding to different position points, or performs fully connected processing and a second nonlinear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. Therefore, in this optical artificial neural network blood glucose detection chip, the optical modulation layer corresponds to the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network; the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; and the processor corresponds to the fully connected layer and output layer of the artificial neural network, or the processor corresponds to the fully connected layer and the second nonlinear activation function in the nonlinear layer of the artificial neural network. The nonlinear activation function and output layer, namely the optical modulation layer and image sensor in this optical artificial neural network blood glucose detection chip, realize the related functions of the input layer, linear layer, and some nonlinear activation functions in the artificial neural network. In other words, the embodiments of the present invention strip away the input layer, linear layer, and some or all of the nonlinear activation functions in the artificial neural network implemented in software in the prior art, and realize these structures in hardware. As a result, when using this optical artificial neural network blood glucose detection chip for artificial neural network intelligent processing, it is not necessary to perform complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some or all of the nonlinear activation functions. Only the processor in the optical artificial neural network blood glucose detection chip needs to perform fully connected processing or fully connected processing of electrical signals and a second nonlinear activation processing. This can significantly reduce the power consumption and latency of artificial neural network processing.Therefore, in this embodiment of the invention, the optical modulation layer is used as the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network; the square detection response of the image sensor is used as the first nonlinear activation function in the nonlinear layer of the artificial neural network; and the processor is used as the fully connected layer and output layer of the artificial neural network, or the processor is used as the fully connected layer, the second nonlinear activation function in the nonlinear layer, and the output layer of the artificial neural network. Thus, this embodiment of the invention can not only eliminate the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some nonlinear activation functions in the prior art, but also significantly reduce the power consumption and latency of artificial neural network processing.

[0049] Furthermore, it is understood that the embodiments of the present invention can utilize the information carried by the incident light of the blood at the site to be detected, such as one or more of image information, spectral information, incident light angle, and incident light phase information. Since the incident light carrying information at different points in the space of the blood at the site to be detected covers the image, composition, shape, three-dimensional depth, and structure of the blood at the site to be detected, when performing identification processing based on the incident light carrying information at different points in the space of the blood at the site to be detected, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the blood at the site to be detected. This can solve the problem mentioned in the background section that it is difficult to accurately achieve non-invasive blood glucose detection. Therefore, the optical artificial neural network blood glucose detection chip provided by the embodiments of the present invention can simultaneously meet the effects of low power consumption, low latency, and high recognition rate, thereby enabling fast and accurate non-invasive detection of blood glucose values.

[0050] Therefore, the embodiments of the present invention provide a novel optoelectronic chip for accurate non-invasive detection of blood glucose levels. This chip embeds an artificial neural network portion into an image sensor containing various light modulation layers, thereby achieving safe, reliable, fast, and accurate non-invasive blood glucose detection. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the structure of the optical artificial neural network blood glucose detection chip provided in the first embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the recognition principle of an optical artificial neural network blood glucose detection chip provided in an embodiment of the present invention;

[0054] Figure 3 This is a disassembly diagram of an optical artificial neural network blood glucose detection chip provided in an embodiment of the present invention;

[0055] Figure 4a This is a schematic diagram of an embodiment of the present invention, which identifies the light transmitted through a user's finger to detect blood glucose levels;

[0056] Figure 4b This is a schematic diagram of an embodiment of the present invention, which identifies light reflected from a user's wrist to detect blood glucose levels;

[0057] Figure 5 This is a top view of an optical modulation layer provided in an embodiment of the present invention;

[0058] Figure 6 This is a top view of another optical modulation layer provided in an embodiment of the present invention;

[0059] Figure 7 This is a top view of another optical modulation layer provided in an embodiment of the present invention;

[0060] Figure 8 This is a top view of yet another optical modulation layer provided in an embodiment of the present invention;

[0061] Figure 9 This is a top view of yet another optical modulation layer provided in an embodiment of the present invention;

[0062] Figure 10 This is a top view of another optical modulation layer provided in one embodiment of the present invention;

[0063] Figure 11 This is a schematic diagram of the broadband filtering effect of a micro / nano structure provided in an embodiment of the present invention;

[0064] Figure 12 This is a schematic diagram of the narrowband filtering effect of a micro / nano structure provided in an embodiment of the present invention;

[0065] Figure 13 This is a schematic diagram of a front-illuminated image sensor structure provided in an embodiment of the present invention;

[0066] Figure 14 This is a schematic diagram of a rear-illuminated image sensor structure provided in an embodiment of the present invention;

[0067] Figure 15 This is a schematic flowchart of the fabrication method of the optical artificial neural network blood glucose detection chip provided in the third embodiment of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] like Figure 1 As shown, the first embodiment of the present invention provides an optical artificial neural network blood glucose detection chip for blood glucose detection tasks, including: an optical modulation layer 1, an image sensor 2, and a processor 3; the optical modulation layer 1 corresponds to the input layer, the linear layer, and the connection weights from the input layer to the linear layer of the artificial neural network; the square detection response of the image sensor 2 corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor 3 corresponds to the fully connected layer and the output layer of the artificial neural network, or the processor corresponds to the fully connected layer, the second nonlinear activation function in the nonlinear layer, and the output layer of the artificial neural network;

[0070] The light modulation layer 1 is disposed on the surface of the image sensor or the surface of the photosensitive area of ​​the image sensor. The light modulation layer 1 includes a light modulation structure. The light modulation layer 1 is used to perform spectral modulation on the incident light reflected and / or transmitted from the human body part to be tested and entering different positions of the light modulation structure, that is, to perform different intensity modulation on the incident light of different wavelengths, so as to obtain the incident light carrying information corresponding to different positions on the surface of the image sensor. The human body part to be tested is a part with blood glucose information, such as the fingers, wrist, forearm, earlobe, etc.

[0071] Therefore, in this embodiment, a light source is used to illuminate the part of the human body to be tested, and then the transmitted light or reflected light enters the light modulation layer of the chip. The light modulation structure in the light modulation layer adjusts the light incident on the light modulation structure. Since the light modulation structure in the light modulation layer is a pre-designed modulation pattern that can detect blood glucose levels, when the transmitted light or reflected light from the part of the human body to be tested enters the light modulation layer of the chip, the light modulation layer, image sensor and processor on the chip together act as an artificial neural network to identify the transmitted light or reflected light, and thus obtain the blood glucose level detection result.

[0072] In this embodiment, the light source used to irradiate the part of the human body to be tested can be a near-infrared light source with a wavelength range of 780nm-2526nm.

[0073] In addition, it should be noted that the modulation pattern in the light modulation layer that can detect blood glucose levels can be set in the following way:

[0074] For human blood glucose testing, optical simulation of the modulation pattern can be performed on a computer to obtain the modulation intensity (transmittance) of the modulation pattern for different wavelength components of incident light. This intensity is then used as the connection weight from the input layer to the linear layer of an artificial neural network. A nonlinear activation function is implemented in the computer. By pre-collecting and training a large amount of spectral signal data corresponding to blood glucose data, the required modulation pattern can be designed and fabricated. The input layer, linear layer, and first nonlinear activation function of the artificial neural network are then implemented on a chip using hardware (optical modulation layer and image sensor). This allows for rapid and accurate detection of the user's blood glucose value by algorithmically reconstructing the electrical signals obtained from the modulation of incident light at different wavelengths during blood glucose testing.

[0075] In this embodiment, the square detection response of the image sensor 2 refers to the intensity information of the incident light field detected by the image sensor. The intensity information of the incident light field is the square of the modulus of the light field signal. That is, the image sensor 2 converts the incident light carrying information corresponding to different position points after modulation by the light modulation layer 1 into electrical signals corresponding to different position points after the first nonlinear activation processing through the square detection response, and sends the electrical signals corresponding to different position points to the processor 3. The electrical signals are the image signals modulated by the light modulation layer. The incident light includes reflected light and / or transmitted light from the blood of the human body to be detected.

[0076] The processor 3 is used to perform fully connected processing on the electrical signals corresponding to different location points, or the processor performs fully connected processing and a second nonlinear activation processing on the electrical signals corresponding to different location points to obtain the output signal of the artificial neural network, that is, to obtain the blood glucose detection result.

[0077] In this embodiment, the incident light carries information including image information of blood from the body part to be detected and / or various optical spatial information. For example, the incident light carries information including at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0078] In this embodiment, the optical modulation layer 1 is disposed on the surface of the image sensor. The optical modulation layer 1 includes an optical modulation structure. The optical modulation layer 1 is used to perform different spectral modulations on the incident light entering different positions of the optical modulation structure, so as to obtain the information carried by the modulated incident light corresponding to different positions on the surface of the image sensor. Therefore, in this embodiment, the modulation effect of the optical modulation structure on the incident light can be regarded as the connection weight from the input layer to the linear layer.

[0079] In this embodiment, when the image sensor 2 performs photoelectric conversion on the modulated incident light carrying information, since the image sensor 2 can only detect the intensity information of the light, the electrical signal obtained by processing the light field distribution signal is proportional to the square of the modulus of the light field distribution signal. Therefore, the image sensor 2 has a square detection response. Thus, the image sensor 2 can be regarded as part of the nonlinear layer of the artificial neural network, that is, the square detection response of the image sensor 2 can be regarded as the first nonlinear activation function of the artificial neural network.

[0080] In this embodiment, the image sensor 2 converts the incident light carrying information corresponding to different position points after modulation by the optical modulation layer 1 into electrical signals corresponding to different position points after the first nonlinear activation processing through the square detection response, that is, the image signals modulated by the optical modulation layer. At the same time, the processor 3 connected to the image sensor 2 is used to perform fully connected processing or fully connected and second nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.

[0081] In this embodiment, the light modulation layer 1 includes a light modulation structure. The light modulation structure modulates the spectrum of incident light (such as reflected light, transmitted light, radiation light and other related action light of the target to be identified) at different positions of the light modulation structure with different intensities, so as to obtain the light field distribution signal corresponding to different positions on the surface of the image sensor 2.

[0082] In this embodiment, it is understood that the modulation intensity is related to the specific structural form of the optical modulation structure. For example, different modulation intensities can be achieved by designing different optical modulation structures (such as changing the shape and / or size parameters of the optical modulation structure).

[0083] In this embodiment, it can be understood that the optical modulation structures at different positions on the optical modulation layer 1 have different spectral modulation effects on the incident light. The modulation intensity of the optical modulation structure on different wavelength components of the incident light corresponds to the connection strength of the linear layer of the artificial neural network, that is, to the input layer and the connection weight from the input layer to the linear layer. It should be noted that the optical modulation layer 1 is composed of multiple optical filter units, and the optical modulation structures at different positions within each optical filter unit are different, thus having different spectral modulation effects on the incident light; the optical modulation structures at different positions between optical filter units can be the same or different, thus having the same or different spectral modulation effects on the incident light.

[0084] In this embodiment, the image sensor 2 converts the light field distribution signal corresponding to different position points after being modulated by the light modulation layer 1 into an electrical signal corresponding to different position points after the first nonlinear activation processing through the square detection response, and sends the electrical signal corresponding to different position points to the processor 3. The image sensor 2 corresponds to a part of the nonlinear layer of the neural network.

[0085] In this embodiment, the processor 3 performs fully connected processing on the electrical signals at different locations and performs a second nonlinear activation processing on the electrical signals at different locations to obtain the output signal of the artificial neural network.

[0086] It is understood that in this embodiment, the image sensor 2 corresponds to a part of the nonlinear layer of the neural network, and the processor 3 corresponds to another part of the nonlinear layer of the neural network and the output layer. It can also be understood as corresponding to the remaining layers (all other layers) in the neural network except for the input layer, the linear layer, and the first nonlinear activation function in the nonlinear layer.

[0087] In this embodiment, it should be noted that the squared detection response of image sensor 2 corresponds to the first nonlinear activation function in the nonlinear layer of the neural network. In this case, the processor may only perform fully connected processing without performing the second nonlinear activation processing, or the processor may perform both fully connected processing and the second nonlinear activation processing. The specific method can be determined according to the actual application scenario of the chip, and this embodiment does not limit it.

[0088] Furthermore, it should be noted that the processor 3 can be located within the optical artificial neural network blood glucose detection chip. That is, the processor 3 can be located within the optical artificial neural network blood glucose detection chip together with the optical modulation layer 1 and the image sensor 2, or it can be located separately outside the optical artificial neural network blood glucose detection chip and connected to the image sensor 2 inside the optical artificial neural network blood glucose detection chip via a data cable or connecting device. This embodiment does not limit this.

[0089] Furthermore, it should be noted that the processor 3 can be implemented using a computer, an ARM or FPGA circuit board with certain computing power, or a microprocessor; this embodiment does not limit its implementation. Additionally, as mentioned above, the processor 3 can be integrated within the optical artificial neural network blood glucose detection chip or can be externally located independently of it. When the processor 3 is externally located, the electrical signals from the image sensor 2 can be read out to the processor 3 via a signal readout circuit, and then the processor 3 performs fully connected processing and nonlinear activation processing on the readout electrical signals.

[0090] In this embodiment, it is understood that when the processor 3 performs the second nonlinear activation process, it can use a nonlinear activation function, such as the Sigmoid function, Tanh function, ReLU function, etc. This embodiment does not limit this.

[0091] In this embodiment, the optical modulation layer 1 corresponds to the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network. The image sensor 2 corresponds to a part of the nonlinear layer of the artificial neural network. That is, the square detection response of the image sensor 2 corresponds to the first nonlinear activation function of the artificial neural network. The image sensor 2 is used to convert the light field distribution signal at different spatial locations into an electrical signal by performing nonlinear activation processing through the square detection response. The processor 3 corresponds to the remaining layers of the artificial neural network and fully connects the electrical signals at different locations. It can also further obtain the output signal of the artificial neural network through a second nonlinear activation function.

[0092] like Figure 2 As shown on the left, the optical artificial neural network blood glucose detection chip includes an optical modulation layer 1, an image sensor 2, and a processor 3. Figure 2 In this system, processor 3 is implemented using a signal readout circuit and a computer. For example... Figure 2As shown on the right, the optical modulation layer 1 in the optical artificial neural network blood glucose detection chip corresponds to the input layer and linear layer of the artificial neural network, the image sensor 2 corresponds to a part of the nonlinear layer of the artificial neural network, and the processor 3 corresponds to the other part of the nonlinear layer and the output layer of the artificial neural network. The filtering effect of the optical modulation layer 1 on the incident light entering the optical modulation layer 1 corresponds to the connection weight from the input layer to the linear layer, and the square detection response of the image sensor 2 corresponds to the first nonlinear activation function of the artificial neural network. It can be seen that the optical modulation layer and the image sensor in the optical artificial neural network blood glucose detection chip provided in this embodiment realize the relevant functions of the input layer, linear layer and some or all nonlinear activation functions in the artificial neural network in hardware. Therefore, when using the optical artificial neural network blood glucose detection chip for intelligent processing, it is not necessary to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer (e.g., omitting calculations such as the connection weight from the input layer to the linear layer). This can significantly reduce the power consumption and latency of the artificial neural network processing. Furthermore, since this embodiment utilizes image information, spectral information, incident light angle information, and incident light phase information at different points in the blood space of the human body to be tested, it can more accurately achieve non-invasive identification of blood glucose levels in the human body to be tested.

[0093] like Figure 2 As shown on the right, the optical modulation layer 1 has different broadband spectral modulation effects on the incident light, thus modulating the incident light spectrum P at the corresponding unit position. λ Projection / connection to the emitted light field E N Above; the square detection response of image sensor 2 corresponds to a portion of the nonlinear activation function of the optical artificial neural network, which modulates the output light field E of optical modulation layer 1. N Converted to photocurrent response I of image sensor N Above. Processor 3 includes a signal readout circuit and a computer. The signal readout circuit in processor 3 reads out the photocurrent response I. N The signal is then transmitted to a computer, where it undergoes either fully connected processing of the electrical signal or further nonlinear activation processing, ultimately outputting the blood glucose level test result.

[0094] like Figure 3 As shown, the optical modulation structure on the optical modulation layer 1 is integrated on the surface of the image sensor 2 to modulate the incident light. The spectral information of the incident light is projected / connected to different pixels of the image sensor 2 to obtain an electrical signal containing the spectral information of the incident light and the image information. That is, after the incident light passes through the optical modulation layer 1, it is converted into an electrical signal by the nonlinear activation of the square detection response of the image sensor 2, forming an image containing the spectral information of the incident light. Finally, the processor 3 connected to the image sensor 2 processes the electrical signal containing the spectral information of the incident light and the image information to obtain the output result.

[0095] In this embodiment, the incident light carrying information may include one or more of the following: light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light (including two of them).

[0096] For example, in one implementation, the incident light carrying information may include light intensity distribution information. In other implementations, multiple information such as image information, spectral information, incident light angle, and incident light phase information of the blood in the human body to be detected can be used simultaneously to identify the target object, thereby enabling more accurate intelligent identification of blood glucose in the blood of the human body to be detected.

[0097] Therefore, the optical artificial neural network blood glucose detection chip provided in this embodiment can utilize one or more of the following: image information, spectral information, incident light angle, and incident light phase information of the blood at the site to be detected in the human body. That is, the incident light carries information at different points in space, and an artificial neural network is embedded in the hardware. From the spatial image, spectrum, angle, and phase information, information such as material composition, image shape, three-dimensional depth, and material composition distribution can be further extracted. This can solve the problem of difficulty in accurately realizing non-invasive blood glucose detection mentioned in the background section. At the same time, the embodiment of the present invention can eliminate the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some nonlinear activation functions in the prior art, thereby achieving low power consumption and low latency. Thus, the optical artificial neural network blood glucose detection chip provided in this embodiment can simultaneously meet the effects of low power consumption, low latency, and high recognition rate, thereby quickly and accurately detecting blood glucose values. In addition, the chip provided in this embodiment is small in size and can be integrated into a small device, making it convenient to carry around.

[0098] This invention implements a novel optical artificial neural network (ALN) blood glucose detection chip capable of performing artificial neural network functions for blood glucose detection tasks. This invention embeds an artificial neural network onto a hardware chip, using the optical modulation layer on the hardware chip as the input and linear layers of the artificial neural network. The filtering effect of the optical modulation layer on the incident light is used as the connection weight from the input layer to the linear layer. The squared detection response of the image sensor on the hardware chip is used as the first nonlinear activation function in the nonlinear layer of the artificial neural network. This invention projects the spatial spectral information of the human body's test site, containing blood glucose information, onto the pre-trained hardware chip. The hardware chip then derives the blood glucose detection result by analyzing the spatial spectral information of the human body's test site. It should be noted that this invention achieves low-power, safe, reliable, fast, accurate, and non-invasive blood glucose detection, thus well meeting users' needs for accurate non-invasive blood glucose detection.

[0099] It is understood that in this optical artificial neural network blood glucose detection chip, the optical modulation layer corresponds to the input layer and linear layer of the artificial neural network, and the image sensor corresponds to a part of the nonlinear layer of the artificial neural network; the processor corresponds to another part of the nonlinear layer and the output layer of the artificial neural network. Specifically, the optical modulation layer is disposed on the surface of the image sensor, and the optical modulation layer includes an optical modulation structure. The optical modulation structure is used to perform different spectral modulations on the incident light entering different positions of the optical modulation structure, so as to obtain the incident light carrying information corresponding to different positions on the surface of the image sensor. In this embodiment of the invention, the modulation effect of the optical modulation structure on the incident light is equivalent to the connection weight from the input layer to the linear layer. Meanwhile, in this embodiment of the invention, the image sensor converts the incident light carrying information corresponding to different position points after modulation by the optical modulation layer into electrical signals corresponding to different position points after a first nonlinear activation process via the square detection response. These electrical signals are then sent to the processor. The processor performs fully connected processing on the electrical signals corresponding to different position points, or performs fully connected processing and a second nonlinear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. Therefore, in this optical artificial neural network blood glucose detection chip, the optical modulation layer corresponds to the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network; the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; and the processor corresponds to the fully connected layer and output layer of the artificial neural network, or the processor corresponds to the fully connected layer and the second nonlinear activation function in the nonlinear layer of the artificial neural network. The nonlinear activation function and output layer, namely the optical modulation layer and image sensor in this optical artificial neural network blood glucose detection chip, realize the related functions of the input layer, linear layer, and some nonlinear activation functions in the artificial neural network. In other words, the embodiments of the present invention strip away the input layer, linear layer, and some or all of the nonlinear activation functions in the artificial neural network implemented in software in the prior art, and realize these structures in hardware. As a result, when using this optical artificial neural network blood glucose detection chip for artificial neural network intelligent processing, it is not necessary to perform complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some or all of the nonlinear activation functions. Only the processor in the optical artificial neural network blood glucose detection chip needs to perform fully connected processing or fully connected processing of electrical signals and a second nonlinear activation processing. This can significantly reduce the power consumption and latency of artificial neural network processing.Therefore, in this embodiment of the invention, the optical modulation layer is used as the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network; the squared detection response of the image sensor is used as the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor is used as the fully connected layer and output layer of the artificial neural network, or the processor is used as the fully connected layer, the second nonlinear activation function in the nonlinear layer, and the output layer of the artificial neural network. Thus, this embodiment of the invention not only eliminates the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some nonlinear activation functions in the prior art, but also simultaneously utilizes the image information, spectral information, incident light angle, and incident light phase information of the blood at the human body's detection site. The incident light at different points in the blood space of the target site carries information. Therefore, since the information carried by the incident light at different points in the blood space of the target site encompasses information such as the image, composition, shape, three-dimensional depth, and structure of the blood, the identification and processing based on this information can cover multi-dimensional information including the image, composition, shape, three-dimensional depth, and structure of the blood. This solves the problem mentioned in the background section of the difficulty in accurately achieving non-invasive blood glucose detection. Thus, the optical artificial neural network blood glucose detection chip provided in this embodiment of the invention can simultaneously achieve low power consumption, low latency, and high recognition rate, thereby enabling rapid and accurate non-invasive blood glucose detection. Therefore, this embodiment of the invention provides a novel optoelectronic chip for accurately detecting blood glucose levels. This chip embeds an artificial neural network portion into an image sensor containing various light modulation layers, achieving safe, reliable, rapid, and accurate non-invasive blood glucose detection.

[0100] It should be noted that the novel optoelectronic chip for accurately detecting blood glucose levels provided in this embodiment of the invention has advantages such as high measurement accuracy, portability, real-time online detection, and simple operation, which can greatly improve the quality of life of diabetic patients and has broad market prospects.

[0101] Based on the above embodiments, in this embodiment, the optical artificial neural network blood glucose detection chip includes a trained optical modulation structure, an image sensor, and a processor;

[0102] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip that satisfies the training convergence condition by training it with input training samples and output training samples corresponding to the blood glucose detection task, including different optical modulation structures, image sensors, and processors with different fully connected parameters; or, the trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip that satisfies the training convergence condition by training it with input training samples and output training samples corresponding to the blood glucose detection task, including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters.

[0103] The input training samples include incident light reflected or transmitted from the human body's test sites with different blood glucose values; the output training samples include the corresponding blood glucose values.

[0104] It is understandable that, since the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer corresponds to the connection weights from the input layer to the linear layer of the artificial neural network, changing the optical modulation structure in the optical modulation layer during training is equivalent to changing the connection weights from the input layer to the linear layer of the artificial neural network. By training convergence conditions, the optical modulation structure suitable for the current recognition task, as well as the fully connected parameters and nonlinear activation parameters suitable for the current recognition task, are determined, thereby completing the training of the optical artificial neural network blood glucose detection chip.

[0105] Understandably, for non-invasive blood glucose testing, spectral signal data corresponding to a large number of human body parts containing blood glucose information can be collected first. Then, a computer can simulate the response of incident light passing through a micro / nano modulation structure. Through data training, the required micro / nano modulation structure can be reverse-engineered and integrated onto an image sensor. During the user's blood glucose testing process, by algorithmically reconstructing the electrical signals obtained from the modulation of incident light at different wavelengths, a rapid and accurate detection of the user's blood glucose value can be achieved.

[0106] Specifically, for human blood glucose detection, optical simulation of micro / nano modulation structures on a computer can be performed to obtain the modulation intensity (transmittance) of the structure for different wavelength components of incident light. This intensity can then be used as the connection weights from the input layer to the linear layer of an artificial neural network. A nonlinear activation function can be implemented in the computer. By pre-collecting and training a large amount of spectral signal data corresponding to blood glucose data, the required micro / nano modulation structure can be designed and fabricated. The input layer, linear layer, and some nonlinear activation functions of the artificial neural network can then be implemented on a chip. During the user's blood glucose detection process, by algorithmically reconstructing the electrical signals obtained from the modulation of incident light at different wavelengths, the user's blood glucose value can be detected quickly and accurately.

[0107] Understandably, after training the optical artificial neural network (ALN) blood glucose detection chip, it can be used to perform recognition tasks. Specifically, the blood from the body to be detected carries image information, spectral information, incident light angle information, and incident light phase information from different points in space. After the incident light enters the optical modulation layer 1 of the trained ANN blood glucose detection chip, the optical modulation structure in the optical modulation layer 1 modulates the incident light. The intensity of the modulated light signal is detected by the image sensor 2 and converted into an electrical signal. Then, the processor 3 performs fully connected processing or simultaneously performs fully connected processing and a second nonlinear activation processing to obtain the blood glucose detection result.

[0108] like Figure 4a As shown, the complete process for identifying blood in a human body to detect blood glucose levels is as follows: a light source 100 (which can be a mobile phone screen light source) shines on the human body to be detected 200 (finger) containing blood glucose information. Then, the transmitted light from the blood in the human body to be detected is collected by the optical artificial neural network blood glucose detection chip 300. After being processed by the light modulation layer, image sensor and processor in the optical artificial neural network blood glucose detection chip, the blood glucose detection result can be obtained.

[0109] For example Figure 4b As shown, the complete process for identifying blood in the body to detect blood glucose levels is as follows: a light source 100 (which can be a mobile phone screen light source) shines on the body to be detected 200 (wrist) containing blood glucose information. Then, the reflected light from the blood in the body to be detected is collected by the optical artificial neural network blood glucose detection chip 300. After processing by the light modulation layer, image sensor and processor in the optical artificial neural network blood glucose detection chip, the blood glucose detection result can be obtained.

[0110] In this embodiment, the light source 100 can be a near-infrared light source with a wavelength range of 780nm-2526nm.

[0111] The human body sites to be tested include a variety of options, including but not limited to fingers, wrists, forearms, and earlobes. Detection methods include, but are not limited to, transmission detection, reflection detection, and a combination of transmission and reflection detection.

[0112] It should be noted that this chip directly fabricates micro / nano modulation structures on the surface of the photosensitive area of ​​the image sensor. Several discrete or continuous micro / nano structures constitute a unit. The micro / nano modulation structures at different locations have different spectral modulation effects on the incident light, collectively forming the optical modulation layer. The modulation intensity of these micro / nano modulation structures on different wavelength components of the incident light corresponds to the connection strength (linear layer weight) of the artificial neural network. Simultaneously, the square detection response of the image sensor, after undergoing a first nonlinear activation process with the incident light carrying information modulated by the optical modulation layer at different locations, converts it into electrical signals corresponding to different locations. These electrical signals are then sent to the processor. The processor performs fully connected processing on the electrical signals corresponding to different locations to realize the artificial neural network for spectral imaging. Alternatively, the processor performs fully connected processing and a second nonlinear activation process on the electrical signals corresponding to different locations to realize the artificial neural network for spectral imaging.

[0113] For non-invasive blood glucose testing, spectral signal data corresponding to a large amount of blood glucose data can be collected first. A computer can then simulate the response of incident light passing through a micro / nano modulation structure. Through data training, the required micro / nano modulation structure can be reverse-engineered and integrated onto an image sensor. During blood glucose testing, the electrical signals modulated by incident light of different wavelengths are algorithmically reconstructed to achieve rapid and accurate detection of the user's blood glucose level. This chip utilizes the spectral information of blood sugar levels in the human body and embeds an artificial neural network in the hardware, improving the speed and accuracy of blood glucose detection. Furthermore, this chip solution can be mass-produced using existing CMOS processes, reducing the device's size, power consumption, and cost.

[0114] In this embodiment, the optical modulation layer 1 is used as the input layer and linear layer of the neural network, and the image sensor 2 is used as part of the nonlinear layer of the neural network (that is, the square detection response of the image sensor 2 is used as the first nonlinear activation function of the neural network). In order to minimize the loss function of the neural network, the modulation intensity of different wavelength components in the incident light of the blood of the human body to be detected by the optical modulation structure in the optical modulation layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the optical modulation layer, the modulation intensity of different wavelength components in the incident light of the blood of the human body to be detected can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, and thus optimizing the training of the neural network.

[0115] Therefore, in this embodiment, the optical modulation structure is obtained based on neural network training. The training samples are optically simulated by a computer to obtain the sample modulation intensity of the optical modulation structure in the training samples on the incident light of blood at different wavelengths in the human body to be detected in the intelligent processing task. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network for nonlinear activation. The neural network is trained using the training samples corresponding to the intelligent processing task until the neural network converges. The corresponding training sample optical modulation structure is then used as the optical modulation layer of the corresponding intelligent processing task.

[0116] Therefore, this embodiment, by implementing the input layer, linear layer (optical modulation layer), and part of the nonlinear layer (the squared detection response of image sensor 2 as the first nonlinear activation function of the neural network) at the physical layer, eliminates the complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some or all of the nonlinear activation functions in the prior art, thereby improving processing speed and reducing latency. Simultaneously, this embodiment utilizes image information, spectral information, incident light angle information, and incident light phase information at different points in the spatial distribution of blood in the human body to be detected. That is, the incident light carries information at different points in the spatial distribution of blood in the human body to be detected. Thus, since the incident light carries information at different points in the spatial distribution of blood in the human body to be detected, encompassing the image, composition, shape, three-dimensional depth, and structure of the blood in the human body to be detected, the identification processing based on this information can cover multi-dimensional information such as image, composition, shape, three-dimensional depth, and structure of the blood in the human body to be detected. This solves the problem mentioned in the background section regarding the inability to achieve accurate non-invasive blood glucose detection.

[0117] Based on the above embodiments, in this embodiment, when training an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters, the different optical modulation structures are designed and implemented by using computer optical simulation design.

[0118] This embodiment designs the optical modulation structure using computer optical simulation. The optical modulation structure is adjusted through simulation until the neural network converges, at which point the corresponding optical modulation structure is determined to be the final required size. This saves prototyping time and costs, improves product efficiency, and easily solves complex optical problems. For example, FDTD software can be used to simulate and design the optical modulation structure. By changing the optical modulation structure in the optical simulation, the modulation intensity of the optical modulation structure for different incident light can be accurately predicted. This prediction is then used as the connection weight between the input layer and the linear layer of the neural network to train the optical artificial neural network for blood glucose detection, accurately obtaining the optical modulation structure.

[0119] Understandably, multiple reflected or transmitted light samples can be collected from different parts of the human body with different blood sugar levels beforehand and the data can be trained. The required micro-nano modulation structure can be designed and fabricated, and the first nonlinear activation function in the input layer, linear layer, and nonlinear layer of the artificial neural network can be realized on the chip.

[0120] Therefore, this embodiment uses computer optical simulation design to design the optical modulation structure, which saves time and cost in the prototype production of the optical modulation structure and improves product efficiency.

[0121] The optical artificial neural network blood glucose detection chip based on micro-nano modulation structure and image sensor provided in this embodiment has the following structural schematic diagram: Figure 2 As shown, the system includes an optical modulation layer 1, an image sensor 2, and a processor 3. The optical modulation layer 1 corresponds to the input layer and linear layer of the optical artificial neural network. Each unit comprises several discrete or continuous micro / nano modulation structures. These modulation structures exhibit different broadband spectral modulation effects on the incident light. Different units may contain the same or different micro / nano modulation structures, and spatial reconstruction can be performed based on the image. Each unit corresponds vertically to multiple photosensitive pixels of the image sensor. The square detection response of the image sensor 2 corresponds to a portion of the nonlinear activation function of the optical artificial neural network, which modulates the output light field E of the optical modulation layer 1. N Converted to photocurrent response I of image sensor N Above. Processor 3 includes a signal readout circuit and a computer. The signal readout circuit in processor 3 reads out the photocurrent response I. N The signal is then transmitted to a computer, where it undergoes either fully connected processing of the electrical signal or further nonlinear activation processing, and finally outputs the blood glucose test result.

[0122] Looking at it vertically, such as Figure 2 As shown, each micro / nano modulation structure in the optical modulation layer is designed through pre-trained artificial neural network training and can be fabricated by directly growing one or more layers of dielectric or metallic material on the image sensor, followed by etching. The overall size of each modulation unit in the optical modulation layer is typically λ.2 ~10 5 λ 2 The thickness is typically 0.1λ to 10λ, where λ is the center wavelength of the target band. Each modulation unit structure in the optical modulation layer corresponds to multiple pixels on the image sensor. The optical modulation layer is fabricated directly on the image sensor, and the image sensor and processor are connected via electrical contacts.

[0123] It is understandable that both the optical modulation layer and the CIS wafer (a special type of image sensor) can be manufactured using semiconductor CMOS integration technology. The optical modulation layer can be monolithically integrated directly on the image sensor at the wafer level. The chip can be fabricated in a single CMOS fabrication process, thus achieving monolithic integration at the wafer level. This helps to reduce the distance between the sensor and the optical modulation layer, shrink the device size, and lower packaging costs.

[0124] Therefore, this embodiment maps the optical modulation layer to the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network, and maps the square detection response of the image sensor to the first nonlinear activation function in the nonlinear layer of the artificial neural network. In this way, the spatial spectral information of the blood in the human body to be tested is projected onto the photocurrent response of the image sensor, and the full connection and second nonlinear activation of the electrical signal are realized in the processor, thereby enabling rapid non-destructive blood glucose detection.

[0125] The optical artificial neural network blood glucose detection chip based on micro-nano modulation structures and image sensors provided in this embodiment has the following advantages: A. By embedding the artificial neural network into an image sensor containing various optical modulation layers, the chip processes reflected or transmitted light from the test area to achieve safe, reliable, fast, and accurate non-destructive blood glucose detection. B. The chip can be fabricated in a single CMOS fabrication process, which helps reduce device failure rate, improve device yield, and reduce costs. C. Monolithic integration at the wafer level minimizes the distance between the sensor and the optical modulation layer, which helps reduce unit size, device volume, and packaging costs, and facilitates later integration into a wearable real-time blood glucose monitor.

[0126] As can be seen, this embodiment uses the optical modulation layer as the input layer and linear layer of the artificial neural network, and the image sensor as the first nonlinear activation in the nonlinear layer of the artificial neural network. The spatial spectral information of blood glucose information is projected onto the photocurrent response of the image sensor, and the full connection of the electrical signal and the second nonlinear activation (or the second nonlinear activation can be omitted) are implemented in the processor, thereby realizing low power consumption, safe and reliable fast, accurate and non-invasive blood glucose detection.

[0127] Based on the above embodiments, in this embodiment, the optical modulation structure in the optical modulation layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical modulation layer includes a discrete structure and / or a continuous structure.

[0128] In this embodiment, the optical modulation structure in the optical modulation layer may include only regular structures, only irregular structures, or both regular and irregular structures.

[0129] In this embodiment, the term "regular structure" in optical modulation structure can mean that the smallest modulation unit contained in the optical modulation structure has a regular structure, such as a rectangle, square, or circle. Furthermore, "regular structure" can also mean that the arrangement of the smallest modulation units in the optical modulation structure is regular, such as a regular array, circle, trapezoid, or polygon. Additionally, "regular structure" can also mean that both the smallest modulation units in the optical modulation structure have a regular structure and a regular arrangement.

[0130] In this embodiment, the term "irregular structure" in optical modulation structure can refer to the following: the minimum modulation unit contained in the optical modulation structure is an irregular structure, such as an irregular polygon, a random shape, or other irregular graphic. Furthermore, "irregular structure" in optical modulation structure can also refer to the following: the arrangement of the minimum modulation units contained in the optical modulation structure is irregular, such as an irregular polygonal form, a random arrangement, etc. Additionally, "irregular structure" in optical modulation structure can also refer to the following: both the minimum modulation units contained in the optical modulation structure are irregular structures, and the arrangement of the minimum modulation units is also irregular.

[0131] In this embodiment, the optical modulation structure in the optical modulation layer may include a discrete structure, a continuous structure, or both.

[0132] In this embodiment, the optical modulation structure includes a continuous structure, which means that the optical modulation structure is composed of continuous modulation patterns; the optical modulation structure includes a discrete structure, which means that the optical modulation structure is composed of discrete modulation patterns.

[0133] It is understandable that the continuous modulation pattern here can refer to linear patterns, wavy patterns, zigzag patterns, and so on.

[0134] It is understandable that the discrete modulation pattern here can refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete squares, discrete irregular polygons, etc.).

[0135] In this embodiment, it should be noted that the optical modulation structure has different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon resonance, and resonance enhancement. By designing different filter structures, the transmission spectrum of light passing through different sets of filter structures will be different.

[0136] Based on the above embodiments, in this embodiment, the optical modulation layer is a single-layer structure or a multi-layer structure.

[0137] In this embodiment, it should be noted that the optical modulation layer can be a single-layer filter structure or a multi-layer filter structure, such as a two-layer, three-layer, four-layer, or other multi-layer structure.

[0138] In this embodiment, as Figure 1 As shown, the optical modulation layer 1 is a single-layer structure. The thickness of the optical modulation layer 1 is related to the target wavelength range. For wavelengths of 400nm to 10μm, the thickness of the grating structure can be 50nm to 5μm.

[0139] It is understandable that since the function of the light modulation layer 1 is to modulate the spectrum of the incident light, it is preferable to use materials with high refractive index and low loss. For example, silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, III-V group materials, etc. can be selected for preparation. Among them, silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, silicon carbide, etc.

[0140] Furthermore, it should be noted that, in order to form more or more complex connection weights between the input layer and the linear layer, the optical modulation layer 1 can preferably be set as a multi-layer structure, and the optical modulation structure corresponding to each layer can be set as a different structure, thereby increasing the spectral modulation capability of the optical modulation layer on the incident light, thereby forming more or more complex connection weights between the input layer and the linear layer, and thus improving the accuracy of the optical artificial neural network blood glucose detection chip in processing intelligent tasks.

[0141] In addition, it should be noted that for an optical modulation layer with a multi-layer structure, the materials of each layer can be the same or different. For example, for an optical modulation layer 1 with two layers, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0142] It should be noted that the thickness of the optical modulation layer 1 is related to the target wavelength range. For wavelengths of 400nm to 10μm, the total thickness of the multilayer structure can be 50nm to 5μm.

[0143] Based on the above embodiments, in this embodiment, the optical modulation structure in the optical modulation layer includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponding to one or more pixels on the image sensor; the structures of each micro-nano unit may be the same or different.

[0144] In this embodiment, to obtain array-distributed connection weights (for connecting the input layer and the linear layer) to facilitate subsequent fully connected and nonlinear activation processing by the processor, preferably, the optical modulation structure is an array structure. Specifically, the optical modulation structure includes a unit array composed of multiple micro / nano units, each micro / nano unit corresponding to one or more pixels on the image sensor. It should be noted that the structures of the various micro / nano units can be the same or different. Furthermore, it should be noted that the structures of the various micro / nano units can be periodic or aperiodic. Additionally, it should be noted that each micro / nano unit may further contain multiple sets of micro / nano structure arrays, with the structures of each set of micro / nano structure arrays being the same or different.

[0145] The following is combined with Figures 5-9 For example, in this embodiment, such as Figure 5 As shown, the optical modulation layer 1 contains multiple repeating continuous or discrete micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit has the same structure (and each micro / nano unit is a non-periodic structure), and each micro / nano unit corresponds to one or more pixels on the image sensor 2; as shown... Figure 6 As shown, the optical modulation layer 1 contains multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66, each with the same structure (as shown). Figure 5 The difference is Figure 6 Each micro / nano unit is a periodic structure, and each micro / nano unit corresponds to one or more pixels on the image sensor 2; for example... Figure 7 As shown, the light modulation layer 1 contains multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit has the same structure (and each micro / nano unit is a periodic structure). Each micro / nano unit corresponds to one or more pixels on the image sensor 2. Figure 6 The difference is Figure 7 The unit shape of the periodic array within each micro / nano unit exhibits fourfold rotational symmetry; such as Figure 8 As shown, the optical modulation layer 1 contains multiple micro / nano units, such as 11, 22, 33, 44, 55, and 66, and... Figure 6The difference lies in the fact that each micro / nano unit has a different structure. Each micro / nano unit corresponds to one or more pixels on the image sensor 2. In this embodiment, the light modulation layer 1 contains multiple different micro / nano units, meaning that different regions on the optical artificial neural network blood glucose detection chip modulate the incident light differently, thereby increasing the design freedom and improving the accuracy of blood glucose detection. Figure 9 As shown, the optical modulation layer 1 contains multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit has the same structure, and... Figure 5 The difference lies in the fact that each micro-nano unit is composed of a discrete non-periodic array structure, and each micro-nano unit corresponds to one or more pixels on the image sensor 2.

[0146] In this embodiment, the micro / nano units exhibit different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon resonance, and resonance enhancement. By designing different filter structures, the transmission spectra of light passing through different sets of filter structures differ.

[0147] Based on the above embodiments, in this embodiment, the micro / nano unit includes a regular structure and / or an irregular structure; and / or, the micro / nano unit includes a discrete structure and / or a continuous structure.

[0148] In this embodiment, the micro / nano unit may include only regular structures, only irregular structures, or both regular and irregular structures.

[0149] In this embodiment, "micro-nano unit including regular structure" can mean that the smallest modulation unit contained in the micro-nano unit has a regular structure, such as a rectangle, square, or circle. Furthermore, "micro-nano unit including regular structure" can also mean that the arrangement of the smallest modulation units contained in the micro-nano unit is regular, such as a regular array, circle, trapezoid, or polygon. Additionally, "micro-nano unit including regular structure" can also mean that both the smallest modulation units contained in the micro-nano unit have a regular structure and a regular arrangement.

[0150] In this embodiment, "micro-nano unit including irregular structure" can mean that the smallest modulation unit contained in the micro-nano unit has an irregular structure, such as an irregular polygon, a random shape, or other irregular graphic. Furthermore, "micro-nano unit including irregular structure" can also mean that the arrangement of the smallest modulation units contained in the micro-nano unit is irregular, such as an irregular polygonal form, a random arrangement, etc. Additionally, "micro-nano unit including irregular structure" can also mean that both the smallest modulation units contained in the micro-nano unit and their arrangement are irregular.

[0151] In this embodiment, the micro-nano units in the optical modulation layer may include discrete structures, continuous structures, or both discrete and continuous structures.

[0152] In this embodiment, the term "micro-nano unit including continuous structure" can mean that the micro-nano unit is composed of continuous modulation patterns; the term "micro-nano unit including discrete structure" can mean that the micro-nano unit is composed of discrete modulation patterns.

[0153] It is understandable that the continuous modulation pattern here can refer to linear patterns, wavy patterns, zigzag patterns, and so on.

[0154] It is understandable that the discrete modulation pattern here can refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0155] In this embodiment, it should be noted that different micro / nano units have different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon resonance, and resonance enhancement. By designing different micro / nano units, the transmission spectrum of light passing through different sets of micro / nano units is different.

[0156] Based on the above embodiments, in this embodiment, the micro-nano unit includes multiple sets of micro-nano structure arrays, and the structures of each set of micro-nano structure arrays may be the same or different.

[0157] In this embodiment, as Figure 5 As shown, the optical modulation layer 1 contains multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit contains multiple sets of micro / nano structure arrays. For example, micro / nano unit 11 contains four different micro / nano structure arrays 110, 111, 112, and 113, and the filtering unit 44 contains four different micro / nano structure arrays 440, 441, 442, and 443. Figure 10As shown, the optical modulation layer 1 contains multiple micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit contains multiple sets of micro-nano structure arrays. For example, micro-nano unit 11 contains four identical micro-nano structure arrays 110, 111, 112, and 113.

[0158] It should be noted that this example only uses a micro-nano unit with four micro-nano structure arrays and is not intended to be limiting. In practical applications, micro-nano units with six, eight, or other numbers of micro-nano structure arrays can be set as needed.

[0159] In this embodiment, each group of micro-nano structure arrays within the micro-nano unit exhibits different modulation effects on light of different wavelengths, and the modulation effects on input light also differ between the groups of filter structures. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon resonance, and resonance enhancement. By designing different micro-nano structure arrays, the transmission spectrum of light passing through different groups of micro-nano structure arrays is made different.

[0160] Based on the above embodiments, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0161] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light from the blood of the human body to be tested as the connection weights of the neural network input layer and linear layer, different micro-nano structure arrays are used to achieve broadband filtering and narrowband filtering. Therefore, in this embodiment, the micro-nano structure array obtains the modulation intensity of different wavelength components of the incident light from the blood of the human body to be tested by performing broadband filtering or narrowband filtering. Figure 11 and Figure 12 As shown, each array of micro-nano structures in the optical modulation layer has the function of broadband filtering or narrowband filtering.

[0162] It is understandable that each group of micro / nano structure arrays can possess broadband filtering capabilities, narrowband filtering capabilities, or a combination of both. Furthermore, the broadband and narrowband filtering ranges of each group of micro / nano structure arrays can be the same or different. For example, by designing the period, duty cycle, radius, and side length of each group of micro / nano structures within a micro / nano unit, it can achieve narrowband filtering, allowing only one (or fewer) wavelengths of light to pass through. Conversely, by designing the period, duty cycle, radius, and side length of each group of micro / nano structures within a micro / nano unit, it can achieve broadband filtering, allowing more or all wavelengths of light to pass through.

[0163] Understandably, in practical applications, the filtering state of each micro-nano structure array can be determined by using broadband filtering, narrowband filtering, or a combination thereof, depending on the application scenario.

[0164] Based on the above embodiments, in this embodiment, each group of micro / nano structure arrays is a periodic structure array or an aperiodic structure array.

[0165] In this embodiment, each group of micro / nano structure arrays can be entirely periodic, entirely aperiodic, or partially periodic and partially aperiodic. Periodic structure arrays are easier to design using optical simulation, while aperiodic structure arrays can achieve more complex modulation effects.

[0166] In this embodiment, as Figure 5 As shown, the optical modulation layer 1 contains multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit consists of multiple micro / nano structure arrays, each with a different structure, and the micro / nano structure arrays are aperiodic. Here, aperiodic structure refers to the fact that the modulation apertures on the micro / nano structure arrays are arranged in a non-periodic pattern. For example... Figure 5 As shown, micro / nano unit 11 comprises four different aperiodic structure arrays 110, 111, 112, and 113, and micro / nano unit 44 comprises four different aperiodic structure arrays 440, 441, 442, and 443. These aperiodic micro / nano structure arrays are designed and trained from neural network data for intelligent processing tasks and are typically irregularly shaped structures. Figure 6 As shown, the optical modulation layer 1 contains multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit consists of multiple sets of micro / nano structure arrays, and the structures of each micro / nano structure array are different from each other. Figure 5 The difference lies in the fact that micro / nano structure arrays are periodic structures. A periodic structure refers to the arrangement of modulation apertures on the micro / nano structure array according to a periodic pattern. The basic unit of the periodic structure array is also designed through training an artificial neural network using a large amount of previously collected blood glucose data, and the period size is typically 20nm to 50μm. For example... Figure 6 As shown, micro / nano unit 11 comprises four different periodic structure arrays 110, 111, 112, and 113, and micro / nano unit 44 comprises four different periodic structure arrays 440, 441, 442, and 443. The periodic filter structures are designed using neural network data trained for a prior blood glucose detection task and are typically irregularly shaped structures. Figure 7As shown, the optical modulation layer 1 contains multiple distinct micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit consists of multiple arrays of micro / nano structures, each with a unique structure, and these arrays are periodic. The periodicity refers to the arrangement of shapes on the filter structure in a periodic pattern, with the period typically ranging from 20 nm to 50 μm. Figure 7 As shown, the micro-nano unit 11 and micro-nano unit 12 have different micro-nano structure arrays. Micro-nano unit 11 contains four different periodic structure arrays 110, 111, 112 and 113, and micro-nano unit 44 contains four different periodic structure arrays 440, 441, 442 and 443. The periodic structure micro-nano structure arrays are designed by training neural network data for blood glucose detection tasks and are usually irregularly shaped structures.

[0167] It should be noted that, Figures 5-9 Each micro / nano unit comprises four sets of micro / nano structure arrays, each formed using a modulation aperture of a different shape. These four arrays provide different modulation effects on the incident light. It should be noted that this example only illustrates a micro / nano unit with four sets of micro / nano structure arrays and is not intended to be limiting. In practical applications, micro / nano units with six, eight, or other numbers of micro / nano structure arrays can be configured as needed. In this embodiment, the four different shapes can be circular, cross-shaped, regular polygonal, and rectangular (not limited to these).

[0168] In this embodiment, each group of micro-nano structure arrays within the micro-nano unit exhibits different modulation effects on light of different wavelengths, and the modulation effects on input light also differ between the groups of micro-nano structure arrays. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon resonance, and resonance enhancement. By designing different micro-nano structure arrays, the transmission spectrum of light passing through different groups of micro-nano structure arrays is made different.

[0169] Based on the above embodiments, in this embodiment, the micro-nano unit includes one or more sets of empty structures among the multiple sets of micro-nano structure arrays.

[0170] The following is combined with Figure 9 The example shown is for illustration purposes. In this embodiment, for example... Figure 9As shown, the optical modulation layer 1 comprises multiple repeating micro / nano units, such as 11, 22, 33, 44, 55, and 66. Each micro / nano unit consists of multiple sets of micro / nano structure arrays, and the structures corresponding to the multiple sets of micro / nano structure arrays are different from each other. The micro / nano structure arrays are periodic structures. Unlike the above embodiments, each micro / nano unit contains one or more sets of empty structures, which are used to pass through the incident light. It can be understood that when multiple sets of micro / nano structure arrays contain one or more sets of empty structures, a richer spectral modulation effect can be formed, thereby meeting the spectral modulation requirements in specific scenarios (or meeting the specific connection weight requirements between the input layer and the linear layer in specific scenarios).

[0171] like Figure 9 As shown, each micro / nano unit includes a set of micro / nano structure arrays and three sets of empty structures. Micro / nano unit 11 includes one aperiodic structure array 111, micro / nano unit 22 includes one aperiodic structure array 221, micro / nano unit 33 includes one aperiodic structure array 331, micro / nano unit 44 includes one aperiodic structure array 441, micro / nano unit 55 includes one aperiodic structure array 551, and micro / nano unit 66 includes one aperiodic structure array 661. The micro / nano structure arrays are used to modulate the incident light in different ways. It should be noted that this is only an example of one set of micro / nano structure arrays and three sets of empty structures, and is not intended to be limiting. In practical applications, micro / nano units including one set of micro / nano structure arrays and five sets of empty structures or other numbers of micro / nano structure arrays can be set as needed. In this embodiment, the micro / nano structure arrays can be made using circular, cross-shaped, regular polygonal, and rectangular modulation apertures (not limited to these).

[0172] It should be noted that the multiple micro-nano unit arrays may not contain any empty structures; that is, the multiple micro-nano unit arrays may be aperiodic or periodic.

[0173] Based on the above embodiments, in this embodiment, the micro / nano unit has polarization-independent characteristics.

[0174] In this embodiment, because the micro / nano units possess polarization-independent characteristics, the optical modulation layer is insensitive to the polarization of the incident light, thereby realizing an optical artificial neural network blood glucose detection chip that is insensitive to both the incident angle and polarization. The optical artificial neural network blood glucose detection chip provided by this embodiment is insensitive to the incident angle and polarization characteristics of the incident light; that is, the measurement results are not affected by the incident angle and polarization characteristics of the incident light, thus ensuring the stability of spectral measurement performance and consequently, the stability of blood glucose detection. It should be noted that the micro / nano units can also possess polarization-dependent characteristics.

[0175] Based on the above embodiments, in this embodiment, the micro / nano unit has fourfold rotational symmetry.

[0176] In this embodiment, it should be noted that quadruple rotational symmetry is a specific case of polarization-independent characteristics. By designing the micro / nano unit as a structure with quadruple rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0177] The following is combined with Figure 7 The example shown is for illustration purposes. In this embodiment, for example... Figure 7 As shown, the optical modulation layer 1 contains multiple repeating micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of multiple sets of micro-nano structure arrays. The structures corresponding to the multiple sets of micro-nano structure arrays are different from each other. The micro-nano structure arrays are periodic structures. Unlike the above embodiments, the structures corresponding to each set of micro-nano structure arrays can be structures with fourfold rotational symmetry, such as circles, crosses, regular polygons, and rectangles. That is, after rotating the structure by 90°, 180°, or 270°, it coincides with the original structure, thereby giving the structure polarization-independent characteristics, so that the same intelligent recognition effect can be achieved when different polarized light is incident.

[0178] Based on the above embodiments, in this embodiment, the optical modulation layer is composed of one or more filter layers;

[0179] The filter layer is prepared from one or more of semiconductor materials, metallic materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, surface plasmon polaritons (SPP) micro / nano structures, and tunable Fabry-perot cavities (FP cavities).

[0180] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanopillar two-dimensional materials, and nanowire two-dimensional materials.

[0181] Among them, photonic crystals, as well as combinations of metasurfaces and random structures, are compatible with CMOS technology and can achieve good modulation effects. Other materials can also be filled into the micropores of micro-nano modulation structures to smooth the surface. Quantum dots and perovskites can utilize the spectral modulation characteristics of the materials themselves to minimize the volume of individual modulation structures. SPP has a small volume and can realize polarization-related light modulation. Liquid crystals can be dynamically controlled by voltage to improve spatial resolution. Tunable Fabry-Perot resonators can be dynamically controlled to improve spatial resolution.

[0182] Based on the above embodiments, in this embodiment, the thickness of the light modulation layer is 0.1λ to 10λ, where λ represents the center wavelength of the incident light.

[0183] In this embodiment, it should be noted that if the thickness of the optical modulation layer is much smaller than the center wavelength of the incident light, it cannot achieve effective spectral modulation; if the thickness of the optical modulation layer is much larger than the center wavelength of the incident light, it is difficult to fabricate and will introduce significant optical losses. Therefore, in this embodiment, to reduce optical losses, facilitate fabrication, and ensure effective spectral modulation, the overall size (area) of each micro / nano unit in the optical modulation layer 1 is typically λ. 2 ~10 5 λ 2 The thickness is typically 0.1λ to 10λ (λ represents the center wavelength of the incident light from the blood at the site of the human body to be tested). For example... Figure 5 As shown, the overall size of each micro / nano unit is 0.5 μm. 2 ~40000μm 2 The dielectric material in the optical modulation layer 1 is polycrystalline silicon with a thickness of 50 nm to 2 μm.

[0184] Understandably, the image sensor 2 can be fabricated on a CIS wafer, and the optical modulation layer 1 can be directly fabricated on the CIS wafer. The optical modulation layer 1 contains multiple repeating modulation units, each of which contains four different continuous aperiodic structure arrays. The basic units of the aperiodic structure arrays are designed by training an artificial neural network with a large amount of previously collected blood glucose information, and are typically irregularly shaped structures. Each aperiodic structure array exhibits different broadband modulation effects on the incident light, and the overall size of each modulation unit is 0.5 μm. 2 ~40000μm 2 The dielectric material in the optical modulation layer 1 is polycrystalline silicon, with a thickness of 50 nm to 2 μm. It is understood that the CIS wafer includes a silicon detector layer and a metal wire layer, with a response range from the visible to the near-infrared band; the CIS wafer is bare, without the Bayer filter array and microlens array fabricated on it. Each modulation unit corresponds to multiple sensor units on the CIS wafer.

[0185] Based on the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0186] CMOS image sensor (CIS), charge-coupled device (CCD), single-photon avalanche diode (SPAD) array, and focal plane array.

[0187] In this embodiment, it should be noted that a wafer-level CMOS image sensor (CIS) is used to achieve monolithic integration at the wafer level, which can minimize the distance between the image sensor and the light modulation layer. This is beneficial for reducing the size of the unit, lowering the device volume and packaging cost. SPAD can be used for weak light detection, and CCD can be used for strong light detection.

[0188] In this embodiment, the optical modulation layer and the image sensor can be fabricated using complementary metal-oxide-semiconductor (CMOS) integration technology, which helps reduce device failure rate, increase device yield, and reduce cost. For example, the optical modulation layer can be prepared by directly growing one or more layers of dielectric material on the image sensor, followed by etching, depositing a metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer.

[0189] Based on the above embodiments, in this embodiment, the type of artificial neural network includes: a feedforward neural network.

[0190] In this embodiment, the feedforward neural network (FNN), also known as a deep feedforward network (DFN) or a multi-layer perceptron (MLP), is the simplest type of neural network, with neurons arranged in layers. Each neuron is connected only to neurons in the previous layer. It receives the output of the previous layer and outputs it to the next layer; there is no feedback between layers. Feedforward neural networks have a simple structure, are easy to implement in hardware, and are widely used. They can approximate any continuous function and square-integrable function with arbitrary precision and can accurately implement any finite training sample set. A feedforward network is a static nonlinear mapping. Through the composite mapping of simple nonlinear processing units, complex nonlinear processing capabilities can be obtained.

[0191] Based on the above embodiments, in this embodiment, a light-transmitting medium layer is disposed between the light modulation layer and the image sensor.

[0192] In this embodiment, it should be noted that by providing a light-transmitting medium layer between the light modulation layer and the image sensor, the light modulation layer and the image sensor layer can be effectively separated, thus avoiding mutual interference between them.

[0193] Based on the above embodiments, in this embodiment, the image sensor is a front-illuminated type, comprising: a metal wire layer and a photodetector layer disposed from top to bottom, wherein the light modulation layer is integrated on the side of the metal wire layer away from the photodetector layer; or,

[0194] The image sensor is back-illuminated and includes a photodetector layer and a metal wire layer arranged from top to bottom, with the light modulation layer integrated on the side of the photodetector layer away from the metal wire layer.

[0195] In this embodiment, as Figure 13 The image sensor shown is a front-illuminated type, with the silicon probe layer 21 located below the metal line layer 22, and the light modulation layer 1 directly integrated onto the metal line layer 22.

[0196] In this embodiment, with Figure 13 The difference is, Figure 14 The image sensor shown is a back-illuminated type, with the silicon probe layer 21 above the metal line layer 22, and the light modulation layer 1 directly integrated onto the silicon probe layer 21.

[0197] It should be noted that, for back-illuminated image sensors, the silicon probe layer 21 is located above the metal line layer 22, which can reduce the influence of the metal line layer on the incident light, thereby improving the quantum efficiency of the device.

[0198] As described above, this embodiment uses the optical modulation layer as the input and linear layers of the artificial neural network, and the image sensor as part of the nonlinear layer (using the squared detection response of the image sensor as the first nonlinear activation function of the artificial neural network). The filtering effect of the optical modulation layer on the incident light entering the optical modulation layer is used as the connection weight from the input layer to the linear layer. The optical modulation layer and image sensor in the optical artificial neural network blood glucose detection chip provided in this embodiment implement the functions of the input layer, linear layer, and some nonlinear activation functions of the artificial neural network in hardware. This eliminates the need for complex signal processing and algorithm processing corresponding to the input layer, linear layer, and some nonlinear activation functions when using this optical artificial neural network blood glucose detection chip for intelligent processing. This significantly reduces the power consumption and latency of the artificial neural network processing. Furthermore, this embodiment utilizes both the image information of the blood at the site of the human body to be detected and the spectral information at different points in space, thus enabling more accurate intelligent detection of blood glucose levels at the site of the human body.

[0199] Therefore, in this embodiment of the invention, the optical modulation layer is used as the input layer and linear layer of the artificial neural network, and the image sensor is used as part of the nonlinear layer of the artificial neural network. The spatial spectral information of the object is projected onto the photocurrent response of the image sensor, and the fully connected electrical signal and secondary nonlinear activation are implemented in the processor, realizing intelligent sensing, recognition, and / or decision-making functions with low power consumption, low latency, and high accuracy. The optical artificial neural network blood glucose detection chip based on optical filters and image sensors in this embodiment of the invention has the following effects: embedding the artificial neural network part into the image sensor containing various optical modulation layers enables fast and accurate intelligent sensing, recognition, and / or decision-making functions. In addition, this embodiment of the invention can also achieve monolithic integration at the wafer level, thereby minimizing the distance between the sensor and the optical modulation layer, which is beneficial to reduce the size of the unit, reduce the device volume, and reduce packaging costs.

[0200] Based on the same inventive concept, another embodiment of the present invention provides an intelligent blood glucose meter, including: an optical artificial neural network blood glucose detection chip as described in the above embodiment.

[0201] Preferably, the blood glucose meter in this embodiment can be a wearable blood glucose meter.

[0202] Since the intelligent blood glucose meter provided in this embodiment includes the optical artificial neural network blood glucose detection chip described in the above embodiments, the intelligent blood glucose meter provided in this embodiment has all the beneficial effects of the optical artificial neural network blood glucose detection chip described in the above embodiments. Since the above embodiments have already described this in detail, this embodiment will not repeat it.

[0203] Based on the same inventive concept, another embodiment of the present invention provides a method for fabricating an optical artificial neural network blood glucose detection chip as described in the above embodiments, such as... Figure 15 As shown, the specific steps include the following:

[0204] Step 1510: Prepare an optical modulation layer containing an optical modulation structure on the surface of the image sensor;

[0205] Step 1520: Generate a processor capable of performing fully connected signal processing or generate a processor capable of performing fully connected signal processing and a second nonlinear activation processing.

[0206] Step 1530: Connect the image sensor and the processor;

[0207] The light modulation layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the image sensor; the incident light carrying information includes light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0208] The image sensor converts the information carried by the incident light, which is modulated by the optical modulation layer at different locations, into electrical signals corresponding to different locations after the first nonlinear activation processing through the square detection response, and sends the electrical signals corresponding to different locations to the processor.

[0209] The processor performs fully connected processing on the electrical signals corresponding to different location points, or the processor performs fully connected processing and a second nonlinear activation processing on the electrical signals corresponding to different location points to obtain the output signal of the artificial neural network.

[0210] In this embodiment, the training process for the optical artificial neural network blood glucose detection chip is also included, specifically comprising:

[0211] Using the input and output training samples corresponding to the blood glucose detection task, an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters is trained to obtain optical modulation structures, image sensors, and processors that meet the training convergence conditions. The optical modulation structures, image sensors, and processors that meet the training convergence conditions are then used as the trained optical modulation structures, image sensors, and processors.

[0212] Alternatively, using the input and output training samples corresponding to the blood glucose detection task, an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters can be trained to obtain optical modulation structures, image sensors, and processors that meet the training convergence conditions, and the optical modulation structures, image sensors, and processors that meet the training convergence conditions can be used as trained optical modulation structures, image sensors, and processors.

[0213] It is understood that when training an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters, the different optical modulation structures are designed and implemented using computer optical simulation design.

[0214] Understandably, various blood glucose value training samples can be collected and trained multiple times in advance, and the required micro-nano modulation structures can be designed and fabricated to realize the first nonlinear activation function in the input layer, linear layer, and nonlinear layer of the artificial neural network on the chip.

[0215] In this embodiment, a light modulation layer comprising a light modulation structure is fabricated on the surface of the photosensitive area of ​​the image sensor, including:

[0216] One or more layers of a predetermined material are grown on the surface of the image sensor;

[0217] Dry etching is performed on one or more layers of the preset material to obtain an optical modulation layer containing an optical modulation structure.

[0218] Alternatively, one or more layers of the preset material can be imprinted and transferred to obtain an optical modulation layer containing an optical modulation structure;

[0219] Alternatively, by applying external dynamic control to one or more preset materials, an optical modulation layer containing an optical modulation structure can be obtained;

[0220] Alternatively, one or more layers of the preset material can be printed in sections to obtain a light modulation layer containing a light modulation structure;

[0221] Alternatively, the one or more layers of the preset material can be partitioned and grown to obtain a light modulation layer containing a light modulation structure;

[0222] Alternatively, quantum dot transfer can be performed on one or more layers of the preset material to obtain an optical modulation layer containing an optical modulation structure.

[0223] When the optical artificial neural network blood glucose detection chip is used for blood glucose level detection at a specific site on the human body, the input and output training samples corresponding to the task are used to train the optical artificial neural network blood glucose detection chip, which includes different optical modulation structures, image sensors, and processors with different fully connected parameters, to obtain optical modulation structures, image sensors, and processors that meet the training convergence conditions; or, the optical artificial neural network blood glucose detection chip, which includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters, is trained to obtain optical modulation structures, image sensors, and processors that meet the training convergence conditions.

[0224] In this embodiment, it should be noted that, as Figure 1 As shown, the optical modulation layer 1 can be fabricated by directly growing one or more layers of dielectric material on the image sensor 2, followed by etching, depositing a metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the optical modulation structure, each unit can have different modulation effects on light of different wavelengths within the target range, and this modulation effect is insensitive to the incident angle and polarization. Each unit in the optical modulation layer 1 corresponds to one or more pixels on the image sensor 2. 1 is fabricated directly on 2.

[0225] In this embodiment, it should be noted that, as Figure 14 As shown, assuming the image sensor 2 is a back-illuminated structure, the light modulation layer 1 can be directly etched on the silicon image sensor layer 21 of the back-illuminated image sensor and then deposited with metal to prepare it.

[0226] Furthermore, it should be noted that the optical modulation structure on the optical modulation layer can be obtained by: dry etching of one or more layers of preset material to create an optical modulation structure pattern; dry etching directly removing unwanted portions of one or more layers of preset material on the surface of the photosensitive area of ​​the image sensor to obtain an optical modulation layer containing an optical modulation structure; or imprint transfer of one or more layers of preset material, where the desired structure is prepared by etching on another substrate and then transferred to the photosensitive area of ​​the image sensor using materials such as PDMS to obtain an optical modulation layer containing an optical modulation structure; or external dynamic modulation of one or more layers of preset material, where active materials are used and external electrodes are applied to modulate the optical modulation characteristics of the corresponding area by changing the voltage to obtain an optical modulation layer containing an optical modulation structure; or partition printing of one or more layers of preset material, where partition printing is a technique used to print in sections to obtain an optical modulation layer containing an optical modulation structure; or partition material growth of one or more layers of preset material to obtain an optical modulation layer containing an optical modulation structure; or quantum dot transfer of one or more layers of preset material to obtain an optical modulation layer containing an optical modulation structure.

[0227] Furthermore, it should be noted that since the preparation method provided in this embodiment is the same as the preparation method of the optical artificial neural network blood glucose detection chip in the above embodiments, for details regarding some principles and structures, please refer to the description in the above embodiments, and this embodiment will not repeat them here.

[0228] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A photonic artificial neural network blood glucose detection chip, characterized in that, For blood glucose detection, the system includes: an optical modulation layer, an image sensor, and a processor; the optical modulation layer corresponds to the input layer, linear layer, and connection weights from the input layer to the linear layer of an artificial neural network; the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor corresponds to the fully connected layer and output layer of the artificial neural network, or the processor corresponds to the fully connected layer, the second nonlinear activation function in the nonlinear layer, and the output layer of the artificial neural network; wherein, the square detection response refers to the image sensor detecting the light field intensity information of the incident light modulated by the optical modulation layer, and the detected electrical signal is proportional to the square of the modulus of the light field intensity information, thereby constituting the first nonlinear activation of the light field distribution signal; The optical modulation layer and the image sensor constitute the hardware computing part, and the processor constitutes the electrical computing part; The optical modulation layer is disposed on the surface of the image sensor. The optical modulation layer includes an optical modulation structure, which is configured to be trained so that the modulation intensity of light of different wavelengths corresponds to the connection weights from the input layer to the linear layer of the artificial neural network. The optical modulation structure is used to perform different spectral modulations on the incident light reflected and / or transmitted from the human body part to be tested and entering different locations of the optical modulation structure, so as to obtain incident light carrying information corresponding to different locations on the surface of the image sensor; the human body part to be tested is a part that has blood glucose information. The image sensor converts the incident light carrying information corresponding to different position points after modulation by the optical modulation layer into electrical signals corresponding to different position points through a square detection response. The electrical signals corresponding to different position points are then sent to the processor. The processor performs fully connected processing on the electrical signals corresponding to different position points to obtain the blood glucose detection result. Alternatively, the processor performs fully connected processing and a second nonlinear activation processing on the electrical signals corresponding to different position points to obtain the blood glucose detection result.

2. The optical artificial neural network blood glucose detection chip according to claim 1, characterized in that, The incident light carries information including at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

3. The optical artificial neural network blood glucose detection chip according to claim 1, characterized in that, The optical artificial neural network blood glucose detection chip includes an optical modulation structure, an image sensor, and a processor; The optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip that satisfies the training convergence condition by training it with input training samples and output training samples corresponding to the blood glucose detection task, which includes different optical modulation structures, image sensors, and processors with different fully connected parameters; or, the optical modulation structure, image sensor, and processor refer to the optical artificial neural network blood glucose detection chip that satisfies the training convergence condition by training it with input training samples and output training samples corresponding to the blood glucose detection task, which includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters. The input training samples include incident light reflected or transmitted from the human body test sites with different blood glucose values; the output training samples include the corresponding blood glucose values.

4. The optical artificial neural network blood glucose detection chip according to claim 3, characterized in that, When training an optical artificial neural network blood glucose detection chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network blood glucose detection chip that includes different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters, the different optical modulation structures are designed and implemented using computer optical simulation design.

5. The optical artificial neural network blood glucose detection chip according to any one of claims 1 to 4, characterized in that, The light source used to irradiate the part of the human body to be tested is a near-infrared light source.

6. The optical artificial neural network blood glucose detection chip according to any one of claims 1 to 4, characterized in that, The optical modulation structure in the optical modulation layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical modulation layer includes a discrete structure and / or a continuous structure.

7. The optical artificial neural network blood glucose detection chip according to any one of claims 1 to 4, characterized in that, The optical modulation structure in the optical modulation layer comprises a unit array consisting of multiple micro-nano units, each micro-nano unit corresponding to one or more pixels on the image sensor; the structures of each micro-nano unit may be the same or different.

8. The optical artificial neural network blood glucose detection chip according to claim 7, characterized in that, The micro / nano unit comprises a regular structure and / or an irregular structure; and / or, the micro / nano unit comprises a discrete structure and / or a continuous structure.

9. The optical artificial neural network blood glucose detection chip according to claim 7, characterized in that, The micro-nano unit contains multiple sets of micro-nano structure arrays, and the structures of each set of micro-nano structure arrays may be the same or different.

10. The optical artificial neural network blood glucose detection chip according to claim 9, characterized in that, Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

11. The optical artificial neural network blood glucose detection chip according to claim 9, characterized in that, Each group of micro / nano structure arrays is either a periodic structure array or an aperiodic structure array.

12. The optical artificial neural network blood glucose detection chip according to claim 9, characterized in that, The micro-nano unit contains one or more sets of empty structures among the multiple arrays of micro-nano structures.

13. The optical artificial neural network blood glucose detection chip according to claim 9, characterized in that, The micro / nano unit has fourfold rotational symmetry.

14. The optical artificial neural network blood glucose detection chip according to claim 1, characterized in that, The optical modulation layer is composed of one or more filter layers; The filter layer is prepared from one or more of semiconductor materials, metallic materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, surface plasmon polariton (SPP) micro / nano structures, and tunable Fabry-Perot resonators.

15. The optical artificial neural network blood glucose detection chip according to claim 14, characterized in that, The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanopillar two-dimensional materials, and nanowire two-dimensional materials.

16. The optical artificial neural network blood glucose detection chip according to claim 1, characterized in that, The thickness of the optical modulation layer is 0.1λ~10λ, where λ represents the center wavelength of the incident light.

17. A smart blood glucose meter, characterized in that, Including the optical artificial neural network blood glucose detection chip as described in any one of claims 1 to 16.

18. A method for fabricating an optical artificial neural network blood glucose detection chip as described in any one of claims 1 to 16, characterized in that, include: An optical modulation layer containing an optical modulation structure is fabricated on the surface of the image sensor; Generate a processor capable of performing fully connected signal processing or a processor capable of performing fully connected signal processing and a second nonlinear activation processing. Connect the image sensor and the processor; The optical modulation layer is used to perform different spectral modulations on the incident light entering different positions of the optical modulation structure through the optical modulation structure, so as to obtain the incident light carrying information corresponding to different positions on the surface of the image sensor; The image sensor converts the information carried by the incident light, which is modulated by the optical modulation layer at different locations, into electrical signals corresponding to different locations after the first nonlinear activation processing through the square detection response, and sends the electrical signals corresponding to different locations to the processor. The processor performs fully connected processing on the electrical signals corresponding to different location points to obtain blood glucose detection results; or, the processor performs fully connected processing on the electrical signals corresponding to different location points and a second nonlinear activation processing to obtain blood glucose detection results.

19. The method for fabricating the optical artificial neural network blood glucose detection chip according to claim 18, characterized in that, Also includes: The training process for the aforementioned optical artificial neural network blood glucose detection chip specifically includes: Using the input training samples and output training samples corresponding to the blood glucose detection task, an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors and processors with different fully connected parameters is trained to obtain optical modulation structures, image sensors and processors that meet the training convergence conditions, and the optical modulation structures, image sensors and processors that meet the training convergence conditions are used as trained optical modulation structures, image sensors and processors. Alternatively, using the input and output training samples corresponding to the blood glucose detection task, an optical artificial neural network blood glucose detection chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters can be trained to obtain optical modulation structures, image sensors, and processors that meet the training convergence conditions, and the optical modulation structures, image sensors, and processors that meet the training convergence conditions can be used as trained optical modulation structures, image sensors, and processors.

20. The method for fabricating the optical artificial neural network blood glucose detection chip according to claim 18, characterized in that, Fabricating an optical modulation layer comprising an optical modulation structure on the surface of the image sensor includes: One or more layers of a predetermined material are grown on the surface of the image sensor; Etching the optical modulation structure pattern on one or more layers of the preset material yields an optical modulation layer containing the optical modulation structure. Alternatively, one or more layers of the preset material can be imprinted and transferred to obtain an optical modulation layer containing an optical modulation structure; Alternatively, by applying external dynamic modulation to one or more preset materials, an optical modulation layer containing an optical modulation structure can be obtained; Alternatively, one or more layers of the preset material can be printed in sections to obtain a light modulation layer containing a light modulation structure; Alternatively, one or more layers of the preset material can be grown in sections to obtain a light modulation layer containing a light modulation structure; Alternatively, quantum dot transfer can be performed on one or more layers of the preset material to obtain an optical modulation layer containing an optical modulation structure.

Images