Optical artificial neural network fingerprint recognition chip, fingerprint recognition device and preparation method

Abstract

The present invention provides an optical artificial neural network fingerprint recognition chip, a fingerprint recognition device and a preparation method. The chip provided by the present invention simulates an artificial neural network in a hardware manner for fingerprint recognition. The present invention uses an optical filter layer as an input layer of the artificial neural network, uses an image sensor as a linear layer of the artificial neural network, and uses the filtering effect of the optical filter layer on incident light entering the optical filter layer as a connection weight from the input layer to the linear layer, so that when the fingerprint recognition chip is used for fingerprint recognition processing later, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, thereby greatly reducing power consumption and delay during artificial neural network processing.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an optical artificial neural network fingerprint recognition chip, a fingerprint recognition device and a preparation method thereof. Background Art

[0002] Fingerprint recognition technology is a biometric recognition technology that is widely used in the fields of smartphone unlocking, access control systems, bank password verification, etc. The main process of fingerprint recognition includes fingerprint collection, fingerprint preprocessing, fingerprint feature extraction and comparison. The acquisition of fingerprint images is the key to fingerprint recognition. The main ways to obtain fingerprint images are optical acquisition, capacitive sensor acquisition, thermal sensor acquisition, ultrasonic acquisition, etc. Among them, capacitive sensors are usually placed on the bottom of the back of the mobile phone to collect fingerprints. When combined with thermal sensors, liveness detection can also be achieved, but it cannot be used for under-screen fingerprint recognition. This is because the thickness of the screen module limits the signal acquisition of the capacitive sensor.

[0003] Existing under-screen fingerprint recognition technologies mainly include two solutions: optical acquisition and ultrasonic acquisition. However, these two solutions require fingerprint imaging first and then transmission to a computer for subsequent neural network recognition model algorithm processing. The transmission and processing of large amounts of data causes high power consumption and delay, and the device is large in size. Summary of the invention

[0004] In view of the problems existing in the prior art, the embodiments of the present invention provide an optical artificial neural network fingerprint recognition chip, a fingerprint recognition device and a preparation method.

[0005] Specifically, the embodiment of the present invention provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides an optical artificial neural network fingerprint recognition chip, comprising: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;

[0007] The optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure. The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light includes reflected light, transmitted light and / or radiated light on the fingertips of the human finger ends;

[0008] The image sensor is used to convert the information carried by the incident light corresponding to different positions after being modulated by the optical filter layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signals are image signals modulated by the optical filter layer;

[0009] The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain fingerprint recognition processing results.

[0010] Furthermore, the information carried by the incident light includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0011] Furthermore, the optical artificial neural network fingerprint recognition chip includes a trained light modulation structure, an image sensor and a processor;

[0012] The trained optical modulation structure, image sensor and processor refer to the optical modulation structure, image sensor and processor that meet the training convergence conditions by training the optical artificial neural network fingerprint recognition chip including different optical modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters using input training samples and output training samples corresponding to the fingerprint recognition task;

[0013] The input training samples include incident light reflected, transmitted or radiated by samples of the end pads of human fingers with different fingerprints; and the output training samples include corresponding fingerprint recognition results.

[0014] Furthermore, when training an optical artificial neural network fingerprint recognition chip comprising different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0015] Furthermore, the light modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or the light modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0016] Furthermore, the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; and the structures of the various micro-nano units are the same or different.

[0017] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or the micro-nano unit includes a discrete structure and / or a continuous structure.

[0018] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0019] Furthermore, each group of micro-nano structure arrays has a broadband filtering or narrowband filtering function.

[0020] Furthermore, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0021] Furthermore, the micro-nano unit includes one or more groups of empty structures in the multiple groups of micro-nano structure arrays.

[0022] Furthermore, the micro-nano unit has polarization-independent characteristics.

[0023] Furthermore, the micro-nano unit has four-fold rotational symmetry.

[0024] Further, the optical filter layer is composed of one or more filter layers;

[0025] The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, and adjustable Fabry-Perot resonant cavities.

[0026] Furthermore, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

[0027] Furthermore, the thickness of the optical filter layer is 0.1λ to 10λ, wherein λ represents the central wavelength of the incident light.

[0028] In a second aspect, an embodiment of the present invention provides a fingerprint recognition device, comprising: at least one optical artificial neural network fingerprint recognition chip as described above.

[0029] In a third aspect, an embodiment of the present invention provides a method for preparing an optical artificial neural network fingerprint recognition chip, comprising:

[0030] Preparing an optical filter layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor;

[0031] Generate a processor capable of fully connected signal processing and nonlinear activation processing;

[0032] connecting the image sensor and the processor;

[0033] The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the optical modulation structure through the optical modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light carrying information includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light;

[0034] The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different position points to obtain fingerprint recognition processing results; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light and / or radiated light on the fingertips at the end of a human finger.

[0035] Furthermore, a light filter layer including a light modulation structure is prepared on the surface of the photosensitive area of ​​the image sensor, comprising:

[0036] Growing one or more layers of a preset material on the surface of the photosensitive region of the image sensor;

[0037] Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light filter layer including the light modulation structure;

[0038] or performing embossing transfer on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure;

[0039] or obtaining an optical filter layer including an optical modulation structure by externally dynamically modulating the one or more layers of the preset material;

[0040] or performing zone printing on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure;

[0041] or performing partitioned growth on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure;

[0042] Alternatively, quantum dots are transferred to the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure.

[0043] Furthermore, the method further includes: a training process for the optical artificial neural network fingerprint recognition chip, specifically including:

[0044] Using input training samples and output training samples corresponding to the fingerprint recognition task, an optical artificial neural network fingerprint recognition chip containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters is trained to obtain light modulation structures, image sensors and processors that meet the training convergence conditions, and the light modulation structures, image sensors and processors that meet the training convergence conditions are used as the trained light modulation structures, image sensors and processors.

[0045] The optical artificial neural network fingerprint recognition chip, fingerprint recognition device and preparation method provided by the embodiment of the present invention simulate the artificial neural network in hardware to identify the user's fingerprint. That is, the embodiment of the present invention realizes a new fingerprint recognition chip that can realize the function of artificial neural network. In the fingerprint recognition chip, the optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure. The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area. Accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions. At the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions to obtain the output signal of the artificial neural network. It can be seen that In the fingerprint recognition chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the fingerprint recognition chip realize the related functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the fingerprint recognition chip is used for artificial neural network fingerprint recognition processing in the future, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and the processor in the fingerprint recognition chip only needs to perform related processing with full connection and nonlinear activation of electrical signals, which can greatly reduce the power consumption and delay of artificial neural network fingerprint recognition. It can be seen that the embodiment of the present invention uses the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor are used to project the spatial spectrum information of the fingertips at the end of the human finger into electrical signals, and then the full connection processing and nonlinear activation processing of the electrical signals are implemented in the processor. It can be seen that the embodiment of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art.

[0046] It can be seen that the embodiment of the present invention provides a new optoelectronic chip for accurate fingerprint recognition, which embeds an artificial neural network part into an image sensor containing various optical filter layers to achieve safe, reliable, fast and accurate fingerprint recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a schematic diagram of the structure of the optical artificial neural network fingerprint recognition chip provided by the first embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the recognition principle of an optical artificial neural network fingerprint recognition chip provided by an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of disassembling an optical artificial neural network fingerprint recognition chip provided by an embodiment of the present invention;

[0051] Figure 4 Schematic diagram of a fingertip recognition process according to an embodiment of the present invention;

[0052] Figure 5 is a top view of an optical filter layer provided by one embodiment of the present invention;

[0053] Figure 6 is a top view of another optical filter layer provided by one embodiment of the present invention;

[0054] Figure 7 is a top view of another optical filter layer provided by an embodiment of the present invention;

[0055] Figure 8 is a top view of another optical filter layer provided by an embodiment of the present invention;

[0056] Fig. 9 is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0057] Fig.10 is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0058] Fig.11 This is a schematic diagram of the broadband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0059] Fig.12 This is a schematic diagram of the narrow-band filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0060] Fig.13 It is a schematic flow chart of a method for preparing an optical artificial neural network fingerprint recognition chip provided by the third embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The existing under-screen fingerprint recognition technology mainly includes two solutions: optical acquisition and ultrasonic acquisition. However, these two solutions require fingerprint imaging first and then transfer to the computer for subsequent neural network recognition model algorithm processing. The transmission and processing of large amounts of data causes large power consumption and delay. Based on this, an embodiment of the present invention provides an optical artificial neural network fingerprint recognition chip, in which the optical filter layer in the fingerprint recognition chip corresponds to the input layer of the artificial neural network, the image sensor corresponds to the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. The embodiment of the present invention uses the optical filter layer and the image sensor to project the spatial spectrum information of the fingertips at the end of the human finger into an electrical signal, and then implements the full connection processing and nonlinear activation processing of the electrical signal in the processor. It can be seen that the embodiment of the present invention can save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art. The embodiment of the present invention strips the input layer and linear layer of the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and linear layer in the artificial neural network by hardware, so that when the smart chip is used for the subsequent intelligent processing of the artificial neural network, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer. Only the processor in the smart chip needs to perform related processing related to full connection and nonlinear activation of electrical signals, which can greatly reduce the power consumption and delay of the artificial neural network processing. The content provided by the present invention will be explained and illustrated in detail through specific embodiments.

[0063] like Figure 1 As shown, the optical artificial neural network fingerprint recognition chip provided by the first embodiment of the present invention is used for fingerprint recognition tasks, including: an optical filter layer 1, an image sensor 2 and a processor 3; the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor 2 corresponds to the linear layer of the artificial neural network; the processor 3 corresponds to the nonlinear layer and the output layer of the artificial neural network;

[0064] The optical filter layer 1 is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer 1 includes a light modulation structure. The optical filter layer 1 is used to perform spectrum modulation of the intensity modulation of the incident light entering different positions of the optical modulation structure through the light modulation structure, that is, different intensity modulation is performed on the incident light of different wavelengths, so as to obtain the incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light carrying information includes image information and / or various optical spatial information of the fingertips at the end of the human finger to be processed by the optical artificial neural network fingerprint recognition chip, for example, the incident light carrying information includes at least one of light intensity distribution information, spectrum information, angle information of the incident light and phase information of the incident light; the incident light includes reflected light, transmitted light and / or radiated light on the fingertips at the end of the human finger;

[0065] The image sensor 2 is used to convert the light intensity distribution signals corresponding to different positions after being modulated by the optical filter layer 1 into electrical signals corresponding to different positions, and send the electrical signals corresponding to different positions to the processor 3; the electrical signals are image signals modulated by the optical filter layer;

[0066] The processor 3 is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain fingerprint recognition processing results.

[0067] In this embodiment, the optical filter layer 1 is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer 1 includes a light modulation structure. The optical filter layer 1 is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the modulated incident light carrying information corresponding to the different positions on the surface of the photosensitive area. Accordingly, the image sensor 2 is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions, that is, image signals modulated by the optical filter layer. At the same time, the processor 3 connected to the image sensor 2 is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions to obtain the output signal of the artificial neural network, that is, the fingerprint recognition processing result.

[0068] In this embodiment, the optical filter layer 1 includes a light modulation structure, through which the spectrum of the incident light entering different positions of the light modulation structure (for example, reflected light, transmitted light, radiated light and other related action light from the fingertips of the human fingers to be identified) is modulated with different intensities, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area of ​​the image sensor 2.

[0069] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the light modulation structure. For example, different modulation intensities can be achieved by designing different light modulation structures (such as changing the shape and / or size parameters of the light modulation structure).

[0070] In this embodiment, it can be understood that the optical modulation structures at different positions on the optical filter layer 1 have different spectrum modulation effects on the incident light, and the modulation intensity of the optical modulation structure on different wavelength components of the incident light corresponds to the connection strength of the artificial neural network, that is, the connection weight of the input layer and the input layer to the linear layer. It should be noted that the optical filter layer 1 is composed of a plurality of optical filter units, and the optical modulation structures at different positions in each optical filter unit are different, so they have different spectrum modulation effects on the incident light; the optical modulation structures at different positions between the optical filter units can be the same or different, so they have the same or different spectrum modulation effects on the incident light.

[0071] In this embodiment, the image sensor 2 converts the information carried by the incident light corresponding to different position points into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to the linear layer of the neural network.

[0072] In this embodiment, the processor 3 performs full connection processing and nonlinear activation processing on the electrical signals at different positions, thereby obtaining the output signal of the artificial neural network.

[0073] It can be understood that processor 3 corresponds to the nonlinear layer and output layer of the neural network, and can also be understood as corresponding to the remaining layers (all other layers) in the neural network except the input layer and the linear layer.

[0074] In addition, it should be noted that the processor 3 can be set in the fingerprint recognition chip, that is, the processor 3 can be set in the fingerprint recognition chip together with the filter layer 1 and the image sensor 2, or it can be set separately outside the fingerprint recognition chip and connected to the image sensor 2 in the fingerprint recognition chip through a data line or a connecting device. This embodiment does not limit this.

[0075] In addition, it should be noted that the processor 3 can be implemented by a computer, or by an ARM or FPGA circuit board with certain computing power, or by a microprocessor, which is not limited in this embodiment. In addition, as mentioned above, the processor 3 can be integrated in the fingerprint recognition chip, or can be independently set outside the fingerprint recognition chip. When the processor 3 is independently set outside the fingerprint recognition chip, the electrical signal in the image sensor 2 can be read out to the processor 3 through the signal readout circuit, and then the processor 3 performs full connection processing and nonlinear activation processing on the read electrical signal.

[0076] In this embodiment, it can be understood that when the processor 3 performs nonlinear activation processing, it can be implemented by using a nonlinear activation function, for example, a Sigmoid function, a Tanh function, a ReLU function, etc., and this embodiment does not limit this.

[0077] In this embodiment, the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor 2 corresponds to the linear layer of the artificial neural network, and converts the information carried by the incident light at different positions in space into electrical signals. The processor 3 corresponds to the nonlinear layer and output layer of the artificial neural network, and fully connects the electrical signals at different positions, and obtains the output signal of the artificial neural network through a nonlinear activation function, thereby realizing fingerprint recognition processing of the fingertips at the end of the human finger.

[0078] like Figure 2 As shown on the left, the optical artificial neural network fingerprint recognition chip includes an optical filter layer 1, an image sensor 2 and a processor 3. Figure 2 In the embodiment, the processor 3 is implemented by a signal readout circuit and a computer. Figure 2 As shown on the right, the optical filter layer 1 in the optical artificial neural network fingerprint recognition chip corresponds to the input layer of the artificial neural network, the image sensor 2 corresponds to the linear layer of the artificial neural network, the processor 3 corresponds to the nonlinear layer and output layer of the artificial neural network, and the filtering effect of the optical filter layer 1 on the incident light entering the optical filter layer 1 corresponds to the connection weight from the input layer to the linear layer. It can be seen that the optical filter layer and the image sensor in the fingerprint recognition chip provided in this embodiment realize the relevant functions of the input layer and the linear layer in the artificial neural network by hardware, so that when the fingerprint recognition chip is used for subsequent recognition processing, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, which can greatly reduce the power consumption and delay of the artificial neural network processing.

[0079] like Figure 2 As shown on the right, the incident light spectrum P at different positions of the optical filter layer 1 λ Projected / connected to the image sensor photocurrent response I N In the figure, processor 3 includes a signal readout circuit and a computer. The signal readout circuit in processor 3 reads out the photocurrent response and transmits it to the computer, which performs full connection processing and nonlinear activation processing on the electrical signal and finally outputs the result.

[0080] like Figure 3As shown, the light modulation structure on the optical filter layer 1 is integrated above the image sensor 2, modulates the incident light, and projects / connects the spectrum information of the incident light to different pixel points of the image sensor 2 to obtain an electrical signal containing the spectrum information of the incident light and the image information. That is, after the incident light passes through the optical filter layer 1, it is converted into an electrical signal by the image sensor 2 to form an image containing the spectrum information of the incident light. Finally, the processor 3 connected to the image sensor 2 processes the electrical signal containing the image information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0081] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0082] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, the fingerprint can be identified by simultaneously utilizing multiple information including image information of the fingertips of human fingers, spectral information, the angle of the incident light, and the phase information of the incident light, thereby achieving more accurate fingerprint identification.

[0083] It can be seen that the optical artificial neural network fingerprint recognition chip provided in this embodiment can simultaneously utilize the image information and spectral information of the fingertips of human fingers, that is, the spectral information at different points in space, and embed the artificial neural network in the hardware. The fingerprint components, image shape, three-dimensional depth and other information can be further extracted from the image information, spectral information, angle information of the incident light and phase information of the incident light, thereby solving the problems mentioned in the background technology part that the fingerprint characteristics of certain specific people are difficult to identify, similar fingerprints are easy to be misidentified, and fingerprints left on the surface of objects are easy to be stolen, resulting in the inability to accurately perform fingerprint recognition, thereby achieving accurate fingerprint recognition, and realizing a spectrum optical artificial neural network fingerprint recognition chip with low power consumption, low latency and high accuracy.

[0084] The optical artificial neural network fingerprint recognition chip provided by the embodiment of the present invention realizes a new intelligent chip capable of realizing the function of an artificial neural network. In the intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the intelligent chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and realizes the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the intelligent chip is used for subsequent intelligent processing of the artificial neural network, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the processor in the intelligent chip needs to perform relevant processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay during artificial neural network processing.

[0085] In addition, it should be noted that in the prior art, only the two-dimensional image information of the fingerprint is used when identifying the fingerprint, but the two-dimensional image information is difficult to ensure the accuracy of the identification. Therefore, based on this, in one implementation, the incident light carries information that can include light intensity distribution information and spectrum information, so that when the optical artificial neural network fingerprint identification chip provided by the present application is used to perform intelligent identification tasks, the light intensity distribution information and spectrum information of the end of the human finger can be used at the same time. It can be seen that since the incident light carries information that covers the image, composition, shape, three-dimensional depth, structure and other information of the end of the human finger, when the identification processing is performed based on the incident light carrying information at different points in the space of the end of the human finger, the image, composition, shape, three-dimensional depth, structure and other multi-dimensional information of the end of the human finger can be covered, thereby solving the problem that it is difficult to ensure the accuracy of identification using the two-dimensional image information of the fingerprint, such as the difficulty in identifying the fingerprint characteristics of certain people, the easy identification error of similar fingerprints, and the easy theft of fingerprints left on the surface of objects, resulting in the inability to accurately identify fingerprints, thereby more accurately realizing the identification of fingerprints. In addition, in another implementation, the information carried by the incident light may further include light intensity distribution information, spectrum information, and angle information of the incident light, so that the image, composition, shape, three-dimensional depth, three-dimensional structure and other information of the fingertips at the end of the human finger can be more comprehensively captured, so that the fingerprint can be more accurately recognized. In addition, in another implementation, the information carried by the incident light may further include light intensity distribution information, spectrum information, angle information of the incident light and phase information of the incident light, so that the image, composition, shape, three-dimensional depth, three-dimensional structure and other information of the fingertips at the end of the human finger can be more comprehensively captured, so that the fingerprint can be more accurately recognized.

[0086] The optical artificial neural network fingerprint recognition chip provided by the embodiment of the present invention includes an optical filter layer, an image sensor and a processor. The optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor. The optical filter layer includes a light modulation structure. The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different positions on the surface of the photosensitive area. Accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different positions into electrical signals corresponding to the different positions. At the same time, the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different positions to obtain the output signal of the artificial neural network. It can be seen that in the fingerprint recognition chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the input layer of the artificial neural network. The sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer. That is, the optical filter layer and the image sensor in the fingerprint recognition chip realize the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiment of the present invention strips the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the fingerprint recognition chip is used for artificial neural network recognition processing in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. The processor in the fingerprint recognition chip only needs to perform relevant processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay of the artificial neural network processing.It can be seen that the embodiment of the present invention uses the optical filter layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor are used to project the spatial spectrum information of the fingertips at the end of the human finger into electrical signals, and then the full connection processing and nonlinear activation processing of the electrical signals are implemented in the processor. It can be seen that the embodiment of the present invention can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present invention actually simultaneously utilizes the image information, spectrum information, angle of the incident light and phase information of the fingertips at the end of the human fingertips, that is, the incident light at different points in the space of the fingertips at the end of the human finger carries information. It can be seen that Since the information carried by the incident light at different points in the space between the fingertips of the human finger includes the image, composition, shape, three-dimensional depth, structure and other information of the fingertips of the human finger, when identification and processing are performed based on the information carried by the incident light at different points in the space between the fingertips of the human finger, multi-dimensional information such as the image, composition, shape, three-dimensional depth, structure and other information of the fingertips of the human finger can be included. This can solve the problems mentioned in the background technology part that the fingerprint features of certain people are difficult to identify, similar fingerprints are easy to be misidentified, and fingerprints left on the surface of objects are easy to be stolen, resulting in the inability to accurately identify fingerprints. It can be seen that the optical artificial neural network fingerprint recognition chip provided in the embodiment of the present invention can not only achieve the effects of low power consumption and low latency, but also achieve the effect of high accuracy, so that it can be prepared for application in fingerprint recognition tasks.

[0087] Based on the contents of the above embodiments, in this embodiment, the optical artificial neural network fingerprint recognition chip includes a trained light modulation structure, an image sensor and a processor;

[0088] The trained optical modulation structure, image sensor and processor refer to the optical modulation structure, image sensor and processor that meet the training convergence conditions by training the optical artificial neural network fingerprint recognition chip including different optical modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters using input training samples and output training samples corresponding to the fingerprint recognition task;

[0089] The input training samples include incident light reflected, transmitted or radiated by samples of the end pads of human fingers with different fingerprints; the output training samples include corresponding fingerprint recognition results. Fingerprint recognition task

[0090] In this embodiment, the optical artificial neural network fingerprint recognition chip can be used for the fingerprint recognition task of the fingertips at the end of human fingers. Fingerprints are the lines formed by the concave and convex skin at the end of human fingers. Everyone's fingerprint is different. Fingerprint recognition is to classify and compare the fingerprints of the identified object to make a judgment.

[0091] In this embodiment, the reflected light, transmitted light and / or radiated light of the fingertips of human fingers enters the trained optical artificial neural network fingerprint recognition chip to obtain the fingerprint recognition result of the fingertips of human fingers.

[0092] It can be understood that since the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the light modulation structure in the optical filter layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence conditions, the light modulation structure suitable for the current fingerprint recognition task is determined, as well as the full connection parameters and nonlinear activation parameters suitable for the current fingerprint recognition task, thereby completing the training of the optical artificial neural network fingerprint recognition chip.

[0093] In this embodiment, the fingerprint recognition task of the fingertips of human fingers is taken as an example for explanation. It can be understood that when using the fingerprint recognition chip to perform recognition tasks, the optical artificial neural network fingerprint recognition chip needs to be trained first. The training of the optical artificial neural network fingerprint recognition chip here refers to collecting fingerprints of a large number of people as a training set in the early stage to determine the optical modulation structure suitable for the current fingerprint recognition task, as well as the fully connected parameters and nonlinear activation parameters suitable for the current fingerprint recognition task.

[0094] It can be understood that since the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the light modulation structure in the optical filter layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence conditions, the light modulation structure suitable for the current recognition task is determined, as well as the fully connected parameters and nonlinear activation parameters suitable for the current recognition task, thereby completing the training of the fingerprint recognition chip.

[0095] It is understandable that after the fingerprint recognition chip is trained, the fingerprint recognition chip can be used to perform recognition tasks. Specifically, after the incident light carrying the image information of the fingertips of the human finger and the spatial spectrum information enters the optical filter layer 1 of the trained fingerprint recognition chip, the light modulation structure in the optical filter layer 1 will modulate the incident light, and the information carried by the modulated incident light is detected by the image sensor 2 and converted into an electrical signal, and then the processor 3 performs full connection processing and nonlinear activation processing, and the fingerprint recognition result of the fingertips of the human finger can be obtained.

[0096] like Figure 4 As shown, the complete process for identifying the end of a human finger is as follows: the light source 200 under the mobile phone screen shines onto the human finger fingerprint 300, and then the reflected light or transmitted light of the end of the human finger is collected by the optical artificial neural network fingerprint recognition chip 100, or the light directly radiated outward from the end of the human finger is collected by the optical artificial neural network fingerprint recognition chip 100, and after being processed by the optical filter layer, image sensor and processor in the fingerprint recognition chip, the recognition result can be obtained.

[0097] Among them, the trained optical artificial neural network fingerprint recognition chip refers to an optical artificial neural network fingerprint recognition chip including a trained optical modulation structure, an image sensor and a processor; the trained optical modulation structure, image sensor and processor refer to the optical artificial neural network fingerprint recognition chip including different optical modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters, which are trained by using input training samples and output training samples corresponding to the recognition processing task, and the optical modulation structure, image sensor and processor that meet the training convergence conditions are obtained.

[0098] For example, for the fingerprint recognition task, the input training sample corresponding to the fingerprint recognition task is a human fingertip sample, and the output training sample corresponding to the fingerprint recognition task is the recognition result of the human fingertip sample. It can be understood that for the fingerprint recognition task, since the advantage of the fingerprint recognition chip provided in this embodiment is that it can obtain image information, spectral information, angle information of the incident light, and phase information of the incident light at different points in the human fingertip space, in order to make full use of this advantage, the human fingertip samples used as input training samples are preferably real human fingertip samples, rather than two-dimensional images of human fingertip samples. Of course, this does not mean that two-dimensional images cannot be used as human fingertip samples.

[0099] In this embodiment, the optical filter layer 1 serves as the input layer of the neural network, and the image sensor 2 serves as the linear layer of the neural network. In order to minimize the loss function of the neural network, the modulation intensity of the light modulation structure in the optical filter layer for the different wavelength components in the incident light at the fingertips of the human body is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the filter, the modulation intensity of the different wavelength components in the incident light at the fingertips of the human body can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, thereby optimizing the training of the neural network.

[0100] Therefore, the light modulation structure in the present embodiment is obtained based on neural network training. The optical simulation of the samples of the fingertips at the end of the human finger is performed by computer, and the sample modulation intensity of the light modulation structure in the samples of the fingertips at the end of the human finger for the different wavelength components of the incident light at the fingertips at the end of the human finger in the fingerprint recognition task is obtained. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network, and nonlinear activation is performed. The neural network is trained using the training samples corresponding to the fingerprint recognition task until the neural network converges, and the corresponding light modulation structure of the samples of the fingertips at the end of the human finger is used as the optical filter layer corresponding to the fingerprint recognition task.

[0101] It can be seen that this embodiment realizes the input layer (optical filter layer) and linear layer (image sensor) of the neural network at the physical layer, which not only saves the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present invention actually simultaneously utilizes the image information, spectral information, angle information of the incident light and phase information of the incident light at the end of the human finger, that is, the information carried by the incident light at different points in the space of the fingertips of the human fingertips, so that the composition, shape, three-dimensional depth and other information of the fingertips of the human fingertips can be further extracted from the information carried by the incident light. It can be seen that the optical artificial neural network fingerprint recognition chip provided by the embodiment of the present invention can not only achieve the effects of low power consumption and low latency, but also achieve the effect of high accuracy.

[0102] Based on the contents of the above embodiments, in this embodiment, when training an optical artificial neural network fingerprint recognition chip comprising different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0103] In this embodiment, the light modulation structure is designed through computer optical simulation, and the light modulation structure is adjusted through optical simulation until the corresponding light modulation structure is determined to be the final size of the light modulation structure to be produced when the neural network converges, thereby saving prototype production time and cost, improving product efficiency, and easily solving complex optical problems.

[0104] For example, FDTD software can be used to simulate and design the light modulation structure. By changing the light modulation structure in the optical simulation, the modulation intensity of the light modulation structure for different incident lights can be accurately predicted, and used as the connection weight between the neural network input layer and the linear layer to train the optical artificial neural network fingerprint recognition chip and accurately obtain the light modulation structure.

[0105] It can be seen that this embodiment designs the light modulation structure by adopting the computer optical simulation design method, which saves the time and cost of prototyping the light modulation structure and improves product efficiency.

[0106] Based on the contents of the above embodiments, in this embodiment, the light modulation structure in the optical filter layer includes a regular structure and / or an irregular structure;

[0107] In this embodiment, the light modulation structure in the optical filter layer may include only regular structures, only irregular structures, or both regular structures and irregular structures.

[0108] In this embodiment, it can be understood that the optical modulation structure here includes a regular structure, which may include: the minimum modulation unit contained in the optical modulation structure is a regular structure, the arrangement of the minimum modulation units contained in the optical modulation structure is regular, and the minimum modulation unit contained in the optical modulation structure is a regular structure, and the arrangement of the minimum modulation units is also regular, etc.

[0109] The minimum modulation unit included in the optical modulation structure may be a regular shape such as a rectangle, a square, or a circle. The minimum modulation unit included in the optical modulation structure may be a minimum modulation unit arranged in a regular array form, a regular circular form, a regular trapezoidal form, a regular polygonal form, etc.

[0110] In this embodiment, the light modulation structure includes an irregular structure, which may refer to: the smallest modulation unit included in the light modulation structure is an irregular structure, such as the smallest modulation unit may be an irregular polygon, a random shape, or other irregular graphics. In addition, the light modulation structure includes an irregular structure, which may refer to: the arrangement of the smallest modulation unit included in the light modulation structure is irregular, such as the arrangement may be an irregular polygon, a random arrangement, etc. In addition, the light modulation structure includes an irregular structure, which may refer to: the smallest modulation unit included in the light modulation structure is an irregular structure, and the arrangement of the smallest modulation unit is also irregular, etc.

[0111] Based on the contents of the above embodiments, in this embodiment, the light modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0112] In this embodiment, the light modulation structure in the optical filter layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0113] In this embodiment, the light modulation structure includes a continuous structure, which may mean that the light modulation structure is composed of a continuous modulation pattern; the light modulation structure includes a discrete structure, which may mean that the light modulation structure is composed of a discrete modulation pattern.

[0114] It can be understood that the continuous modulation pattern here may refer to a straight line pattern, a wavy line pattern, a broken line pattern, etc.

[0115] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0116] In this embodiment, it should be noted that the light modulation structure has different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra of light passing through different groups of filter structures are different.

[0117] Based on the contents of the above embodiments, in this embodiment, the optical filter layer is a single-layer structure or a multi-layer structure.

[0118] In this embodiment, it should be noted that the optical filter layer may be a single-layer filter structure or a multi-layer filter structure, for example, a two-layer, three-layer, four-layer or other multi-layer structure.

[0119] In this embodiment, if Figure 1 As shown, the optical filter layer 1 is a single-layer structure, and the thickness of the optical filter layer 1 is related to the target wavelength range. For a wavelength of 400 nm to 10 μm, the thickness of the grating structure can be 50 nm to 5 μm.

[0120] It can be understood that since the function of the optical filter layer 1 is to perform spectral modulation on the incident light, it is preferably prepared from materials with high refractive index and low loss. For example, silicon, germanium, germanium silicon materials, silicon compounds, germanium compounds, III-V materials, etc. can be selected for preparation, wherein silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, etc.

[0121] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical filter layer 1 can be set to a multi-layer structure, and the optical modulation structure corresponding to each layer can be set to a different structure, thereby increasing the spectrum modulation capability of the optical filter layer to the incident light, so that more or more complex connection weights can be formed between the input layer and the linear layer, thereby improving the accuracy of the fingerprint recognition chip when processing fingerprint recognition tasks.

[0122] In addition, it should be noted that, for a filter layer including a multi-layer structure, the material of each layer can be the same or different. For example, for an optical filter layer 1 having two layers, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0123] It should be noted that the thickness of the optical filter layer 1 is related to the target wavelength range. For a wavelength of 400 nm to 10 μm, the total thickness of the multilayer structure can be 50 nm to 5 μm.

[0124] Based on the contents of the above embodiments, in this embodiment, the light modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of each micro-nano unit are the same or different.

[0125] In this embodiment, in order to obtain connection weights distributed in an array (connection weights for connecting the input layer and the linear layer) so that the processor can perform subsequent full connection and nonlinear activation processing, preferably, in this embodiment, the light modulation structure is in the form of an array structure. Specifically, the light modulation structure includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixels on the image sensor. It should be noted that the structures of each micro-nano unit can be the same or different. In addition, it should be noted that the structure of each micro-nano unit can be periodic or non-periodic. In addition, it should be noted that each micro-nano unit can further include multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different. It should be noted that the structure of each micro-nano unit is a structure designed according to the corresponding fingerprint recognition task, and the structure of each micro-nano unit is based on the incident light reflected, transmitted or radiated by the fingertip samples of the end of the human finger with different fingerprints as input, and the corresponding fingerprint recognition result as output training is obtained. It can be seen that in this embodiment, different micro-nano modulation structures can be designed according to different fingerprint recognition tasks, so as to accurately and quickly perform fingerprint recognition.

[0126] Combine the following Figures 5 to 9 For example, in this embodiment, Figure 5 As shown, the optical filter layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, 66, each micro-nano unit has the same structure (and each micro-nano unit is a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; Figure 6 As shown, the optical filter layer 1 comprises a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, each of which has the same structure (with Figure 5 The difference is Figure 6 Each micro-nano unit is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; Figure 7 As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure (and each of which is a periodic structure), and each of which corresponds to one or more pixels on the image sensor 2. Figure 6 The difference is Figure 7The unit shape of the periodic array in each micro-nano unit has four-fold rotational symmetry; Figure 8 As shown, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66, and Figure 6 The difference is that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixels on the image sensor 2. In this embodiment, the optical filter layer 1 includes multiple different micro-nano units, that is, different areas on the fingerprint recognition chip have different modulation effects on the incident light, thereby increasing the degree of freedom of design and further improving the accuracy of recognition. Fig. 9 As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure. Figure 5 The difference is that each micro-nano unit is composed of a discrete non-periodic array structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0127] In this embodiment, the micro-nano unit has different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra of light passing through different groups of filter structures are different.

[0128] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0129] In this embodiment, the micro-nano unit includes a regular structure, which may refer to: the minimum modulation unit included in the micro-nano unit is a regular structure, the arrangement of the minimum modulation unit included in the micro-nano unit is regular, the minimum modulation unit included in the micro-nano unit is a regular structure, and the arrangement of the minimum modulation unit is also regular, etc. Among them, the minimum modulation unit can be a regular shape such as a rectangle, a square and a circle, and the arrangement of the minimum modulation unit included in the micro-nano unit can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc.

[0130] In this embodiment, the micro-nano unit here includes an irregular structure, which may mean: the minimum modulation unit contained in the micro-nano unit is an irregular structure, the arrangement of the minimum modulation unit contained in the micro-nano unit is irregular, the minimum modulation unit contained in the micro-nano unit is an irregular structure, and the arrangement of the minimum modulation unit is also irregular, etc.

[0131] Among them, the smallest modulation unit included in the micro-nano unit can be an irregular polygon, a random shape or other irregular graphics, and the arrangement of the smallest modulation unit included in the micro-nano unit can be an irregular polygon form, a random arrangement form, etc.

[0132] In this embodiment, the micro-nano units in the optical filter layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0133] In this embodiment, the micro-nano unit including a continuous structure may refer to that the micro-nano unit is composed of a continuous modulation pattern; the micro-nano unit including a discrete structure may refer to that the micro-nano unit is composed of a discrete modulation pattern.

[0134] It can be understood that the continuous modulation pattern here may refer to a straight line pattern, a wavy line pattern, a broken line pattern, etc.

[0135] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0136] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano units, the corresponding transmission spectra of light passing through different groups of micro-nano units are different.

[0137] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0138] In this embodiment, if Figure 5 As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which includes a plurality of micro-nano structure arrays, such as the micro-nano unit 11 includes four different micro-nano structure arrays 110, 111, 112, and 113, and the filter unit 44 includes four different micro-nano structure arrays 440, 441, 442, and 443. Fig.10 As shown, the optical filter layer 1 includes multiple micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0139] It should be noted that the micro-nano unit including four groups of micro-nano structure arrays is only used as an example here, which does not serve as a limitation. In practical applications, the micro-nano unit can also be provided with six groups, eight groups or other number of groups of micro-nano structure arrays as needed.

[0140] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and each group of filter structures has different modulation effects on input light, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra of light passing through different groups of micro-nano structure arrays are different.

[0141] Based on the contents of the above embodiments, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0142] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light at the end of the fingertips of the human body as the connection weights of the neural network input layer and the linear layer, different micro-nano structure arrays are used to achieve broadband filtering and narrowband filtering. Therefore, in this embodiment, the micro-nano structure array performs broadband filtering or narrowband filtering on the incident light at the end of the fingertips of the human body to obtain the modulation intensity of different wavelength components of the incident light at the end of the fingertips of the human body. Fig.11 and Fig.12 As shown, each group of micro-nano structure arrays in the optical filter layer has the function of broadband filtering or narrowband filtering.

[0143] It is understandable that, for each group of micro-nano structure arrays, they may all have broadband filtering effects, or they may all have narrowband filtering effects, or they may partially have broadband filtering effects and partially have narrowband filtering effects. In addition, the broadband filtering range and narrowband filtering range of each group of micro-nano structure arrays may also be the same or different. For example, by designing the period, duty cycle, radius, side length and other dimensional parameters of each group of micro-nano structures in the micro-nano unit, it has a narrowband filtering effect, that is, only one (or fewer) wavelengths of light can pass through. For another example, by designing the period, duty cycle, radius, side length and other dimensional parameters of each group of micro-nano structures in the micro-nano unit, it has a broadband filtering effect, that is, allowing more wavelengths or all wavelengths of light to pass through.

[0144] It is understandable that, in specific use, the filtering state of each group of micro-nano structure arrays can be determined by performing broadband filtering, narrowband filtering or a combination thereof according to the application scenario.

[0145] Based on the contents of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0146] In this embodiment, each group of micro-nano structure arrays can be periodic structure arrays, non-periodic structure arrays, or partly periodic structure arrays and partly non-periodic structure arrays. Among them, the periodic structure array is easy to perform optical simulation design, and the non-periodic structure array can achieve more complex modulation effects.

[0147] In this embodiment, if Figure 5 As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The structures of each micro-nano structure array are different from each other, and the micro-nano structure array is a non-periodic structure. The non-periodic structure refers to that the shapes of the modulated holes on the micro-nano structure array are arranged in a non-periodic arrangement. Figure 5 As shown, the micro-nano unit 11 includes four different non-periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different non-periodic structure arrays 440, 441, 442 and 443. The micro-nano structure arrays with non-periodic structures are designed by training neural network data for recognition processing tasks in the early stage, and are usually irregularly shaped structures. Figure 6 As shown, the optical filter layer 1 comprises a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The structures of the micro-nano structure arrays are different from each other. Figure 5 The difference is that the micro-nano structure array is a periodic structure. The periodic structure refers to the arrangement of the modulated holes on the micro-nano structure array in a periodic arrangement, and the period size is usually 20nm to 50μm. Figure 6 As shown, the micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442 and 443. The filter structure of the periodic structure is obtained by training and designing the neural network data for the recognition processing task in the early stage, and is usually an irregular structure. Figure 7 As shown, the optical filter layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures. The periodic structure refers to the arrangement of the shapes on the filter structure in a periodic arrangement, and the size of the period is usually 20nm to 50μm. Figure 7As shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different. The micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442 and 443. The micro-nano structure arrays with periodic structures are obtained through the early design of neural network data training for recognition and processing tasks, and are usually irregularly shaped structures.

[0148] It should be noted that Figures 5 to 9 Each micro-nano unit includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are formed by modulation holes of four different shapes, and the four groups of micro-nano structure arrays are used to have different modulation effects on the incident light. It should be noted that here, only the micro-nano unit including four groups of micro-nano structure arrays is used as an example, which does not play a limiting role. In practical applications, micro-nano units including six groups, eight groups or other number groups of micro-nano structure arrays can also be set as needed. In this embodiment, the four different shapes can be circular, cross, regular polygon and rectangular (not limited to this).

[0149] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and each group of micro-nano structure arrays has different modulation effects on input light, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra of light passing through different groups of micro-nano structure arrays are different.

[0150] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes a plurality of micro-nano structure arrays, and there is one or more groups of empty structures.

[0151] Combine the following Fig. 9 The example shown is used for illustration. In this embodiment, Fig. 9 As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which is composed of a plurality of micro-nano structure arrays, the structures corresponding to the plurality of micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures. Different from the above-mentioned embodiments, any micro-nano unit includes one or more groups of empty structures, and the empty structures are used to pass incident light directly. It can be understood that when a plurality of micro-nano structure arrays include one or more groups of empty structures, a richer spectrum modulation effect can be formed, thereby meeting the spectrum modulation requirements in a specific scenario (or meeting the specific connection weight requirements between the input layer and the linear layer in a specific scenario).

[0152] like Fig. 9As shown, each micro-nano unit includes a group of micro-nano structure arrays and three groups of empty structures, micro-nano unit 11 includes a non-periodic structure array 111, micro-nano unit 22 includes a non-periodic structure array 221, micro-nano unit 33 includes a non-periodic structure array 331, micro-nano unit 44 includes a non-periodic structure array 441, micro-nano unit 55 includes a non-periodic structure array 551, and micro-nano unit 66 includes a non-periodic structure array 661, wherein the micro-nano structure array is used to modulate the incident light in different ways. It should be noted that here, only one group of micro-nano structure arrays and three groups of empty structures are used as an example, which does not play a limiting role. In practical applications, a micro-nano unit including a group of micro-nano structure arrays and five groups of empty structures or other number groups of micro-nano structure arrays can also be set according to needs. In this embodiment, the micro-nano structure array can be made of circular, cross-shaped, regular polygonal and rectangular modulation holes (not limited to this).

[0153] It should be noted that the multiple groups of micro-nano structure arrays included in the micro-nano unit may not include any empty structures, that is, the multiple groups of micro-nano structure arrays may be non-periodic structure arrays or periodic structure arrays.

[0154] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit has polarization-independent characteristics.

[0155] In this embodiment, since the micro-nano unit has polarization-independent characteristics, the optical filter layer is insensitive to the polarization of the incident light, thereby realizing an optical artificial neural network fingerprint recognition chip that is insensitive to both the incident angle and polarization. The optical artificial neural network fingerprint recognition chip provided by the embodiment of the present invention is insensitive to the incident angle and polarization characteristics of the incident light, that is, the measurement result will not be affected by the incident angle and polarization characteristics of the incident light, thereby ensuring the stability of the spectral measurement performance, and further ensuring the stability of the fingerprint recognition process. It should be noted that the micro-nano unit may also have polarization-related characteristics.

[0156] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0157] In this embodiment, it should be noted that four-fold rotational symmetry is a specific case of polarization-independent characteristics. By designing the micro-nano unit to have a structure with four-fold rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0158] Combine the following Figure 7 The example shown is used for illustration. In this embodiment, Figure 7As shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which is composed of a plurality of groups of micro-nano structure arrays. The structures corresponding to the plurality of groups of micro-nano structure arrays are different from each other. The micro-nano structure arrays are periodic structures. Different from the above-mentioned embodiments, the structure corresponding to each group of micro-nano structure arrays can be a structure with four-fold rotational symmetry, such as a circle, a cross, a regular polygon, a rectangle, etc., that is, after the structure is rotated 90°, 180°, and 270°, it overlaps with the original structure, so that the structure has the characteristic of being independent of polarization, so that the same fingerprint recognition effect can be achieved under different polarized light incidence.

[0159] Based on the contents of the above embodiments, in this embodiment, the optical filter layer is composed of one or more filter layers;

[0160] The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is a filter layer made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP (Surface Plasmon Polaritons, SPP) micro-nano structures, and adjustable Fabry-Perot cavity (Fabry-perot Cavity, FP cavity).

[0161] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

[0162] Among them, photonic crystals, as well as the combination of metasurfaces and random structures are compatible with CMOS technology and can have better modulation effects. Other materials can also be filled in the micropores of the micro-nano modulation structure to smooth the surface. Quantum dots and perovskites can utilize the spectral modulation characteristics of the material itself to minimize the volume of a single modulation structure. SPP has a small volume and can achieve polarization-dependent light modulation. Liquid crystals can be dynamically controlled by voltage to improve spatial resolution. Adjustable Fabry-Perot cavities can be dynamically controlled to improve spatial resolution.

[0163] Based on the content of the above embodiment, in this embodiment, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0164] In this embodiment, it should be noted that if the thickness of the optical filter layer is much smaller than the central wavelength of the incident light, it cannot play an effective spectrum modulation role; if the thickness of the optical filter layer is much larger than the central wavelength of the incident light, it is difficult to prepare in terms of technology, and will introduce large optical losses. Therefore, in this embodiment, in order to reduce optical losses, facilitate preparation, and ensure effective spectrum modulation, the overall size (area) of each micro-nano unit in the optical filter layer 1 is usually λ 2 ~10 5 λ 2 The thickness is usually 0.1λ~10λ (λ represents the central wavelength of the incident light at the end of the finger of the human body). Figure 5 As shown, the overall size of each micro-nano unit is 0.5 μm 2 ~40000μm 2 The dielectric material in the optical filter layer 1 is polysilicon with a thickness of 50nm to 2μm.

[0165] Based on the contents of the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0166] CMOS image sensor (Contact Image Sensor, CIS), Charge Coupled Device (Charge Coupled Device, CCD), Single Photon Avalanche Diode (Single Photon Avalanche Diode, SPAD) array and focal plane photodetector array.

[0167] In this embodiment, it should be noted that the use of a wafer-level CMOS image sensor CIS and monolithic integration at the wafer level can minimize the distance between the image sensor and the optical filter layer, which is beneficial for reducing the size of the unit, reducing the device volume and packaging cost. SPAD can be used for weak light detection, and CCD can be used for strong light detection.

[0168] In this embodiment, the optical filter layer and the image sensor can be manufactured by a complementary metal oxide semiconductor (CMOS) integrated process, which is conducive to reducing device failure rate, improving device yield and reducing cost. For example, the optical filter layer can be prepared by directly growing one or more layers of dielectric material on the image sensor, then etching, depositing metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer.

[0169] Based on the contents of the above embodiments, in this embodiment, the types of the artificial neural network include: feedforward neural network.

[0170] In this embodiment, the feedforward neural network (FNN), also known as the deep feedforward network (DFN) and the multi-layer perceptron (MLP), is the simplest neural network, in which neurons are arranged in layers. Each neuron is only connected to the neurons in the previous layer. It receives the output of the previous layer and outputs it to the next layer, and there is no feedback between the layers. The feedforward neural network has a simple structure, is easy to implement on hardware, and is widely used. It can approximate any continuous function and square integrable function with arbitrary precision, and can accurately implement any finite training sample set. The feedforward network is a static nonlinear mapping. Complex nonlinear processing capabilities can be obtained through the composite mapping of simple nonlinear processing units.

[0171] Based on the contents of the above embodiments, in this embodiment, a light-transmitting medium layer is arranged between the optical filter layer and the image sensor.

[0172] In this embodiment, it should be noted that a light-transmitting medium layer is provided between the optical filter layer and the image sensor, so as to effectively separate the optical filter layer from the image sensor layer and avoid mutual interference between the two.

[0173] Based on the contents of the above embodiments, in this embodiment, the image sensor is front-illuminated, comprising: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on a side of the metal wire layer away from the light detection layer; or,

[0174] The image sensor is back-illuminated and comprises: a light detection layer and a metal wire layer arranged from top to bottom, and the optical filter layer is integrated on a side of the light detection layer away from the metal wire layer.

[0175] In this embodiment, in a front-illuminated image sensor, the silicon detection layer is below the metal wire layer, and the optical filter layer 1 is directly integrated onto the metal wire layer.

[0176] In this embodiment, the back-illuminated image sensor is different from the front-illuminated image sensor in that the silicon detection layer is above the metal wire layer, and the optical filter layer 1 is directly integrated onto the silicon detection layer.

[0177] It should be noted that, for a back-illuminated image sensor, the silicon detection layer is above the metal wire layer, which can reduce the influence of the metal wire layer on the incident light, thereby improving the quantum efficiency of the device.

[0178] According to the above content, this embodiment uses the optical filter layer as the input layer of the artificial neural network, uses the image sensor as the linear layer of the artificial neural network, and uses the filtering effect of the optical filter layer on the incident light entering the optical filter layer as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor in the optical artificial neural network fingerprint recognition chip provided by this embodiment realize the relevant functions of the input layer and the linear layer in the artificial neural network by hardware, so that when the fingerprint recognition chip is used for recognition processing later, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, which can greatly reduce the power consumption and delay of the artificial neural network processing. In addition, since this embodiment simultaneously utilizes the image information of the fingertips at the end of the human finger and the spectral information at different points in space, it can more accurately realize the recognition processing of fingerprints.

[0179] Based on the same inventive concept, another embodiment of the present invention provides a fingerprint recognition device, comprising: at least one optical artificial neural network fingerprint recognition chip as described in the above embodiment.

[0180] In this embodiment, it should be noted that, due to the fast chip recognition speed, the fingerprint recognition device in this embodiment includes at least one optical artificial neural network fingerprint recognition chip described in the above embodiment. For example, the fingerprint recognition device may include two of the above chips, so that the fingerprints of the fingertips of the user's left and right hands can be recognized at the same time, further improving the efficiency of fingerprint recognition.

[0181] Since the fingerprint recognition device provided in this embodiment includes the optical artificial neural network fingerprint recognition chip described in the above embodiments, the fingerprint recognition device provided in this embodiment has all the beneficial effects of the optical artificial neural network fingerprint recognition chip described in the above embodiments. Since the above embodiments have already described this in detail, this embodiment will not repeat it.

[0182] Based on the same inventive concept, another embodiment of the present invention provides a method for preparing an optical artificial neural network fingerprint recognition chip as described in the above embodiment, such as Fig.13 As shown, the specific steps include:

[0183] Step 1310: preparing an optical filter layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor;

[0184] Step 1320, generating a processor capable of performing full connection processing and nonlinear activation processing on the signal;

[0185] Step 1330: connecting the image sensor and the processor;

[0186] The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the optical modulation structure through the optical modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light carrying information includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light;

[0187] The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different position points to obtain fingerprint recognition processing results; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light and / or radiated light on the fingertips at the end of a human finger.

[0188] This embodiment also includes a training process for the optical artificial neural network fingerprint recognition chip, which specifically includes:

[0189] Using input training samples and output training samples corresponding to the fingerprint recognition task, an optical artificial neural network fingerprint recognition chip containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters is trained to obtain light modulation structures, image sensors and processors that meet the training convergence conditions, and the light modulation structures, image sensors and processors that meet the training convergence conditions are used as the trained light modulation structures, image sensors and processors.

[0190] A light filter layer including a light modulation structure is prepared on the surface of the photosensitive area of ​​the image sensor, comprising:

[0191] Growing one or more layers of a preset material on the surface of the photosensitive region of the image sensor;

[0192] Dry-etching the one or more layers of the preset material to form a light modulation structure pattern to obtain an optical filter layer including the light modulation structure;

[0193] or performing embossing transfer on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure;

[0194] Or by externally dynamically regulating the one or more layers of the preset material, an optical filter layer including a light modulation structure is obtained;

[0195] or performing zone printing on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure;

[0196] or performing partitioned material growth on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure;

[0197] Alternatively, quantum dots are transferred to the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure.

[0198] When the optical artificial neural network fingerprint recognition chip is used for a fingerprint recognition processing task, the optical artificial neural network fingerprint recognition chip including different optical modulation structures, image sensors and processors with different full connection parameters and nonlinear activation parameters is trained using input training samples and output training samples corresponding to the recognition processing task to obtain an optical modulation structure, image sensor and processor that meet the training convergence conditions.

[0199] In this embodiment, it should be noted that Figure 1 As shown, the optical filter layer 1 can be prepared by directly growing one or more layers of dielectric material on the image sensor 2, then etching, depositing metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the optical modulation structure, each unit can have different modulation effects on light of different wavelengths within the target range, and the modulation effect is insensitive to the incident angle and polarization. Each unit in the optical filter layer 1 corresponds to one or more pixels on the image sensor 2. 1 is prepared directly on 2.

[0200] In this embodiment, it should be noted that, assuming that the image sensor 2 is a back-illuminated structure, the optical filter layer 1 can be prepared by directly etching the silicon detector layer of the back-illuminated image sensor and then depositing metal.

[0201] In addition, it should be noted that the light modulation structure on the optical filter layer can be obtained by dry etching the light modulation structure pattern of one or more layers of preset materials. Dry etching is to directly remove the unnecessary parts of one or more layers of preset materials on the surface of the photosensitive area of ​​the image sensor to obtain the optical filter layer containing the light modulation structure; or one or more layers of preset materials are subjected to imprint transfer. Imprint transfer is to prepare the required structure by etching on other substrates, and then transfer the structure to the photosensitive area of ​​the image sensor through materials such as PDMS to obtain the optical filter layer containing the light modulation structure; or by external dynamic regulation of one or more layers of preset materials. The external dynamic regulation is to use active materials, and then the external electrodes regulate the light modulation characteristics of the corresponding areas by changing the voltage to obtain the optical filter layer containing the light modulation structure; or one or more layers of preset materials are subjected to zone printing. Zone printing is to use printing technology in zones to obtain the optical filter layer containing the light modulation structure; or one or more layers of preset materials are subjected to zone material growth to obtain the optical filter layer containing the light modulation structure; or one or more layers of preset materials are subjected to quantum dot transfer to obtain the optical filter layer containing the light modulation structure.

[0202] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the optical artificial neural network fingerprint recognition chip in the above-mentioned embodiment, for detailed contents on some principles and structures, etc., please refer to the introduction of the above-mentioned embodiment, and this embodiment will not go into details.

[0203] Based on this, the optical artificial neural network fingerprint recognition chip provided by the embodiment of the present invention realizes a new intelligent chip that can realize the function of an artificial neural network. In the intelligent chip, the optical filter layer serves as the input layer of the artificial neural network, and the image sensor serves as the linear layer of the artificial neural network. At the same time, the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the input layer to the linear layer, that is, the optical filter layer and the image sensor in the intelligent chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present invention peels off the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and realizes the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the intelligent chip is used for subsequent intelligent processing of the artificial neural network, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, and only the processor in the intelligent chip needs to perform full connection with the electrical signal and nonlinear processing. The relevant processing of sexual activation is sufficient, which can greatly reduce the power consumption and delay during artificial neural network processing. In addition, the embodiment of the present invention can also simultaneously utilize the image information, spectral information, angle of incident light and phase information of the fingertips at the end of the human finger, that is, the incident light carries information at different points in the space of the fingertips at the end of the human finger. It can be seen that since the incident light carries information at different points in the space of the fingertips at the end of the human finger, it covers the image, composition, shape, three-dimensional depth, structure and other information of the fingertips at the end of the human finger. Therefore, when the identification and processing are performed based on the incident light carrying information at different points in the space of the fingertips at the end of the human finger, it can cover the image, composition, shape, three-dimensional depth, structure and other multi-dimensional information of the fingertips at the end of the human finger, thereby improving the accuracy of the fingerprint. It can be seen that the optical artificial neural network fingerprint recognition chip provided by the embodiment of the present invention can not only achieve the effects of low power consumption and low delay, but also improve the accuracy of fingerprint recognition, so that it can be better applied in the field of fingerprint recognition processing.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An optical artificial neural network fingerprint recognition chip, characterized in that: Used for fingerprint recognition tasks, comprising: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network; The optical filter layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the optical filter layer includes a light modulation structure. The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light carrying information includes light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light; the incident light includes reflected light, transmitted light and / or radiated light on the fingertips at the end of a human finger; The image sensor is used to convert the information carried by the incident light corresponding to different positions after being modulated by the optical filter layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signals are image signals modulated by the optical filter layer; The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain fingerprint recognition processing results; The optical artificial neural network fingerprint recognition chip includes a trained light modulation structure, an image sensor and a processor; The trained optical modulation structure, image sensor and processor refer to the optical modulation structure, image sensor and processor that meet the training convergence conditions by training the optical artificial neural network fingerprint recognition chip including different optical modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters using input training samples and output training samples corresponding to the fingerprint recognition task; The input training samples include incident light reflected, transmitted or radiated by samples of the end pads of human fingers with different fingerprints; and the output training samples include corresponding fingerprint recognition results.

2. The optical artificial neural network fingerprint recognition chip according to claim 1, characterized in that: When training an optical artificial neural network fingerprint recognition chip including different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

3. The optical artificial neural network fingerprint recognition chip according to any one of claims 1 to 2, characterized in that: The light modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the light modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

4. The optical artificial neural network fingerprint recognition chip according to any one of claims 1 to 2, characterized in that: The light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, each of which corresponds to one or more pixel points on the image sensor; and the structures of the various micro-nano units are the same or different.

5. The optical artificial neural network fingerprint recognition chip according to claim 4, characterized in that: The micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

6. The optical artificial neural network fingerprint recognition chip according to claim 4, characterized in that: The micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

7. The optical artificial neural network fingerprint recognition chip according to claim 6, characterized in that: Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

8. The optical artificial neural network fingerprint recognition chip according to claim 6, characterized in that: Each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

9. The optical artificial neural network fingerprint recognition chip according to claim 6, characterized in that: The micro-nano unit includes a plurality of micro-nano structure arrays, and there is one or more groups of empty structures.

10. The optical artificial neural network fingerprint recognition chip according to claim 6, characterized in that: The micro-nano unit has polarization-independent characteristics.

11. The optical artificial neural network fingerprint recognition chip according to claim 10, characterized in that: The micro-nano unit has four-fold rotational symmetry.

12. The optical artificial neural network fingerprint recognition chip according to claim 1, characterized in that: The optical filter layer is composed of one or more filter layers; The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, and adjustable Fabry-Perot resonant cavities.

13. The optical artificial neural network fingerprint recognition chip according to claim 12, characterized in that: The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

14. The optical artificial neural network fingerprint recognition chip according to claim 1, characterized in that: The thickness of the optical filter layer is 0.1λ~10λ, where λ represents the central wavelength of the incident light.

15. A fingerprint recognition device, characterized in that: include: At least one optical artificial neural network fingerprint recognition chip as described in any one of claims 1 to 14.

16. A method for preparing an optical artificial neural network fingerprint recognition chip according to any one of claims 1 to 14, characterized in that: include: Preparing an optical filter layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor; Generate a processor capable of fully connected signal processing and nonlinear activation processing; connecting the image sensor and the processor; The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the optical modulation structure through the optical modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the photosensitive area; the incident light carrying information includes light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light; The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to the different position points to obtain the fingerprint recognition processing result; the electrical signal is an image signal modulated by the optical filter layer, and the incident light includes reflected light, transmitted light and / or radiated light on the fingertips at the end of the human finger; The method further includes: a training process for the optical artificial neural network fingerprint recognition chip, specifically including: Using input training samples and output training samples corresponding to the fingerprint recognition task, an optical artificial neural network fingerprint recognition chip containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters is trained to obtain light modulation structures, image sensors and processors that meet the training convergence conditions, and the light modulation structures, image sensors and processors that meet the training convergence conditions are used as the trained light modulation structures, image sensors and processors.

17. The method for preparing the optical artificial neural network fingerprint recognition chip according to claim 16, characterized in that: A light filter layer including a light modulation structure is prepared on the surface of the photosensitive area of ​​the image sensor, comprising: Growing one or more layers of a preset material on the surface of the photosensitive region of the image sensor; Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light filter layer including the light modulation structure; or performing embossing transfer on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure; or obtaining an optical filter layer including an optical modulation structure by externally dynamically modulating the one or more layers of the preset material; or performing zone printing on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure; or performing partitioned growth on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure; Alternatively, quantum dots are transferred to the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure.

Images