Optical artificial neural network smelting end point monitoring chip and preparation method

Abstract

The present invention provides an optical artificial neural network smelting endpoint monitoring chip and a preparation method thereof, which are used for smelting endpoint monitoring tasks. The present invention uses an optical filter layer as an input layer and a linear layer of an artificial neural network, uses the filtering effect of the optical filter layer on incident light as a connection weight from the input layer to the linear layer, uses the square detection response of an image sensor as a first nonlinear activation function in a nonlinear layer of the artificial neural network, and uses a processor as a fully connected, second nonlinear activation function in a nonlinear layer and an output layer of the artificial neural network, so that the optical filter layer and the image sensor realize related functions of the input layer, the linear layer and the nonlinear activation function in the artificial neural network in a hardware manner, thereby greatly reducing power consumption and delay during artificial neural network processing.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an optical artificial neural network smelting endpoint monitoring chip and a preparation method thereof. Background Technology

[0002] Converter steelmaking is the most widely used and most efficient steelmaking method in the world. End point control is one of the key technologies in converter production. Accurate determination of the end point is of great significance in improving the quality of molten steel and shortening the smelting cycle. However, due to the instability of the raw materials entering the furnace, complex chemical reactions and strict requirements for the types of steel being produced, accurate control of the end point of smelting remains difficult. Accurate online detection of the end point carbon content and molten steel temperature has always been a difficult problem that needs to be solved urgently in the metallurgical industry around the world. At present, the industry mainly relies on manual experience or complex large-scale instruments and equipment to measure the furnace mouth temperature and the qualitative measurement of slag residues for end point control, which has low accuracy and high cost. SUMMARY OF THE INVENTION

[0003] Aiming at the problems existing in the prior art, the present invention provides an optical artificial neural network smelting endpoint monitoring chip and a preparation method.

[0004] Specifically, the embodiment of the present invention provides the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides an optical artificial neural network smelting endpoint monitoring chip for smelting endpoint monitoring tasks, comprising: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer, the linear layer and the connection weight from the input layer to the linear layer of the artificial neural network, the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the full connection of the artificial neural network, the second nonlinear activation function in the nonlinear layer and the output layer;

[0006] The optical filter layer is arranged on the surface of the image sensor, and the optical filter layer includes an optical modulation structure, and the optical modulation structure is used to perform different spectrum modulations on the incident light entering different positions of the optical modulation structure, so as to obtain the incident light carrying information corresponding to different positions on the surface of the image sensor; the incident light includes reflected light, transmitted light and / or radiated light from the steelmaking furnace mouth;

[0007] The image sensor converts the information carried by the incident light corresponding to different positions after being modulated by the optical filter layer into electrical signals corresponding to different positions through the square detection response, and sends the electrical signals corresponding to different positions to the processor;

[0008] The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process on the electrical signals corresponding to different position points and a second non-linear activation process to obtain a smelting end-point monitoring result;

[0009] Among them, the smelting end-point monitoring task includes identifying the smelting end-point, and the smelting end-point monitoring result includes a smelting end-point identification result.

[0010] Furthermore, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0011] Furthermore, the smelting end-point monitoring task further includes identifying the carbon content and / or molten steel temperature during the smelting process, and the smelting end-point monitoring result includes the carbon content and / or molten steel temperature identification result during the smelting process.

[0012] Furthermore, the optical artificial neural network smelting end-point monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;

[0013] The trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters that are trained using input training samples and output training samples corresponding to the smelting end-point monitoring task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions; or, the trained optical modulation structure, image sensor, and processor refer to the optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters that are trained using input training samples and output training samples corresponding to the smelting end-point monitoring task to obtain an optical modulation structure, image sensor, and processor that meet the training convergence conditions;

[0014] Among them, the input training samples include the incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth that has been smelted to the end-point and that has not been smelted to the end-point; the output training samples include the determination result of whether it has been smelted to the end-point.

[0015] Furthermore, when the smelting end-point monitoring task further includes identifying the carbon content and / or molten steel temperature during the smelting process, correspondingly, the input training samples further include the incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth smelted to different carbon contents and / or molten steel temperatures, and the output training samples further include the corresponding carbon content and molten steel temperature.

[0016] Further, when training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0017] Further, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0018] Further, the optical modulation structure in the optical filter layer includes a unit array composed of multiple micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0019] Further, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0020] Further, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different.

[0021] Further, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0022] Further, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0023] Further, one or more groups of the multiple groups of micro-nano structure arrays included in the micro-nano unit are empty structures.

[0024] Further, the micro-nano unit has four-fold rotational symmetry.

[0025] Further, the optical filter layer is composed of one or more layers of structures;

[0026] Each layer of structure is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP nanostructures, tunable Fabry-Perot resonators.

[0027] Further, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0028] Further, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0029] In a second aspect, an embodiment of the present invention provides an intelligent smelting control device, including the optical artificial neural network smelting endpoint monitoring chip as described above.

[0030] In a third aspect, an embodiment of the present invention provides a method for manufacturing the optical artificial neural network smelting endpoint monitoring chip as described above, including:

[0031] Preparing an optical filter layer including an optical modulation structure on the surface of the image sensor;

[0032] Generating a processor with a fully connected signal processing function or generating a processor with a fully connected signal processing function and a second non-linear activation processing function;

[0033] Connecting the image sensor and the processor;

[0034] Wherein, the optical filter layer is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively through the optical modulation structure, so as to obtain an optical field distribution signal corresponding to different position points on the surface of the image sensor;

[0035] The image sensor converts the optical field distribution signal corresponding to different position points after being modulated by the optical filter layer into an electrical signal corresponding to different position points through square-law detection response, and sends the electrical signal corresponding to different position points to the processor;

[0036] The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points, and obtains a smelting endpoint monitoring result.

[0037] Further, it further includes: the training process of the optical artificial neural network smelting endpoint monitoring chip, specifically including:

[0038] Using the input training samples and output training samples corresponding to the smelting end-point monitoring task, training an optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor;

[0039] Or, using the input training samples and output training samples corresponding to the smelting end-point monitoring task, training an optical artificial neural network smelting end-point monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0040] Further, preparing an optical filter layer including an optical modulation structure on the surface of the image sensor, including:

[0041] Growing one or more layers of a preset material on the surface of the image sensor;

[0042] Etching a pattern of the optical modulation structure on the one or more layers of the preset material to obtain an optical filter layer including an optical modulation structure;

[0043] Or performing imprint transfer on the one or more layers of the preset material to obtain an optical filter layer including an optical modulation structure;

[0044] Or obtaining an optical filter layer including an optical modulation structure by applying external dynamic modulation to the one or more layers of the preset material;

[0045] Or performing zone printing on the one or more layers of the preset material to obtain an optical filter layer including an optical modulation structure;

[0046] Or performing zone growth on the one or more layers of the preset material to obtain an optical filter layer including an optical modulation structure;

[0047] Or performing quantum dot transfer on the one or more layers of the preset material to obtain an optical filter layer including an optical modulation structure.

[0048] An embodiment of the present invention realizes a brand-new optical artificial neural network smelting end-point monitoring chip capable of implementing the functions of an artificial neural network, which is used for the smelting end-point monitoring task. In this optical artificial neural network smelting end-point monitoring chip, the optical filter layer corresponds to the input layer and the linear layer of the artificial neural network, and the image sensor corresponds to a part of the non-linear layer of the artificial neural network; the processor corresponds to another part of the non-linear layer of the artificial neural network and the output layer. Specifically, the optical filter layer is disposed on the surface of the image sensor. The optical filter layer includes an optical modulation structure, and the optical modulation structure is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor. In the embodiment of the present invention, the modulation effect of the optical modulation structure on the optical filter layer on the incident light is equivalent to the connection weight from the input layer to the linear layer. At the same time, in the embodiment of the present invention, the image sensor performs the first non-linear activation process on the information carried by the incident light corresponding to different position points after modulation by the optical filter layer through square-law detection response and converts it into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor. The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in this optical artificial neural network smelting end-point monitoring chip, the optical filter layer corresponds to the input layer, the linear layer of the artificial neural network, and the connection weight from the input layer to the linear layer. The square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor corresponds to the fully connected and output layer of the artificial neural network, or the processor corresponds to the fully connected, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. That is to say, the optical filter layer and the image sensor in this optical artificial neural network smelting end-point monitoring chip realize the related functions of the input layer, the linear layer, and a part of the non-linear activation function in the artificial neural network. That is to say, the embodiment of the present invention strips the input layer, the linear layer, and a part or all of the non-linear activation functions in the artificial neural network implemented by software in the prior art, and uses hardware to implement the input layer, the linear layer, and a part or all of the non-linear activation functions in the artificial neural network. As a result, when using this optical artificial neural network smelting end-point monitoring chip for artificial neural network intelligent processing later, it is no longer necessary to perform complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and a part or all of the non-linear activation functions. It only needs to perform a fully connected process or a fully connected and second non-linear activation process on the electrical signals by the processor in the optical artificial neural network smelting end-point monitoring chip. In this way, the power consumption and delay during artificial neural network processing can be greatly reduced.

[0049] As can be seen, the embodiments of the present invention provide a novel optoelectronic chip for quickly and accurately identifying the smelting end point. This chip embeds the artificial neural network part into the hardware device to achieve safe, reliable, fast, and accurate control of the smelting end point. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0051] Figure 1 It is a schematic structural diagram of an optical artificial neural network smelting end point monitoring chip provided by the first embodiment of the present invention;

[0052] Figure 2 It is a schematic diagram of the recognition principle of an optical artificial neural network smelting end point monitoring chip provided by an embodiment of the present invention;

[0053] Figure 3 It is a schematic diagram of disassembling an optical artificial neural network smelting end point monitoring chip provided by an embodiment of the present invention;

[0054] Figure 4 It is a schematic diagram of identifying the furnace mouth during the smelting process to determine the smelting end point provided by an embodiment of the present invention;

[0055] Figure 5 It is a top view of an optical filter layer provided by an embodiment of the present invention;

[0056] Figure 6 It is a top view of another optical filter layer provided by an embodiment of the present invention;

[0057] Figure 7 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0058] Figure 8 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0059] Figure 9 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0060] Figure 10 It is a top view of yet another optical filter layer provided by an embodiment of the present invention;

[0061] Figure 11 It is a schematic diagram of the broadband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0062] Figure 12 It is a schematic diagram of the narrowband filtering effect of the micro-nano structure provided by an embodiment of the present invention;

[0063] Figure 13 It is a schematic diagram of the structure of a front-illuminated image sensor provided by an embodiment of the present invention;

[0064] Figure 14 It is a schematic diagram of the structure of a back-illuminated image sensor provided by an embodiment of the present invention;

[0065] Figure 15 It is a schematic flow chart of the preparation method of the optical artificial neural network end-point monitoring chip provided by the third embodiment of the present invention. Detailed implementation manners

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] Converter steelmaking is the most widely used and most efficient steelmaking method in the world. Smelting endpoint control is one of the key technologies in converter production. Accurate judgment of the endpoint is of great significance in improving the quality of molten steel and shortening the smelting cycle. However, due to the instability of the raw materials entering the furnace, complex chemical reactions and the strictness of the steel grades, accurate control of the smelting endpoint is still difficult. Accurate online detection of the endpoint carbon content and molten steel temperature has always been a problem that needs to be solved urgently in the metallurgical industry around the world. At present, the endpoint control in the industry mainly relies on manual experience or complex large-scale instruments and equipment to measure the furnace mouth temperature and slag residue qualitative measurement, which has low accuracy and high cost. Therefore, realizing an online smelting endpoint control chip with micro volume, low cost, safety and reliability and easy to integrate the control system later is of great significance to the development of the steel smelting industry and the automation of the industry. It is a true industry 4.0. The embodiment of the present invention is based on the spectral radiation information feature analysis of the converter steelmaking furnace mouth, and carries out online non-contact measurement of converter steelmaking smelting endpoint, molten steel temperature, carbon content, etc., thereby realizing accurate online control of the smelting endpoint. Specifically, the embodiment of the present invention provides an optical artificial neural network smelting endpoint monitoring chip for smelting endpoint monitoring tasks, that is, the chip can accurately and contactlessly identify the smelting endpoint, and the structure of the chip is as follows: the optical filter layer in the chip corresponds to the input layer, linear layer and the connection weight from the input layer to the linear layer of the artificial neural network, and the square detection response of the image sensor in the chip corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor in the chip corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the full connection, the second nonlinear activation function in the nonlinear layer of the artificial neural network and the output layer. The embodiment of the present invention uses the optical filter layer and the image sensor to project the spatial spectrum information of the steelmaking furnace mouth into an electrical signal, and then implements the full connection processing or the full connection processing and the second nonlinear activation processing of the electrical signal in the processor. It can be seen that the embodiment of the present invention uses a hardware-type optical artificial neural network chip to achieve accurate identification of the smelting endpoint. The embodiment of the present invention can omit the complex signal processing and algorithm processing corresponding to the input layer, the linear layer and part or all of the nonlinear activation functions in the prior art, thereby greatly reducing the delay.In addition, embodiments of the present invention can also utilize one or more of the image information, spectral information, incident light angle, and incident light phase information of the steelmaking furnace mouth, that is, the incident light at different points in the space of the steelmaking furnace mouth carries information. It can be seen that since the incident light at different points in the space of the steelmaking furnace mouth carries information that covers the image, composition, shape, three-dimensional depth, structure, etc. of the steelmaking furnace mouth, when performing recognition processing based on the information carried by the incident light at different points in the space of the steelmaking furnace mouth, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, and structure of the steelmaking furnace mouth, thereby improving the accuracy of smelting end point recognition. It can be seen that the optical artificial neural network smelting end point monitoring chip provided by the embodiments of the present invention well solves the world-wide problem of difficult monitoring of the smelting end point, realizes non-contact accurate monitoring. At the same time, due to the use of the optical artificial neural network chip, complex signal processing and algorithm processing related to the input layer, linear layer, and part or all of the non-linear activation functions in the optical artificial neural network can be replaced by pre-prepared hardware (optical filter layer and image sensor), thereby greatly reducing processing correspondence and delay, and enabling fast, accurate, safe, and reliable smelting end point detection. The content provided by the present invention will be explained and described in detail through specific embodiments below.

[0068] As Figure 1 shown, the optical artificial neural network smelting end point monitoring chip provided by the first embodiment of the present invention is used for the smelting end point monitoring task, and includes: an optical filter layer 1, an image sensor 2, and a processor 3; the optical filter layer 1 corresponds to the input layer, linear layer, and connection weights from the input layer to the linear layer of the artificial neural network, and the square-law detection response of the image sensor 2 corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor 3 corresponds to the fully connected layer and output layer of the artificial neural network, or, the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network;

[0069] The optical filter layer 1 is disposed on the surface of the image sensor or the surface of the photosensitive area of the image sensor. The optical filter layer 1 includes an optical modulation structure. The optical filter layer 1 is used to perform spectral modulation of intensity modulation with wavelength variation on the incident light entering different position points of the optical modulation structure through the optical modulation structure, that is, perform different intensity modulations on incident light of different wavelengths, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor;

[0070] In this embodiment, the square-law detection response of the image sensor 2 means that the image sensor detects the intensity information of the incident light field, and the intensity information of the incident light field is the square of the modulus of the optical field signal. That is, the image sensor 2 converts the incident light-carrying information corresponding to different position points after being modulated by the optical filter layer 1 into electrical signals corresponding to different position points through the square-law detection response after the first non-linear activation process, and sends the electrical signals corresponding to different position points to the processor 3; the electrical signal is the image signal after being modulated by the optical filter layer; wherein, the incident light includes reflected light, transmitted light, and / or radiation light from the steelmaking furnace mouth.

[0071] The processor 3 is configured to perform a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network, and further obtain the smelting end-point monitoring result.

[0072] In this embodiment, the incident light-carrying information contains image information and / or various optical space information of the target object to be processed by the optical artificial neural network intelligent chip. For example, the incident light-carrying information includes at least one of light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light.

[0073] In this embodiment, the optical filter layer 1 is disposed on the surface of the image sensor. The optical filter layer 1 includes an optical modulation structure. The optical filter layer 1 is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain a modulated optical field distribution signal corresponding to different position points on the surface of the image sensor. Thus, it can be seen that in the embodiment, the modulation effect of the optical modulation structure on the optical filter layer on the incident light can be regarded as the connection weight from the input layer to the linear layer.

[0074] In this embodiment, when the image sensor 2 performs photoelectric conversion on the incident light-carrying information after modulation, since the image sensor 2 can only detect the intensity information of light, the electrical signal obtained by processing the optical field distribution signal is proportional to the square of the modulus of the optical field distribution signal. Therefore, the image sensor 2 has a square-law detection response, so the image sensor 2 can be regarded as a part of the non-linear layer of the artificial neural network. That is, the square-law detection response of the image sensor 2 can be regarded as the first non-linear activation function of the artificial neural network.

[0075] In this embodiment, the image sensor 2 converts the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer 1 into electrical signals corresponding to different position points through a first non-linear activation process using square-law detection response, that is, the image signals after being modulated by the optical filter layer. At the same time, the processor 3 connected to the image sensor 2 is used to perform a fully connected process or a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signals of the artificial neural network.

[0076] In this embodiment, the optical filter layer 1 includes an optical modulation structure. The incident light (such as the reflected light, transmitted light, radiation light, etc. of the target to be recognized) entering different position points of the optical modulation structure is subjected to spectral modulation of different intensities by the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor 2.

[0077] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the optical modulation structure. For example, different modulation intensities can be achieved by designing different optical modulation structures (such as changing the shape and / or size parameters of the optical modulation structure).

[0078] In this embodiment, it can be understood that the optical modulation structures at different positions on the optical filter layer 1 have different spectral modulation effects on the incident light. The modulation intensity of the optical modulation structure on different wavelength components of the incident light corresponds to the connection strength of the linear layer of the artificial neural network, that is, the connection weights corresponding to the input layer and from the input layer to the linear layer. It should be noted that the optical filter layer 1 is composed of multiple optical filter units. The optical modulation structures at different positions within each optical filter unit are different, so they have different spectral modulation effects on the incident light; the optical modulation structures at different positions between the optical filter units can be the same or different, so they have the same or different spectral modulation effects on the incident light.

[0079] In this embodiment, the image sensor 2 converts the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer 1 into electrical signals corresponding to different position points through a first non-linear activation process using square-law detection response, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to a part of the non-linear layer of the neural network.

[0080] In this embodiment, the processor 3 performs a fully connected process on the electrical signals at different position points or, the processor 3 performs a fully connected process and a second non-linear activation process on the electrical signals at different position points, and then obtains the output signals of the artificial neural network.

[0081] It can be understood that in this embodiment, the image sensor 2 corresponds to a part of the non-linear layer of the neural network, and the processor 3 corresponds to another part of the non-linear layer of the neural network and the output layer. It can also be understood as corresponding to the remaining layers (all other layers) in the neural network except the input layer, the linear layer, and the first non-linear activation function in the non-linear layer.

[0082] In this embodiment, it should be noted that the square-law detection response of the image sensor 2 corresponds to the first non-linear activation function in the non-linear layer of the neural network. In this case, in the processor, only full connection processing may be performed without performing a second non-linear activation processing, or both full connection processing and a second non-linear activation processing may be performed. It can be specifically determined according to the actual application scenario of the chip, and this embodiment does not make any limitations in this regard.

[0083] In addition, it should be added that the processor 3 can be disposed within the optical artificial neural network tapping end monitoring chip, that is, the processor 3 can be disposed together with the filter layer 1 and the image sensor 2 within the optical artificial neural network tapping end monitoring chip, or can be separately disposed outside the optical artificial neural network tapping end monitoring chip and connected to the image sensor 2 within the optical artificial neural network tapping end monitoring chip through a data line or a connecting device. This embodiment does not make any limitations in this regard.

[0084] In addition, it should be noted that the processor 3 can be implemented by a computer, or can be implemented by an ARM or FPGA circuit board with certain computing capabilities, or can also be implemented by a microprocessor. This embodiment does not make any limitations in this regard. In addition, as described above, the processor 3 can be integrated within the optical artificial neural network tapping end monitoring chip, or can be disposed independently outside the optical artificial neural network tapping end monitoring chip. When the processor 3 is disposed independently outside the optical artificial neural network tapping end monitoring chip, the electrical signal in the image sensor 2 can be read out to the processor 3 through a signal readout circuit, and then the processor 3 performs full connection processing and non-linear activation processing on the read electrical signal.

[0085] In this embodiment, it can be understood that when the processor 3 performs the second non-linear activation processing, it can be implemented by a non-linear activation function. For example, it can use a Sigmoid function, a Tanh function, a ReLU function, etc. This embodiment does not make any limitations in this regard.

[0086] In this embodiment, the optical filter layer 1 corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The image sensor 2 corresponds to a part of the non-linear layer of the artificial neural network. That is, the square-law detection response of the image sensor 2 corresponds to the first non-linear activation function of the artificial neural network. The image sensor 2 is used to perform non-linear activation processing on the information carried by the incident light at different spatial positions through the square-law detection response and then convert it into an electrical signal. The processor 3 corresponds to the remaining layers of the artificial neural network, fully connects the electrical signals at different positions, and can further obtain the output signal of the artificial neural network through the second non-linear activation function.

[0087] As Figure 2 shown on the left, the optical artificial neural network end-point monitoring chip for smelting includes an optical filter layer 1, an image sensor 2, and a processor 3. In Figure 2 it, the processor 3 is implemented by a signal readout circuit and a computer. As Figure 2 shown on the right, the optical filter layer 1 in the optical artificial neural network end-point monitoring chip for smelting corresponds to the input layer and the linear layer of the artificial neural network. The image sensor 2 corresponds to a part of the non-linear layer of the artificial neural network. The processor 3 corresponds to another part of the non-linear layer and the output layer of the artificial neural network. The filtering effect of the optical filter layer 1 on the incident light entering the optical filter layer 1 corresponds to the connection weights from the input layer to the linear layer. The square-law detection response of the image sensor 2 corresponds to the first non-linear activation function of the artificial neural network. It can be seen that the optical filter layer and the image sensor in the optical artificial neural network end-point monitoring chip provided in this embodiment implement the related functions of the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network in a hardware manner, so that subsequent complex signal processing and algorithm processing corresponding to the input layer and the linear layer (such as omitting calculations such as the connection weights from the input layer to the linear layer) are not required when using this optical artificial neural network end-point monitoring chip for intelligent processing. This can significantly reduce the power consumption and delay during the processing of the artificial neural network. In addition, since this embodiment simultaneously utilizes the image information, spectral information, incident light angle information, and incident light phase information at different points in the space of the steelmaking furnace mouth, the intelligent processing of the steelmaking furnace mouth can be more accurately realized.

[0088] As Figure 2 shown on the right, the optical filter layer 1 has different broadband spectral modulation effects on the incident light, projects / connects the incident light spectrum P λ onto the outgoing light field E N ; the square-law detection response of the image sensor 2 corresponds to a part of the non-linear activation function of the optical artificial neural network, and converts the outgoing light field E N of the optical filter layer 1 into the photocurrent response I NProcessor 3 includes a signal readout circuit and a computer. The signal readout circuit in processor 3 reads out the photocurrent response I N Then it is transmitted to the computer, which performs full connection processing of the electrical signal or performs nonlinear activation processing again, and finally outputs the result.

[0089] If Figure 3 As shown in FIG. 1 , the light modulation structure on the optical filter layer 1 is integrated on the surface of the image sensor 2 to modulate the incident light, project / connect the spectrum information of the incident light to different pixel points of the image sensor 2, and obtain an electrical signal containing the spectrum information of the incident light and the image information. That is, after the incident light passes through the optical filter layer 1, it is converted into an electrical signal after nonlinear activation by the square detection response of the image sensor 2, forming an image containing the spectrum information of the incident light. Finally, the processor 3 connected to the image sensor 2 processes the electrical signal containing the image information, spectrum information, angle information of the incident light, and phase information of the incident light at different points in space, and then obtains the output result.

[0090] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0091] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, the target object may be identified by using multiple information including the image information, spectrum information, angle of the incident light, and phase information of the incident light at the same time, thereby realizing more accurate intelligent identification of the target object.

[0092] It can be seen that the optical artificial neural network smelting endpoint monitoring chip provided in this embodiment can actually use one or more information of the image information, spectrum information, angle of incident light and phase information of incident light at the steelmaking furnace mouth, that is, the incident light at different points in space carries information, and an artificial neural network is embedded in the hardware. From the spatial image, spectrum, angle, and phase information, material composition, image shape, three-dimensional depth and other information can be further extracted, thereby solving the problem of difficulty in accurately identifying the smelting endpoint mentioned in the background technology part. At the same time, the embodiment of the present invention can save the complex signal processing and algorithm processing corresponding to the input layer, linear layer and part of the nonlinear activation function in the prior art, thereby achieving low power consumption and low latency. It can be seen that the optical artificial neural network smelting endpoint monitoring chip provided in the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, thereby quickly and accurately identifying the smelting endpoint.

[0093] ​​The optical artificial neural network smelting endpoint monitoring chip and preparation method provided by the embodiments of the present invention realize a brand-new optical artificial neural network smelting endpoint monitoring chip capable of realizing the functions of an artificial neural network, which is used for the smelting endpoint monitoring task. In this optical artificial neural network smelting endpoint monitoring chip, the optical filter layer corresponds to the input layer and the linear layer of the artificial neural network, and the image sensor corresponds to a part of the non-linear layer of the artificial neural network; the processor corresponds to another part of the non-linear layer of the artificial neural network and the output layer. Specifically, the optical filter layer is arranged on the surface of the image sensor, and the optical filter layer includes an optical modulation structure. The optical modulation structure is used to perform different spectral modulations on the incident light entering different position points of the optical modulation structure respectively, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor. In the embodiments of the present invention, the modulation effect of the optical modulation structure on the optical filter layer on the incident light is equivalent to the connection weight from the input layer to the linear layer. At the same time, in the embodiments of the present invention, the image sensor converts the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer into electrical signals corresponding to different position points through square-law detection response for the first non-linear activation process, and sends the electrical signals corresponding to different position points to the processor. The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in this optical artificial neural network smelting endpoint monitoring chip, the optical filter layer corresponds to the input layer, the linear layer, and the connection weight from the input layer to the linear layer of the artificial neural network, and the square-law detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network; the processor corresponds to the fully connected and output layers of the artificial neural network, or the processor corresponds to the fully connected, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. That is to say, the optical filter layer and the image sensor in this optical artificial neural network smelting endpoint monitoring chip realize the related functions of the input layer, the linear layer, and part of the non-linear activation function in the artificial neural network. That is to say, the embodiments of the present invention strip the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network implemented by software in the prior art, and use hardware to implement these structures of the input layer, the linear layer, and part or all of the non-linear activation functions in the artificial neural network. As a result, when using this optical artificial neural network smelting endpoint monitoring chip for artificial neural network intelligent processing subsequently, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part or all of the non-linear activation functions. It only needs to perform a fully connected process or a fully connected and second non-linear activation process on the electrical signals by the processor in the optical artificial neural network smelting endpoint monitoring chip. In this way, the power consumption and delay during artificial neural network processing can be greatly reduced.It can be seen that in the embodiments of the present invention, the optical filter layer serves as the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The square detection response of the image sensor serves as the first non-linear activation function in the non-linear layer of the artificial neural network. The processor serves as the fully connected layer and the output layer of the artificial neural network, or the processor corresponds to the fully connected layer, the second non-linear activation function in the non-linear layer, and the output layer of the artificial neural network. It can be seen that the embodiments of the present invention can not only eliminate the complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and a part of the non-linear activation function in the prior art, but also actually utilize the image information, spectral information, the angle of the incident light, and the phase information of the incident light of the steelmaking furnace mouth at the same time, that is, the incident light at different points in the space of the steelmaking furnace mouth carries information. It can be seen that since the incident light at different points in the space of the steelmaking furnace mouth carries information covering the image, composition, shape, three-dimensional depth, structure, etc. of the steelmaking furnace mouth, when performing identification processing based on the information carried by the incident light at different points in the space of the steelmaking furnace mouth, it can cover multi-dimensional information such as the image, composition, shape, three-dimensional depth, structure, etc. of the steelmaking furnace mouth, thereby solving the problem of difficult accurate identification of the smelting end point mentioned in the background art. It can be seen that the optical artificial neural network smelting end point monitoring chip provided by the embodiments of the present invention can simultaneously achieve the effects of low power consumption, low latency, and high recognition rate, and thus can quickly and accurately identify the smelting end point.

[0094] It can be seen that the embodiments of the present invention provide a new type of optoelectronic chip for accurately identifying the smelting end point. This chip embeds the artificial neural network part into the hardware device to achieve safe, reliable, fast, and accurate smelting end point control.

[0095] Based on the content of the above embodiments, in this embodiment, the smelting end point monitoring task further includes identifying the carbon content and / or the molten steel temperature during the smelting process, and the smelting end point monitoring result includes the identification results of the carbon content and / or the molten steel temperature during the smelting process.

[0096] Based on the content of the above embodiments, in this embodiment, the optical artificial neural network smelting end point monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;

[0097] The trained optical modulation structure, image sensor, and processor refer to the optical modulation structure, image sensor, and processor that meet the training convergence conditions obtained by training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task; or, the trained optical modulation structure, image sensor, and processor refer to the optical modulation structure, image sensor, and processor that meet the training convergence conditions obtained by training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task.

[0098] Among them, the input training samples include incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth during smelting to the endpoint and not smelting to the endpoint; the output training samples include the determination results of whether smelting has reached the endpoint.

[0099] Based on the content of the above embodiments, in this embodiment, when the smelting endpoint monitoring task further includes identifying the carbon content and / or molten steel temperature during smelting, correspondingly, the input training samples further include incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth during smelting to different carbon contents and / or molten steel temperatures, and the output training samples further include the corresponding carbon content and / or molten steel temperature.

[0100] It can be seen that the detectable samples in this embodiment include, but are not limited to, the endpoint control of converter steelmaking, and can also introduce artificial neural network training to detect the temperature and material elements of the smelting furnace mouth, which is extremely easy to integrate with the backend industrial control system, has high recognition accuracy, and relatively accurate qualitative analysis.

[0101] In this embodiment, it can be understood that the input training samples include incident light reflected, transmitted, and / or radiated by the steelmaking furnace mouth in the corresponding intelligent processing task; the output training samples include the intelligent processing results of the steelmaking furnace mouth, such as smelting endpoint recognition results, temperature recognition results during smelting, carbon content recognition results during smelting, etc.

[0102] In this embodiment, the reflected light, transmitted light, and / or radiated light of the steelmaking furnace mouth enter the trained optical artificial neural network smelting endpoint monitoring chip to obtain intelligent processing results such as whether it is the smelting endpoint, the furnace mouth temperature, and the content of material elements.

[0103] In this embodiment, taking the recognition task of the smelting end point as an example for illustration, it can be understood that when using the optoelectronic artificial neural network smelting end point monitoring chip for the recognition task, first, the optoelectronic artificial neural network smelting end point monitoring chip needs to be trained. Here, training the optoelectronic artificial neural network smelting end point monitoring chip means determining the optical modulation structure applicable to the current recognition task, as well as the fully connected parameters and non-linear activation parameters applicable to the current recognition task through training.

[0104] It can be understood that since the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weights from the input layer to the linear layer of the artificial neural network, therefore, during training, changing the optical modulation structure in the optical filter layer is equivalent to changing the connection weights from the input layer to the linear layer of the artificial neural network. By the training convergence condition, the optical modulation structure applicable to the current recognition task, as well as the fully connected parameters and non-linear activation parameters applicable to the current recognition task, are determined, thus completing the training of the optoelectronic artificial neural network smelting end point monitoring chip.

[0105] It can be understood that after training the optoelectronic artificial neural network smelting end point monitoring chip, the optoelectronic artificial neural network smelting end point monitoring chip can be used to perform the recognition task. Specifically, the incident light carrying the image information and spatial spectral information of the steelmaking furnace mouth enters the optical filter layer 1 of the trained optoelectronic artificial neural network smelting end point monitoring chip. The optical modulation structure in the optical filter layer 1 modulates the incident light. The intensity of the modulated optical signal is detected by the image sensor 2 and converted into an electrical signal, and then the processor 3 performs a fully connected process or simultaneously performs a fully connected and second non-linear activation process to obtain the recognition result of whether it is the smelting end point currently.

[0106] As Figure 4 shown, the complete process of recognizing the steelmaking furnace mouth to determine whether it is the smelting end point is as follows: The light emitted by the furnace mouth flame 100 of the steelmaking converter 200 is collected by the optoelectronic artificial neural network smelting end point monitoring chip 300. After being processed by the optical filter layer, image sensor, and processor in the optoelectronic artificial neural network smelting end point monitoring chip, the recognition result of whether it is the smelting end point currently can be obtained.

[0107] Thus, it can be seen that the purpose of this embodiment is to provide a novel optoelectronic chip for smelting end point control. This chip consists of an optical filter layer forming the input layer and linear layer of the optoelectronic artificial neural network, and an image sensor forming a part of the non-linear activation function of the non-linear layer of the optoelectronic artificial neural network. It can collect the image information and spectral information of the furnace mouth during the smelting process online, and can achieve fast, accurate, safe, and reliable detection and qualitative analysis of the smelting end point, furnace mouth temperature, carbon element, etc.

[0108] The chip directly fabricates micro-nano modulation structures on the surface of the photosensitive area of the image sensor. Several discrete or continuous micro-nano structures form a unit. The micro-nano modulation structures at different positions have different spectral modulation effects on the incident light, jointly constituting an optical filter layer. The modulation intensities of these micro-nano modulation structures on different wavelength components of the incident light correspond to the connection strengths (linear layer weights) of the artificial neural network. At the same time, the square detection response of the image sensor performs the first non-linear activation process on the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer and converts it into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor. The processor performs a fully connected process on the electrical signals corresponding to different position points, or the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. For the control of the smelting end point, the images and spectral information of the converter mouth at the end point can be collected first, and the weights of the linear layer, that is, the system function of the optical filter layer, can be obtained through data training, and then the required structure of the optical filter layer can be reversely designed and integrated above the image sensor. During actual use, during the actual smelting end point control process, by further training and optimizing the weights of the fully connected layer of the electrical signals using the output of the fabricated optical filter layer, a high-accuracy optical artificial neural network can be realized to complete the accurate judgment of the end point moment during the smelting process.

[0109] It can be understood that the chip actually utilizes both the image information and spectral information of the converter mouth at the end point moment, thereby obtaining accurate online mouth temperature and element content, improving the accuracy of the judgment of the smelting end point moment; and partially implementing the artificial neural network in hardware, improving the speed of the judgment of the smelting end point moment. In addition, the chip solution can be mass-produced using the existing CMOS process, reducing the volume, power consumption and cost of the device, and facilitating integration with subsequent control systems.

[0110] It can be understood that for the recognition task, since the advantage of the optical artificial neural network smelting end point monitoring chip provided in this embodiment also lies in being able to obtain the image information, spectral information, incident light angle information and incident light phase information at different points in the space of the recognition object, therefore, to make full use of this advantage, the real recognition object is preferably used as the recognition object sample for the input training sample, rather than the two-dimensional image of the recognition object. Of course, this does not mean that the two-dimensional image cannot be used as the recognition object sample.

[0111] In this embodiment, the optical filter layer 1 is used as the input layer and the linear layer of the neural network, and the image sensor 2 is used as part of the non-linear layer of the neural network (i.e., the square-law detection response of the image sensor 2 is used as the first non-linear activation function of the neural network). In order to minimize the loss function of the neural network, the modulation intensity of different wavelength components in the incident light at the steelmaking furnace opening by the optical modulation structure in the optical filter layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the optical filter layer, the modulation intensity of different wavelength components in the incident light at the steelmaking furnace opening can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, and further optimizing the training of the neural network.

[0112] Therefore, in this embodiment, the optical modulation structure is obtained based on the training of the neural network. Through computer optical simulation of the training samples, the sample modulation intensity of different wavelength components of the incident light at the steelmaking furnace opening by the optical modulation structure in the training samples is obtained. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network for non-linear activation, and the neural network is trained using the training samples corresponding to the intelligent processing task until the neural network converges, and the corresponding training sample optical modulation structure is used as the optical filter layer for the corresponding intelligent processing task.

[0113] It can be seen that in this embodiment, by implementing the input layer and the linear layer (optical filter layer) of the neural network and part of the non-linear layer (the square-law detection response of the image sensor 2 as the first non-linear activation function of the neural network) at the physical layer, the complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part or all of the non-linear activation functions in the prior art can be omitted, thereby improving the processing speed and reducing the time delay. At the same time, the embodiments of the present invention actually utilize the image information, spectral information, angle of the incident light, and phase information of the incident light at the steelmaking furnace opening at the same time, that is, the incident light at different points in the space of the steelmaking furnace opening carries information. It can be seen that since the incident light at different points in the space of the steelmaking furnace opening carries information covering the image, composition, shape, three-dimensional depth, structure, etc. of the steelmaking furnace opening, when performing recognition processing based on the information carried by the incident light at different points in the space of the steelmaking furnace opening, multi-dimensional information such as the image, composition, shape, three-dimensional depth, structure, etc. of the steelmaking furnace opening can be covered, thereby solving the problem that it is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the steelmaking furnace opening mentioned in the background art.

[0114] Based on the content of the above embodiments, in this embodiment, when training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0115] In this embodiment, the optical modulation structure is designed through computer optical simulation, and the optical modulation structure is adjusted through optical simulation until the neural network converges, and then the corresponding optical modulation structure is determined as the size of the optical modulation structure to be finally fabricated, saving prototype production time and cost, improving product efficiency, and easily solving complex optical problems. For example, the optical modulation structure can be simulated and designed by FDTD software, and the optical modulation structure can be changed in optical simulation, so that the modulation intensity of the optical modulation structure on different incident lights can be accurately predicted, and it can be used as the connection weight between the input layer and the linear layer of the neural network to train the optical artificial neural network smelting endpoint monitoring chip and accurately obtain the optical modulation structure.

[0116] It can be understood that the images and spectral information of the converter mouth at the end point can be collected multiple times in advance and data training can be carried out, and the required micro-nano modulation structure can be designed and fabricated, and the input layer, linear layer, and the first non-linear activation function in the non-linear layer of the artificial neural network can be realized on the chip.

[0117] Thus, in this embodiment, by designing the optical modulation structure by means of computer optical simulation design, the prototype production time and cost of the optical modulation structure are saved, and the product efficiency is improved.

[0118] It can be understood that this chip actually utilizes the image information, spectral information, incident light angle information, and incident light phase information at different spatial points at the converter mouth at the same time, improving the accuracy of smelting endpoint recognition. This embodiment partially realizes the artificial neural network in hardware, improving the real-time performance of smelting endpoint recognition. In addition, this chip solution can be mass-produced using existing CMOS processes, reducing the volume, power consumption, and cost of the device.

[0119] The structural schematic diagram of the optical artificial neural network smelting endpoint monitoring chip based on the micro-nano modulation structure and image sensor provided in this embodiment is as Figure 2As shown in the figure, it includes an optical filter layer 1, an image sensor 2, and a processor 3. The optical filter layer 1 corresponds to the input layer and the linear layer of the optical artificial neural network. Among them, several discrete or continuous micro-nano modulation structures form a unit. These modulation structures have different broadband spectral modulation effects on the incident light. The micro-nano modulation structures included in different units can be the same or different, or can be spatially reconstructed according to the image. Each unit corresponds to multiple photosensitive pixels of the image sensor in the vertical direction. The square detection response of the image sensor 2 corresponds to a part of the non-linear activation function of the optical artificial neural network, and converts the outgoing light field E of the optical filter layer 1 N into the photocurrent response I of the image sensor N . The processor 3 includes a signal readout circuit and a computer. The signal readout circuit in the processor 3 reads out the photocurrent response I N and transmits it to the computer. The computer performs a fully connected processing of the electrical signal or performs a non-linear activation processing again, and finally outputs the result

[0120] It can be understood that for the control of the smelting end point, by performing optical simulation on the micro-nano modulation structure on the computer, the modulation intensity (transmittance) of the modulation structure on different wavelength components of the incident light can be obtained, which is used as the connection weight from the input layer to the linear layer of the artificial neural network. At the same time, the square detection response of the image sensor is used to perform the first non-linear activation processing on the information carried by the incident light corresponding to different position points after being modulated by the optical filter layer, and convert it into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor. The processor performs a fully connected processing on the electrical signals corresponding to different position points, or the processor performs a fully connected processing on the electrical signals corresponding to different position points and a second non-linear activation processing to obtain the output signal of the artificial neural network. By collecting the converter mouth image and spectral information at the end point multiple times and performing data training in advance, the required micro-nano modulation structure can be designed and prepared, and the input layer, linear layer, and the first non-linear activation function in the non-linear layer of the artificial neural network can be implemented on the chip

[0121] Viewed longitudinally, as Figure 2 shown, each micro-nano modulation structure in the optical filter layer is designed through pre-artificial neural network training, and can be prepared by directly growing one or more layers of dielectric or metal materials on the image sensor and then etching. The overall size of each modulation unit in the optical filter layer is usually λ 2 ~10 5 λ 2 , and the thickness is usually 0.1λ~10λ, where λ is the central wavelength of the target band. Each modulation unit structure in the optical filter layer corresponds to multiple pixels on the image sensor. The optical filter layer is directly prepared on the image sensor, and the image sensor and the processor are connected through electrical contacts

[0122] It can be understood that both the optical filter layer and the CIS wafer (the CIS wafer is a special image sensor) can be manufactured by semiconductor CMOS integration process. The optical filter layer is monolithically integrated on the image sensor directly at the wafer level. Using the CMOS process, the preparation of the chip can be completed in one tape-out, so that monolithic integration can be achieved at the wafer level, which is beneficial to reducing the distance between the sensor and the optical filter layer, reducing the volume of the device, and lowering the packaging cost.

[0123] Thus, in this embodiment, the optical filter layer corresponds to the input layer, the linear layer of the artificial neural network, and the connection weights from the input layer to the linear layer. The square detection response of the image sensor corresponds to the first non-linear activation function in the non-linear layer of the artificial neural network. Furthermore, the spatial spectral information of the converter mouth is projected into the photocurrent response of the image sensor, and the full connection of the electrical signal and the second non-linear activation are realized in the processor, so that the identification of whether it is the end point of smelting can be quickly achieved.

[0124] The optical artificial neural network smelting end point monitoring chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A. Embed the artificial neural network part into the image sensor including various optical filter layers to achieve safe, reliable, fast and accurate smelting end point control. B. The detectable samples include but are not limited to the end point control of converter steelmaking. Introduce the artificial neural network to train and detect the temperature and material elements of the smelting furnace mouth, which is extremely easy to integrate with the backend industrial control system, and has high identification accuracy and accurate qualitative analysis. C. The preparation of the chip can be completed by one tape-out through the CMOS process, which is beneficial to reducing the device failure rate, improving the finished product yield of the device, and reducing the cost. D. Monolithic integration is achieved at the wafer level, which can minimize the distance between the sensor and the optical filter layer to the greatest extent, is beneficial to reducing the size of the unit, and reducing the device volume and packaging cost.

[0125] Based on the content of the above embodiment, in this embodiment, the optical modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the optical modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

[0126] In this embodiment, the optical modulation structure in the optical filter layer may only include a regular structure, or only include an irregular structure, or may include both a regular structure and an irregular structure.

[0127] In this embodiment, the fact that the optical modulation structure includes a regular structure may mean that: the smallest modulation unit included in the optical modulation structure is a regular structure, such as regular shapes like rectangles, squares, and circles. In addition, the fact that the optical modulation structure includes a regular structure may also mean that: the arrangement pattern of the smallest modulation units included in the optical modulation structure is regular, such as a regular array form, circular form, trapezoidal form, polygonal form, etc. In addition, the fact that the optical modulation structure includes a regular structure may also mean that: the smallest modulation unit included in the optical modulation structure is a regular structure, and at the same time, the arrangement pattern of the smallest modulation units is also regular, etc.

[0128] In this embodiment, the fact that the optical modulation structure includes an irregular structure may mean that: the smallest modulation unit included in the optical modulation structure is an irregular structure, such as irregular polygons, random shapes, and other irregular patterns. In addition, the fact that the optical modulation structure includes an irregular structure may also mean that: the arrangement pattern of the smallest modulation units included in the optical modulation structure is irregular, such as an irregular polygonal form, random arrangement form, etc. In addition, the fact that the optical modulation structure includes an irregular structure may also mean that: the smallest modulation unit included in the optical modulation structure is an irregular structure, and at the same time, the arrangement pattern of the smallest modulation units is also irregular, etc.

[0129] In this embodiment, the optical modulation structure in the optical filter layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0130] In this embodiment, the fact that the optical modulation structure includes a continuous structure may mean that: the optical modulation structure is composed of continuous modulation patterns; the fact that the optical modulation structure includes a discrete structure may mean that: the optical modulation structure is composed of discrete modulation patterns.

[0131] It can be understood that the continuous modulation patterns here may refer to linear patterns, wavy patterns, zigzag patterns, and so on.

[0132] It can be understood that the discrete modulation patterns here may refer to modulation patterns formed by discrete graphics (such as discrete points, discrete squares, discrete irregular polygons, etc.).

[0133] In this embodiment, it should be noted that the optical modulation structure has different modulation effects on lights of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after light passes through different groups of filter structures.

[0134] Based on the content of the above embodiments, in this embodiment, the optical filter layer is a single-layer structure or a multi-layer structure.

[0135] In this embodiment, it should be noted that the optical filter layer can be a single-layer filter structure or a multi-layer filter structure. For example, it can be a multi-layer structure such as two layers, three layers, four layers, etc.

[0136] In this embodiment, as Figure 1 shown, the optical filter layer 1 is a single-layer structure. The thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths from 400 nm to 10 μm, the thickness of the grating structure can be from 50 nm to 5 μm.

[0137] It can be understood that since the function of the optical filter layer 1 is to perform spectral modulation on the incident light, it is preferably prepared from materials with high refractive index and low loss. For example, silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, group III-V materials, etc. can be selected for preparation. Among them, silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, silicon carbide, etc.

[0138] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical filter layer 1 can be set as a multi-layer structure, and the corresponding optical modulation structures of each layer can be set as different structures, so as to increase the spectral modulation ability of the optical filter layer for the incident light, so that more or more complex connection weights can be formed between the input layer and the linear layer, and thus the accuracy of the optical artificial neural network end-point monitoring chip in processing intelligent tasks can be improved.

[0139] In addition, it should be noted that for the filter layer containing a multi-layer structure, the materials of each layer structure can be the same or different. For example, for the two-layer optical filter layer 1, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0140] It should be noted that the thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths from 400 nm to 10 μm, the total thickness of the multi-layer structure can be from 50 nm to 5 μm.

[0141] Based on the content of the above embodiment, in this embodiment, the optical modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the respective micro-nano units are the same or different.

[0142] In this embodiment, in order to obtain connection weights distributed in an array (the connection weights for connecting the input layer and the linear layer) to facilitate subsequent fully connected and non-linear activation processing by the processor, preferably, in this embodiment, the optical modulation structure is in the form of an array structure. Specifically, the optical modulation structure includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor. It should be noted that the structures of the respective micro-nano units may be the same or different. In addition, it should be noted that the structures of the respective micro-nano units may be periodic or non-periodic. In addition, it should be noted that each micro-nano unit may further include multiple groups of micro-nano structure arrays, and the structures of the respective groups of micro-nano structure arrays are the same or different, etc.

[0143] The following will be described by way of example. In this embodiment, as Figures 5 - 9 shown, the optical filter layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (and each micro-nano unit is a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; as Figure 5 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (the difference from Figure 6 is that Figure 5 each micro-nano unit in Figure 6 is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; as Figure 7 shown, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit has the same structure (and each micro-nano unit is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2. The difference from Figure 6 is that Figure 7 the unit shape of the periodic array in each micro-nano unit in Figure 8 has four-fold rotational symmetry; as Figure 6 shown, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66. The difference from Figure 9 is that the structures of each micro-nano unit are different from each other, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2. In this embodiment, the optical filter layer 1 includes a plurality of mutually different micro-nano units, that is, the modulation effects of different regions on the optical artificial neural network smelting end-point monitoring chip on the incident light are different, thereby improving the design freedom, and further improving the recognition accuracy. As Figure 5The difference is that each micro-nano unit is composed of a discrete aperiodic array structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0144] In this embodiment, the micro-nano units have different modulation effects on light of different wavelengths. The specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different filter structures, the corresponding transmission spectra are different after light passes through different groups of filter structures.

[0145] Based on the content of the above embodiment, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0146] In this embodiment, the micro-nano unit may only include a regular structure, or only include an irregular structure, or may include both a regular structure and an irregular structure.

[0147] In this embodiment, the micro-nano unit including a regular structure here may mean that the smallest modulation unit included in the micro-nano unit is a regular structure, such as the smallest modulation unit may be a regular shape such as a rectangle, a square, and a circle. In addition, the micro-nano unit including a regular structure here may also mean that the arrangement manner of the smallest modulation unit included in the micro-nano unit is regular, such as the arrangement manner may be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc. In addition, the micro-nano unit including a regular structure here may also mean that the smallest modulation unit included in the micro-nano unit is a regular structure, and at the same time the arrangement manner of the smallest modulation unit is also regular, etc.

[0148] In this embodiment, the micro-nano unit including an irregular structure here may mean that the smallest modulation unit included in the micro-nano unit is an irregular structure, such as the smallest modulation unit may be an irregular polygon, a random shape, etc. In addition, the micro-nano unit including an irregular structure here may also mean that the arrangement manner of the smallest modulation unit included in the micro-nano unit is irregular, such as the arrangement manner may be an irregular polygonal form, a random arrangement form, etc. In addition, the micro-nano unit including an irregular structure here may also mean that the smallest modulation unit included in the micro-nano unit is an irregular structure, and at the same time the arrangement manner of the smallest modulation unit is also irregular, etc.

[0149] In this embodiment, the micro-nano units in the optical filter layer may include a discrete structure, or may include a continuous structure, or may include both a discrete structure and a continuous structure.

[0150] In this embodiment, the micro-nano unit including a continuous structure may mean that the micro-nano unit is composed of a continuous modulation pattern; the micro-nano unit including a discrete structure may mean that the micro-nano unit is composed of a discrete modulation pattern.

[0151] It can be understood that the continuous modulation pattern here may refer to a linear pattern, a wavy pattern, a broken line pattern, etc.

[0152] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0153] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano units, the corresponding transmission spectra are different after light passes through different groups of micro-nano units.

[0154] Based on the content of the above embodiment, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0155] In this embodiment, as Figure 5 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 different micro-nano structure arrays 110, 111, 112, and 113, and the filtering unit 44 includes 4 different micro-nano structure arrays 440, 441, 442, and 443. As Figure 10 shown, the optical filter layer 1 includes multiple micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0156] It should be noted that here only a micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of micro-nano structure arrays can also be set according to needs.

[0157] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and the modulation effects of each group of filtering structures on the input light are also different. The specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after light passes through different groups of micro-nano structure arrays.

[0158] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

[0159] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light at the steelmaking furnace mouth as the connection weights between the input layer and the linear layer of the neural network, broadband filtering and narrowband filtering are achieved by using different micro-nano structure arrays. Therefore, in this embodiment, the micro-nano structure arrays obtain the modulation intensity of different wavelength components of the incident light at the steelmaking furnace mouth by performing broadband filtering or narrowband filtering on the incident light at the steelmaking furnace mouth. As Figure 11 and Figure 12 shown, each group of micro-nano structure arrays in the optical filter layer has the function of broadband filtering or narrowband filtering.

[0160] It can be understood that for each group of micro-nano structure arrays, they can all have the function of broadband filtering, or they can all have the function of narrowband filtering, or some can have the function of broadband filtering and some can have the function of narrowband filtering. In addition, the broadband filtering range and narrowband filtering range of each group of micro-nano structure arrays can also be the same or different. For example, by designing the size parameters such as the period, duty cycle, radius, and side length of each group of micro-nano structures in the micro-nano unit, it can have the function of narrowband filtering, that is, only light of one (or a few) wavelengths can pass through. Another example is that by designing the size parameters such as the period, duty cycle, radius, and side length of each group of micro-nano structures in the micro-nano unit, it can have the function of broadband filtering, that is, light of more wavelengths or all wavelengths is allowed to pass through.

[0161] It can be understood that in specific use, the filtering state of each group of micro-nano structure arrays can be determined by means of broadband filtering, narrowband filtering, or a combination thereof according to the application scenario.

[0162] Based on the content of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

[0163] In this embodiment, each group of micro-nano structure arrays can all be periodic structure arrays, or they can all be aperiodic structure arrays, or some can be periodic structure arrays and some can be aperiodic structure arrays. Among them, periodic structure arrays are easy to perform optical simulation design, and aperiodic structure arrays can achieve more complex modulation effects.

[0164] In this embodiment, as Figure 5 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and the micro-nano structure arrays are aperiodic structures. Among them, the aperiodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in an aperiodic arrangement. As Figure 5As shown, the micro-nano unit 11 includes four different aperiodic structure arrays 110, 111, 112, and 113, and the micro-nano unit 44 includes four different aperiodic structure arrays 440, 441, 442, and 443. The micro-nano structure arrays of the aperiodic structures are designed through neural network data training for intelligent processing tasks in the early stage, and are usually structures with irregular shapes. As Figure 6 shown, the optical filter layer 1 includes multiple repeating micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other. Different from Figure 5 this, the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes of the modulation holes on the micro-nano structure array are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. As Figure 6 shown, the micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112, and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442, and 443. The filter structures of the periodic structures are designed through neural network data training for intelligent processing tasks in the early stage, and are usually structures with irregular shapes. As Figure 7 shown, the optical filter layer 1 includes multiple different micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the structures of each micro-nano structure array are different from each other, and the micro-nano structure array is a periodic structure. Among them, the periodic structure means that the shapes on the filter structure are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 μm. As Figure 7 shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other. The micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112, and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442, and 443. The micro-nano structure arrays of the periodic structures are designed through neural network data training for intelligent processing tasks in the early stage, and are usually structures with irregular shapes.

[0165] It should be noted that Figures 5 - 9 each micro-nano unit includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are formed by using four different shapes of modulation holes respectively. The four groups of micro-nano structure arrays have different modulation effects on the incident light. It should be noted that only the micro-nano unit including four groups of micro-nano structure arrays is taken as an example here, which does not play a restrictive role. In actual applications, micro-nano units including six groups, eight groups or other numbers of groups of micro-nano structure arrays can also be set according to needs. In this embodiment, the four different shapes can be circular, cross-shaped, regular polygon, and rectangular (not limited to this).

[0166] In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on lights of different wavelengths, and the modulation effects of different groups of micro-nano structure arrays on the input light are also different. The specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon, resonance enhancement, etc. By designing different micro-nano structure arrays, the corresponding transmission spectra are different after the light passes through different groups of micro-nano structure arrays.

[0167] Based on the content of the above embodiment, in this embodiment, one or more groups of the multi-group micro-nano structure arrays included in the micro-nano unit are empty structures.

[0168] The following Figure 9 is illustrated with the examples shown. In this embodiment, as Figure 9 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure arrays are periodic structures. Different from the above embodiment, for any micro-nano unit, one or more groups of empty structures are included, and the empty structures are used to directly transmit the incident light. It can be understood that when one or more groups of empty structures are included in the multiple groups of micro-nano structure arrays, a richer spectrum modulation effect can be formed, so as to meet the spectrum modulation requirements in specific scenarios (or meet the specific connection weight requirements between the input layer and the linear layer in specific scenarios).

[0169] As Figure 9 shown, each micro-nano unit includes a group of micro-nano structure arrays and three groups of empty structures. The micro-nano unit 11 includes 1 non-periodic structure array 111, the micro-nano unit 22 includes 1 non-periodic structure array 221, the micro-nano unit 33 includes 1 non-periodic structure array 331, the micro-nano unit 44 includes 1 non-periodic structure array 441, the micro-nano unit 55 includes 1 non-periodic structure array 551, and the micro-nano unit 66 includes 1 non-periodic structure array 661, where the micro-nano structure arrays are used to perform different modulations on the incident light. It should be noted that only the example of including a group of micro-nano structure arrays and three groups of empty structures is used here for illustration, which does not play a restrictive role. In actual applications, micro-nano units including a group of micro-nano structure arrays and five groups of empty structures or other numbers of groups of micro-nano structure arrays can also be set according to needs. In this embodiment, the micro-nano structure arrays can be made of modulation holes in the shapes of circles, crosses, regular polygons, and rectangles (not limited to this).

[0170] It should be noted that none of the multiple groups of micro-nano structure arrays included in the micro-nano unit need to include empty structures, that is, the multiple groups of micro-nano structure arrays can be non-periodic structure arrays or periodic structure arrays.

[0171] Based on the content of the above embodiments, in this embodiment, the micro-nano unit has polarization-independent characteristics.

[0172] In this embodiment, since the micro-nano unit has polarization-independent characteristics, the optical filter layer is insensitive to the polarization of incident light, thereby realizing an optical artificial neural network smelting end-point monitoring chip that is insensitive to both the incident angle and polarization. The optical artificial neural network smelting end-point monitoring chip provided by the embodiment of the present invention is insensitive to the incident angle and polarization characteristics of incident light, that is, the measurement result is not affected by the incident angle and polarization characteristics of incident light, so as to ensure the stability of the spectral measurement performance, and further ensure the stability of intelligent processing, such as the stability of intelligent perception, the stability of intelligent recognition, the stability of intelligent decision-making, etc. It should be noted that the micro-nano unit can also have polarization-dependent characteristics.

[0173] Based on the content of the above embodiments, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0174] In this embodiment, it should be noted that four-fold rotational symmetry belongs to a specific case of polarization-independent characteristics. By designing the micro-nano unit into a structure with four-fold rotational symmetry, the requirements of polarization-independent characteristics can be met.

[0175] The following Figure 7 is illustrated by the following example. In this embodiment, as Figure 7 shown, the optical filter layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, and the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other. The micro-nano structure array is a periodic structure. Different from the above embodiments, the corresponding structure of each group of micro-nano structure arrays can be a structure with four-fold rotational symmetry, such as a circle, a cross, a regular polygon, a rectangle, etc., that is, after the structure is rotated by 90°, 180°, and 270°, it coincides with the original structure, so that the structure has polarization-independent characteristics, and the same intelligent recognition effect can be obtained when different polarized lights are incident.

[0176] Based on the content of the above embodiments, in this embodiment, the optical filter layer is composed of one or more layers of structures;

[0177] Each layer of the structure is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, perovskite materials; and / or, the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon polaritons (SPP) micro-nano structures, tunable Fabry-perot cavities (FP cavities).

[0178] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of two-dimensional nanodot materials, two-dimensional nanocolumn materials, and two-dimensional nanowire materials.

[0179] Among them, the photonic crystal, and the combination of the metasurface and the random structure can be compatible with the CMOS process, and can have a good modulation effect. Other materials can also be filled in the micropores of the micro-nano modulation structure for surface smoothing; the quantum dots and perovskites can minimize the volume of a single modulation structure by utilizing the spectral modulation characteristics of the materials themselves; the SPP has a small volume and can achieve polarization-related optical modulation; the liquid crystal can be dynamically regulated by voltage to improve the spatial resolution; the tunable Fabry-Perot resonator can be dynamically regulated to improve the spatial resolution.

[0180] Based on the content of the above embodiments, in this embodiment, the thickness of the optical filter layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

[0181] In this embodiment, it should be noted that if the thickness of the optical filter layer is much smaller than the central wavelength of the incident light, it cannot play an effective spectral modulation role; if the thickness of the optical filter layer is much larger than the central wavelength of the incident light, it is difficult to fabricate in terms of technology and will introduce large optical losses. Therefore, in this embodiment, in order to reduce optical losses and be easy to fabricate, and to ensure an effective spectral modulation effect, the overall size (area) of each micro-nano unit in the optical filter layer 1 is usually λ 2 ~10 5 λ 2 , and the thickness is usually 0.1λ to 10λ (λ represents the central wavelength of the incident light at the steelmaking furnace mouth). As Figure 5 shown, the overall size of each micro-nano unit is 0.5μm 2 ~40000μm 2 , and the dielectric material in the optical filter layer 1 is polysilicon with a thickness of 50nm to 2μm.

[0182] Based on the content of the above embodiments, in this embodiment, the image sensor is any one or more of the following:

[0183] CMOS image sensor (Contact Image Sensor, CIS), charge coupled device (Charge Coupled Device, CCD), single photon avalanche diode (Single Photon Avalanche Diode, SPAD) array, and focal plane optoelectronic image sensor array.

[0184] In this embodiment, it should be noted that a wafer-level CMOS image sensor (CIS) is adopted to achieve monolithic integration at the wafer level, which can minimize the distance between the image sensor and the optical filter layer, facilitate reducing the size of the unit, and reducing the device volume and packaging cost. The SPAD can be used for low-light detection, and the CCD can be used for high-light detection.

[0185] In this embodiment, the optical filter layer and the image sensor can be manufactured by complementary metal oxide semiconductor (CMOS) integrated process, which is beneficial to reducing the device failure rate, improving the device yield and reducing the cost. For example, a layer or multiple layers of dielectric materials can be directly grown on the image sensor and then etched. Before removing the sacrificial layer used for etching, metal materials are deposited, and finally the sacrificial layer is removed to prepare the optical filter layer.

[0186] Based on the content of the above embodiment, in this embodiment, the types of the artificial neural network include: feedforward neural network.

[0187] In this embodiment, a feedforward neural network (FNN), also known as a deep feedforward network (DFN) and a multi-layer perceptron (MLP), is the simplest neural network, and the neurons are arranged in layers. Each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer, and there is no feedback between layers. The feedforward neural network has a simple structure, is easy to implement on hardware, has a wide range of applications, can approximate any continuous function and square-integrable function with arbitrary precision, and can accurately implement any finite training sample set. The feedforward network is a static non-linear mapping. Through the composite mapping of simple non-linear processing units, complex non-linear processing capabilities can be obtained.

[0188] Based on the content of the above embodiment, in this embodiment, a light-transmitting dielectric layer is provided between the optical filter layer and the image sensor.

[0189] In this embodiment, it should be noted that setting a light-transmitting dielectric layer between the optical filter layer and the image sensor can effectively separate the optical filter layer and the image sensor layer and avoid mutual interference between the two.

[0190] Based on the content of the above embodiment, in this embodiment, the image sensor is front-illuminated and includes: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on the side of the metal wire layer away from the light detection layer; or,

[0191] The image sensor is a back-illuminated type, including: a light detection layer and a metal wire layer arranged from top to bottom, and the optical filter layer is integrated on a side of the light detection layer away from the metal wire layer.

[0192] In this embodiment, as Figure 13 shown is a front-illuminated image sensor, where the silicon detection layer 21 is below the metal wire layer 22, and the optical filter layer 1 is directly integrated onto the metal wire layer 22.

[0193] In this embodiment, different from Figure 13 that, Figure 14 shown is a back-illuminated image sensor, where the silicon detection layer 21 is above the metal wire layer 22, and the optical filter layer 1 is directly integrated onto the silicon detection layer 21.

[0194] It should be noted that for the back-illuminated image sensor, with the silicon detection layer 21 above the metal wire layer 22, the influence of the metal wire layer on the incident light can be reduced, thereby improving the quantum efficiency of the device.

[0195] According to the above content, in this embodiment, the optical filter layer is used as the input layer and the linear layer of the artificial neural network, the image sensor is used as a part of the non-linear layer of the artificial neural network (the square detection response of the image sensor is used as the first non-linear activation function of the artificial neural network), and the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer. The optical filter layer and the image sensor in the optical artificial neural network smelting end-point monitoring chip provided in this embodiment implement the related functions of the input layer, the linear layer, and part of the non-linear activation function in the artificial neural network through hardware, so that subsequent complex signal processing and algorithm processing corresponding to the input layer, the linear layer, and part of the non-linear activation function are no longer required when using this optical artificial neural network smelting end-point monitoring chip for intelligent processing. In this way, the power consumption and delay during the processing of the artificial neural network can be significantly reduced. In addition, since this embodiment simultaneously utilizes the image information of the steelmaking furnace mouth and the spectral information at different points in space, the intelligent processing of the steelmaking furnace mouth can be more accurately achieved.

[0196] As can be seen, in the embodiment of the present invention, the optical filter layer is used as the input layer and the linear layer of the artificial neural network, and the image sensor is used as part of the non-linear layer of the artificial neural network. The spatial spectral information of the object is projected into the photocurrent response of the image sensor, and the full connection and second-order non-linear activation of the electrical signal are realized in the processor, realizing functions such as intelligent perception, recognition, and / or decision-making with low power consumption, low latency, and high accuracy. The optical artificial neural network smelting endpoint monitoring chip based on the optical filter and the image sensor in the embodiment of the present invention has the following effects: embedding the artificial neural network part into the image sensor including various optical filter layers to realize fast and accurate intelligent perception, recognition, and / or decision-making functions. In addition, the embodiment of the present invention can also achieve monolithic integration at the wafer level, thereby minimizing the distance between the sensor and the optical filter layer to the greatest extent, which is beneficial to reducing the size of the unit and the device volume and packaging cost.

[0197] Based on the same inventive concept, another embodiment of the present invention provides an intelligent smelting control device, including: the optical artificial neural network smelting endpoint monitoring chip as described in the above embodiment. The intelligent smelting control device may include various devices related to smelting process control, which are not limited in this embodiment.

[0198] Since the intelligent smelting control device provided in this embodiment includes the optical artificial neural network smelting endpoint monitoring chip described in the above embodiment, therefore, the intelligent smelting control device provided in this embodiment has all the beneficial effects of the optical artificial neural network smelting endpoint monitoring chip described in the above embodiment. Since the above embodiment has described this in detail, this embodiment will not be repeated here.

[0199] Based on the same inventive concept, another embodiment of the present invention provides a preparation method of the optical artificial neural network smelting endpoint monitoring chip as described in the above embodiment, as Figure 15 shown, specifically including the following steps:

[0200] Step 1510: Prepare an optical filter layer containing an optical modulation structure on the surface of the image sensor;

[0201] Step 1520: Generate a processor with the function of fully connecting signals or generate a processor with the functions of fully connecting signals and second non-linear activation processing;

[0202] Step 1530: Connect the image sensor and the processor;

[0203] Wherein, the optical filter layer is configured to perform different spectral modulations on the incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain the information carried by the incident light corresponding to different position points on the surface of the image sensor; the information carried by the incident light includes light intensity distribution information, spectral information, the angle information of the incident light, and the phase information of the incident light;

[0204] The image sensor performs a first non-linear activation process on the information carried by the incident light corresponding to different position points modulated by the optical filter layer through square-law detection response, and converts it into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor;

[0205] The processor performs a fully connected process on the electrical signals corresponding to different position points, or, the processor performs a fully connected process and a second non-linear activation process on the electrical signals corresponding to different position points, to obtain the output signal of the artificial neural network.

[0206] In this embodiment, it further includes the training process of the optical artificial neural network smelting endpoint monitoring chip, specifically including:

[0207] Using the input training samples and output training samples corresponding to the smelting endpoint monitoring task, training the optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor;

[0208] Or, using the input training samples and output training samples corresponding to the smelting endpoint monitoring task, training the optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters and different second non-linear activation parameters to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions, and using the optical modulation structure, image sensor, and processor that meet the training convergence conditions as the trained optical modulation structure, image sensor, and processor.

[0209] It can be understood that when training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters, or when training an optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters, the different optical modulation structures are designed and implemented by means of computer optical simulation design.

[0210] It can be understood that the converter mouth images and spectral information at the end point can be collected multiple times in advance and data training can be carried out. The required micro-nano modulation structure can be designed and prepared, and the input layer, linear layer, and the first non-linear activation function in the non-linear layer of the artificial neural network can be implemented on the chip.

[0211] In this embodiment, an optical filter layer including an optical modulation structure is prepared on the surface of the photosensitive area of the image sensor, including:

[0212] Grow one or more layers of preset materials on the surface of the image sensor;

[0213] Perform dry etching on the one or more layers of preset materials with an optical modulation structure pattern to obtain an optical filter layer including an optical modulation structure;

[0214] Or perform imprint transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0215] Or obtain an optical filter layer including an optical modulation structure by applying external dynamic regulation to the one or more layers of preset materials;

[0216] Or perform zone printing on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0217] Or perform zone material growth on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure;

[0218] Or perform quantum dot transfer on the one or more layers of preset materials to obtain an optical filter layer including an optical modulation structure.

[0219] When the optical artificial neural network smelting endpoint monitoring chip is used for the intelligent processing task at the steelmaking furnace mouth, the optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, and processors with different fully connected parameters is trained using the input training samples and output training samples corresponding to the intelligent processing task to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions; or, the optical artificial neural network smelting endpoint monitoring chip including different optical modulation structures, image sensors, processors with different fully connected parameters, and different second non-linear activation parameters is trained to obtain an optical modulation structure, an image sensor, and a processor that meet the training convergence conditions.

[0220] In this embodiment, it should be noted that, as Figure 1 shown, the optical filter layer 1 can be prepared by directly growing one or more layers of dielectric materials on the image sensor 2, then etching, depositing metal materials before removing the sacrificial layer used for etching, and finally removing the sacrificial layer. By designing the size parameters of the optical modulation structure, each unit can have different modulation effects on light with different wavelengths within the target range, and this modulation effect is insensitive to the incident angle and polarization. Each unit in the optical filter layer 1 corresponds to one or more pixels on the image sensor 2. 1 is directly prepared on 2.

[0221] In this embodiment, it should be noted that, as Figure 14 shown, assuming that the image sensor 2 is a back-illuminated structure, the optical filter layer 1 can be directly etched on the silicon image sensor layer 21 of the back-illuminated image sensor, and then metal is deposited for preparation.

[0222] In addition, it should be noted that the optical modulation structure on the optical filter layer can be obtained by dry etching the pattern of the optical modulation structure on one or more preset materials. Dry etching is to directly remove the unnecessary parts of one or more preset materials on the surface of the photosensitive area of the image sensor, so as to obtain an optical filter layer containing the optical modulation structure; or by imprint transfer of one or more preset materials. Imprint transfer is to prepare the required structure by etching on other substrates, and then transfer the structure to the photosensitive area of the image sensor through materials such as PDMS, so as to obtain an optical filter layer containing the optical modulation structure; or by applying external dynamic regulation to one or more preset materials. External dynamic regulation is to use active materials, and then apply electrodes to regulate the optical modulation characteristics of the corresponding area by changing the voltage, so as to obtain an optical filter layer containing the optical modulation structure; or by zone printing of one or more preset materials. Zone printing is to use printing technology in zones to obtain an optical filter layer containing the optical modulation structure; or by zone material growth of one or more preset materials to obtain an optical filter layer containing the optical modulation structure; or by quantum dot transfer of one or more preset materials to obtain an optical filter layer containing the optical modulation structure.

[0223] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the optical artificial neural network endpoint monitoring chip in the above-mentioned embodiment, for the detailed content of some principles and structures, etc., reference can be made to the introduction of the above-mentioned embodiment, and this embodiment will not be elaborated here.

[0224] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An optical artificial neural network smelting endpoint monitoring chip, characterized in that: Used for smelting endpoint monitoring tasks, comprising: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer, the linear layer and the connection weight from the input layer to the linear layer of the artificial neural network, the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor corresponds to the full connection and the output layer of the artificial neural network, or the processor corresponds to the full connection of the artificial neural network, the second nonlinear activation function in the nonlinear layer and the output layer; The optical filter layer is arranged on the surface of the image sensor, and the optical filter layer includes an optical modulation structure, and the optical modulation structure is used to perform different spectrum modulations on the incident light entering different positions of the optical modulation structure, so as to obtain incident light carrying information corresponding to different positions on the surface of the image sensor; the incident light carrying information includes light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light; the incident light includes reflected light, transmitted light and / or radiated light from the steelmaking furnace mouth; The image sensor converts the information carried by the incident light corresponding to different positions after being modulated by the optical filter layer into electrical signals corresponding to different positions through the square detection response, and sends the electrical signals corresponding to different positions to the processor; The processor performs full connection processing on the electrical signals corresponding to different position points, or the processor performs full connection processing and second nonlinear activation processing on the electrical signals corresponding to different position points to obtain a smelting endpoint monitoring result; The smelting endpoint monitoring task includes identifying the smelting endpoint, and the smelting endpoint monitoring result includes a smelting endpoint identification result.

2. The optical artificial neural network smelting endpoint monitoring chip according to claim 1 is characterized in that: The smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, and the smelting endpoint monitoring result includes the carbon content and / or molten steel temperature identification result during the smelting process.

3. The optical artificial neural network smelting endpoint monitoring chip according to claim 1 or 2, characterized in that: The optical artificial neural network smelting endpoint monitoring chip includes a trained light modulation structure, an image sensor and a processor; The trained light modulation structure, image sensor and processor refer to the light modulation structure, image sensor and processor that meet the training convergence conditions by training an optical artificial neural network smelting endpoint monitoring chip that includes different light modulation structures, image sensors and processors with different fully connected parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task; or, the trained light modulation structure, image sensor and processor refer to the light modulation structure, image sensor and processor that meet the training convergence conditions by training an optical artificial neural network smelting endpoint monitoring chip that includes different light modulation structures, image sensors and processors with different fully connected parameters and different second nonlinear activation parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task; The input training samples include incident light reflected, transmitted and / or radiated from a steelmaking furnace mouth that has been smelted to an end point or not; and the output training samples include a determination result of whether the steel has been smelted to an end point.

4. The optical artificial neural network smelting endpoint monitoring chip according to claim 3 is characterized in that: When the smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, accordingly, the input training sample also includes incident light reflected, transmitted and / or radiated by the steelmaking furnace mouth smelting to different carbon contents and / or molten steel temperatures, and the output training sample also includes the corresponding carbon content and / or molten steel temperature.

5. The optical artificial neural network smelting endpoint monitoring chip according to claim 3 is characterized in that: When training an optical artificial neural network smelting endpoint monitoring chip comprising different light modulation structures, image sensors and processors with different fully connected parameters, or when training an optical artificial neural network smelting endpoint monitoring chip comprising different light modulation structures, image sensors and processors with different fully connected parameters and different second nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

6. The optical artificial neural network smelting endpoint monitoring chip according to claim 1, characterized in that: The light modulation structure in the optical filter layer includes a regular structure and / or an irregular structure; and / or, the light modulation structure in the optical filter layer includes a discrete structure and / or a continuous structure.

7. The optical artificial neural network smelting endpoint monitoring chip according to claim 1, characterized in that: The light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, each of which corresponds to one or more pixel points on the image sensor; and the structures of the various micro-nano units are the same or different.

8. The optical artificial neural network smelting endpoint monitoring chip according to claim 7, characterized in that: The micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

9. The optical artificial neural network smelting endpoint monitoring chip according to claim 7, characterized in that: The micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

10. The optical artificial neural network smelting endpoint monitoring chip according to claim 9, characterized in that: Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

11. The optical artificial neural network smelting endpoint monitoring chip according to claim 9, characterized in that: Each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

12. The optical artificial neural network smelting endpoint monitoring chip according to claim 9, characterized in that: The micro-nano unit includes a plurality of micro-nano structure arrays, and there is one or more groups of empty structures.

13. The optical artificial neural network smelting endpoint monitoring chip according to claim 9, characterized in that: The micro-nano unit has four-fold rotational symmetry.

14. The optical artificial neural network smelting endpoint monitoring chip according to claim 1, characterized in that: The optical filter layer is composed of one or more layers; Each layer of the structure is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is a filter layer made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structures, and adjustable Fabry-Perot resonant cavities.

15. The optical artificial neural network smelting endpoint monitoring chip according to claim 14, characterized in that: The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

16. The optical artificial neural network smelting endpoint monitoring chip according to claim 1, characterized in that: The thickness of the optical filter layer is 0.1λ to 10λ, wherein λ represents the central wavelength of the incident light.

17. An intelligent smelting control device, characterized in that: It comprises the optical artificial neural network smelting endpoint monitoring chip as described in any one of claims 1 to 16.

18. A method for preparing an optical artificial neural network smelting endpoint monitoring chip as claimed in any one of claims 1 to 16, characterized in that: include: Preparing an optical filter layer including a light modulation structure on the surface of the image sensor; Generate a processor with a function of fully connecting the signal or generate a processor with a function of fully connecting the signal and performing a second nonlinear activation processing; connecting the image sensor and the processor; The optical filter layer is used to perform different spectrum modulations on the incident light entering different positions of the optical modulation structure through the optical modulation structure, so as to obtain light field distribution signals corresponding to different positions on the surface of the image sensor; The image sensor converts the light field distribution signals corresponding to different positions after being modulated by the optical filter layer into electrical signals corresponding to different positions after the first nonlinear activation processing through the square detection response, and sends the electrical signals corresponding to different positions to the processor; The processor performs full connection processing on the electrical signals corresponding to different position points, or the processor performs full connection processing and a second nonlinear activation processing on the electrical signals corresponding to different position points to obtain a smelting endpoint monitoring result.

19. The method for preparing the optical artificial neural network smelting endpoint monitoring chip according to claim 18, characterized in that: Also includes: The training process of the optical artificial neural network smelting endpoint monitoring chip specifically includes: Using input training samples and output training samples corresponding to the smelting endpoint monitoring task, an optical artificial neural network smelting endpoint monitoring chip including different light modulation structures, image sensors and processors with different fully connected parameters is trained to obtain light modulation structures, image sensors and processors that meet the training convergence conditions, and the light modulation structures, image sensors and processors that meet the training convergence conditions are used as the trained light modulation structures, image sensors and processors; Alternatively, using input training samples and output training samples corresponding to the smelting endpoint monitoring task, an optical artificial neural network smelting endpoint monitoring chip including different light modulation structures, image sensors, and processors with different fully connected parameters and different second nonlinear activation parameters is trained to obtain light modulation structures, image sensors, and processors that meet the training convergence conditions, and the light modulation structures, image sensors, and processors that meet the training convergence conditions are used as the trained light modulation structures, image sensors, and processors.

20. The method for preparing the optical artificial neural network smelting endpoint monitoring chip according to claim 18, characterized in that: An optical filter layer including a light modulation structure is prepared on the surface of the image sensor, comprising: Growing one or more layers of a preset material on the surface of the image sensor; Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light filter layer including the light modulation structure; or performing embossing transfer on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure; or obtaining an optical filter layer including an optical modulation structure by externally dynamically modulating the one or more layers of the preset material; or performing zone printing on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure; or performing partitioned growth on the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure; Alternatively, quantum dots are transferred to the one or more layers of the preset material to obtain an optical filter layer including a light modulation structure.

Images