Smelting endpoint monitoring chip, intelligent smelting control equipment and preparation method

Abstract

The present invention provides a smelting endpoint monitoring chip, intelligent smelting control equipment and a preparation method. The present invention embeds an artificial neural network on a hardware chip, uses the light modulation layer on the hardware chip as the input layer of the artificial neural network, uses the image sensor as the linear layer of the artificial neural network, and uses the filtering effect of the light modulation layer on the incident light as the connection weight from the input layer to the linear layer. Therefore, when the smelting endpoint monitoring chip is subsequently used for smelting endpoint monitoring, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer, which can greatly reduce the power consumption and delay during artificial neural network processing.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a smelting endpoint monitoring chip, intelligent smelting control equipment and a preparation method. Background Art

[0002] Converter steelmaking is currently the most widely used and efficient steelmaking method in the world. Endpoint control is a key technology in converter production. Accurately determining the endpoint is crucial for improving molten steel quality and shortening the smelting cycle. However, due to factors such as unstable raw materials entering the furnace, complex chemical reactions, and the stringent requirements for steel grades, accurate control of the smelting endpoint remains challenging. Accurate online monitoring of endpoint carbon content and molten steel temperature has long been a pressing challenge for the global metallurgical industry. Currently, endpoint control in the industry relies primarily on manual experience or complex, large-scale instrumentation to measure furnace mouth temperature and slag residue, resulting in low accuracy and high cost. Summary of the Invention

[0003] In response to the problems existing in the prior art, embodiments of the present invention provide a smelting endpoint monitoring chip, intelligent smelting control equipment and a preparation method.

[0004] Specifically, the embodiments of the present invention provide the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides a smelting endpoint monitoring chip for smelting endpoint monitoring tasks, comprising: an optical modulation layer and an image sensor, wherein the image sensor is connected to a processor; the optical modulation layer corresponds to the input layer of an artificial neural network and the connection weights from the input layer to the linear layer; the image sensor corresponds to the linear layer of the artificial neural network; and the processor corresponds to the nonlinear layer and output layer of the artificial neural network.

[0006] The light modulation layer is disposed on the surface of the photosensitive area of ​​the image sensor, and includes a light modulation structure. The light modulation structure is used to perform different spectrum modulations on incident light entering different positions of the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area. The incident light includes reflected light, transmitted light, and / or radiated light from the steelmaking furnace mouth.

[0007] The image sensor is used to convert information carried by incident light corresponding to different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signals are image signals modulated by the light modulation layer;

[0008] The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the smelting endpoint monitoring result;

[0009] The smelting endpoint monitoring task includes identifying the smelting endpoint, and the smelting endpoint monitoring result includes a smelting endpoint identification result.

[0010] Furthermore, the information carried by the incident light includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0011] Furthermore, the smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, and the smelting endpoint monitoring result also includes the carbon content and / or molten steel temperature identification result during the smelting process.

[0012] Furthermore, the smelting endpoint monitoring chip includes a trained light modulation structure and an image sensor, and the processor is a trained processor;

[0013] The trained optical modulation structure, image sensor, and processor refer to optical modulation structures, image sensors, and processors that meet the training convergence conditions, obtained by training a smelting endpoint monitoring chip comprising different optical modulation structures, image sensors, and processors with different fully connected parameters and nonlinear activation parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task;

[0014] The input training samples include incident light reflected, transmitted and / or radiated from the mouth of a steelmaking furnace that has been smelted to the end point or not; the output training samples include the determination result of whether the steelmaking furnace has been smelted to the end point.

[0015] Furthermore, when the smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, accordingly, the input training sample also includes incident light reflected, transmitted and / or radiated by the steelmaking furnace mouth smelting to different carbon contents and / or molten steel temperatures, and the output training sample also includes the corresponding carbon content and / or molten steel temperature.

[0016] Furthermore, when training a smelting endpoint monitoring chip comprising different light modulation structures and image sensors and a processor having different fully connected parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0017] Furthermore, the light modulation structure in the light modulation layer includes a regular structure and / or an irregular structure; and / or the light modulation structure in the light modulation layer includes a discrete structure and / or a continuous structure.

[0018] Furthermore, the light modulation structure in the light modulation layer includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of each micro-nano unit are the same or different.

[0019] Furthermore, the micro-nano unit includes a regular structure and / or an irregular structure; and / or the micro-nano unit includes a discrete structure and / or a continuous structure.

[0020] Furthermore, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0021] Furthermore, each group of micro-nanostructure arrays has the function of broadband filtering or narrowband filtering.

[0022] Furthermore, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0023] Furthermore, the micro-nano unit includes one or more groups of empty structures in the multiple groups of micro-nano structure arrays.

[0024] Furthermore, the micro-nano unit has four-fold rotational symmetry.

[0025] Furthermore, the light modulation layer is composed of one or more filter layers;

[0026] The filter layer is made of one or more semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nanostructures, and adjustable Fabry-Perot resonant cavities.

[0027] Furthermore, the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

[0028] Furthermore, the thickness of the light modulation layer is 0.1λ~10λ, where λ represents the central wavelength of the incident light.

[0029] In a second aspect, an embodiment of the present invention further provides an intelligent smelting control device, comprising the smelting endpoint monitoring chip as described in the first aspect.

[0030] In a third aspect, an embodiment of the present invention further provides a method for preparing the smelting endpoint monitoring chip as described in the first aspect, comprising:

[0031] Preparing a light modulation layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor;

[0032] Generate a processor capable of fully connected signal processing and nonlinear activation processing;

[0033] connecting the image sensor and the processor;

[0034] The light modulation layer is used to perform different spectrum modulation on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area;

[0035] The image sensor is used to convert the information carried by the incident light corresponding to different position points after modulation by the light modulation layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the smelting end point monitoring results.

[0036] Furthermore, the method for preparing the smelting endpoint monitoring chip further includes: a training process for the smelting endpoint monitoring chip, specifically including:

[0037] Using input training samples and output training samples corresponding to the smelting endpoint monitoring task, a smelting endpoint monitoring chip including different optical modulation structures and image sensors and a processor with different fully connected parameters are trained to obtain an optical modulation structure, image sensor and processor that meet the training convergence conditions, and the optical modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained optical modulation structure, image sensor and processor.

[0038] Furthermore, a light modulation layer including a light modulation structure is prepared on the surface of the image sensor, including:

[0039] growing one or more layers of a preset material on the surface of the image sensor;

[0040] Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light modulation layer including the light modulation structure;

[0041] or performing embossing transfer on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0042] or obtaining a light modulation layer including a light modulation structure by externally dynamically modulating the one or more layers of the preset material;

[0043] or performing zone printing on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0044] or performing partitioned growth on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0045] Alternatively, quantum dot transfer is performed on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure.

[0046] The embodiment of the present invention realizes a new smelting endpoint monitoring chip that can realize the function of an artificial neural network, which is used for smelting endpoint monitoring tasks. In the smelting endpoint monitoring chip, an artificial neural network is embedded in a hardware manner, wherein a light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure. The light modulation layer is used to perform different spectrum modulations on the incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area. Accordingly, the image sensor is used to convert the incident light carrying information corresponding to different position points into electrical signals corresponding to different position points, and the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in the smelting endpoint monitoring chip, the light modulation layer is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. The optical modulation layer serves as the input layer of the artificial neural network, the image sensor serves as the linear layer of the artificial neural network, and the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer serves as the connection weight from the input layer to the linear layer. That is, the optical modulation layer and the image sensor in the smelting end point monitoring chip realize the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiment of the present invention strips the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the smelting end point monitoring chip is used for smelting end point identification in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. The processor in the smelting end point monitoring chip only needs to perform relevant processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay during smelting end point identification.

[0047] It can be seen that the embodiment of the present invention provides a new optoelectronic chip for accurately identifying the smelting endpoint. The chip embeds the artificial neural network part into the image sensor containing various light modulation layers to achieve safe, reliable, fast and accurate smelting endpoint control. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a schematic structural diagram of a smelting endpoint monitoring chip provided by one embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the identification principle of a smelting endpoint monitoring chip provided by one embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of disassembling a smelting endpoint monitoring chip provided by one embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of identifying a furnace mouth during a smelting process to determine a smelting endpoint, provided by one embodiment of the present invention;

[0053] Figure 5 is a top view of a light modulation layer provided by one embodiment of the present invention;

[0054] Figure 6 is a top view of another light modulation layer provided by one embodiment of the present invention;

[0055] Figure 7 is a top view of another light modulation layer provided by an embodiment of the present invention;

[0056] Figure 8 is a top view of another light modulation layer provided by an embodiment of the present invention;

[0057] Figure 9 is a top view of yet another light modulation layer provided by an embodiment of the present invention;

[0058] Figure 10 is a top view of yet another light modulation layer provided by an embodiment of the present invention;

[0059] Figure 11 This is a schematic diagram of the broadband filtering effect of a micro-nano structure provided by an embodiment of the present invention;

[0060] Figure 12 This is a schematic diagram of the narrowband filtering effect of the micro-nano structure provided by one embodiment of the present invention;

[0061] Figure 13 1 is a schematic structural diagram of a front-illuminated image sensor provided by an embodiment of the present invention;

[0062] Figure 141 is a schematic structural diagram of a back-illuminated image sensor provided by an embodiment of the present invention;

[0063] Figure 15 It is a flow chart of a method for preparing a smelting endpoint monitoring chip provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0065] Converter steelmaking is currently the most widely used and efficient steelmaking process in the world. Endpoint control is a key technology in converter production. Accurately determining the endpoint is crucial for improving molten steel quality and shortening the smelting cycle. However, due to factors such as unstable raw materials entering the furnace, complex chemical reactions, and the stringent requirements of the steel grades being produced, accurate endpoint control remains challenging. Accurate online monitoring of endpoint carbon content and molten steel temperature has been a pressing challenge for the global metallurgical industry. Currently, endpoint control in the industry relies primarily on manual experience or complex, large-scale instrumentation to measure furnace mouth temperature and slag residue, resulting in low accuracy and high cost. Therefore, developing an online smelting endpoint control chip that is miniaturized, low-cost, secure, reliable, and easily integrated with control systems is of great significance to the development and automation of the steelmaking industry, representing a true example of Industry 4.0. The present invention utilizes spectral radiation characteristic analysis at the converter mouth to perform online, non-contact measurement of converter steelmaking endpoint, molten steel temperature, carbon content, and other parameters, thereby achieving accurate online control of the smelting endpoint. Specifically, an embodiment of the present invention provides a smelting endpoint monitoring chip for smelting endpoint monitoring tasks, in which an artificial neural network is embedded in a hardware manner, wherein a light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure, and the light modulation layer is used to perform different spectrum modulations on the incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to the different position points on the surface of the photosensitive area, and accordingly, the image sensor is used to convert the incident light carrying information corresponding to the different position points into electrical signals corresponding to the different position points, and the processor connected to the image sensor is used to convert the incident light carrying information corresponding to the different position points into electrical signals corresponding to the different position points. The electrical signal corresponding to the point is fully connected and nonlinearly activated to obtain the output signal of the artificial neural network. It can be seen that in the smelting end point monitoring chip, the input layer and the linear layer in the artificial neural network implemented by software in the prior art are stripped off, and the two-layer structure of the input layer and the linear layer in the artificial neural network is implemented by hardware, so that when the smelting end point monitoring chip is used for smelting end point identification in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. The processor in the smelting end point monitoring chip only needs to perform related processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay during smelting end point identification.In addition, the embodiment of the present invention can also use the information carried by the incident light of the steelmaking furnace mouth, including one or more information of image information, spectral information, angle of incident light and phase information of incident light. Since the information carried by the incident light at different points of the steelmaking furnace mouth covers the image, composition, shape, three-dimensional depth, structure, three-dimensional color and other information of the steelmaking furnace mouth, when the steelmaking end point is identified based on the information carried by the incident light at different points, the image, composition, shape, three-dimensional depth, three-dimensional color and other information of the steelmaking furnace mouth can be covered, so that the available information can be fully utilized to identify the steelmaking end point, thereby solving the problem mentioned in the background technology part that it is difficult to accurately identify the smelting end point. Point problem, it can be seen that the smelting endpoint monitoring chip provided by the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, so that the smelting endpoint can be quickly and accurately identified. The smelting endpoint monitoring chip provided by the embodiment of the present invention has well solved the global problem of difficult monitoring of the smelting endpoint, and has achieved contactless and accurate monitoring. At the same time, due to the use of optical artificial neural network chips, the complex signal processing and algorithm processing corresponding to the input layer and linear layer in the optical artificial neural network can be replaced by pre-prepared hardware (optical modulation layer and image sensor), thereby greatly reducing the processing delay and achieving fast, accurate, safe and reliable smelting endpoint monitoring. The contents provided by the present invention will be explained and illustrated in detail through specific embodiments below.

[0066] like Figure 1 As shown, a smelting endpoint monitoring chip provided by an embodiment of the present invention is used for smelting endpoint monitoring tasks, including: an optical modulation layer 1 and an image sensor 2, wherein the image sensor 2 is connected to a processor 3, and the processor 3 can be set in the chip or independently of the chip (the processor 3 can be part of the chip or a separate device); the optical modulation layer 1 corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor 2 corresponds to the linear layer of the artificial neural network; the processor 3 corresponds to the nonlinear layer and output layer of the artificial neural network;

[0067] The light modulation layer 1 is provided on the surface of the photosensitive area of ​​the image sensor 2. The light modulation layer includes a light modulation structure. The light modulation structure is used to perform different spectrum modulations on the incident light entering different positions of the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area. The incident light includes reflected light, transmitted light and / or radiated light from the steelmaking furnace mouth.

[0068] In this embodiment, the image sensor 2 is used to convert information carried by incident light corresponding to different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signals are image signals modulated by the light modulation layer;

[0069] In this embodiment, the processor 3 is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the smelting endpoint monitoring result;

[0070] The smelting endpoint monitoring task includes identifying the smelting endpoint, and the smelting endpoint monitoring result includes a smelting endpoint identification result.

[0071] In this embodiment, the information carried by the incident light includes image information and / or various optical spatial information of the target object to be processed by the optical artificial neural network intelligent chip. For example, the information carried by the incident light includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.

[0072] It can be seen that in this embodiment, it is necessary to use a light source to illuminate the converter steelmaking furnace mouth, and then allow the transmitted light or reflected light to enter the light modulation layer of the chip, and the light modulation structure in the light modulation layer adjusts the light incident on the light modulation structure. Since the light modulation structure in the light modulation layer is a pre-designed modulation pattern that can be used for smelting end point monitoring, when the reflected light, transmitted light and / or radiated light from the steelmaking furnace mouth enters the light modulation layer of the chip, the light modulation layer, image sensor and processor on the chip together act as a whole of the artificial neural network to identify the reflected light, transmitted light and / or radiated light, and then obtain the smelting end point monitoring result.

[0073] In addition, it should be noted that the modulation pattern in the light modulation layer that can be used for smelting endpoint monitoring can be set in the following manner:

[0074] For smelting end point monitoring, the modulation pattern can be optically simulated on a computer to obtain the modulation intensity (transmittance) of the modulation pattern for different wavelength components of the incident light, which is used as the connection weight from the input layer to the linear layer of the artificial neural network. A nonlinear activation function is implemented in the computer, and a large amount of reflected, transmitted and / or radiated light from the steelmaking furnace mouth is collected in advance. At the same time, the corresponding monitoring results of whether it is the smelting end point are collected. The collected input sample data and output sample data are used to train the artificial neural network. The required modulation pattern can be designed and prepared, and the input layer and linear layer of the artificial neural network are realized on the chip through hardware (optical modulation layer and image sensor).

[0075] The processor 3 is used to perform full-connection processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network, that is, to obtain the smelting endpoint monitoring result.

[0076] In this embodiment, a light modulation layer 1 is disposed on the surface of the image sensor and includes a light modulation structure. The light modulation layer 1 is used to modulate the spectrum of incident light entering different locations of the light modulation structure through the light modulation structure, thereby obtaining information carried by the modulated incident light corresponding to different locations on the surface of the image sensor. Thus, in this embodiment, the light modulation layer corresponds to the input layer of the artificial neural network, and the modulation effect of the light modulation structure on the incident light on the light modulation layer can be regarded as the connection weight from the input layer to the linear layer.

[0077] In this embodiment, the light modulation layer 1 includes a light modulation structure, which performs spectral modulation of different intensities on the incident light (such as reflected light, transmitted light, radiated light and other related action light of the target to be identified) entering different position points of the light modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the image sensor 2; the incident light carrying information includes light intensity distribution information, spectral information, angle information of the incident light and phase information of the incident light.

[0078] In this embodiment, it can be understood that the modulation intensity is related to the specific structural form of the light modulation structure. For example, different modulation intensities can be achieved by designing different light modulation structures (such as changing the shape and / or size parameters of the light modulation structure).

[0079] In this embodiment, it is understood that the optical modulation structures at different locations on the optical modulation layer 1 have different spectral modulation effects on the incident light. The modulation intensities of the optical modulation structures on different wavelength components of the incident light correspond to the connection strengths of the linear layer of the artificial neural network, that is, to the connection weights of the input layer and the connection weights from the input layer to the linear layer. It should be noted that the optical modulation layer 1 is composed of multiple optical filter units, and the optical modulation structures at different locations within each optical filter unit are different, thus having different spectral modulation effects on the incident light. The optical modulation structures at different locations between optical filter units can be the same or different, thus having the same or different spectral modulation effects on the incident light.

[0080] In this embodiment, the image sensor 2 converts the light intensity distribution signals corresponding to different position points into electrical signals corresponding to different position points, and sends the electrical signals corresponding to different position points to the processor 3. The image sensor 2 corresponds to the linear layer of the neural network.

[0081] In this embodiment, the processor 3 performs full connection processing and nonlinear activation processing on the electrical signals at different locations to obtain the output signal of the artificial neural network.

[0082] It can be understood that processor 3 corresponds to the nonlinear layer and output layer of the neural network, and can also be understood as corresponding to the remaining layers (all other layers) in the neural network except the input layer and the linear layer.

[0083] In addition, it should be noted that the processor 3 can be set in the smelting end point monitoring chip, that is, the processor 3 can be set in the smelting end point monitoring chip together with the optical modulation layer 1 and the image sensor 2, or it can be set separately outside the smelting end point monitoring chip and connected to the image sensor 2 in the smelting end point monitoring chip through a data line or a connecting device. This embodiment does not limit this.

[0084] In addition, it should be noted that the processor 3 can be implemented using a computer, an ARM or FPGA circuit board with certain computing power, or a microprocessor, and this embodiment does not limit this. In addition, as previously described, the processor 3 can be integrated into the smelting endpoint monitoring chip, or can be independently provided outside the smelting endpoint monitoring chip. When the processor 3 is provided independently outside the smelting endpoint monitoring chip, the electrical signal in the image sensor 2 can be read out to the processor 3 through the signal readout circuit, and then the processor 3 performs full connection processing and nonlinear activation processing on the readout electrical signal.

[0085] In this embodiment, it can be understood that when the processor 3 performs nonlinear activation processing, it can be implemented using a nonlinear activation function, such as a Sigmoid function, a Tanh function, a ReLU function, etc., which is not limited in this embodiment.

[0086] In this embodiment, the light modulation layer 1 corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer. The image sensor 2 corresponds to the linear layer of the artificial neural network, converts the light intensity distribution signal at different spatial positions into an electrical signal. The processor 3 corresponds to the nonlinear layer and output layer of the artificial neural network, fully connects the electrical signals at different positions, and obtains the output signal of the artificial neural network through a nonlinear activation function to realize the monitoring of the smelting end point.

[0087] like Figure 2 As shown on the left, the smelting endpoint monitoring chip includes a light modulation layer 1 and an image sensor 2. The processor 3 is connected to the image sensor 2 in the smelting endpoint monitoring chip. Figure 2 In the embodiment, the processor 3 is implemented by a signal readout circuit and a computer. Figure 2As shown on the right, the optical modulation layer 1 in the smelting endpoint monitoring chip corresponds to the input layer of the artificial neural network, the image sensor 2 corresponds to the linear layer of the artificial neural network, the processor 3 corresponds to the nonlinear layer and output layer of the artificial neural network, and the filtering effect of the optical modulation layer 1 on the incident light entering the optical modulation layer 1 corresponds to the connection weight from the input layer to the linear layer. It can be seen that the optical modulation layer and the image sensor in the smelting endpoint monitoring chip provided in this embodiment realize the relevant functions of the input layer and the linear layer in the artificial neural network by means of hardware, so that when the smelting endpoint monitoring chip is used for subsequent identification processing, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer (for example, calculations such as the connection weight from the input layer to the linear layer are omitted), which can greatly reduce the power consumption and delay during artificial neural network processing. In addition, the embodiment of the present invention simultaneously utilizes the image information, spectral information, angle of incident light and phase information of incident light at different spatial positions of the steelmaking furnace mouth, that is, the information carried by the incident light at different points in the space of the steelmaking furnace mouth. Since the information carried by the incident light at different points in the space of the steelmaking furnace mouth covers the image, composition, shape, three-dimensional depth, structure, three-dimensional color and other information of different spatial positions of the steelmaking furnace mouth, when the smelting end point monitoring is performed based on the information carried by the incident light at different points in the steelmaking furnace mouth, the image, composition, shape, three-dimensional depth, structure, three-dimensional color and other information of different spatial positions of the steelmaking furnace mouth can be covered, thereby solving the problem of difficulty in accurately realizing smelting end point monitoring mentioned in the background technology part. It can be seen that the smelting end point monitoring chip provided by the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate.

[0088] like Figure 2 As shown on the right, the incident light spectrum at different positions of the light modulation layer 1 Projected / connected to the photocurrent response of the image sensor In the figure, processor 3 includes a signal readout circuit and a computer. The signal readout circuit in processor 3 reads out the photocurrent response and transmits it to the computer. The computer performs full connection processing and nonlinear activation processing on the electrical signal and finally outputs the result.

[0089] like Figure 3As shown, the light modulation structure on the light modulation layer 1 is integrated above the image sensor 2, modulating the incident light reflected, transmitted and / or radiated by the steelmaking furnace mouth, and projecting / connecting the spectral information of the incident light to different pixel points of the image sensor 2 to obtain an electrical signal containing the spectral information and image information of the incident light. That is, after the incident light passes through the light modulation layer 1, it is converted into an electrical signal by the image sensor 2 to form an image containing the spectral information of the incident light. Finally, the processor 3 connected to the image sensor 2 processes the electrical signal containing the spectral information and image information of the incident light. It can be seen that the smelting end point monitoring chip provided by the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, so that the smelting end point can be monitored quickly and accurately.

[0090] In this embodiment, the information carried by the incident light may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

[0091] For example, in one implementation, the information carried by the incident light may include light intensity distribution information. In other implementations, the target object can be identified by simultaneously utilizing multiple information including the target object's image information, spectral information, angle of the incident light, and phase information of the incident light, thereby achieving more accurate intelligent identification of the target object.

[0092] The present invention projects one or more of the image information, spectral information, angle of incident light, and phase information of incident light at different positions at the steelmaking furnace mouth onto the photocurrent response of the hardware chip, making full use of the light-related information in various dimensions of the steelmaking furnace mouth, thereby enabling accurate smelting endpoint monitoring.

[0093] The embodiment of the present invention realizes a new smelting endpoint monitoring chip that can realize the function of an artificial neural network, which is used for smelting endpoint monitoring tasks. In the smelting endpoint monitoring chip, an artificial neural network is embedded in a hardware manner, wherein a light modulation layer is arranged on the surface of the photosensitive area of ​​the image sensor, and the light modulation layer includes a light modulation structure. The light modulation layer is used to perform different spectrum modulations on the incident light entering different position points of the light modulation structure through the light modulation structure, so as to obtain the incident light carrying information corresponding to different position points on the surface of the photosensitive area. Accordingly, the image sensor is used to convert the incident light carrying information corresponding to different position points into electrical signals corresponding to different position points, and the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. It can be seen that in the smelting endpoint monitoring chip, the light modulation layer is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. The optical modulation layer serves as the input layer of the artificial neural network, the image sensor serves as the linear layer of the artificial neural network, and the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer serves as the connection weight from the input layer to the linear layer. That is, the optical modulation layer and the image sensor in the smelting end point monitoring chip realize the relevant functions of the input layer and the linear layer in the artificial neural network. That is, the embodiment of the present invention strips the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and implements the two-layer structure of the input layer and the linear layer in the artificial neural network by hardware, so that when the smelting end point monitoring chip is used for smelting end point identification in the future, there is no need to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer. The processor in the smelting end point monitoring chip only needs to perform relevant processing with full connection and nonlinear activation of the electrical signal, which can greatly reduce the power consumption and delay during smelting end point identification. It can be seen that the embodiment of the present invention uses the light modulation layer as the input layer of the artificial neural network, the image sensor as the linear layer of the artificial neural network, and the filtering effect of the light modulation layer on the incident light entering the light modulation layer as the connection weight from the input layer to the linear layer. The light modulation layer and the image sensor are used to project the light-carrying information at different points of the steelmaking furnace mouth into electrical signals, and then the full connection processing and nonlinear activation processing of the electrical signals are implemented in the processor. It can be seen that the embodiment of the present invention can eliminate the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, thereby greatly improving the processing speed.

[0094] In addition, the embodiment of the present invention can utilize one or more information including image information, spectral information, angle of incident light and phase information of incident light at different points in the space of the steelmaking furnace mouth, that is, utilize the information carried by the incident light at different points in the space of the steelmaking furnace mouth. Since the information carried by the incident light at different points in the steelmaking furnace mouth covers the image, composition, shape, three-dimensional depth, structure, three-dimensional color and other information of the steelmaking furnace mouth, when the steelmaking end point is identified based on the information carried by the incident light at different points, the image, composition, shape, three-dimensional depth, three-dimensional color and other information of the steelmaking furnace mouth can be covered, so that the available information can be fully utilized to identify the steelmaking end point, thereby solving the problem of difficulty in accurately identifying the smelting end point mentioned in the background technology part. It can be seen that the smelting end point monitoring chip provided by the embodiment of the present invention can simultaneously meet the effects of low power consumption, low latency and high recognition rate, so that the smelting end point can be identified quickly and accurately.

[0095] It can be seen that the embodiment of the present invention provides a new optoelectronic chip for accurately identifying the smelting endpoint. The chip embeds the artificial neural network part into the image sensor containing various light modulation layers to achieve safe, reliable, fast and accurate smelting endpoint control.

[0096] Based on the contents of the above embodiments, in this embodiment, the smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, and the smelting endpoint monitoring result also includes the carbon content and / or molten steel temperature identification result during the smelting process.

[0097] Based on the content of the above embodiment, in this embodiment, the smelting endpoint monitoring chip includes a trained light modulation structure and an image sensor, and the processor is a trained processor;

[0098] The trained optical modulation structure, image sensor, and processor refer to optical modulation structures, image sensors, and processors that meet the training convergence conditions, obtained by training a smelting endpoint monitoring chip comprising different optical modulation structures, image sensors, and processors with different fully connected parameters and nonlinear activation parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task;

[0099] The input training samples include incident light reflected, transmitted and / or radiated from the mouth of a steelmaking furnace that has been smelted to the end point or not; the output training samples include the determination result of whether the steelmaking furnace has been smelted to the end point.

[0100] Based on the contents of the above embodiments, in this embodiment, when the smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, accordingly, the input training sample also includes the incident light reflected, transmitted and / or radiated by the steelmaking furnace mouth smelted to different carbon contents and / or molten steel temperatures, and the output training sample also includes the corresponding carbon content and / or molten steel temperature.

[0101] It can be seen that the monitorable samples of this embodiment include but are not limited to the endpoint control of converter steelmaking. Artificial neural network training can also be introduced to monitor the smelting furnace mouth temperature and material elements. It is very easy to integrate with the back-end industrial control system, and has high recognition accuracy and relatively accurate qualitative analysis.

[0102] In this embodiment, it can be understood that the input training samples include incident light reflected, transmitted and / or radiated by the steelmaking furnace mouth in the corresponding intelligent processing task; the output training samples include the intelligent processing results of the steelmaking furnace mouth, such as the smelting end point recognition results, the temperature recognition results during the smelting process, the carbon content recognition results during the smelting process, etc.

[0103] In this embodiment, the reflected light, transmitted light and / or radiated light from the steelmaking furnace mouth enters the trained smelting endpoint monitoring chip to obtain intelligent processing results such as whether it is the smelting endpoint, the furnace mouth temperature and the material element content.

[0104] In this embodiment, the identification task of the smelting end point is taken as an example. It can be understood that when using the smelting end point monitoring chip to perform the identification task, the smelting end point monitoring chip needs to be trained first. Training the smelting end point monitoring chip here refers to determining the optical modulation structure suitable for the current identification task through training, as well as the fully connected parameters and nonlinear activation parameters suitable for the current identification task.

[0105] It can be understood that since the filtering effect of the optical modulation layer on the incident light entering the optical modulation layer corresponds to the connection weight from the input layer to the linear layer of the artificial neural network, during training, changing the optical modulation structure in the optical modulation layer is equivalent to changing the connection weight from the input layer to the linear layer of the artificial neural network. Through the training convergence conditions, the optical modulation structure suitable for the current recognition task is determined, as well as the fully connected parameters and nonlinear activation parameters suitable for the current recognition task, thereby completing the training of the smelting end point monitoring chip.

[0106] It is understood that after training the smelting endpoint monitoring chip, it can be used to perform recognition tasks. Specifically, after the incident light carrying the steelmaking furnace mouth image information and light-carrying information enters the trained smelting endpoint monitoring chip's light modulation layer 1, the light modulation structure in the light modulation layer 1 modulates the incident light. The modulated light signal intensity is detected by the image sensor 2 and converted into an electrical signal. The processor 3 then performs full-connection processing and nonlinear activation processing to obtain the recognition result of whether the current smelting endpoint is reached.

[0107] like Figure 4 As shown, the complete process for identifying the mouth of the steelmaking furnace to determine whether it is the end point of smelting is as follows: the light emitted by the mouth flame 100 of the steelmaking converter 200 is collected by the smelting end point monitoring chip 300, and after being processed by the light modulation layer, image sensor and processor in the smelting end point monitoring chip, the identification result of whether the current state is the end point of smelting can be obtained.

[0108] It can be seen that the purpose of this embodiment is to provide a new optoelectronic chip for smelting endpoint control. The chip collects the image information, spectral information, angle of incident light and phase information of the furnace mouth during the smelting process online, and can realize fast, accurate, safe and reliable smelting endpoint, furnace mouth temperature measurement and detection and qualitative analysis of carbon elements, etc.

[0109] This chip features micro-nanomodulation structures fabricated directly on the surface of the image sensor's photosensitive area. Several discrete or continuous micro-nanomodulation structures form a unit. These structures at different locations modulate the incident light spectrum differently, collectively forming a light modulation layer. The modulation intensities of these micro-nanomodulation structures for different wavelength components of the incident light correspond to the connection strengths (linear layer weights) of the artificial neural network. The image sensor converts the information carried by the incident light modulated by the light modulation layer at different locations into electrical signals corresponding to these locations. These signals are then sent to the processor, which performs fully connected processing and nonlinear activation on these signals to generate the output signal of the artificial neural network. For smelting endpoint control, images and spectral information of the converter mouth at the endpoint are first collected. Through data training, the weights of the linear layer (i.e., the system function of the light modulation layer) are derived. This allows the desired light modulation layer structure to be reverse engineered and integrated above the image sensor. In actual use, during the actual smelting endpoint control process, the output of the completed optical modulation layer is used to further train and optimize the weights of the fully connected layer of electrical signals, thereby realizing a high-accuracy optical artificial neural network and accurately judging the endpoint moment of the smelting process.

[0110] As can be understood, this chip utilizes both image and spectral information from the converter mouth at the endpoint to accurately determine the online mouth temperature and elemental content, improving the accuracy of determining the smelting endpoint. It also partially implements an artificial neural network in hardware, speeding up the determination of the smelting endpoint. Furthermore, this chip solution can be mass-produced using existing CMOS processes, reducing device size, power consumption, and cost, and facilitating integration with subsequent control systems.

[0111] It is understandable that for recognition tasks, the advantage of the smelting endpoint monitoring chip provided in this embodiment is its ability to obtain light-borne information about the object being recognized. Therefore, to fully utilize this advantage, actual objects are preferably used as input training samples, rather than two-dimensional images of the objects. Of course, this does not mean that two-dimensional images cannot be used as recognition object samples.

[0112] In this embodiment, the optical modulation layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network. In order to minimize the loss function of the neural network, the modulation intensity of the different wavelength components in the incident light from the steelmaking furnace mouth by the optical modulation structure in the optical modulation layer is used as the connection weight from the input layer to the linear layer of the neural network. By adjusting the structure of the optical modulation layer, the modulation intensity of the different wavelength components in the incident light from the steelmaking furnace mouth can be adjusted, thereby realizing the adjustment of the connection weight from the input layer to the linear layer, and then optimizing the training of the neural network.

[0113] Therefore, in this embodiment, the optical modulation structure is obtained based on neural network training. The training samples are optically simulated by a computer to obtain the sample modulation intensity of the optical modulation structure in the training samples for different wavelength components of the incident light at the steelmaking furnace mouth in the intelligent processing task. The sample modulation intensity is used as the connection weight from the input layer to the linear layer of the neural network to perform nonlinear activation, and the neural network is trained using the training samples corresponding to the intelligent processing task until the neural network converges and the corresponding training sample optical modulation structure is used as the optical modulation layer of the corresponding intelligent processing task.

[0114] Thus, by implementing the input layer and linear layer of the neural network at the physical layer, this embodiment can eliminate the complex signal processing and algorithm processing corresponding to the input layer and linear layer in the prior art, thereby improving processing speed and reducing latency. At the same time, this embodiment of the present invention actually utilizes the image information, spectral information, angle of incident light, and phase information of the steelmaking furnace mouth simultaneously, that is, the information carried by the incident light at different points in the space of the steelmaking furnace mouth. This allows further extraction of information such as the composition, shape, and three-dimensional depth of the steelmaking furnace mouth from the spatial image, spectrum, angle, and phase information, thereby resolving the problem mentioned in the background technology section that it is difficult to ensure recognition accuracy using two-dimensional image information of the steelmaking furnace mouth.

[0115] Based on the contents of the above embodiments, in this embodiment, when training a smelting endpoint monitoring chip comprising different optical modulation structures and image sensors and a processor having different fully connected parameters and nonlinear activation parameters, the different optical modulation structures are designed and implemented by adopting computer optical simulation design.

[0116] This embodiment designs the optical modulation structure through computer optical simulation, and adjusts the optical modulation structure through optical simulation until the neural network converges, determining that the corresponding optical modulation structure is the final optical modulation structure size required. This saves prototyping time and cost, improves product efficiency, and easily solves complex optical problems. For example, the optical modulation structure can be simulated and designed using FDTD software. By changing the optical modulation structure during optical simulation, the modulation intensity of the optical modulation structure for different incident light can be accurately predicted. This can be used as the connection weight between the input layer and the linear layer of the neural network to train the smelting endpoint monitoring chip and accurately obtain the optical modulation structure.

[0117] It is understandable that the converter mouth image and spectral information at the end moment can be collected multiple times in advance and data training can be performed, the required micro-nano modulation structure can be designed and prepared, and the input layer and linear layer of the artificial neural network can be realized on the chip.

[0118] It can be seen that this embodiment designs the light modulation structure by adopting computer optical simulation design, which saves the time and cost of light modulation structure prototype production and improves product efficiency.

[0119] It is understood that this chip utilizes both image information and light-carrying information at the furnace mouth, improving the accuracy of smelting endpoint identification. This embodiment partially implements an artificial neural network in hardware, improving the real-time nature of smelting endpoint identification. Furthermore, this chip solution can be mass-produced using existing CMOS processes, reducing device size, power consumption, and cost.

[0120] The optical artificial neural network smelting endpoint monitoring chip based on micro-nano modulation structure and image sensor provided in this embodiment has a structural schematic diagram as shown in FIG. Figure 2 As shown, it includes a light modulation layer 1, an image sensor 2 and a processor 3. The light modulation layer corresponds to the input layer of the artificial neural network and the connection weights from the input layer to the linear layer. The image sensor corresponds to the linear layer of the artificial neural network. A number of discrete or continuous micro-nano modulation structures constitute a unit. These modulation structures have different broadband spectrum modulation effects on the incident light. The micro-nano modulation structures contained in different units can be the same or different, and can also be spatially reconstructed according to the image. Each unit corresponds to multiple image sensor photosensitive pixels in the vertical direction. The image sensor 2 converts the outgoing light field of the light modulation layer 1 into a single unit. Converted to the photocurrent response of the image sensor The processor 3 includes a signal readout circuit and a computer. The signal readout circuit in the processor 3 reads out the photocurrent response. The data is transmitted to the computer, which performs full connection processing and nonlinear activation processing on the electrical signal and finally outputs the result.

[0121] It can be understood that for smelting end point control, by performing optical simulation of the micro-nano modulation structure on a computer, the modulation intensity (transmittance) of the modulation structure for different wavelength components of the incident light can be obtained, and it is used as the connection weight from the input layer to the linear layer of the artificial neural network. The image sensor is used to convert the information carried by the incident light corresponding to different position points after modulation by the optical modulation layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the electrical signal is the image signal modulated by the optical modulation layer; the processor performs full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network. The light-carrying information of the converter mouth at the end point is collected multiple times in advance and data training is performed. The required micro-nano modulation structure can be designed and prepared, and the input layer and linear layer of the artificial neural network can be realized on the chip.

[0122] From a vertical perspective, Figure 2 As shown in Figure 1, each micro-nano modulation structure in the light modulation layer is designed by pre-training an artificial neural network and can be prepared by directly growing one or more layers of dielectric or metal materials on the image sensor and then etching them. The overall size of each modulation unit in the light modulation layer is usually λ 2 ~10 5 λ 2 The thickness is typically 0.1λ to 10λ, where λ is the center wavelength of the target wavelength. Each modulation unit structure in the optical modulation layer corresponds to multiple pixels on the image sensor. The optical modulation layer is fabricated directly on the image sensor, and the image sensor and processor are connected via electrical contacts.

[0123] It is understandable that both the light modulation layer and the CIS wafer (CIS wafer is a special image sensor) can be manufactured by the semiconductor CMOS integration process. The light modulation layer can be monolithically integrated on the image sensor directly at the wafer level, and the chip can be prepared by a single tape-out using the CMOS process, thereby achieving monolithic integration at the wafer level, which is beneficial to reducing the distance between the sensor and the light modulation layer, reducing the size of the device, and reducing packaging costs.

[0124] The optical artificial neural network smelting endpoint monitoring chip based on micro-nano modulation structure and image sensor provided by this embodiment has the following effects: A. The artificial neural network part is embedded in the image sensor containing various optical modulation layers to achieve safe, reliable, fast and accurate smelting endpoint control. B. The samples that can be monitored include but are not limited to the endpoint control of converter steelmaking. The introduction of artificial neural network training to monitor the smelting furnace mouth temperature and material elements is very easy to integrate with the back-end industrial control system, and has high recognition accuracy and accurate qualitative analysis. C. The preparation of the chip can be completed in one-time tape-out through the CMOS process, which is conducive to reducing the device failure rate, improving the finished product yield of the device, and reducing costs. D. The realization of monolithic integration at the wafer level can minimize the distance between the sensor and the optical modulation layer, which is conducive to reducing the size of the unit, reducing the device volume and packaging cost.

[0125] Based on the contents of the above embodiments, in this embodiment, the light modulation structure in the light modulation layer includes a regular structure and / or an irregular structure; and / or, the light modulation structure in the light modulation layer includes a discrete structure and / or a continuous structure.

[0126] In this embodiment, the light modulation structure in the light modulation layer may include only regular structures, only irregular structures, or both regular structures and irregular structures.

[0127] In this embodiment, the optical modulation structure comprising a regular structure may refer to: the smallest modulation unit included in the optical modulation structure is a regular structure, such as a regular shape such as a rectangle, square, or circle. Furthermore, the optical modulation structure comprising a regular structure may also refer to: the smallest modulation unit included in the optical modulation structure is arranged in a regular manner, such as a regular array, a circle, a trapezoid, or a polygon. Furthermore, the optical modulation structure comprising a regular structure may also refer to: the smallest modulation unit included in the optical modulation structure is a regular structure, and the arrangement of the smallest modulation unit is also regular.

[0128] In this embodiment, the light modulation structure comprising an irregular structure may mean that the smallest modulation unit included in the light modulation structure is an irregular structure, such as an irregular polygon, a random shape, or other irregular pattern. Furthermore, the light modulation structure comprising an irregular structure may also mean that the arrangement of the smallest modulation units included in the light modulation structure is irregular, such as an irregular polygon, a random arrangement, or the like. Furthermore, the light modulation structure comprising an irregular structure may also mean that the smallest modulation units included in the light modulation structure are irregular structures, and the arrangement of the smallest modulation units is also irregular.

[0129] In this embodiment, the light modulation structure in the light modulation layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0130] In this embodiment, the light modulation structure including a continuous structure may mean that the light modulation structure is composed of a continuous modulation pattern; the light modulation structure including a discrete structure may mean that the light modulation structure is composed of a discrete modulation pattern.

[0131] It can be understood that the continuous modulation pattern here may refer to a straight line pattern, a wavy line pattern, a broken line pattern, etc.

[0132] It is understandable that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete squares, discrete irregular polygons, etc.).

[0133] In this embodiment, it should be noted that the optical modulation structure has different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, and resonance enhancement. By designing different filter structures, light passing through different groups of filter structures will have different corresponding transmission spectra.

[0134] Based on the contents of the above embodiments, in this embodiment, the light modulation layer is a single-layer structure or a multi-layer structure.

[0135] In this embodiment, it should be noted that the light modulation layer can be a single-layer filter structure or a multi-layer filter structure, for example, a two-layer, three-layer, four-layer or other multi-layer structure.

[0136] In this embodiment, if Figure 1 As shown, the light modulation layer 1 is a single-layer structure. The thickness of the light modulation layer 1 is related to the target wavelength range. For a wavelength of 400 nm to 10 μm, the thickness of the grating structure can be 50 nm to 5 μm.

[0137] It can be understood that since the function of the optical modulation layer 1 is to perform spectral modulation on the incident light, it is preferably prepared from materials with high refractive index and low loss. For example, silicon, germanium, germanium silicon materials, silicon compounds, germanium compounds, III-V materials, etc. can be selected for preparation, among which silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, etc.

[0138] In addition, it should be noted that in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical modulation layer 1 can be set to a multi-layer structure, and the optical modulation structure corresponding to each layer can be set to a different structure, thereby increasing the optical modulation layer's ability to modulate the spectrum of the incident light, so that more or more complex connection weights can be formed between the input layer and the linear layer, thereby improving the accuracy of the smelting endpoint monitoring chip when processing intelligent tasks.

[0139] In addition, it should be noted that for a filter layer comprising a multi-layer structure, the material of each layer can be the same or different. For example, for a two-layer optical modulation layer 1, the first layer can be a silicon layer and the second layer can be a silicon nitride layer.

[0140] It should be noted that the thickness of the light modulation layer 1 is related to the target wavelength range. For a wavelength of 400 nm to 10 μm, the total thickness of the multilayer structure can be 50 nm to 5 μm.

[0141] Based on the contents of the above embodiments, in this embodiment, the light modulation structure in the light modulation layer includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of each micro-nano unit are the same or different.

[0142] In this embodiment, in order to obtain connection weights (used to connect the input layer and the linear layer) distributed in an array so that the processor can perform subsequent full connection and nonlinear activation processing, preferably, in this embodiment, the light modulation structure is in the form of an array structure. Specifically, the light modulation structure includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponding to one or more pixels on the image sensor. It should be noted that the structures of each micro-nano unit can be the same or different. In addition, it should be noted that the structure of each micro-nano unit can be periodic or non-periodic. In addition, it should be noted that each micro-nano unit can further include multiple groups of micro-nano structure arrays, each group of micro-nano structure arrays having the same or different structures.

[0143] The following combination Figures 5 to 9 For example, in this embodiment, Figure 5As shown, the light modulation layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit has the same structure (and each micro-nano unit is a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; Figure 6 As shown, the light modulation layer 1 comprises a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure (similar to Figure 5 The difference is Figure 6 Each micro-nano unit is a periodic structure), each micro-nano unit corresponds to one or more pixel points on the image sensor 2; Figure 7 As shown, the light modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit has the same structure (and each micro-nano unit is a periodic structure). Each micro-nano unit corresponds to one or more pixels on the image sensor 2. Figure 6 The difference is Figure 7 The unit shape of the periodic array in each micro-nano unit has four-fold rotational symmetry; Figure 8 As shown, the light modulation layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, and 66, and Figure 6 The difference is that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixels on the image sensor 2. In this embodiment, the light modulation layer 1 includes multiple different micro-nano units, that is, different areas on the smelting endpoint monitoring chip have different modulation effects on the incident light, thereby increasing the degree of freedom of design and thus improving the accuracy of recognition. Figure 9 As shown, the light modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit has the same structure. Figure 5 The difference is that each micro-nano unit is composed of a discrete non-periodic array structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2.

[0144] In this embodiment, the micro-nano units have different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, and resonance enhancement. By designing different filter structures, light passing through different groups of filter structures will have different corresponding transmission spectra.

[0145] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes a regular structure and / or an irregular structure; and / or, the micro-nano unit includes a discrete structure and / or a continuous structure.

[0146] In this embodiment, the micro-nano unit may include only regular structures, only irregular structures, or both regular structures and irregular structures.

[0147] In this embodiment, the micro-nano unit comprising a regular structure may refer to: the smallest modulation unit contained in the micro-nano unit is a regular structure, such as a regular shape such as a rectangle, square, or circle. Furthermore, the micro-nano unit comprising a regular structure may also refer to: the smallest modulation unit contained in the micro-nano unit is arranged in a regular manner, such as a regular array, circle, trapezoid, polygon, etc. Furthermore, the micro-nano unit comprising a regular structure may also refer to: the smallest modulation unit contained in the micro-nano unit is a regular structure, and the arrangement of the smallest modulation unit is also regular.

[0148] In this embodiment, the micro-nano unit including an irregular structure may refer to: the smallest modulation unit contained in the micro-nano unit is an irregular structure, such as the smallest modulation unit can be an irregular polygon, a random shape, or other irregular pattern. Furthermore, the micro-nano unit including an irregular structure may also refer to: the arrangement of the smallest modulation units contained in the micro-nano unit is irregular, such as the arrangement can be an irregular polygon, a random arrangement, etc. Furthermore, the micro-nano unit including an irregular structure may also refer to: the smallest modulation unit contained in the micro-nano unit is an irregular structure, and the arrangement of the smallest modulation units is also irregular, etc.

[0149] In this embodiment, the micro-nano units in the light modulation layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.

[0150] In this embodiment, the micro-nano unit including a continuous structure may refer to that the micro-nano unit is composed of a continuous modulation pattern; the micro-nano unit including a discrete structure may refer to that the micro-nano unit is composed of a discrete modulation pattern.

[0151] It can be understood that the continuous modulation pattern here may refer to a straight line pattern, a wavy line pattern, a broken line pattern, etc.

[0152] It can be understood that the discrete modulation pattern here may refer to a modulation pattern formed by discrete graphics (such as discrete points, discrete triangles, discrete stars, etc.).

[0153] In this embodiment, it should be noted that different micro-nano units have different modulation effects on light of different wavelengths. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, and resonance enhancement. By designing different micro-nano units, light passing through different groups of micro-nano units will have different corresponding transmission spectra.

[0154] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

[0155] In this embodiment, if Figure 5 As shown, the light modulation layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes four different micro-nano structure arrays 110, 111, 112, and 113, and the filter unit 44 includes four different micro-nano structure arrays 440, 441, 442, and 443. Figure 10 As shown, the optical modulation layer 1 includes multiple micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit includes multiple groups of micro-nano structure arrays. For example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.

[0156] It should be noted that the example here is only to illustrate the micro-nano unit including four groups of micro-nano structure arrays, which does not serve as a limitation. In actual applications, the micro-nano unit can also be provided with six groups, eight groups or other numbers of micro-nano structure arrays as needed.

[0157] In this embodiment, each micro-nanostructure array within the micro-nano unit modulates light of different wavelengths differently, and each filter structure group modulates the input light differently. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, and resonance enhancement. By designing different micro-nanostructure arrays, light passing through different micro-nanostructure arrays produces different transmission spectra.

[0158] Based on the contents of the above embodiments, in this embodiment, each group of micro-nanostructure arrays has the function of broadband filtering or narrowband filtering.

[0159] In this embodiment, in order to obtain the modulation intensity of different wavelength components of the incident light from the steelmaking furnace mouth as the connection weights of the neural network input layer and the linear layer, different micro-nanostructure arrays are used to achieve broadband filtering and narrowband filtering. Therefore, in this embodiment, the micro-nanostructure array performs broadband filtering or narrowband filtering on the incident light from the steelmaking furnace mouth to obtain the modulation intensity of different wavelength components of the incident light from the steelmaking furnace mouth. Figure 11 and Figure 12 As shown, each group of micro-nanostructure arrays in the light modulation layer has the function of broadband filtering or narrowband filtering.

[0160] It is understood that each group of micro-nanostructure arrays may all have broadband filtering capabilities, all have narrowband filtering capabilities, or some may have broadband filtering capabilities and some may have narrowband filtering capabilities. Furthermore, the broadband and narrowband filtering ranges of each group of micro-nanostructure arrays may be the same or different. For example, by designing the period, duty cycle, radius, side length, and other dimensional parameters of each group of micro-nanostructures within a micro-nanounit, they may achieve narrowband filtering capabilities, allowing only one (or fewer) wavelengths of light to pass through. Another example is by designing the period, duty cycle, radius, side length, and other dimensional parameters of each group of micro-nanostructures within a micro-nanounit, they may achieve broadband filtering capabilities, allowing light of more or all wavelengths to pass through.

[0161] It is understandable that, in specific use, the filtering state of each group of micro-nano structure arrays can be determined by performing broadband filtering, narrowband filtering or a combination thereof according to the application scenario.

[0162] Based on the contents of the above embodiments, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

[0163] In this embodiment, each group of micro-nanostructure arrays can be all periodic structure arrays, all non-periodic structure arrays, or some periodic structure arrays and some non-periodic structure arrays. Of these, periodic structure arrays are easier to perform optical simulation design, while non-periodic structure arrays can achieve more complex modulation effects.

[0164] In this embodiment, if Figure 5 As shown, the light modulation layer 1 includes multiple repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures of each micro-nano structure array are different from each other, and the micro-nano structure array is a non-periodic structure. Among them, the non-periodic structure refers to the arrangement of the modulation holes on the micro-nano structure array in a non-periodic arrangement. Figure 5 As shown, the micro-nano unit 11 includes four different non-periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different non-periodic structure arrays 440, 441, 442 and 443. The micro-nano structure arrays with non-periodic structures are designed by neural network data training for intelligent processing tasks in the early stage, and are usually irregularly shaped structures. Figure 6 As shown, the light modulation layer 1 comprises a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of micro-nano structure arrays. The structures of the micro-nano structure arrays are different from each other. Figure 5The difference is that the micro-nano structure array is a periodic structure. The periodic structure refers to the arrangement of the modulated holes on the micro-nano structure array in a periodic manner, and the period size is usually 20nm~50μm. Figure 6 As shown, the micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442 and 443. The filter structure of the periodic structure is obtained by training and designing the neural network data for intelligent processing tasks in the early stage, and is usually an irregular structure. Figure 7 As shown, the optical modulation layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures of each micro-nano structure array are different from each other, and the micro-nano structure array is a periodic structure. The periodic structure refers to the shape on the filter structure arranged in a periodic arrangement, and the period size is usually 20nm~50μm. Figure 7 As shown, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other. The micro-nano unit 11 includes four different periodic structure arrays 110, 111, 112 and 113, and the micro-nano unit 44 includes four different periodic structure arrays 440, 441, 442 and 443. The micro-nano structure arrays with periodic structures are obtained through early training and design of neural network data for intelligent processing tasks, and are usually irregularly shaped structures.

[0165] It should be noted that Figures 5 to 9 Each micro-nano unit comprises four micro-nanostructure arrays, each formed with four different shapes of modulation apertures. These four micro-nanostructure arrays are designed to achieve different modulation effects on incident light. It should be noted that the micro-nano unit comprising four micro-nanostructure arrays is used for illustrative purposes only and is not intended to be limiting. In practical applications, a micro-nano unit comprising six, eight, or other numbers of micro-nanostructure arrays may be configured as needed. In this embodiment, the four different shapes may be, but are not limited to, a circle, a cross, a regular polygon, and a rectangle.

[0166] In this embodiment, each micro-nanostructure array within a micro-nano unit modulates light of different wavelengths in a different manner, and each micro-nanostructure array modulates input light differently. Specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, and resonance enhancement. By designing different micro-nanostructure arrays, light passing through different micro-nanostructure arrays produces different transmission spectra.

[0167] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit includes a plurality of micro-nano structure arrays, and there is one or more groups of empty structures.

[0168] The following combination Figure 9 The example shown is used to illustrate, in this embodiment, Figure 9 As shown, the optical modulation layer 1 includes multiple repeating micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures corresponding to the multiple groups of micro-nano structure arrays are different. The micro-nano structure arrays are periodic structures. Unlike the above embodiment, each micro-nano unit includes one or more groups of empty structures, which are used to directly pass incident light. It can be understood that when multiple groups of micro-nano structure arrays include one or more groups of empty structures, a richer spectrum modulation effect can be achieved, thereby meeting the spectrum modulation requirements in specific scenarios (or meeting the specific connection weight requirements between the input layer and the linear layer in specific scenarios).

[0169] like Figure 9 As shown, each micro-nano unit comprises a micro-nano structure array and three groups of empty structures. Micro-nano unit 11 comprises a non-periodic structure array 111, micro-nano unit 22 comprises a non-periodic structure array 221, micro-nano unit 33 comprises a non-periodic structure array 331, micro-nano unit 44 comprises a non-periodic structure array 441, micro-nano unit 55 comprises a non-periodic structure array 551, and micro-nano unit 66 comprises a non-periodic structure array 661. The micro-nano structure arrays are used to modulate incident light in various ways. It should be noted that this example, which includes one micro-nano structure array and three groups of empty structures, is not intended to be limiting. In practical applications, micro-nano units can also include one micro-nano structure array and five groups of empty structures, or another number of micro-nano structure arrays, as needed. In this embodiment, the micro-nano structure arrays can be made using modulation apertures in circular, cross-shaped, regular polygonal, or rectangular shapes (but not limited to these).

[0170] It should be noted that the multiple groups of micro-nano structure arrays included in the micro-nano unit may not include any empty structures, that is, the multiple groups of micro-nano structure arrays may be non-periodic structure arrays or periodic structure arrays.

[0171] Based on the content of the above embodiment, in this embodiment, the micro-nano unit has a polarization-independent characteristic.

[0172] In this embodiment, because the micro-nano unit has polarization-independent characteristics, the optical modulation layer is insensitive to the polarization of the incident light, thereby realizing a smelting endpoint monitoring chip that is insensitive to both the incident angle and polarization. The smelting endpoint monitoring chip provided by the embodiment of the present invention is insensitive to the incident angle and polarization characteristics of the incident light, that is, the measurement results are not affected by the incident angle and polarization characteristics of the incident light, thereby ensuring the stability of the spectral measurement performance, and further ensuring the stability of intelligent processing, such as the stability of intelligent perception, the stability of intelligent identification, the stability of intelligent decision-making, and so on. It should be noted that the micro-nano unit may also have polarization-dependent characteristics.

[0173] Based on the contents of the above embodiments, in this embodiment, the micro-nano unit has four-fold rotational symmetry.

[0174] In this embodiment, it should be noted that four-fold rotational symmetry is a specific case of polarization-independent characteristics. By designing the micro-nano unit into a structure with four-fold rotational symmetry, the requirement of polarization-independent characteristics can be met.

[0175] The following combination Figure 7 The example shown is used to illustrate, in this embodiment, Figure 7 As shown, the optical modulation layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66. Each micro-nano unit is composed of a plurality of groups of micro-nano structure arrays. The structures corresponding to the plurality of groups of micro-nano structure arrays are different from each other. The micro-nano structure arrays are periodic structures. Unlike the above-mentioned embodiments, the structure corresponding to each group of micro-nano structure arrays can be a structure with four-fold rotational symmetry, such as a circle, a cross, a regular polygon, or a rectangle. That is, after the structure is rotated 90°, 180°, and 270°, it coincides with the original structure, thereby making the structure polarization-independent, so that the same intelligent recognition effect can be achieved when different polarized light is incident.

[0176] Based on the content of the above embodiment, in this embodiment, the light modulation layer is composed of one or more filter layers;

[0177] The filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP (Surface Plasmon Polaritons, SPP) micro-nanostructures, and adjustable Fabry-Perot cavity (Fabry-perot Cavity, FP cavity).

[0178] The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

[0179] Among them, photonic crystals, as well as the combination of metasurfaces and random structures, are compatible with CMOS processes and can achieve better modulation effects. Other materials can also be filled in the micropores of micro-nano modulation structures to smooth the surface. Quantum dots and perovskites can utilize the spectral modulation characteristics of the material itself to minimize the volume of a single modulation structure. SPPs are small in size and can achieve polarization-dependent light modulation. Liquid crystals can be dynamically controlled by voltage to improve spatial resolution. Adjustable Fabry-Perot cavities can be dynamically controlled to improve spatial resolution.

[0180] Based on the content of the above embodiment, in this embodiment, the thickness of the light modulation layer is 0.1λ~10λ, where λ represents the central wavelength of the incident light.

[0181] In this embodiment, it should be noted that if the thickness of the optical modulation layer is much smaller than the central wavelength of the incident light, it will not be able to effectively modulate the spectrum; if the thickness of the optical modulation layer is much larger than the central wavelength of the incident light, it will be difficult to manufacture and will introduce large optical losses. Therefore, in this embodiment, in order to reduce optical losses, facilitate manufacture, and ensure effective spectrum modulation, the overall size (area) of each micro-nano unit in the optical modulation layer 1 is generally λ 2 ~10 5 λ 2 , the thickness is usually 0.1λ~10λ (λ represents the central wavelength of the incident light at the steelmaking furnace mouth). Figure 5 As shown, the overall size of each micro-nano unit is 0.5 μm 2 ~40000μm 2 The dielectric material in the light modulation layer 1 is polysilicon with a thickness of 50nm~2μm.

[0182] Based on the content of the above embodiment, in this embodiment, the image sensor is any one or more of the following:

[0183] CMOS image sensor (Contact Image Sensor, CIS), Charge Coupled Device (Charge Coupled Device, CCD), Single Photon Avalanche Diode (Single Photon Avalanche Diode, SPAD) array and focal plane photoelectric image sensor array.

[0184] In this embodiment, it should be noted that the use of a wafer-level CMOS image sensor CIS and monolithic integration at the wafer level can minimize the distance between the image sensor and the light modulation layer, which is beneficial for reducing the size of the unit, reducing the device volume and packaging cost. SPAD can be used for weak light detection, and CCD can be used for strong light detection.

[0185] In this embodiment, the light modulation layer and image sensor can be manufactured using a complementary metal oxide semiconductor (CMOS) integrated process, which helps reduce device failure rates, improve device yield, and lower costs. For example, the light modulation layer can be fabricated by directly growing one or more layers of dielectric material on the image sensor, then etching it, depositing a metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer.

[0186] Based on the content of the above embodiment, in this embodiment, the type of the artificial neural network includes: a feedforward neural network.

[0187] In this embodiment, a feedforward neural network (FNN), also known as a deep feedforward network (DFN) or multi-layer perceptron (MLP), is the simplest type of neural network, with neurons arranged in layers. Each neuron is connected only to neurons in the previous layer. It receives the output of the previous layer and outputs it to the next layer, with no feedback between layers. Feedforward neural networks have a simple structure, are easy to implement in hardware, and have a wide range of applications. They can approximate any continuous function and square-integrable function with arbitrary precision, and can accurately implement any finite training sample set. A feedforward network is a static nonlinear mapping. Complex nonlinear processing capabilities can be achieved through the composite mapping of simple nonlinear processing units.

[0188] Based on the content of the above embodiment, in this embodiment, a light-transmitting medium layer is provided between the light modulation layer and the image sensor.

[0189] In this embodiment, it should be noted that providing a light-transmitting medium layer between the light modulation layer and the image sensor can effectively separate the light modulation layer from the image sensor layer to avoid mutual interference between the two.

[0190] Based on the contents of the above embodiments, in this embodiment, the image sensor is front-illuminated and includes: a metal wire layer and a light detection layer arranged from top to bottom, and the light modulation layer is integrated on a side of the metal wire layer away from the light detection layer; or,

[0191] The image sensor is back-illuminated and includes a light detection layer and a metal wire layer arranged from top to bottom. The light modulation layer is integrated on a side of the light detection layer away from the metal wire layer.

[0192] In this embodiment, if Figure 13 The image sensor shown is a front-illuminated image sensor, in which the silicon detection layer 21 is below the metal wire layer 22 , and the light modulation layer 1 is directly integrated onto the metal wire layer 22 .

[0193] In this embodiment, Figure 13 The difference is, Figure 14 The image sensor shown is a back-illuminated image sensor, in which the silicon detection layer 21 is above the metal line layer 22 , and the light modulation layer 1 is directly integrated onto the silicon detection layer 21 .

[0194] It should be noted that, for a back-illuminated image sensor, the silicon detection layer 21 is above the metal wire layer 22 , which can reduce the influence of the metal wire layer on the incident light, thereby improving the quantum efficiency of the device.

[0195] Thus, in the embodiments of the present invention, the optical modulation layer is used as the input layer of the artificial neural network, the filtering effect of the optical modulation layer on the incident light is used as the connection weight from the input layer to the linear layer, and the image sensor is used as the linear layer of the artificial neural network. The embodiments of the present invention project the light-carrying information of the object into the photocurrent response of the image sensor, and implement full connection and nonlinear activation of the electrical signal in the processor, thereby achieving low power consumption, low latency, and high accuracy intelligent perception, recognition, and / or decision-making functions. The optical artificial neural network smelting endpoint monitoring chip based on the optical filter and image sensor in the embodiments of the present invention has the following effects: the artificial neural network is partially embedded in the image sensor containing various optical modulation layers, achieving fast and accurate intelligent perception, recognition, and / or decision-making functions. In addition, the embodiments of the present invention can also achieve monolithic integration at the wafer level, thereby minimizing the distance between the sensor and the optical modulation layer, which is conducive to reducing the size of the unit, reducing the device volume and packaging cost.

[0196] Based on the same inventive concept, another embodiment of the present invention provides an intelligent smelting control device, including: the smelting endpoint monitoring chip as described in the above embodiment. The intelligent smelting control device can include various devices related to smelting process control, which are not limited in this embodiment.

[0197] Since the intelligent smelting control equipment provided in this embodiment includes the smelting endpoint monitoring chip described in the above embodiment, the intelligent smelting control equipment provided in this embodiment has all the beneficial effects of the smelting endpoint monitoring chip described in the above embodiment. Since the above embodiment has already described this in more detail, this embodiment will not repeat it.

[0198] Based on the same inventive concept, another embodiment of the present invention provides a method for preparing a smelting endpoint monitoring chip as described in the above embodiment, such as Figure 15 As shown, the specific steps include:

[0199] Step 1510: preparing a light modulation layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor;

[0200] Step 1520: Generate a processor capable of performing full connection processing and nonlinear activation processing on the signal;

[0201] Step 1530: Connect the image sensor and the processor;

[0202] The light modulation layer is used to perform different spectral modulations on incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive area; the incident light-carrying information includes light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;

[0203] The image sensor is used to convert the information carried by the incident light corresponding to different position points after modulation by the light modulation layer into electrical signals corresponding to different position points, and send the electrical signals corresponding to different position points to the processor; the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the smelting end point monitoring results.

[0204] In this embodiment, a training process for the optical artificial neural smelting endpoint monitoring chip is also included, specifically including:

[0205] Using input training samples and output training samples corresponding to the smelting endpoint monitoring task, a smelting endpoint monitoring chip containing different optical modulation structures and image sensors and a processor with different fully connected parameters are trained to obtain an optical modulation structure, image sensor and processor that meet the training convergence conditions, and the optical modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained optical modulation structure, image sensor and processor.

[0206] It can be understood that when training an optical artificial neural smelting endpoint monitoring chip comprising different light modulation structures, image sensors and processors with different fully connected parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

[0207] It is understandable that the converter mouth image and spectral information at the end moment can be collected multiple times in advance and data training can be performed, the required micro-nano modulation structure can be designed and prepared, and the input layer and linear layer of the artificial neural network can be realized on the chip.

[0208] In this embodiment, a light modulation layer including a light modulation structure is prepared on the surface of the photosensitive area of ​​the image sensor, including:

[0209] growing one or more layers of a preset material on the surface of the image sensor;

[0210] performing dry etching of a light modulation structure pattern on the one or more layers of the preset material to obtain a light modulation layer including the light modulation structure;

[0211] or performing embossing transfer on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0212] Or, a light modulation layer including a light modulation structure is obtained by externally and dynamically regulating the one or more layers of the preset material;

[0213] or performing zone printing on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0214] or performing partitioned material growth on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure;

[0215] Alternatively, quantum dot transfer is performed on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure.

[0216] When the smelting endpoint monitoring chip is used for the intelligent processing task of the steelmaking furnace mouth, the smelting endpoint monitoring chip containing different optical modulation structures, image sensors and processors with different fully connected parameters is trained using the input training samples and output training samples corresponding to the intelligent processing task to obtain the optical modulation structure, image sensor and processor that meet the training convergence conditions.

[0217] In this embodiment, it should be noted that Figure 1 As shown, the light modulation layer 1 can be fabricated by directly growing one or more layers of dielectric material on the image sensor 2, then etching it. Before removing the sacrificial layer used for etching, metal material is deposited, and finally the sacrificial layer is removed. By designing the dimensional parameters of the light modulation structure, each unit can modulate light of different wavelengths within the target range differently, and this modulation effect is insensitive to the angle of incidence and polarization. Each unit in the light modulation layer 1 corresponds to one or more pixels on the image sensor 2. 1 is fabricated directly on 2.

[0218] In this embodiment, it should be noted that Figure 14 As shown, assuming that the image sensor 2 is a back-illuminated structure, the light modulation layer 1 can be prepared by directly etching the silicon image sensor layer 21 of the back-illuminated image sensor and then depositing metal.

[0219] In addition, it should be noted that the light modulation structure on the light modulation layer can be obtained by dry etching the light modulation structure pattern of one or more layers of preset materials. Dry etching is to directly remove unnecessary portions of one or more layers of preset materials on the surface of the photosensitive area of ​​the image sensor to obtain a light modulation layer containing a light modulation structure; or imprint transfer is performed on one or more layers of preset materials. Imprint transfer is to prepare the required structure by etching on another substrate and then transfer the structure to the photosensitive area of ​​the image sensor using materials such as PDMS to obtain a light modulation layer containing a light modulation structure; or by external dynamic regulation of one or more layers of preset materials. External dynamic regulation uses active materials and then applies external electrodes to regulate the light modulation characteristics of corresponding areas by changing the voltage to obtain a light modulation layer containing a light modulation structure; or by zone printing of one or more layers of preset materials. Zone printing is to use a printing technology to obtain a light modulation layer containing a light modulation structure; or by zone material growth of one or more layers of preset materials to obtain a light modulation layer containing a light modulation structure; or by quantum dot transfer of one or more layers of preset materials to obtain a light modulation layer containing a light modulation structure.

[0220] In addition, it should be noted that since the preparation method provided in this embodiment is the preparation method of the smelting endpoint monitoring chip in the above embodiment, for detailed contents on some principles and structures, please refer to the introduction of the above embodiment, and this embodiment will not go into details.

[0221] It can be seen that the present invention embeds an artificial neural network on the hardware chip, uses the light modulation layer on the hardware chip as the input layer of the artificial neural network, uses the image sensor as the linear layer of the artificial neural network, and uses the filtering effect of the light modulation layer on the incident light as the connection weight from the input layer to the linear layer. This makes it unnecessary to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer when the smelting endpoint monitoring chip is subsequently used for smelting endpoint monitoring, which can greatly reduce the power consumption and delay during artificial neural network processing. The present invention projects the image information, spectral information, angle of incident light, and phase information of incident light at different positions at the steelmaking furnace mouth into the photocurrent response of the hardware chip, making full use of the light-related information in various dimensions of the steelmaking furnace mouth, thereby enabling accurate smelting endpoint monitoring. It can be seen that the present application has made improvements in terms of processing speed and accuracy, and can quickly and efficiently solve the problem of accurate control of the smelting endpoint.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A smelting endpoint monitoring chip, characterized in that: Used for smelting endpoint monitoring tasks, including: a light modulation layer and an image sensor, the image sensor is connected to a processor; the light modulation layer corresponds to the input layer of the artificial neural network and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and output layer of the artificial neural network; The light modulation layer is disposed on the surface of the photosensitive area of ​​the image sensor, and includes a light modulation structure. The light modulation structure is used to perform different spectrum modulations on incident light entering different positions of the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area. The incident light includes reflected light, transmitted light, and / or radiated light from the steelmaking furnace mouth. The image sensor is used to convert information carried by incident light corresponding to different positions after being modulated by the light modulation layer into electrical signals corresponding to the different positions, and send the electrical signals corresponding to the different positions to the processor; the electrical signals are image signals modulated by the light modulation layer; The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the smelting endpoint monitoring result; The smelting endpoint monitoring task includes identifying the smelting endpoint, and the smelting endpoint monitoring result includes a smelting endpoint identification result.

2. The smelting endpoint monitoring chip according to claim 1, characterized in that: The information carried by the incident light includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.

3. The smelting endpoint monitoring chip according to claim 1, characterized in that: The smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, and the smelting endpoint monitoring result also includes the carbon content and / or molten steel temperature identification result during the smelting process.

4. The smelting endpoint monitoring chip according to any one of claims 1 to 3, characterized in that: The smelting endpoint monitoring chip includes a trained light modulation structure and an image sensor, and the processor is a trained processor; The trained optical modulation structure, image sensor, and processor refer to optical modulation structures, image sensors, and processors that meet the training convergence conditions, obtained by training a smelting endpoint monitoring chip comprising different optical modulation structures, image sensors, and processors with different fully connected parameters and nonlinear activation parameters using input training samples and output training samples corresponding to the smelting endpoint monitoring task; The input training samples include incident light reflected, transmitted and / or radiated from the mouth of a steelmaking furnace that has been smelted to the end point or not; the output training samples include the determination result of whether the steelmaking furnace has been smelted to the end point.

5. The smelting endpoint monitoring chip according to claim 4, characterized in that: When the smelting endpoint monitoring task also includes identifying the carbon content and / or molten steel temperature during the smelting process, accordingly, the input training sample also includes the incident light reflected, transmitted and / or radiated by the steelmaking furnace mouth smelted to different carbon contents and / or molten steel temperatures, and the output training sample also includes the corresponding carbon content and / or molten steel temperature.

6. The smelting endpoint monitoring chip according to claim 4, characterized in that: When training a smelting endpoint monitoring chip comprising different light modulation structures and image sensors and a processor having different full connection parameters and nonlinear activation parameters, the different light modulation structures are designed and implemented by adopting computer optical simulation design.

7. The smelting endpoint monitoring chip according to claim 1, characterized in that: The light modulation structure in the light modulation layer includes a regular structure and / or an irregular structure; and / or the light modulation structure in the light modulation layer includes a discrete structure and / or a continuous structure.

8. The smelting endpoint monitoring chip according to claim 1, characterized in that: The light modulation structure in the light modulation layer includes a unit array composed of multiple micro-nano units, each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the various micro-nano units are the same or different.

9. The smelting endpoint monitoring chip according to claim 8, characterized in that: The micro-nano unit includes a regular structure and / or an irregular structure; and / or the micro-nano unit includes a discrete structure and / or a continuous structure.

10. The smelting endpoint monitoring chip according to claim 8, characterized in that: The micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.

11. The smelting endpoint monitoring chip according to claim 10, characterized in that: Each group of micro-nano structure arrays has the function of broadband filtering or narrowband filtering.

12. The smelting endpoint monitoring chip according to claim 10, characterized in that: Each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.

13. The smelting endpoint monitoring chip according to claim 10, characterized in that: The micro-nano unit comprises a plurality of micro-nano structure arrays, wherein one or more groups of empty structures are included.

14. The smelting endpoint monitoring chip according to claim 10, characterized in that: The micro-nano unit has four-fold rotational symmetry.

15. The smelting endpoint monitoring chip according to claim 1, characterized in that: The light modulation layer is composed of one or more filter layers; The filter layer is made of one or more semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and / or, the filter layer is made of one or more photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nanostructures, and adjustable Fabry-Perot resonant cavities.

16. The smelting endpoint monitoring chip according to claim 15, characterized in that: The semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset proportion, and a direct bandgap compound semiconductor material; and / or, the nanostructure includes one or more of nanodot two-dimensional materials, nanocolumn two-dimensional materials, and nanowire two-dimensional materials.

17. The smelting endpoint monitoring chip according to claim 1, characterized in that: The thickness of the light modulation layer is 0.1λ to 10λ, where λ represents the central wavelength of the incident light.

18. An intelligent smelting control device, characterized in that: It comprises the smelting endpoint monitoring chip as described in any one of claims 1 to 17.

19. A method for preparing a smelting endpoint monitoring chip according to any one of claims 1 to 17, characterized in that: include: Preparing a light modulation layer including a light modulation structure on the surface of the photosensitive area of ​​the image sensor; Generate a processor capable of fully connected signal processing and nonlinear activation processing; connecting the image sensor and the processor; The light modulation layer is used to perform different spectral modulations on incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive area; the incident light-carrying information includes light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light; The image sensor is used to convert the information carried by the incident light corresponding to different position points after being modulated by the light modulation layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processor; The processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the smelting endpoint monitoring result.

20. The method for preparing a smelting endpoint monitoring chip according to claim 19, characterized in that: Also includes: The training process of the smelting endpoint monitoring chip specifically includes: Using input training samples and output training samples corresponding to the smelting endpoint monitoring task, a smelting endpoint monitoring chip including different optical modulation structures and image sensors and a processor with different fully connected parameters are trained to obtain an optical modulation structure, image sensor and processor that meet the training convergence conditions, and the optical modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained optical modulation structure, image sensor and processor.

21. The method for preparing a smelting endpoint monitoring chip according to claim 19, wherein: A light modulation layer including a light modulation structure is prepared on the surface of the image sensor, comprising: growing one or more layers of a preset material on the surface of the image sensor; Etching the one or more layers of the preset material into a light modulation structure pattern to obtain a light modulation layer including the light modulation structure; or performing embossing transfer on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure; or obtaining a light modulation layer including a light modulation structure by externally dynamically modulating the one or more layers of the preset material; or performing zone printing on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure; or performing partitioned growth on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure; Alternatively, quantum dot transfer is performed on the one or more layers of the preset material to obtain a light modulation layer including a light modulation structure.

Images