Method and device for determining working condition in reduction furnace and detection model training method and device

Abstract

The invention discloses a method and device for determining working conditions in a reduction furnace and a method and device for training a detection model, and relates to the technical field of polycrystalline silicon production. The method for determining the working condition in the reduction furnace comprises the following steps: acquiring a first reduction furnace in-furnace image of a target reduction furnace at a first moment; detecting the in-furnace image of the first reduction furnace by adopting a target detection model to obtain a detection result; and according to the detection result, determining the target in-furnace working condition of the target reduction furnace at the second moment. According to the method provided by the embodiment of the invention, the target in-furnace working condition of the target reduction furnace can be pre-judged in advance, and the production quality of polycrystalline silicon can be effectively improved finally by adjusting the in-furnace working condition of the reduction furnace in time when no abnormity occurs in the reduction furnace.

Description

Technical Field

[0001] The present application belongs to the technical field of polysilicon production, and specifically relates to a method and device for determining the working conditions in a reduction furnace, and a method and device for training a detection model. Background Art

[0002] During the 110-hour long polysilicon production process, how to timely determine the working conditions in the furnace and make adjustments in advance to prevent abnormal phenomena such as atomization is currently the most concerned issue in this field. However, as a gray box system, some data related to the working conditions in the furnace are difficult or even impossible to obtain directly through sensors.

[0003] The existing technology usually manually judges the working conditions in the reduction furnace, but manual judgment has a lag, that is, when the judgment is abnormal, atomization occurs in the furnace, and low-quality polysilicon has been deposited on the silicon rods. Although normal production can be restored through timely adjustments, some low-quality polysilicon has already been deposited, which will seriously affect the subsequent silicon participation reaction, and ultimately lead to a significant decrease in the quality of this batch of polysilicon. Therefore, how to timely determine the working conditions in the reduction furnace to improve the quality of polysilicon is a research hotspot for those skilled in the art. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a method and device for determining the working condition inside a reduction furnace, and a method and device for training a detection model in view of the above-mentioned deficiencies in the prior art. By using the method for determining the working condition inside a reduction furnace, the target working condition inside a target reduction furnace can be predicted in advance, and the working condition inside the reduction furnace can be adjusted in time when no abnormality occurs in the reduction furnace, thereby effectively improving the production quality of polysilicon.

[0005] In a first aspect, an embodiment of the present application provides a method for determining an operating condition in a reduction furnace, comprising: obtaining an image of a first reduction furnace in a target reduction furnace at a first moment; detecting the image of the first reduction furnace in a target detection model to obtain a detection result, wherein the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model, and a segmentation model; determining a target operating condition in the target reduction furnace at a second moment based on the detection result, wherein the target operating condition in the target furnace includes at least one of a target atomization value, a target temperature value, and a target silicon rod diameter, and the target operating condition in the target furnace corresponds one-to-one to the target detection model, and the first moment is earlier than the second moment.

[0006] Based on the same inventive concept, in the second aspect, the embodiment of the present application also provides a detection model training method, using a target detection model as any one of the first aspect; the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model and a segmentation model, and the method includes: in the case where the target detection model includes a reduction furnace base detection model, obtaining a first training set, the first training set includes multiple first training samples, the first training sample includes historical reduction furnace in-furnace images and their corresponding historical detection frames; using multiple first training samples to train the first preset model to obtain a reduction furnace base detection model; and / or, in the case where the target detection model includes a temperature detection model, obtaining a second training set, the second training set includes multiple second training samples, the second training sample includes historical reduction furnace in-furnace images, second furnace in-furnace information and multiple different historical sequences, the second furnace in-furnace information The information includes at least one of a second current value trend, a second voltage value trend and a second raw material flow rate trend of the reduction furnace before a third moment, and the historical sequence includes historical time, historical silicon rod temperature values ​​and historical ambient temperature values; a second preset model is trained using multiple second training samples to obtain a temperature detection model; and / or, when the target detection model includes a segmentation model, a third training set and a fourth training set are obtained, the third training set includes a target image and its corresponding target category label, the fourth training set includes multiple fourth training samples, the third training sample includes a historical reduction furnace image, and a fifth pixel point position covered in the historical reduction furnace image and a sixth pixel point position on its edge; the third preset model is trained using multiple third training samples to obtain an initial segmentation model; the initial segmentation model is trained using multiple fourth training samples to obtain a segmentation model.

[0007] Based on the same inventive concept, in the third aspect, an embodiment of the present application also provides a device for determining the operating condition inside a reduction furnace, comprising: a first acquisition module, used to acquire an image inside a first reduction furnace of a target reduction furnace at a first moment; a first detection module, connected to the first acquisition module, used to detect the image inside the first reduction furnace using a target detection model to obtain a detection result, wherein the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model and a segmentation model; a first determination module, connected to the first detection module, used to determine, based on the detection result, a target operating condition inside the target reduction furnace at a second moment, wherein the target operating condition inside the target furnace includes at least one of a target atomization value, a target temperature value and a target silicon rod diameter, and the target operating condition inside the target furnace corresponds one-to-one to the target detection model, and the first moment is earlier than the second moment.

[0008] Based on the same inventive concept, in a fourth aspect, an embodiment of the present application further provides a detection model training device, which adopts a target detection model as in any one of the first aspect; the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model and a segmentation model, and the device includes: a second acquisition module, which is used to acquire a first training set when the target detection model includes a reduction furnace base detection model, the first training set includes multiple first training samples, and the first training samples include historical reduction furnace in-furnace images and their corresponding historical detection frames; a first training module, connected to the second acquisition module, is used to train the first preset model with multiple first training samples to obtain a reduction furnace base detection model; and / or a third acquisition module, which is used to acquire a second training set when the target detection model includes a temperature detection model, the second training set includes multiple second training samples, the second training samples include historical reduction furnace in-furnace images, second furnace in-furnace information and multiple different historical sequences, and the second furnace in-furnace information includes the second reduction furnace before the third moment. At least one of the current value trend, the second voltage value trend and the second raw material flow rate trend, the historical sequence includes historical time, historical silicon rod temperature value and historical ambient temperature value; a second training module, connected to the third acquisition module, used to train the second preset model with multiple second training samples to obtain a temperature detection model; and / or, a fourth acquisition module, used to acquire a third training set and a fourth training set when the target detection model includes a segmentation model, the third training set includes a target image and its corresponding target category label, the fourth training set includes multiple fourth training samples, the third training sample includes a historical reduction furnace image, and a fifth pixel position covered in the historical reduction furnace image and a sixth pixel position on its edge; a third training module, used to be connected to the fourth acquisition module, used to train the third preset model with multiple third training samples to obtain an initial segmentation model; a fourth training module, used to be connected to the third training module, used to train the initial segmentation model with multiple fourth training samples to obtain a segmentation model.

[0009] According to the method and device for determining the operating condition in a reduction furnace and the method and device for training a detection model provided in the embodiments of the present application, a target detection model is used to detect the image in the first reduction furnace to obtain a detection result, and then based on the detection result, the target operating condition in the target reduction furnace at the second moment is determined. The target operating condition in the target reduction furnace can be predicted in advance, and the operating condition in the reduction furnace can be adjusted in time when no abnormality occurs in the reduction furnace, thereby effectively improving the quality of polysilicon production. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram showing a flow chart of a method for determining the working condition in a reduction furnace provided in an embodiment of the present application;

[0011] Figure 2A flow chart showing a deep learning reduction furnace base detection algorithm of a reduction furnace base detection model provided in an embodiment of the present application is shown;

[0012] Figure 3 A network structure diagram of the first CNN feature extraction layer provided in an embodiment of the present application is shown;

[0013] Figure 4 Another schematic flow chart of a method for determining the working condition in a reduction furnace provided in an embodiment of the present application is shown;

[0014] Figure 5 A network structure diagram of a segmentation model provided in an embodiment of the present application is shown;

[0015] Figure 6 An algorithm structure diagram of a segmentation model provided in an embodiment of the present application is shown;

[0016] Figure 7 A structural diagram of a silicon rod segmentation algorithm provided in an embodiment of the present application is shown;

[0017] Figure 8 An image of a silicon rod pixel diameter measured according to a calibration object after edge calculation segmentation of an image provided by an embodiment of the present application is shown;

[0018] Fig. 9 Another silicon rod pixel diameter picture measured according to a calibration object after edge calculation segmentation of an image provided by an embodiment of the present application is shown;

[0019] Fig.10 The temperature image and RGB image provided by the embodiment of the present application are shown, and the temperature calibration is performed by pixel alignment;

[0020] Fig.11 A structural schematic diagram of a device for determining the operating conditions in a reduction furnace provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is further described in detail below with reference to the accompanying drawings and embodiments.

[0022] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0023] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0024] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0025] Before introducing the method for determining the operating conditions in a reduction furnace provided in an embodiment of the present application, related technologies that may be involved in the embodiment of the present application are first described.

[0026] As a gray box system, many controlled variables in the reduction furnace are unmeasurable, or the measurement accuracy is far from meeting the requirements of fine control. Taking the three core controlled variables (surface temperature, growth rate and atomization degree) as an example, there is currently no sensor solution that can directly obtain the surface temperature and growth rate during the production process. Although a dust detector for the exhaust gas outlet pipeline has been introduced to the atomization degree in the furnace, the atomization level can only output three levels, which cannot meet the fine control requirements of polysilicon reduction production. Computer vision technology can solve the problem of unmeasurable controlled variables in the reduction furnace both in terms of phenomenon capture and analysis.

[0027] Although the introduction of the dust detector for the tail gas outlet pipeline of the polysilicon reduction furnace has divided the degree of atomization, the granularity of the division is too coarse to meet the fine control requirements of polysilicon reduction production. The degree of atomization in the polysilicon reduction production process should be a continuous value to achieve fine control. The reason is that this method has a large judgment error and cannot further divide the threshold.

[0028] The reason why this method has a large judgment error is that: 1) Dust detection and atomization degree are indirectly related, and many directly related data dimensions in the middle cannot be collected through sensors, resulting in the inability to accurately convert the relationship between the two through intermediate quantities, and the complex nonlinear relationship between the two is simplified to a linear relationship; 2) The dust detector cannot be directly installed in the furnace, but can only be installed in the exhaust pipe far away from the reduction furnace, resulting in some special conditions in the furnace affecting the actual dust detection, such as: the exhaust port is blocked, resulting in dust accumulation and not entering the exhaust pipe. 3) The dust detector sensor signal is weak, at the picoampere level, and is extremely susceptible to electromagnetic interference.

[0029] For a long time, key parameters that reflect the growth conditions in the furnace, such as silicon rod surface temperature, silicon rod diameter, and furnace clarity, have not been measured, which is the main factor that troubles the high-quality control of the reduction furnace. Only by solving the problem of measurable controlled variables can the energy efficiency of the reduction furnace be further improved.

[0030] Example 1

[0031] The method for determining the working condition in a reduction furnace provided in the embodiment of the present application can be executed by a device for determining the working condition in a reduction furnace and an electronic device, etc. The following takes the method for determining the working condition in a reduction furnace executed by an electronic device as an example for description.

[0032] like Figure 1 As shown, the method for determining the operating conditions in a reduction furnace provided in an embodiment of the present application may include steps S110 to S130.

[0033] S110, obtaining an image inside a first reduction furnace of a target reduction furnace at a first moment.

[0034] S120. Use a target detection model to detect the image inside the first reduction furnace to obtain a detection result. The target detection model includes at least one of a reduction furnace base detection model, a temperature detection model, and a segmentation model.

[0035] S130. Determine a target furnace operating condition of the target reduction furnace at a second moment according to the detection result. The target furnace operating condition includes at least one of a target atomization value, a target temperature value, and a target silicon rod diameter. The target furnace operating condition corresponds to the target detection model one-to-one, and the first moment is earlier than the second moment.

[0036] According to the method for determining the operating condition in a reduction furnace provided in an embodiment of the present application, a target detection model is used to detect the image in the first reduction furnace to obtain a detection result, and then the target operating condition in the target reduction furnace at the second moment is determined based on the detection result. The target operating condition in the target reduction furnace can be predicted in advance and the operating condition in the reduction furnace can be determined through timely adjustment when no abnormality occurs in the reduction furnace, which can ultimately effectively improve the quality of polysilicon production.

[0037] In step S110, the target reduction furnace may be any reduction furnace for which the reduction furnace operating condition determination is required. The first moment is any moment that has occurred. The first reduction furnace internal image may be the internal image of the target reduction furnace at the first moment.

[0038] Exemplarily, the electronic device includes an industrial camera, and the image inside the first reduction furnace of the target reduction furnace at the first moment can be obtained through the industrial camera. The industrial camera can be nested on the observation window of the target reduction furnace and connected to the industrial intelligent computer (i.e., the electronic device) to realize the photo taking function. After taking the photo, the image inside the first reduction furnace can be transferred to the target detection model through the memory inside the industrial intelligent computer.

[0039] Exemplarily, when capturing images of the interior of the first reduction furnace, the center of the industrial camera lens coincides with the center of the observation window, and the industrial camera lens is close to the observation window. If the observation window is tilted downward so that the furnace base of the target reduction furnace can be directly observed, the industrial camera angle needs to be adjusted to be perpendicular to the glass of the observation window and parallel to the channel in the observation window. If the observation window cannot observe the furnace base of the target reduction furnace, the industrial camera needs to be installed with a downward tilt angle to ensure that the silicon rods and the furnace base appear in the center of the field of view at the same time.

[0040] For example, the industrial camera can be a 25-megapixel CMOS 10 Gigabit Ethernet industrial area array camera with a GMAX3265 sensor, which has excellent image quality. Through the 10 Gigabit Ethernet interface, uncompressed data can be transmitted quickly and in real time, with a maximum frame rate of 17.2fps at full resolution.

[0041] Exemplarily, the electronic device may pre-store the image of the interior of the first reduction furnace for subsequent direct retrieval.

[0042] In step S120, after acquiring the first reduction furnace interior image of the target reduction furnace at the first moment, the electronic device may also use the target detection model to detect the first reduction furnace interior image to obtain a detection result.

[0043] After acquiring the first reduction furnace interior image, the first reduction furnace interior image may be input into a target detection model, and the target detection model is used to detect the first reduction furnace interior image to obtain a detection result.

[0044] The target detection model can be a model that can detect the image inside the first reduction furnace and obtain the detection result. The reduction furnace base detection model can be a model that can detect the image inside the first reduction furnace and obtain n target detection frames and n target confidences in the image inside the first reduction furnace; the temperature detection model can be a model that can detect the image inside the first reduction furnace and obtain multiple different first detection sequences; the segmentation model can be a model that can detect the image inside the first reduction furnace and obtain the first pixel position covered by the silicon rod in the image inside the first reduction furnace and the second pixel position of the edge thereof.

[0045] As an example, the target detection model includes a reduction furnace base detection model; as another example, the target detection model includes a temperature detection model and a segmentation model; as yet another example, the target detection model includes a reduction furnace base detection model, a temperature detection model, and a segmentation model.

[0046] Accordingly, as an example, the detection result includes n target detection frames and n target confidences in the first reduction furnace image; as another example, the detection result includes multiple different first detection sequences, and the first pixel position covered by the silicon rod in the first reduction furnace image and the second pixel position at its edge; as another example, the detection result includes n target detection frames and n target confidences in the first reduction furnace image, multiple different first detection sequences, and the first pixel position covered by the silicon rod in the first reduction furnace image and the second pixel position at its edge. Among them, the target detection frame can be determined by the upper left corner coordinates and the lower right corner coordinates of the target detection frame.

[0047] In some embodiments, the target detection model includes a reduction furnace base detection model, and the detection result includes n target detection frames and n target confidences in the first reduction furnace image, the target detection frames and the target confidences correspond one to one, and n is a positive integer; the target furnace working condition includes a target atomization value;

[0048] The target detection model is used to detect the image inside the first reduction furnace, and the detection results are obtained, including:

[0049] The reduction furnace base detection model is used to detect the first reduction furnace interior image, and n target detection frames and n target confidences in the first reduction furnace interior image are obtained.

[0050] The value of n can be set according to actual conditions and is not limited here. For example, taking the target reduction furnace as a 40-rod reduction furnace as an example, n can be 8.

[0051] In some embodiments, the reduction furnace base detection model includes a first CNN feature extraction layer, a Transformer encoder, a Transformer decoder, and a feedforward fully connected layer connected in sequence; the size of the first reduction furnace image is W*H, wherein W represents the width of the first reduction furnace image, and H represents the height of the first reduction furnace image;

[0052] The reduction furnace base detection model is used to detect the first reduction furnace image to obtain n target detection frames and n target confidences in the first reduction furnace image, which may include:

[0053] The first CNN feature extraction layer is used to extract features from the image in the first reduction furnace, and 128a first-category feature maps are obtained. The size of the first-category feature map is W / 32*H / 32, and a is a positive integer.

[0054] The Transformer encoder is used to process 128a first-class feature maps to obtain the second-class feature maps, and the size of the second-class feature maps is WH / 1024*256;

[0055] The Transformer decoder is used to solve the second type of feature map to obtain p first feature vectors, where p is a positive integer;

[0056] The p first eigenvectors are processed using a feedforward fully connected layer to obtain n target detection frames and n target confidences in the first reduction furnace image.

[0057] Exemplarily, the value of a can be set according to actual conditions and is not limited here; for example, the value of a can be multiple powers of 2 such as 4, 8, and 16.

[0058] For example, the first CNN feature extraction layer can convert a color image with 3 channels, a width of W, and a height of H (i.e., the image in the first reduction furnace) into a first-class feature map with 2048 channels, a width of W / 32, and a height of H / 32. In other words, the image feature extraction layer (i.e., the first CNN feature extraction layer) is established based on the CNN structure, and the original three-channel image is converted into a feature map with thousands of channels.

[0059] For example, Figure 4As shown, after the industrial camera collects the image inside the first reduction furnace, the image inside the first reduction furnace is stored in the image memory, and the image inside the first reduction furnace is input into the base recognition algorithm (i.e., the reduction furnace base detection model) to obtain the recognition result (i.e., the detection result), and then the degree of atomization is determined according to the recognition result to obtain the atomization degree value (i.e., the target atomization value), and the atomization degree value is input into the intelligent control system to generate and output the control adjustment strategy to the DCS (distributed control system, distributed control) system. Among them, the industrial intelligent computer (i.e., electronic device) may include image memory, base recognition algorithm, atomization degree judgment and intelligent system. The base recognition algorithm may include a CNN feature layer (i.e., the first CNN feature extraction layer), a Transformer Encoder (i.e., a transformer encoder), a Transformer Decoder (i.e., a transformer decoder) and a feedforward fully connected layer.

[0060] In this embodiment, the reduction furnace base detection model may include a deep learning reduction furnace base detection algorithm based on Transformer technology. The efficient parallel computing capability of the reduction furnace base detection model can make the reduction furnace base detection model training and reasoning faster, and can achieve powerful long-distance dependency learning capabilities through the self-attention mechanism, which is suitable for a variety of tasks.

[0061] In other words, the reduction furnace base detection problem can be solved by adopting the CNN+Transformer neural network structure and using an end-to-end approach.

[0062] CNN, or convolutional neural network, is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. The CNN part extracts the image feature map. The Transformer uses the self-attention mechanism to explicitly model the relationship between all the feature elements in the feature map. The feature map is the feature map of each channel in the original image (i.e., the image inside the first reduction furnace) or the image after convolution kernel filtering. In the original image or after each convolution, the data exists in a three-dimensional form. By dividing the data according to the channel, the data can be regarded as the superposition of many two-dimensional images, and each two-dimensional image is called a feature map. The RGB original image is input into the input layer of CNN. At this time, the original image can be regarded as the superposition of three feature maps (red, green, and blue). There will be several convolution kernel filters between the layers of the CNN structure. Each layer of convolution kernel filters will convolve with each feature map of its input and generate several new feature maps as the input of the next layer. Since the parameter weights of the convolution kernel are trained according to the actual position of the target in the image, after CNN reaches the target accuracy, all the extracted feature maps are visualized as grayscale images with pixels around all targets highlighted and the rest gradually darkened.

[0063] The role of the self-attention mechanism is to allow the model to assign its own weights to the variables. This mechanism repeatedly solves the correlation between data features to find out what kind of correlation exists between data features when the target effect is closest, and then fixes the correlation to solve the desired target on the new data after the new data is input into the model.

[0064] The attention calculation formula is as follows:

[0065]

[0066] Where Q represents the query matrix of the input data, K represents the index matrix of the input data, and V represents the content matrix of the input data. Represents the vector length of the input data. The Softmax function calculation formula is as follows:

[0067]

[0068] Among them, S i represents the softmax value of the i-th element in the sequence, e i Represents the index of the i-th element in the sequence, e jRepresents the index of the jth element in the sequence. Therefore, the softmax value is the ratio of the index of this element to the sum of the indices of all elements.

[0069] In this embodiment, the recognition accuracy of the neural network structure using CNN+Transformer is better than that of CNN structure neural networks such as FastRCNN and Yolo.

[0070] Fast RCNN is a neural network model based on the fast region convolutional network method to deal with target detection problems. It implements target detection through the following steps: 1) Extract features from the image through CNN to obtain the feature map of the image; 2) Obtain the region of interest of the image through the selective search algorithm; 3) Pool the obtained region of interest, that is, through the coordinate projection method, obtain the feature area corresponding to the region of interest in the input image on the feature map, and perform maximum pooling on the region, so that the features of the region of interest are obtained and the feature size is unified; 4) Connect the feature vector of the region of interest to the fully connected layer, and define a multi-task loss function to obtain the coordinate bounding box of the current region of interest; 5) Perform non-maximum suppression on all the obtained bounding boxes to obtain the final detection result.

[0071] Yolo is an end-to-end target detection neural network. The core idea is to transform target detection into a regression problem. The entire image is used as the input of the network. Through the neural network, the location of the bounding box and its category are obtained. It implements target detection through the following steps: 1) Divide the input image into a grid of fixed size; 2) For each grid, YOLO predicts a fixed number of bounding boxes. Each bounding box is described by 5 main attributes: the location of the bounding box (center coordinates and width and height), the confidence of the target contained in the bounding box, and the category of the predicted target; 3) Perform a single forward pass through a CNN to predict the location and category of all bounding boxes at the same time; 4) Use a multi-task loss function to train the network. The loss function includes position loss, confidence loss and category loss. The position loss measures the position difference between the predicted bounding box and the true bounding box. The confidence loss measures whether the bounding box correctly predicts the target and penalizes the confidence of the background box. The category loss measures the prediction accuracy of the target category; 5) Use the non-maximum suppression algorithm to select the best bounding box based on confidence and overlap.

[0072] For example, Figure 2As shown, the deep learning reduction furnace base detection algorithm of the reduction furnace base detection model provided in the embodiment of the present application includes an input (i.e., the first reduction furnace furnace image) which is sequentially extracted through CNN feature extraction (i.e., the first CNN feature extraction layer) and a Transformer codec to obtain a target position output (i.e., n target detection boxes and n target confidences in the first reduction furnace furnace image).

[0073] In some examples, such as Figure 3 As shown, the first CNN feature extraction layer includes a first convolutional layer, a first normalization layer, a first customized module, a first downsampling layer, a second customized module, a second downsampling layer, a third customized module, a third downsampling layer, a fourth customized layer, a global average pooling layer and a second normalization layer connected in sequence;

[0074] The first CNN feature extraction layer is used to extract features from the image inside the first reduction furnace to obtain m first-class feature maps, including:

[0075] The first convolutional layer is used to extract features of the first reduction furnace image to obtain a first feature map;

[0076] A first normalization layer is used to normalize a first feature map to obtain a second feature map;

[0077] The first customized module is used to extract features from a second feature maps to obtain 4a third feature maps;

[0078] The first downsampling layer is used to compress the size of the 4a third feature maps to obtain 4a fourth feature maps;

[0079] The second customized module is used to extract features from 4a fourth feature maps to obtain 16a fifth feature maps;

[0080] The second downsampling layer is used to compress the size of the 16a fifth feature maps to obtain 16a sixth feature maps;

[0081] The third customized module is used to extract features from 16a sixth feature maps to obtain 64a seventh feature maps;

[0082] The third downsampling layer is used to compress the size of 64a seventh feature maps to obtain 64a eighth feature maps;

[0083] The fourth customized module is used to extract features from 64a eighth feature maps to obtain 128a ninth feature maps;

[0084] A global average pooling layer is used to compress the size of 128a ninth feature maps to obtain 128a tenth feature maps;

[0085] The second normalization layer is used to normalize the 128a tenth feature maps to obtain 128a first-category feature maps.

[0086] Exemplarily, if the value of a is 16, the number of the first feature map and the second feature map is 16, the number of the third feature map and the fourth feature map is 64, the number of the fifth feature map and the sixth feature map is 256, the number of the seventh feature map and the eighth feature map is 1024, and the number of the ninth feature map and the tenth feature map is 2048.

[0087] In some examples, the target customized module includes at least one of a first customized module, a second customized module, a third customized module, and a fourth customized module;

[0088] The target customized module has a third normalization layer, a second convolutional layer, a fourth normalization layer, a third convolutional layer, a GELU activation function, a fourth convolutional layer and a fifth normalization layer connected in sequence.

[0089] As an example, the target customized module includes a first customized module; as another example, the target customized module includes a second customized module and a third customized module; as yet another example, the target customized module includes a second customized module, a third customized module and a fourth customized module; as yet another example, the target customized module includes a first customized module, a second customized module, a third customized module and a fourth customized module.

[0090] like Figure 3As shown in the figure, the first CNN feature extraction layer is inspired by the visual Transformer algorithm, and the traditional CNN is transformed in five parts: 1) The balanced distribution structure of the traditional 4-section convolutional neural network is changed to a concentrated distribution structure, that is, the convolution stacking of the third section structure (i.e., the first customized module) is increased, and the convolution stacking of the 1st, 2nd, and 4th layers (i.e., the convolution layer and normalization layer in the target customized module) is reduced; 2) The convolution kernel size is increased and the feature overlap is reduced like the transformer (i.e., when the convolution kernel performs sliding convolution on the image, the traditional convolution will have overlap (overlap), and this part is to minimize the length of the overlap); 3) The traditional convolution structure is modified, and the convolution is replaced by a group convolution with the same number of groups and channels, so that it is close to the weighted summation of the self-attention in the transformer; 4) The Bottleneck with small ends and large middle of the network structure is used to imitate the MLP module in the Transformer block; 5) The ReLU activation function commonly used in CNN is replaced by the GELU activation function commonly used in Transformer, and the Batch activation function commonly used in CNN is replaced by the GELU activation function commonly used in Transformer. Normalization is replaced with LayerNormalization commonly used in Transformer and the number of Normalization is reduced to keep consistent with Transformer. The features extracted by the above-mentioned modified CNN can better adapt to the Transformer codec, thereby effectively improving the recognition accuracy.

[0091] In some examples, the Transformer encoder includes a multi-head self-attention mechanism and a feed-forward neural network.

[0092] The Transformer encoder is used to process 128a first-class feature maps to obtain the second-class feature maps, including:

[0093] A multi-head self-attention mechanism is used to compress 128a first-category feature maps to obtain 16a first-category compressed feature maps;

[0094] A feedforward neural network is used to planarize and reorganize the 16a first compressed feature maps to obtain a first reorganized feature map;

[0095] A feedforward neural network is used to mark the positions of b feature elements in the first reorganized feature map to obtain b position codes;

[0096] The b position codes and the first reorganized feature map are combined into a second type feature map.

[0097] That is to say, taking the value of a as 16 as an example, the Transformer encoder first compresses the number of channels of the first type of feature map from 2048 to 256 through a 1*1 convolutional layer to obtain a first compressed feature map, and then planarizes the first compressed feature map into a first reorganized feature map of size (WH / 1024,256); then, each feature element in the first reorganized feature map is marked with a position, that is, each feature element has a position code, from (0, 0) to (WH / 1024-1,255), and b position codes are obtained, and then the b position codes and the first reorganized feature map are combined into the second type of feature map.

[0098] In this example, the Transformer encoder part is a normal Transformer structure, in which the Transformer encoder layer is composed of multiple encoding modules in series, and the unified structure of each encoding module is: multi-head self-attention mechanism + residual ADD (i.e. residual addition) + layer normalization Layer Normalization + feedforward network FFN (i.e. feedforward neural network) + residual ADD + layer normalization Layer Normalization.

[0099] In some examples, the Transformer decoder includes a multi-head self-attention mechanism and a multi-head cross-attention mechanism.

[0100] The Transformer decoder is used to solve the second type of feature map to obtain p first feature vectors, including:

[0101] A multi-head self-attention mechanism is used to solve the b Q vectors and b K vectors corresponding to the b feature elements in the second type of feature map;

[0102] For each Q vector of the b characteristic elements, solve the b first correlation values ​​corresponding to the b K vectors one by one to obtain b*b first correlation values;

[0103] The b*b first correlation values ​​are encoded according to b positions to form p first eigenvectors.

[0104] That is to say, the input of the Transformer decoder is a new feature map that has position codes for each feature element, i.e., the second-type feature map. The Transformer decoder uses the principle of self-attention neural network to split each feature element in the new feature map (i.e., the second-type feature map) into a query vector and a key vector, and solves the pairwise correlation between each element query vector and itself and other element key vectors, and encodes all the correlation values ​​into WH / 4 first feature vectors of length WH / 4 according to the position code.

[0105] In this example, the decoder layer is composed of multiple decoding modules connected in series, and the unified structure of each decoding module is: Masked multi-head self-attention mechanism + residual ADD + layer normalization Layer Normalization + multi-head cross attention mechanism + residual ADD + layer normalization Layer Normalization + feedforward network FFN + residual ADD + layer normalization LayerNormalization.

[0106] In some examples, the feed-forward fully connected layer includes a first linear layer and a first softmax layer,

[0107] The p first feature vectors are processed by a feedforward fully connected layer to obtain n target detection frames and n target confidences in the first reduction furnace image, including:

[0108] Using a first linear layer to compress the p first eigenvectors to obtain p first single point values, and combining the p first single point values ​​into a second eigenvector;

[0109] The first softmax layer is used to map the second feature vector into n target detection boxes and n target confidences in the first reduction furnace image.

[0110] That is, the feed-forward fully connected layer compresses each of the WH / 4 first feature vectors into a single point value (i.e., the first single point value), combines the single point values ​​into a vector of length WH / 4 (i.e., the second feature vector), and then maps the vector into two coordinate values ​​(i.e., the upper left corner coordinates and the lower right corner coordinates of the target detection box) and a confidence level (i.e., the target confidence level). In other words, the first linear layer converts the feature map (i.e., the first feature vector) into a one-dimensional feature (i.e., the second feature vector), and the first softmax layer probabilistically normalizes the one-dimensional feature to obtain the target position coordinates (i.e., the target detection box).

[0111] In other embodiments, the target detection model includes a temperature detection model, the detection result includes a plurality of different first detection sequences, and the target furnace operating condition includes a target temperature value;

[0112] The target detection model is used to detect the image inside the first reduction furnace, and the detection results are obtained, including:

[0113] A temperature detection model is used to detect the first reduction furnace image and the first furnace information to obtain multiple different first detection sequences. The first furnace information includes at least one of the first current value trend, the first voltage value trend and the first raw material flow rate trend of the target reduction furnace before the first moment.

[0114] As an example, the first in-furnace information includes a first current value trend of the target reduction furnace before the first moment; as another example, the first in-furnace information includes a first current value trend and a first voltage value trend of the target reduction furnace before the first moment; as yet another example, the first in-furnace information includes a first current value trend, a first voltage value trend and a first raw material flow rate trend of the target reduction furnace before the first moment.

[0115] In some examples, the temperature detection model includes a sparse autoencoder structure, the sparse autoencoder structure includes a first input layer, a first hidden layer, and a first output layer connected in sequence; the first input layer includes a convolution substructure, the first hidden layer includes a sparse substructure, and the first output layer includes a decoding substructure;

[0116] The temperature detection model is used to detect the first reduction furnace image and the first furnace information to obtain a plurality of different first detection sequences, including:

[0117] A convolution substructure is used to transform the first reduction furnace image and the first furnace information into a plurality of third-category feature maps;

[0118] A sparse substructure is used to constrain the sparsity of multiple third-category feature maps to obtain multiple fourth-category feature maps;

[0119] A decoding substructure is used to decode multiple feature maps of the fourth category to obtain multiple different first detection sequences.

[0120] That is to say, the deep learning image regression algorithm (i.e., the temperature detection model) is a sparse autoencoder structure, in which the encoder transforms the RGB image (i.e., the image inside the first reduction furnace) into several feature maps (i.e., the third type of feature map) with the help of a convolution structure (i.e., a convolution substructure), and reduces the amount of calculation of the model (i.e., the temperature detection model) with the help of a sparse structure (i.e., a sparse substructure), and increases the weight of the temperature stable area; decodes all feature maps (i.e., the fourth type of feature map) to obtain an output image (i.e., multiple different first detection sequences). That is, an association relationship layer is established with the Transformer structure as the main body, and the relationship between the feature elements in the feature map and the relationship between the feature map and the first current value trend, the first voltage value trend, and the first raw material flow rate are learned.

[0121] In some further embodiments, the target detection model includes a segmentation model, the detection result includes a first pixel position covered by the silicon rod in the first reduction furnace image and a second pixel position at its edge, and the target furnace operating condition includes a target silicon rod diameter;

[0122] The target detection model is used to detect the image inside the first reduction furnace, and the detection results are obtained, including:

[0123] The segmentation model is used to detect the image inside the first reduction furnace, and the first pixel point position covered by the silicon rod and the second pixel point position of the edge of the image inside the first reduction furnace are obtained.

[0124] In some examples, the segmentation model includes an image encoder, a hint encoder, and a mask decoder;

[0125] The segmentation model is used to detect the image in the first reduction furnace, and the first pixel position covered by the silicon rod in the image in the first reduction furnace and the second pixel position of the edge thereof are obtained, including:

[0126] Using an image encoder to process the image in the first reduction furnace to obtain a first image embedding;

[0127] Encoding the first type of information using a hint encoder to obtain a first embedding vector, wherein the first type of information includes at least one of a background point, a mask, and a bounding box;

[0128] A mask decoder is used to predict and segment the mask of the first image embedding and the first embedding vector, so as to obtain the first pixel point position covered by the silicon rod in the first reduction furnace image and the second pixel point position of the edge thereof.

[0129] As an example, the first type of information includes background points; as another example, the first type of information includes masks and bounding boxes; as yet another example, the first type of information includes background points, masks, and bounding boxes.

[0130] In some examples, the image encoder includes a mask encoder connected to a visual transformer, and the visual transformer includes a second CNN feature extraction layer of the CSP-Darknet structure;

[0131] The image encoder is used to process the image in the first reduction furnace to obtain a first image embedding, including:

[0132] Mask coding is used to cover part of the information of the first reduction furnace interior image to obtain the second reduction furnace interior image;

[0133] The second CNN feature extraction layer is used to extract features from the image inside the second reduction furnace to obtain the fifth type of feature map;

[0134] The visual Transformer is used to reconstruct the features in the fifth feature map to obtain the first image embedding.

[0135] In this example, the image encoder includes a visual Transformer, which includes a second CNN feature extraction layer of a CSP-Darknet structure, which can enhance the learning ability of the second CNN feature extraction layer and reduce computational bottlenecks and memory consumption.

[0136] CSP-Darknet is Darknet based on CSP structure. The original intention of CSP structure is to reduce the amount of calculation and enhance the performance of gradient. The main idea is: before entering the current layer, the input data is divided into two parts, one of which is calculated by the current layer, and the other is directly spliced ​​with the current layer result by cropping. This not only ensures that the input data will not be unable to splice the complete target with the original data due to excessive deviation after passing through the network layer, but also reduces the amount of calculation of the network layer. Darknet is a lightweight target detection model with the structure of the first layer: convolution; the second layer: downsampling convolution + 1 residual block; the third layer: downsampling convolution + 2 residual blocks; the fourth layer: downsampling convolution + 8 residual blocks; the fifth layer: downsampling convolution + 8 residual blocks; the sixth layer: downsampling convolution + 4 residual blocks. Its core feature is the use of residual blocks. The design of residual blocks is to help solve the problems of gradient disappearance and gradient explosion in deep neural network training, and improve the performance and convergence speed of the network. The core idea of ​​the residual block is to learn the residual function by introducing skip connections, so that the network can learn the residual mapping more easily.

[0137] The core structure of CSP consists of convolution function, normalization function and Leaky ReLU function.

[0138] The convolution function formula is as follows:

[0139] Assume that the input feature map size is W in ×H in ×D in , the output feature map size is W out ×H out ×D out , where W is the feature map width, H is the feature map height, and D is the feature map depth (i.e. the number of channels), then:

[0140] W out =(W in +2p-w) / s+1

[0141] H out =(H in +2p-h) / s+1

[0142] D out =k

[0143] Where k is the number of convolution kernels, w and h are the width and height of the convolution kernel respectively, s is the convolution step size, and p is the padding width.

[0144] The normalization function formula is as follows:

[0145]

[0146] where a i is the original activation value of the ith neuron, is the normalized value after the normalization operation. The whole normalization process can be decomposed into two steps. The first step is to normalize the activation value to a normal distribution range with a mean of 0 and a variance of 1; i is the first adjustment factor, and its value range is a real number; β i is the second adjustment factor, and its value range is a real number. Where μ is the mean of the activation values ​​of the m neurons contained in the neuron set, and the formula is as follows:

[0147]

[0148] σ is the standard deviation of the activation value calculated based on the mean and the activation values ​​of each neuron in the set. The formula is as follows:

[0149]

[0150] Here, ε is a small constant data added to increase the stability of training, and its value range is a real number.

[0151] The second step is to let each neuron learn the corresponding two adjustment factors γ during the training process. i and β i , fine-tune the values ​​that are normalized to a mean of 0 and a variance of 1. Because the nonlinear expression ability of the neural network may be reduced after the first step, this method will be used to compensate for the expression ability of the neural network after the first step.

[0152] The formula of the Leaky ReLU function is as follows:

[0153]

[0154] Where α is a parameter that adjusts the negative gradient and is usually 0.01.

[0155] The image encoder is a standard transformer encoder, that is, a neural network composed of repeatedly stacked encoder blocks (EncoderBlock), which is used to generate a one-time image embedding, that is, after the input image (i.e., the image inside the first reduction furnace) is divided into non-overlapping pixel blocks, linear projection is performed, and then position embedding based on trigonometric function operations is performed to generate a one-time image with embedded position information (i.e., one-time image embedding).

[0156] The hint encoder is used to embed hints, that is, to use two-dimensional convolution to perform convolution operations to reduce the spatial resolution of the mask. Then use two-dimensional normalization to perform layer normalization. Then pass the GELU activation function and finally map the features of the mask to the embedding dimension space through a 1x1 convolution layer. The hint encoder encodes background points, masks, and bounding boxes into embedding vectors in real time. In order to enable the algorithm to be more accurate in segmentation and meet the accuracy of silicon rod diameter measurement, dense hints are chosen, that is, masks are made based on whether each pixel covers the target. Dense hints are embedded through convolution and summed element by element through image embedding.

[0157] The mask decoder combines the output results of the hint encoder and the image encoder, and outputs the target segmentation result based on the hint (i.e., the first pixel position covered by the silicon rod in the image of the first reduction furnace and the second pixel position of its edge), that is, the last layer of the mask decoder is a linear mapping, and the number of output channels is the same as the number of pixels in a pixel block, in order to facilitate reconstruction, and then reconstruct the image; the loss function uses the mean square error (MSE) loss function, and the loss function is only calculated for the pixel block after the mask; the mean and standard deviation of the pixel values ​​in a pixel block are calculated at the same time, and then the pixel block is normalized, and finally the segmentation mask of the image is output. The mask decoder predicts the segmentation mask based on the embedding of the image encoder and the hint encoder. It maps the image embedding, hint embedding, and output token to the mask. All embeddings are updated by the decoder block, which uses immediate self-attention and cross-attention on the path from hint to image embedding and back. The mask is annotated and used to update the model weights. This layout enhances the dataset and allows the model (i.e., the segmentation model) to learn and improve over time, making it efficient and flexible. For example, the mask decoder can be a lightweight mask decoder. The input is the mask tokens and the encoded visual pixel patches with position encoding.

[0158] Exemplarily, mask encoding randomly masks part of the information of the input image, and the visual Transformer attempts to reconstruct these missing pixels based on the remaining target features, which can increase the robustness of the target encoding so that the target will not be missed by the model when it is occluded or at the edge of an incomplete image. The prompt encoder, as the basis for the visual Transformer to reconstruct the image, encodes the background points, masks, and bounding boxes into embedding vectors (i.e., the first embedding vector) in real time. The encoder divides the prompts into two groups: sparse (points, boxes) and dense (masks). Points and boxes are represented by position encodings, and a learned embedding vector is added for each prompt type. Dense prompts (such as masks) are embedded through convolution and summed element by element through image embedding. These two types of prompts enable the visual Transformer to understand the location and approximate boundary information of the target when facing an incomplete image, which is more conducive to the reconstruction of missing pixels. The mask decoder predicts the segmentation mask based on the embedding of the image and the prompt encoder. It maps image embeddings, prompt embeddings, and outputs to masks. All embeddings are updated by a decoder block that uses immediate self-attention and cross-attention in both directions (from prompts to image embeddings and back). Masks are annotated and used to update model weights. This layout enhances the dataset and allows the model to learn and improve over time, making it efficient and flexible.

[0159] For example, the network structure of the segmentation model is as follows Figure 5 The image (i.e., the image inside the first reduction furnace) is input into the image encoder, the output result is combined with the output result of the mask input convolution prompt encoder, and the combined result is combined with the output result of the feature point and border input prompt encoder to obtain the final output result.

[0160] For example, the algorithm structure diagram of the segmentation model is as follows: Figure 6 As shown, the prompt is input into the prompt encoder, the image (i.e., the image inside the first reduction furnace) is input into the image encoder, and the input results of both the prompt encoder and the image encoder are input into the lightweight mask decoder (i.e., the mask decoder) to obtain the target segmentation mask (i.e., the first pixel position covered by the silicon rod in the image inside the first reduction furnace and the second pixel position of the edge thereof).

[0161] In some examples, the segmentation model further includes a verification module, wherein the verification module is connected to the image encoder and the mask decoder respectively;

[0162] After using a mask decoder to predict and segment the mask of the first image embedding and the first embedding vector to obtain the first pixel point position covered by the silicon rod in the first reduction furnace image and the second pixel point position of the edge thereof, the method further includes:

[0163] The verification module is used to convert the image inside the first reduction furnace into the HSV space, and the binary dividing point is determined in the HSV space;

[0164] Based on the verification module, the image inside the first reduction furnace is binarized by using the binarization cutoff point to obtain a binarized image;

[0165] Based on the verification module, edge extraction operators are used to extract the edges of the binary image to obtain edge features;

[0166] The first image embedding and edge features are fused by using a verification module, and the second edge features with a higher confidence level are retained by using a first rule, so as to obtain the third pixel point position covered by the silicon rod in the first reduction furnace image and the fourth pixel point position of the edge thereof;

[0167] The third pixel position covered by the silicon rod in the first reduction furnace image and the fourth pixel position on its edge are respectively used as the first pixel position covered by the silicon rod in the first reduction furnace image and the second pixel position on its edge.

[0168] In this embodiment, the edge extraction algorithm is used to verify that the accuracy of the first pixel position covered by the silicon rods and the second pixel position of the edge thereof in the first reduction furnace in-furnace image can be improved.

[0169] Exemplarily, the first rule may be an interval progressive rule; the second higher confidence may be that the second confidence is greater than a confidence threshold, and the confidence threshold may be set according to actual conditions, for example, the confidence threshold may be 0.98.

[0170] That is to say, according to the color change of each image (i.e., the image in the first reduction furnace), the appropriate binary demarcation point is adaptively found in the HSV space, and the image (i.e., the image in the first reduction furnace) is binarized; the edge extraction operator (sobel, canny, etc.) is used to extract the edge of the binary image. When the image (i.e., the image in the first reduction furnace) is input into the algorithm, the segmentation algorithm and edge detection are used to extract the edge of the silicon rod in the image, and the results of the two are fused. The edge information (i.e., edge features) with a higher confidence (i.e., the second confidence) is retained using the interval progressive rule to complete the output of the final result. Among them, the binary demarcation point confirmation process is to create a visualization tool that can adjust the three parameters of HSV space h, s, and v. In the process of looping through the training data, the threshold that can accurately divide the most data edges is found through continuous adjustment. Pixels greater than the threshold are set to 1, and pixels below the threshold are set to 0, and the binarization can be completed; HSV space is converted from the image RGB space, and the conversion formula is as follows:

[0171]

[0172] V=max

[0173] Among them, r is the red color data value of a certain pixel of the image (i.e., the image inside the first reduction furnace), g is the green color data value of a certain pixel of the image, b is the blue color data value of a certain pixel of the image, max is the maximum value of r, g, and b at the pixel, and min is the minimum value of r, g, and b at the pixel. The values ​​of r, g, and b can be obtained based on the image inside the first reduction furnace.

[0174] In step S130, after the electronic device detects the image inside the first reduction furnace using the target detection model and obtains the detection result, it can also determine the target furnace operating condition of the target reduction furnace at the second moment based on the detection result.

[0175] As an example, the target furnace operating conditions include a target atomization value; as another example, the target furnace operating conditions include a target temperature value and a target silicon rod diameter; as yet another example, the target furnace operating conditions include a target atomization value, a target temperature value, and a target silicon rod diameter.

[0176] In some embodiments, determining the target furnace operating condition of the target reduction furnace at the second moment according to the detection result includes:

[0177] According to the n target detection frames and the n target confidences, a target atomization value of the target reduction furnace at the second moment is determined.

[0178] In some examples, determining a target atomization value of a target reduction furnace at a second moment according to n target detection frames and n target confidences includes:

[0179] When the n target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the first atomization value; or,

[0180] When only one target confidence value among the n target confidence values ​​is less than or equal to the first confidence threshold value and greater than the second confidence threshold value, and n-1 target confidence values ​​are all greater than the first confidence threshold value, the target atomization value of the target reduction furnace at the second moment is determined to be the second atomization value;

[0181] In the case that only one of the n target confidences is less than the second confidence threshold, and the n-1 target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the third atomization value; or,

[0182] In the case where two target confidences among the n target confidences are both less than or equal to the first confidence threshold, one of the two target confidences is less than the second confidence threshold, and n-2 target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the fourth atomization value; or,

[0183] In the case where only two target confidences among the n target confidences are less than the second confidence threshold, and n-2 target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the fifth atomization value; or,

[0184] By analogy, when there are c target confidences among the n target confidences that are all less than or equal to the first confidence threshold, c-1 of the c target confidences are all less than the second confidence threshold, and nc target confidences are all greater than the first confidence threshold, the target atomization value of the target reduction furnace at the second moment is determined to be the 2cth atomization value; or,

[0185] When only c target confidences among n target confidences are less than the second confidence threshold, and nc target confidences are all greater than the first confidence threshold, the target atomization value of the target reduction furnace at the second moment is determined to be the 2c+1th atomization value, c is less than or equal to n, and c is a positive integer.

[0186] The first confidence threshold is greater than the second confidence threshold, and the first confidence threshold and the second confidence threshold can be set according to actual conditions and are not limited here. For example, the first confidence threshold is 0.9 and the second confidence threshold is 0.8.

[0187] The first atomization value to the 2c+1th atomization value can be set according to actual conditions and stored in the electronic device in advance for subsequent direct call.

[0188] Exemplarily, taking the target reduction furnace as a 40-pair rod reduction furnace as an example, there are 8 pairs of bases along the direction facing the observation window field of view, and 6-7 pairs can be captured within the field of view of the industrial camera; when the image (i.e. the image inside the first reduction furnace) can clearly identify all the bases within the field of view (i.e. the confidence levels of the 8 targets are all greater than 0.9), it is considered that the current atomization degree (i.e. the target atomization value) is 0 (i.e. the first atomization value), which is the clearest case; for every pair less identified, the atomization degree value (i.e. the target atomization value) is increased by 1, and when five pairs or more are less identified, the maximum atomization degree value is 5 (i.e. the 2c+1th atomization value).

[0189] On this basis, the recognition confidence (i.e., target confidence) is introduced to further refine the atomization degree (i.e., target atomization value). For example, all base pairs can be recognized at present, but the recognition confidence of the last pair of bases is reduced from 90% (i.e., the first confidence threshold) to 80% (i.e., the second confidence threshold), then the current atomization degree value (i.e., target atomization value) changes from 0 (i.e., the first atomization value) to 0.5 (i.e., the second atomization value), and so on. The atomization degree value is divided into 11 levels from 0-5. The atomization degree classification is refined from 4 levels in the prior art to 11 levels, which increases the control accuracy of the polysilicon reduction production process by 3 times.

[0190] That is to say, the current atomization situation in the furnace (i.e., target atomization value) is determined according to the number of recognized bases and the average confidence of the farthest base pair (i.e., target confidence): Taking the target reduction furnace as a 40-pair rod reduction furnace as an example, when all bases within the field of view can be clearly identified in the image (i.e., the image in the first reduction furnace) (i.e., the 8 target confidences are all greater than 0.9), the current atomization degree (i.e., target atomization value) is 0 (i.e., the first atomization value), which is the clearest situation; when all bases within the field of view can be identified in the image but the average confidence of the last group of bases is <90% (i.e., only one of the 8 target confidences has a target confidence less than or equal to 0.9 and greater than 0.8, and 7 target confidences are all greater than 0. 8), the fogging degree (i.e., the target fogging value) is 0.5 (i.e., the second fogging value); when the last pair of bases in the image cannot be recognized (average confidence <80%), the fogging degree is 1 (i.e., the third fogging value); when the average confidence of the second-to-last group of bases in the image is <90% (i.e., among the 8 target confidences, there are two target confidences that are both less than or equal to 0.9, one of the two target confidences is less than 0.8, and all 6 target confidences are greater than 0.9), the fogging degree is 1.5 (i.e., the fourth fogging value); and so on, when the number of base pairs that cannot be recognized from far to near from the camera exceeds five pairs, the fogging degree (i.e., the target fogging value) is 5 (i.e., the 2c+1th fogging value), which is the highest fogging degree.

[0191] In other embodiments, determining the target furnace operating condition of the target reduction furnace at the second moment according to the detection result includes:

[0192] According to a plurality of different first detection sequences, a target temperature value of the target reduction furnace at a second moment is determined.

[0193] In some examples, the first detection sequence includes a predicted time, a predicted silicon rod temperature value, and a predicted ambient temperature value, and the target temperature value includes at least one of a target silicon rod temperature value and a target ambient temperature value;

[0194] Determining a target temperature value of a target reduction furnace at a second moment according to a plurality of different first detection sequences includes:

[0195] Determine at least two detection sequences with higher first confidences among the multiple different first detection sequences as at least two second detection sequences;

[0196] The predicted silicon rod temperature value corresponding to the predicted time closest to the first moment in at least two second detection sequences is determined as the target silicon rod temperature value, and / or the predicted ambient temperature value corresponding to the predicted time closest to the first moment in at least two second detection sequences is determined as the target ambient temperature value.

[0197] The second detection sequence includes predicted time, predicted silicon rod temperature value and predicted ambient temperature value.

[0198] That is to say, the output of the temperature detection model is sorted according to the confidence level (i.e., target confidence level) of the output results (i.e., the first detection sequence), and at least two second detection sequences with higher confidence levels select the label combination (i.e., the second detection sequence) corresponding to the time output closest to the current moment (i.e., the first moment) as the final output result (i.e., the target silicon rod temperature value and / or the target ambient temperature value), and infer the current silicon rod temperature (i.e., the target silicon rod temperature value) and ambient temperature (i.e., the target ambient temperature value).

[0199] Exemplarily, the error of the target silicon rod temperature value and / or the target ambient temperature value is plus or minus 10 degrees Celsius.

[0200] In some other embodiments, determining the target furnace operating condition of the target reduction furnace at the second moment according to the detection result includes:

[0201] The target silicon rod diameter of the target reduction furnace at the second moment is determined according to the first pixel point positions covered by the silicon rods and the second pixel point positions on the edge of the first reduction furnace in the image inside the first reduction furnace.

[0202] In some examples, determining a target silicon rod diameter of a target reduction furnace at a second moment according to a first pixel position covered by the silicon rod in an image of the first reduction furnace and a second pixel position of an edge thereof includes:

[0203] Determine the pixel diameter of the silicon rod in the first reduction furnace image according to the first pixel position covered by the silicon rod and the second pixel position of the edge thereof in the first reduction furnace image;

[0204] When the type of the silicon rod is the first type, determining a target silicon rod diameter of the silicon rod according to a first mapping relationship and a pixel diameter, the first mapping relationship including a one-to-one mapping relationship between a plurality of preset pixel diameters and a plurality of preset silicon rod diameters;

[0205] Alternatively, when the type of the silicon rod is the second type, the target actual spacing is determined according to the second mapping relationship and the pixel diameter, and the target silicon rod diameter of the silicon rod is determined according to the target actual spacing, and the second mapping relationship includes a one-to-one mapping relationship between multiple preset pixel diameters and multiple preset actual spacings.

[0206] The target silicon rod diameter is the actual diameter of the silicon rod. The target actual spacing is the spacing between two silicon rods before production starts. The first type is the type in which the silicon rods are in the center of the field of view; the second type is the type in which the silicon rods are arranged on both sides of the field of view.

[0207] That is to say, the segmentation results are converted into physical space: after the silicon rod edge is segmented out by the large model (i.e. segmentation model), the physical quantity measurement of the actual silicon rod diameter needs to be completed by comparing with the marker. The reduction furnace (i.e. target reduction furnace) can be divided into two categories according to the different fields of view of the observation window: the complete silicon rod is in the center of the field of view (i.e. the first type), and the silicon rods are arranged on both sides of the field of view (i.e. the second type). For silicon rods in the center of the field of view, it is necessary to take an image for calibration before the reduction furnace starts working. At this time, the diameter of the silicon rod is a known quantity. A mapping relationship (i.e., the first mapping relationship) is established between the pixel diameter of the silicon rod in the image (i.e., the preset pixel diameter) and the actual diameter (i.e., the preset silicon rod diameter). This mapping relationship can be used in the production process, combined with the change in the angle from the cylindrical observation surface to the lens to convert the pixel diameter in the segmentation result into the actual diameter in the physical space (i.e., the target silicon rod diameter) in real time; for silicon rods on both sides of the field of view, the calibration object is the spacing between the two silicon rods before production starts (i.e., the target actual spacing). In this way, the mapping relationship (i.e., the second mapping relationship) established with the pixel (i.e., the preset pixel diameter) can measure the actual change in the spacing in real-time production, thereby inferring the growth of the silicon rods on both sides, and then inferring the diameters of the silicon rods on both sides.

[0208] For example, the silicon rod segmentation algorithm structure diagram provided in the embodiment of the present application is as follows: Figure 7 As shown, the input (i.e., the image inside the first reduction furnace) is sequentially passed through Layer Normalization (i.e., layer normalization), multi-head attention mechanism, and discarding low weight parameters (i.e., discarding low weight parameter module) to obtain a first result, and the second result obtained by combining the first structure with the input is sequentially input into Layer Normalization, MLP module, and discarding low weight parameters to obtain a third result, and the second result and the third result are combined to obtain an output (i.e., the first pixel position covered by the silicon rod in the image inside the first reduction furnace and the second pixel position of its edge). Among them, the MPL module sequentially includes a linear layer, a GELU activation function, discarding low weight parameters, a linear layer, and discarding low weight parameters.

[0209] For example, the silicon rod pixel diameter image calculated based on the calibration object after edge calculation segmentation of the image (i.e., the image in the first reduction furnace) provided in the embodiment of the present application is as follows: Figure 8 and Fig. 9 As shown, the pixel diameter is 870 and the target actual spacing is 621.

[0210] In order to better understand the method for determining the operating conditions in a reduction furnace provided in an embodiment of the present application, a specific implementation method is described below.

[0211] The method for determining the working condition of a reduction furnace provided in an embodiment of the present application, that is, a method for detecting the atomization degree, silicon rod surface temperature, and silicon rod diameter of a polysilicon reduction furnace based on computer vision technology, includes:

[0212] An industrial camera is used to capture the polysilicon reduction growth process, and the degree of atomization is divided based on how many pairs of electrode bases can be clearly seen in the image.

[0213] by Figure 8 and Fig. 9 After edge computing and segmentation, the diameter of the silicon rod is measured based on the calibration object, and the error requirement is millimeter level.

[0214] by Fig.10 The temperature images and RGB images collected alternately are used, and temperature calibration is performed through pixel alignment. A model is established to calculate the surface temperature of the silicon rod with an error requirement of ±10℃.

[0215] The camera is a 25-megapixel CMOS 10 Gigabit Ethernet industrial area scan camera with a GMAX3265 sensor, which has excellent image quality. Through the 10 Gigabit Ethernet interface, it can quickly transmit uncompressed data in real time, and the maximum frame rate can reach 17.2fps at full resolution.

[0216] The collected images are marked with running time and the number of bases to form a feature map.

[0217] Taking a 40-pair rod reduction furnace as an example, there are 8 pairs of bases along the direction facing the observation window, and 6-7 pairs can be captured within the field of view of the industrial camera; when all the bases within the field of view can be clearly identified in the image, the current atomization degree is considered to be 0, which is the clearest situation; for each pair less identified, the atomization degree value is increased by 1, and when five pairs or more are less identified, the maximum atomization degree value is 5.

[0218] On this basis, the recognition confidence is introduced to further refine the atomization level. For example, all base pairs can be recognized at present, but the recognition confidence of the last pair of bases drops from 90% to 80%, then the current atomization level value changes from 0 to 0.5, and so on, the atomization level value is divided into 11 levels from 0-5. Technical effect: The atomization level classification is refined from 4 levels in the existing technology to 11 levels, which increases the control accuracy of the polysilicon reduction production process by 3 times.

[0219] A deep learning reduction furnace base detection algorithm based on Transformer technology.

[0220] The efficient parallel computing capability of the Transformer model can make model training and reasoning faster, and can achieve powerful long-distance dependency learning capabilities through the self-attention mechanism, making it suitable for a variety of tasks.

[0221] The CNN+Transformer neural network structure is adopted to solve the detection problem in an end-to-end manner.

[0222] The CNN network is a convolutional neural network, which is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning.

[0223] The CNN part extracts the image feature map, and the Transformer uses the self-attention mechanism to explicitly model the relationship between all the feature elements in the feature map.

[0224] Feature map, that is, the feature map of each channel in the original image or the image after convolution kernel filtering. In the original image or after each convolution, the data exists in three-dimensional form. By dividing the data according to the channel, the data can be regarded as the superposition of many two-dimensional images, and each two-dimensional image is called a feature map. Input the RGB original image in the input layer of CNN, and the original image can be regarded as the superposition of 3 feature maps (red, green and blue). There will be several convolution kernel filters between the layers of the CNN structure. Each layer of convolution kernel filters will convolve with each feature map of its input and generate several new feature maps as the input of the next layer. Since the parameter weights of the convolution kernel are trained according to the actual position of the target in the image, after CNN reaches the target accuracy, the extracted feature maps should be visualized as grayscale images with pixels around all targets highlighted and the rest gradually darkened.

[0225] Self-attention, or the self-attention mechanism, allows the model to assign weights to variables. This mechanism repeatedly solves the correlation between data features to find out what kind of correlation exists between data features when the target effect is closest, and then fixes the correlation to solve the desired target on new data after new data is input into the model.

[0226] The attention calculation formula is as follows:

[0227]

[0228] Where Q represents the query matrix of the input data, K represents the index matrix of the input data, and V represents the content matrix of the input data. Represents the vector length of the input data. The Softmax function calculation formula is as follows:

[0229]

[0230] Where S i represents the softmax value of the i-th element in the sequence, e i Represents the index of the i-th element in the sequence, e j Represents the index of the jth element in the sequence. Therefore, the softmax value is the ratio of the index of this element to the sum of the indices of all elements.

[0231] In terms of base detection, the recognition accuracy is better than Fast RCNN, Yolo and other CNN structure neural networks.

[0232] The Fast RCNN is a neural network model based on the fast region convolutional network method to deal with the target detection problem. It implements target detection through the following steps: 1) extracting features from the image through CNN to obtain the feature map of the image; 2) obtaining the region of interest of the image through the selective search algorithm; 3) pooling the obtained region of interest, that is, through the coordinate projection method, obtaining the feature region corresponding to the region of interest in the input image on the feature map, and performing maximum pooling on the region, so that the features of the region of interest are obtained and the feature size is unified; 4) connecting the feature vector of the region of interest to the fully connected layer, and defining a multi-task loss function, so as to obtain the coordinate bounding box of the current region of interest; 5) performing non-maximum suppression on all the obtained bounding boxes to obtain the final detection result.

[0233] Yolo is an end-to-end target detection neural network. The core idea is to transform target detection into a regression problem. The entire image is used as the input of the network. Through the neural network, the location of the bounding box and its category are obtained. It implements target detection through the following steps: 1) Divide the input image into a grid of fixed size; 2) For each grid, YOLO predicts a fixed number of bounding boxes. Each bounding box is described by 5 main attributes: the location of the bounding box (center coordinates and width and height), the confidence of the target contained in the bounding box, and the category of the predicted target; 3) Perform a single forward pass through a CNN to predict the location and category of all bounding boxes at the same time; 4) Use a multi-task loss function to train the network. The loss function includes position loss, confidence loss and category loss. The position loss measures the position difference between the predicted bounding box and the true bounding box. The confidence loss measures whether the bounding box correctly predicts the target and penalizes the confidence of the background box. The category loss measures the prediction accuracy of the target category; 5) Use the non-maximum suppression algorithm to select the best bounding box based on confidence and overlap.

[0234] The image feature extraction layer is established based on the CNN structure, which converts the original three-channel image into a feature map with thousands of channels.

[0235] The association relationship layer is established based on the Transformer structure to learn the feature elements in the feature graph.

[0236] The relationship between the characteristic graph and the current value trend, voltage value trend, and raw material flow rate trend.

[0237] The current algorithm is trained using production time, silicon rod temperature category, and ambient temperature category as labels.

[0238] When the algorithm is trained and put into actual use, the input data of the algorithm is consistent with that during training, that is, the image at the current moment, the trend of the current value, the trend of the voltage value, and the trend of the raw material flow rate in the period before the current moment. The output of the algorithm is sorted by the confidence level of the output result. The top three label combinations [time, silicon rod temperature category, ambient temperature category] with the highest confidence level are selected as the final output result, and the label combination corresponding to the time output closest to the current moment is inferred as the type of the current silicon rod temperature and ambient temperature.

[0239] Segmentation algorithm part: Use the CNN feature extraction layer of the CSP-Darknet structure to enhance the learning ability of CNN and reduce computing bottlenecks and memory consumption; then use feature fusion and feature enhancement to enhance the accuracy of contour detection.

[0240] CSP-Darknet is Darknet based on CSP structure. The original intention of CSP structure is to reduce the amount of calculation and enhance the performance of gradient. The main idea is: before entering the current layer, the input data is divided into two parts, one of which is calculated by the current layer, and the other is directly spliced ​​with the current layer result by cropping. This ensures that the input data will not be unable to splice the complete target with the original data due to excessive deviation after passing through the network layer, and also reduces the amount of calculation of the network layer. Darknet is a lightweight target detection model, and its structure is the first layer: convolution; the second layer: downsampling convolution + 1 residual block; the third layer: downsampling convolution + 2 residual blocks; the fourth layer: downsampling convolution + 8 residual blocks; the fifth layer: downsampling convolution + 8 residual blocks; the sixth layer: downsampling convolution + 4 residual blocks. Its core feature is the use of residual blocks. The design of residual blocks is to help solve the problems of gradient disappearance and gradient explosion in deep neural network training, and improve the performance and convergence speed of the network. The core idea of ​​the residual block is to learn the residual function by introducing skip connections, so that the network can learn the residual mapping more easily.

[0241] The core structure of CSP consists of convolution function, normalization function and Leaky ReLU function.

[0242] The convolution function formula is as follows:

[0243] Assume that the input feature map size is W in ×H in ×D in , the output feature map size is W out ×H out ×D out , where W is the feature map width, H is the feature map height, and D is the feature map depth (i.e. the number of channels), then:

[0244] W out =(W in +2p-w) / s+1

[0245] H out =(H in +2p-h) / s+1

[0246] D out =k

[0247] Where k is the number of convolution kernels, w and h are the width and height of the convolution kernel respectively, s is the convolution step size, and p is the padding width.

[0248] The normalization function formula is as follows:

[0249]

[0250] where a iis the original activation value of the ith neuron, The normalized value after the normalization operation. The whole normalization process can be decomposed into two steps. The first step is to normalize the activation value to a normal distribution range with a mean of 0 and a variance of 1. Where μ is the mean of the activation values ​​of the m neurons contained in the neuron set, and the formula is as follows:

[0251]

[0252] σ is the standard deviation of the activation value calculated based on the mean and the activation values ​​of each neuron in the set. The formula is as follows:

[0253]

[0254] Here, ε is a small constant data added to increase the stability of training.

[0255] The second step is to let each neuron learn the corresponding two adjustment factors γ during the training process. i and β i , fine-tune the values ​​that are normalized to a mean of 0 and a variance of 1. Because the nonlinear expression ability of the neural network may be reduced after the first step, this method will be used to compensate for the expression ability of the neural network after the first step.

[0256] The formula of the Leaky ReLU function is as follows:

[0257]

[0258] Where α is a parameter that adjusts the negative gradient and is usually 0.01.

[0259] Edge detection part: According to the color changes in each image, adaptively find the appropriate binary demarcation point in the HSV space and binarize the image; use edge extraction operators (sobel, canny, etc.) to extract the edges of the binary image.

[0260] The HSV space is converted from the image RGB space, and the conversion formula is as follows:

[0261]

[0262] v=max

[0263] Among them, r, g, and b are the red, green, and blue color data values ​​of a pixel in the image, respectively. Max is the maximum value of r, g, and b at the pixel, and min is the minimum value of r, g, and b at the pixel.

[0264] After the image is input into the algorithm, the segmentation algorithm and edge detection are used to extract the edges of the silicon rods in the image respectively; the results of the two are fused, and the edge information with higher confidence is retained using the interval progressive rule to complete the output of the final result.

[0265] A method for identifying the working conditions in a reduction furnace based on machine vision technology, the implementation scheme of which is as follows:

[0266] An industrial camera that can be nested in the reduction furnace observation window is installed on the reduction furnace observation window and connected to an industrial intelligent computer to realize a photo taking function.

[0267] After taking the photo, the picture is transmitted to the reduction furnace base detection algorithm through the memory inside the industrial intelligent computer to identify the number of base pairs in the current picture.

[0268] The recognition algorithm is a deep learning neural network algorithm, and the overall architecture consists of four parts.

[0269] The first part is the CNN feature extraction layer, which converts a 3-channel color image of size W and H into a feature map of 2048 channels and size W / 32 and H / 32.

[0270] The second part is the Transformer Encoder, which first compresses the number of channels of the feature map from 2048 to 256 through a 1*1 convolution kernel, and then flattens it into a new feature map of size (WH / 1024,256). Then, it marks the position of each feature element in the new feature map, that is, each element has a position code from (0, 0) to (WH / 1024-1,255).

[0271] The third part is the Transformer Decoder, which takes as input the new feature map that has positional encoding for each element. It uses the principle of self-attention neural network to split each element in the new feature map into a query vector and a key vector, and solves the pairwise correlation between each element query vector and itself and other element key vectors, and encodes all the correlation values ​​into WH / 4 feature vectors of length WH / 4 according to the element encoding.

[0272] The fourth part is the feedforward fully connected layer, which compresses each of the WH / 4 feature vectors into a single point value, combines the single point values ​​into a vector of length WH / 4, and then maps the vector into two coordinate values ​​and a confidence as the final output, which are the upper left corner coordinates, lower right corner coordinates and detection result confidence of the target detection box.

[0273] The model parameters of the algorithm are trained from the training data. The training data is a picture of the reduction furnace with a marked base detection frame. The marking method is to use the labelImg tool to draw the base frame in the picture by pulling.

[0274] The current atomization situation in the furnace is determined based on the number of recognized base pairs and the average confidence level of the farthest base pair: taking a 40-pair rod reduction furnace as an example, when all bases within the field of view can be clearly recognized in the image, the current atomization level is 0, which is the clearest situation; when all bases within the field of view can be recognized in the image but the average confidence level of the last group of bases is <90%, the atomization level is 0.5; when the last pair of bases in the image cannot be recognized (confidence level <80%), the atomization level is 1; when the average confidence level of the second-to-last group of bases in the image is <90%, the atomization level is 1.5; and so on. When the number of base pairs that cannot be recognized from far to near from the camera exceeds five pairs, the atomization level is 5, which is the highest atomization level.

[0275] To sum up, the specific base calculation algorithm model is as follows:

[0276] like Figure 2 As shown, its main structure is CNN+Transformer encoder-decoder+target position prediction output

[0277] like Figure 3 As shown in the figure, the CNN part is inspired by the visual Transformer algorithm, and the traditional CNN is modified in five parts: 1) the balanced distribution structure of the traditional 4-section convolutional neural network is changed to a concentrated distribution structure, that is, the convolution stacking of the third section structure is increased, and the convolution stacking of the 1st, 2nd, and 4th layers is reduced; 2) the convolution kernel size is increased and the feature overlap is reduced like the transformer; 3) the traditional convolution structure is modified, and the convolution is replaced by a group convolution with the same number of groups and channels, so that it is close to the weighted sum of self-attention in the transformer; 4) the Bottleneck with small ends and large middle of the network structure is used to imitate the MLP module in the Transformer block; 5) the ReLU activation function commonly used in CNN is replaced by the GELU activation function commonly used in Transformer, and the Batch Normalization commonly used in CNN is replaced by the LayerNormalization commonly used in Transformer and the number of Normalization is reduced to be consistent with the Transformer. The features extracted by the above-mentioned modified CNN can better adapt to the Transformer codec, thereby effectively improving the recognition accuracy.

[0278] The Transformer encoder-decoder part is a normal Transformer structure, where the encoder layer is composed of multiple encoding modules in series, and the unified structure of each encoding module is: multi-head self-attention mechanism + residual ADD + layer normalization LayerNormalization + feedforward network FFN + residual ADD + layer normalization Layer Normalization. The decoder layer is composed of multiple decoding modules in series, and the unified structure of each decoding module is: Masked multi-head self-attention mechanism + residual ADD + layer normalization Layer Normalization + multi-head cross attention mechanism + residual ADD + layer normalization LayerNormalization + feedforward network FFN + residual ADD + layer normalization Layer Normalization.

[0279] The target position output consists of a linear layer + a softmax layer. The linear layer converts the feature map into a one-dimensional feature, and the softmax layer probabilistically normalizes the one-dimensional feature to obtain the target position coordinates.

[0280] Silicon rod diameter measurement method based on pre-trained target segmentation large model:

[0281] Build a large model for target segmentation: Use the structure of image encoder + prompt encoder + mask decoder to build a large model for target segmentation. The image encoder consists of mask encoder + visual transformer. The mask encoder will randomly cover up part of the information of the input image. The visual transformer will try to reconstruct these missing pixels based on the remaining target features. This can increase the robustness of the target encoding, so that the target will not be missed by the model when it is occluded or incomplete at the edge of the image; the prompt encoder, as the basis for the visual transformer to reconstruct the image, encodes the background points, masks, and bounding boxes into embedded vectors in real time. The encoder divides the prompts into two groups: sparse (points, boxes) and dense (masks). Points and boxes are represented by position encoding, and a learned embedding vector is added for each prompt type. Dense prompts (such as masks) are embedded through convolution and summed element by element through image embedding. These two types of prompts enable the visual transformer to understand the location and approximate boundary information of the target when facing an incomplete image, which is more conducive to the effect of reconstructing missing pixels; the mask decoder predicts the segmentation mask based on the embedding of the image and prompt encoder. It maps image embeddings, prompt embeddings, and outputs to masks. All embeddings are updated by a decoder block that uses immediate self-attention and cross-attention in both directions (from prompts to image embeddings and back). Masks are annotated and used to update model weights. This layout enhances the dataset and allows the model to learn and improve over time, making it efficient and flexible.

[0282] Use a large amount of data to complete the pre-training of the target segmentation large model: The large model of this structure requires a large amount of data as training support to achieve good target segmentation effects. However, the reduction furnace scenario cannot provide millions of images and hundreds of millions of segmentation targets. Therefore, the internationally open 1 billion segmentation target data set is used to complete the pre-training of the target segmentation large model, so that the large model can understand what is the correct target segmentation in various scenarios, and then migrate to the segmentation scenario of silicon rod images in the reduction furnace. This dataset includes image data from six continents around the world, covering a wide range of fields, objects, and scenarios, ensuring that the model can be well generalized to different tasks. It includes images from various sources, such as natural scenes, urban environments, medical images, satellite images, etc.

[0283] Use reduction furnace image data to fine-tune the target segmentation large model: collect tens of thousands of reduction furnace image data as fine-tuning data sets, and calibrate silicon rods in the format of the original training data of the large model. After locking the core parameters of the large model, use these data to train the edge output parameters of the large model, thereby achieving migration fine-tuning of the reduction furnace scene. The main function of this fine-tuning is to weaken the impact of the color contrast in the large data set on the large model, because the silicon rods will be connected in the field of view after growing to a certain extent, and the color difference is small due to the close temperature of the silicon rods. Therefore, in order to complete effective segmentation under such conditions, it is necessary to use the real calibration data of the reduction force scene for fine-tuning training to correct the output of the large model.

[0284] Physical space conversion of segmentation results: After the edge of the silicon rod is segmented out with a large model, it is necessary to complete the physical measurement of the actual silicon rod diameter by comparing it with a marker. The reduction furnace can be divided into two categories according to the different fields of view of the observation window: the complete silicon rod is in the center of the field of view, and the silicon rods are arranged on both sides of the field of view. For the silicon rod in the center of the field of view, it is necessary to take an image as a calibration before the reduction furnace starts working. At this time, the diameter of the silicon rod is a known quantity. A mapping relationship is established between the pixel diameter of the silicon rod in the image and the actual diameter. This mapping relationship can be used in the production process, combined with the change in the angle between the cylindrical observation surface and the lens to convert the pixel diameter in the segmentation result into the actual diameter in the physical space in real time; for the silicon rods on both sides of the field of view, the calibration object is the spacing between the two silicon rods before the start of production. In this way, the mapping relationship established between the pixels can measure the real change of the spacing in real-time production, thereby inferring the growth of the silicon rods on both sides, and then inferring the diameter of the silicon rods on both sides.

[0285] In summary, the specific algorithm of the silicon rod diameter measurement method of the target segmentation large model is as follows: Figure 7As shown in the figure, the pre-trained large model consists of three core components: an image encoder, which is used to generate a one-time image embedding; a prompt encoder, which is used to embed prompts; and a mask decoder, which combines the prompt kernel image and outputs a prompt-based target segmentation result.

[0286] The image encoder is a standard Transformer encoder, that is, a neural network composed of multiple repeated stacking of EncoderBlocks. Its network structure is as follows: Figure 5 As shown:

[0287] The hint encoder encodes background points, masks, and bounding boxes into embedded vectors in real time. In order to make the algorithm more accurate in segmentation and meet the accuracy of silicon rod diameter measurement, dense hints are chosen, that is, masks are made based on whether each pixel covers the target. Dense hints are embedded through convolution and summed element by element through image embedding.

[0288] A lightweight mask decoder predicts a segmentation mask based on the embeddings of the image and hint encoders. It maps the image embedding, hint embedding, and output token to the mask. All embeddings are updated by the decoder block, which uses immediate self-attention and criss-cross attention on the path from hint to image embedding and back again. The mask is annotated and used to update the model weights. This layout enhances the dataset and allows the model to learn and improve over time, making it efficient and flexible.

[0289] Visible light camera temperature measurement method based on short-wave infrared camera temperature measurement calibration:

[0290] The experimental reduction furnace is used to calibrate and map the short-wave infrared camera and the visible light camera: the observation window structure of the reduction furnace is a double-layer quartz glass, and there is circulating cooling water in the middle of the quartz glass. Ordinary medium- and long-wave infrared cameras cannot measure the temperature within the field of view. Therefore, the method of using short-wave infrared temperature measurement to calibrate RGB images is considered, and the temperature measurement of RGB images is realized by modeling the relationship between the two. First, an experimental furnace that can adjust the current at will is selected to heat and cool the silicon rod, and infrared images and RGB images are collected at the same time during the heating and cooling process; the temperature information on the infrared image is calibrated on the RGB image in a pixel-level mapping manner.

[0291] Use image regression algorithm to model temperature prediction for RGB images: Use tens of thousands of RGB images with calibrated temperature to train deep learning image regression algorithm, with temperature labels as labels. The deep learning image regression algorithm is a sparse autoencoder structure, in which the encoder transforms the RGB image into several feature maps with the help of convolution structure, and uses the sparse structure to reduce the amount of model calculation and increase the weight of the temperature stable area; decode all feature maps, and compare the pixels with the temperature label image after decoding the output image to solve the approximation. Repeat this process until the approximation is highest and stable, at which time the trained model parameters are the optimal parameters.

[0292] The parameters of the RGB camera with the optimal parameters trained are fixed, and the field of view is corrected on other reduction furnaces. After completing the above two steps, the model can be used to complete the temperature inference of RGB images in the production process of other reduction furnaces.

[0293] Compared with the prior art, the embodiment of the present application improves the accuracy of atomization degree characterization by 3 times, and can predict the optimal position based on the measurement of silicon rod surface temperature and growth rate, optimizes the existing control strategy for polysilicon reduction production, and maintains the polysilicon production process in a state of stable reaction and efficient adhesion, thereby reducing the unit power consumption of polysilicon production compared with the prior art.

[0294] Example 2

[0295] The detection model training method provided in the embodiment of the present application can be executed by a reduction furnace internal working condition determination device and an electronic device, etc. The following takes the detection model training method executed by an electronic device as an example for explanation.

[0296] The detection model training method provided in the embodiment of the present application adopts the target detection model as in any one of the embodiments 1, the target detection model includes at least one of the reduction furnace base detection model, the temperature detection model and the segmentation model, and the method includes:

[0297] In the case where the target detection model includes a reduction furnace base detection model, a first training set is obtained, where the first training set includes a plurality of first training samples, and the first training samples include historical reduction furnace in-furnace images and their corresponding historical detection frames;

[0298] Using a plurality of first training samples to train the first preset model to obtain a reduction furnace base detection model; and / or,

[0299] In the case where the target detection model includes a temperature detection model, a second training set is obtained, the second training set includes a plurality of second training samples, the second training samples include a historical reduction furnace image, second furnace information, and a plurality of different historical sequences, the second furnace information includes at least one of a second current value trend, a second voltage value trend, and a second raw material flow rate trend of the reduction furnace before a third moment, and the historical sequence includes historical time, historical silicon rod temperature values, and historical ambient temperature values;

[0300] The second preset model is trained using a plurality of second training samples to obtain a temperature detection model; and / or, when the target detection model includes a segmentation model, a third training set and a fourth training set are obtained, the third training set includes a target image and its corresponding target category label, the fourth training set includes a plurality of fourth training samples, the third training sample includes an image inside a historical reduction furnace, and a fifth pixel point position covered in the image inside the historical reduction furnace and a sixth pixel point position on its edge;

[0301] Using a plurality of third training samples to train the third preset model to obtain an initial segmentation model;

[0302] The initial segmentation model is trained using a plurality of fourth training samples to obtain a segmentation model.

[0303] According to the target detection model training method provided in the embodiment of the present application, the preset model is trained using training samples, which can improve the accuracy of the target detection model and provide a basis for the subsequent determination of the operating conditions inside the reduction furnace.

[0304] For example, the historical detection frame can use the labelImg tool to draw the base frame in the picture (i.e., the historical reduction furnace image) by pulling. The first training sample is the reduction furnace image (i.e., the historical reduction furnace image) with the base detection frame (i.e., the historical detection frame) marked.

[0305] Exemplarily, a first preset model is trained using a plurality of first training samples to obtain a reduction furnace base detection model, including: using the first preset model to train the historical reduction furnace in-furnace image to obtain n predicted detection frames and n predicted confidences in the historical reduction furnace in-furnace image; determining the first loss value between the n predicted detection frames and their one-to-one corresponding historical detection frames according to the first preset loss function, and adjusting the model parameters of the first preset model when the first loss value is greater than the first preset loss value or the first training times are less than the first preset times, until the first loss value is less than or equal to the first preset loss value or the first training times are greater than or equal to the first preset times, the iterated first preset model is determined as the reduction furnace base detection model. The first training times are the training times of the first preset model. The first preset model may include a first Transformer model. The first preset loss function may include a mean square error loss function, etc. The first preset loss value and the first preset times may be set according to actual conditions, and are not limited here.

[0306] Exemplarily, the experimental reduction furnace is used to calibrate and map the short-wave infrared camera and the visible light camera: the observation window structure of the reduction furnace is a double-layer quartz glass, and circulating cooling water passes through the middle of the quartz glass. Ordinary medium and long-wave infrared cameras cannot measure the temperature within the field of view. Therefore, the method of using short-wave infrared temperature measurement to calibrate the RGB image (i.e., the image inside the historical reduction furnace) is considered, and the temperature measurement of the RGB image is realized by modeling the relationship between the two. First, an experimental furnace that can adjust the current at will is selected to heat and cool the silicon rod, and infrared images and RGB images are collected at the same time during the heating and cooling process; the temperature information on the infrared image is calibrated on the RGB image in a pixel-level mapping manner to obtain the historical silicon rod temperature value and the historical ambient temperature value. Among them, the pixel-level mapping method calibration may include: by compressing the RGB image to the same size as the infrared temperature image (i.e., the infrared image), and then mapping the temperature value on each pixel to the compressed RGB image, and then restoring the RGB image to the original size, and the extra empty value points are filled in by the nearest filling method.

[0307] Exemplarily, before training the temperature detection model, the field of view of the shortwave infrared camera can be adjusted to be consistent with the field of view of the RGB camera (i.e., the visible light camera) to ensure that the local temperature labels of the shortwave infrared image (i.e., the image inside the historical restoration furnace) can be accurately matched on the RGB camera to ensure the accuracy of the temperature detection model training data.

[0308] Exemplarily, the second preset model is trained using multiple second training samples to obtain a temperature detection model, including: using the second preset model to train the image inside the historical restoration furnace to obtain a second target sequence corresponding to the image inside the historical restoration furnace; according to the second preset loss function, determine the second loss value between the second target sequence and its one-to-one corresponding historical sequence, and when the second loss value is greater than the second preset loss value, or the second training number is less than the second preset number, adjust the model parameters of the second preset model until the second loss value is less than or equal to the second preset loss value, or the second training number is greater than or equal to the second preset number, the iterated second preset model is determined as the temperature measurement model. The second training number is the training number of the second preset model. The second preset model may include a second Transformer model. The second preset loss function may include a mean square error loss function and a cosine approximation function, etc. The second preset loss value and the second preset number can be set according to actual conditions, and are not limited here.

[0309] That is to say, the image regression algorithm is used to perform temperature prediction modeling on RGB images: tens of thousands of RGB images with calibrated temperatures are used to train the deep learning image regression algorithm, and the labels are temperature labels (i.e., historical silicon rod temperature values ​​and historical ambient temperature values). The deep learning image regression algorithm is a sparse autoencoder structure, in which the encoder uses the convolution structure to convert the RGB image (i.e., the image inside the historical reduction furnace) into several feature maps, and uses the sparse structure to reduce the amount of model calculation and increase the weight of the temperature stable area; decode all feature maps, and after decoding the output image, compare the pixels with the temperature label image to solve the approximation (i.e., the second loss value). Repeat this process until the approximation is highest and stable (the approximation iteration fluctuation amplitude does not exceed 0.01 and the approximation absolute value is higher than 0.95), at which time the trained model parameters are the optimal parameters. The parameters of the RGB camera that has trained the optimal parameters are fixed, and the field of view correction is performed on other reduction furnaces. After completing the above two steps, the model can be used to complete the temperature reasoning of RGB images in other reduction furnace production processes.

[0310] For example, a deep learning image regression algorithm is trained using tens of thousands of RGB images with calibrated temperatures, with the labels being temperature labels. The deep learning image regression algorithm is a sparse autoencoder structure, in which the encoder transforms the RGB image into several feature maps by means of a convolutional structure.

[0311] The current algorithm (i.e., image regression algorithm) has been tested on multiple furnaces, and the temperature prediction values ​​are compared with the infrared measurement values. The average temperature error is within 10 degrees Celsius. However, due to the older type of reduction furnace, the observation window of the infrared camera currently collected will turn black as the production progresses, resulting in shielding of thermal radiation in the later stage, causing the measured temperature to drop rapidly and be distorted. The subsequent new reduction furnace window will have a guide ring to avoid this phenomenon, making the measurement more accurate and the prediction error smaller.

[0312] Exemplarily, the third training set can be a 1 billion segmentation target data set. Specifically, a large amount of data is used to complete the pre-training of the target segmentation large model (i.e., the segmentation model): the large model of this structure requires a large amount of data as training support to achieve a good target segmentation effect, but the reduction furnace scenario cannot provide millions of pictures and hundreds of millions of segmentation targets, so the internationally disclosed 1 billion segmentation target data set is used to complete the pre-training of the target segmentation large model, so that the large model can understand what is the correct target segmentation in various scenarios, and then migrate to the segmentation scene of the silicon rod image in the reduction furnace. This dataset includes image data from six continents around the world, covering a wide range of fields, objects, and scenes, ensuring that the model can be well generalized to different tasks. It includes images (i.e., target images) from various sources, such as natural scenes, urban environments, medical images, satellite images, etc.

[0313] Exemplarily, the reduction furnace image data is used to fine-tune the target segmentation large model: tens of thousands of reduction furnace image data are collected as the fine-tuning data set (i.e., the fourth training set), and the silicon rods are calibrated in the format of the original training data of the large model (i.e., the segmentation model). After the core parameters of the large model are locked, the edge output parameters of the large model are trained with these data, thereby achieving the migration fine-tuning of the reduction furnace scene. The main function of this fine-tuning is to weaken the influence of edge detection formed by color contrast in the large data set on the large model (i.e., the boundary of the segmentation model), because the silicon rods will be connected in the field of view after growing to a certain extent, and the color difference is small due to the close temperature of the silicon rods. Therefore, in order to complete effective segmentation under such conditions, it is necessary to use the real calibration data of the reduction force scene for fine-tuning training to correct the output of the large model.

[0314] Exemplarily, the third preset model is trained using a plurality of third training samples to obtain an initial segmentation model, including: training the target image using the third preset model to obtain a predicted category label; determining a third loss value between the predicted category label and its corresponding target category label according to a third preset loss function, and when the third loss value is greater than the third preset loss value, or the third training times are less than the third preset times, adjusting the model parameters of the third preset model until the third loss value is less than or equal to the third preset loss value, or the third training times are greater than or equal to the third preset times, the iterated third preset model is determined as the initial segmentation model. The third training times are the training times of the third preset model. The third preset model may include a third Transformer model. The third preset loss function may include a mean square error loss function, etc.

[0315] Exemplarily, the fourth preset model is trained using a plurality of fourth training samples to obtain an initial segmentation model, including: using the initial segmentation model to train the image inside the historical reduction furnace to obtain the first predicted pixel position covered in the image inside the historical reduction furnace and the second predicted pixel position on its edge; according to the fourth preset loss function, the fourth loss value between the first predicted pixel position and the second predicted pixel position on its edge and the one-to-one corresponding fifth pixel position and the sixth pixel position on its edge is determined, and when the fourth loss value is greater than the fourth preset loss value, or the fourth training times is less than the fourth preset times, the model parameters of the initial segmentation model are adjusted until the fourth loss value is less than or equal to the fourth preset loss value, or the fourth training times is greater than or equal to the fourth preset times, the iterated initial segmentation model is determined as the segmentation model. The fourth training times are the training times of the initial segmentation model. The fourth preset loss function may include a mean square error loss function, etc.

[0316] It should be noted that the specific training process of the target detection model is similar to the specific application process of the target detection model in Example 1, and will not be repeated here.

[0317] Example 3

[0318] like Fig.11 As shown, the device for determining the operating condition in a reduction furnace provided in an embodiment of the present application may include a first acquisition module 21 , a first detection module 22 and a first determination module 23 .

[0319] The first acquisition module 21 is used to acquire the first reduction furnace interior image of the target reduction furnace at the first moment;

[0320] A first detection module 22, connected to the first acquisition module 21, is used to detect the image inside the first reduction furnace using a target detection model to obtain a detection result, wherein the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model, and a segmentation model;

[0321] The first determination module 23 is connected to the first detection module 22, and is used to determine the target furnace operating conditions of the target reduction furnace at the second moment according to the detection results. The target furnace operating conditions include at least one of a target atomization value, a target temperature value, and a target silicon rod diameter. The target furnace operating conditions correspond to the target detection model one by one, and the first moment is earlier than the second moment.

[0322] According to the device for determining the operating condition in a reduction furnace provided in an embodiment of the present application, a target detection model is used to detect the image in the first reduction furnace to obtain a detection result, and then the target operating condition in the target reduction furnace at the second moment is determined based on the detection result. The target operating condition in the target reduction furnace can be predicted in advance, that is, the operating condition in the reduction furnace can be determined in time to improve the quality of polysilicon.

[0323] In some embodiments, the target detection model includes a reduction furnace base detection model, and the detection result includes n target detection frames and n target confidences in the first reduction furnace image, the target detection frames and the target confidences correspond one to one, and n is a positive integer; the target furnace working condition includes a target atomization value;

[0324] The first detection module 22 is specifically used for:

[0325] The reduction furnace base detection model is used to detect the first reduction furnace interior image, and n target detection frames and n target confidences in the first reduction furnace interior image are obtained;

[0326] The first determining module 23 is specifically used for:

[0327] According to the n target detection frames and the n target confidences, a target atomization value of the target reduction furnace at the second moment is determined.

[0328] In some embodiments, the target detection model includes a temperature detection model, the detection result includes a plurality of different first detection sequences, and the target furnace operating condition includes a target temperature value;

[0329] The first detection module 22 is specifically used for:

[0330] Using a temperature detection model to detect the first reduction furnace image and first furnace information to obtain a plurality of different first detection sequences, wherein the first furnace information includes at least one of a first current value trend, a first voltage value trend, and a first raw material flow rate trend of the target reduction furnace before a first moment;

[0331] The first determining module 23 is specifically used for:

[0332] According to a plurality of different first detection sequences, a target temperature value of the target reduction furnace at a second moment is determined.

[0333] In some embodiments, the target detection model includes a segmentation model, the detection result includes a first pixel position covered by the silicon rod in the first reduction furnace image and a second pixel position of its edge, and the target furnace working condition includes a target silicon rod diameter;

[0334] The first detection module 22 is specifically used for:

[0335] The first reduction furnace image is detected by using a segmentation model to obtain the first pixel point position covered by the silicon rod in the first reduction furnace image and the second pixel point position of the edge thereof;

[0336] The first determining module 23 is specifically used for:

[0337] The target silicon rod diameter of the target reduction furnace at the second moment is determined according to the first pixel point positions covered by the silicon rods and the second pixel point positions on the edge of the first reduction furnace in the image inside the first reduction furnace.

[0338] The device for determining the operating condition inside a reduction furnace provided in the embodiment of the present application can be used to execute the method for determining the operating condition inside a reduction furnace, that is, it has the beneficial effects and implementation methods of the method for determining the operating condition inside a reduction furnace provided in Example 1 of the present application. For details, please refer to the specific description of the method for determining the operating condition inside a reduction furnace in the above Example 1, which will not be repeated in this embodiment.

[0339] Example 4

[0340] The detection model training device provided in the embodiment of the present application can adopt the target detection model of any one of the embodiments 1, the target detection model includes at least one of the reduction furnace base detection model, the temperature detection model and the segmentation model, and the device includes:

[0341] A second acquisition module is used to acquire a first training set when the target detection model includes a reduction furnace base detection model, the first training set includes a plurality of first training samples, and the first training samples include historical reduction furnace in-furnace images and their corresponding historical detection frames;

[0342] A first training module is connected to the second acquisition module and is used to train the first preset model using a plurality of first training samples to obtain a reduction furnace base detection model; and / or,

[0343] A third acquisition module is used to acquire a second training set when the target detection model includes a temperature detection model, the second training set includes multiple second training samples, the second training samples include historical reduction furnace images, second furnace information and multiple different historical sequences, the second furnace information includes at least one of a second current value trend, a second voltage value trend and a second raw material flow rate trend of the reduction furnace before a third moment, and the historical sequence includes historical time, historical silicon rod temperature values ​​and historical ambient temperature values;

[0344] A second training module, connected to the third acquisition module, is used to train the second preset model with multiple second training samples to obtain a temperature detection model; and / or, a fourth acquisition module is used to obtain a third training set and a fourth training set when the target detection model includes a segmentation model, the third training set includes a target image and its corresponding target category label, the fourth training set includes multiple fourth training samples, and the third training sample includes an image inside the historical reduction furnace, and a fifth pixel point position covered in the image inside the historical reduction furnace and a sixth pixel point position on its edge;

[0345] A third training module, connected to the fourth acquisition module, is used to train the third preset model using a plurality of third training samples to obtain an initial segmentation model;

[0346] The fourth training module is used to connect with the third training module, and is used to train the initial segmentation model with a plurality of fourth training samples to obtain a segmentation model.

[0347] The detection model training device provided in the embodiment of the present application can be used to execute the detection model training method, that is, it has the beneficial effects and implementation methods of the detection model training method provided in Example 2 of the present application. For details, please refer to the specific description of the detection model training method in the above Example 2, which will not be repeated in this embodiment.

[0348] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A method for determining the operating conditions in a reduction furnace, characterized in that: include: Acquire a first reduction furnace interior image of the target reduction furnace at a first moment; Using a target detection model to detect the image inside the first reduction furnace to obtain a detection result, the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model, and a segmentation model; According to the detection result, a target furnace operating condition of the target reduction furnace at a second moment is determined, the target furnace operating condition includes at least one of a target atomization value, a target temperature value, and a target silicon rod diameter, the target furnace operating condition corresponds one-to-one to the target detection model, and the first moment is earlier than the second moment.

2. The method according to claim 1, characterized in that The target detection model includes a reduction furnace base detection model, and the detection result includes n target detection frames and n target confidences in the first reduction furnace in-furnace image, the target detection frames and target confidences correspond one to one, and n is a positive integer; The target furnace operating conditions include a target atomization value; The target detection model is used to detect the first reduction furnace image to obtain the detection result, including: Using the reduction furnace base detection model to detect the first reduction furnace interior image, and obtaining n target detection frames and n target confidences in the first reduction furnace interior image; Determining the target furnace operating condition of the target reduction furnace at the second moment according to the detection result includes: According to the n target detection frames and the n target confidences, a target atomization value of the target reduction furnace at a second moment is determined.

3. The method according to claim 2, characterized in that Determining a target atomization value of the target reduction furnace at a second moment according to the n target detection frames and the n target confidences, comprising: When the n target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the first atomization value; or, When only one target confidence value among the n target confidence values ​​is less than or equal to the first confidence threshold value and greater than the second confidence threshold value, and n-1 target confidence values ​​are all greater than the first confidence threshold value, the target atomization value of the target reduction furnace at the second moment is determined to be the second atomization value; In the case that only one of the n target confidences is less than the second confidence threshold, and the n-1 target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the third atomization value; or, In the case where two target confidences among the n target confidences are both less than or equal to the first confidence threshold, one of the two target confidences is less than the second confidence threshold, and n-2 target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the fourth atomization value; or, In the case where only two target confidences among the n target confidences are less than the second confidence threshold, and n-2 target confidences are all greater than the first confidence threshold, determining the target atomization value of the target reduction furnace at the second moment to be the fifth atomization value; or, By analogy, when there are c target confidences among the n target confidences that are all less than or equal to the first confidence threshold, c-1 of the c target confidences are all less than the second confidence threshold, and nc target confidences are all greater than the first confidence threshold, the target atomization value of the target reduction furnace at the second moment is determined to be the 2cth atomization value; or, When only c target confidences among n target confidences are less than the second confidence threshold, and nc target confidences are all greater than the first confidence threshold, the target atomization value of the target reduction furnace at the second moment is determined to be the 2c+1th atomization value, c is less than or equal to n, and c is a positive integer.

4. The method according to claim 2, characterized in that: The reduction furnace base detection model includes a first CNN feature extraction layer, a Transformer encoder, a Transformer decoder, and a feedforward fully connected layer connected in sequence; the size of the first reduction furnace image is W*H, where W represents the width of the first reduction furnace image, and H represents the height of the first reduction furnace image; The method of using the reduction furnace base detection model to detect the first reduction furnace interior image to obtain n target detection frames and n target confidences in the first reduction furnace interior image includes: The first CNN feature extraction layer is used to perform feature extraction on the image in the first reduction furnace to obtain 128a first-category feature maps, where the size of the first-category feature map is W / 32*H / 32, and a is a positive integer; The Transformer encoder is used to process 128a first-class feature maps to obtain second-class feature maps, and the size of the second-class feature maps is WH / 1024*256; The Transformer decoder is used to solve the second type of feature map to obtain p first feature vectors, where p is a positive integer; The feedforward fully connected layer is used to process the p first feature vectors to obtain n target detection frames and n target confidences in the first reduction furnace interior image.

5. The method according to claim 4, characterized in that The first CNN feature extraction layer includes a first convolution layer, a first normalization layer, a first customization module, a first downsampling layer, a second customization module, a second downsampling layer, a third customization module, a third downsampling layer, a fourth customization layer, a global average pooling layer and a second normalization layer connected in sequence; The first CNN feature extraction layer is used to extract features from the image in the first reduction furnace to obtain m first-category feature maps, including: The first convolutional layer is used to extract features of the first reduction furnace image to obtain a first feature map; A first normalization layer is used to normalize a first feature map to obtain a second feature map; The first customized module is used to extract features from a second feature maps to obtain 4a third feature maps; The first downsampling layer is used to compress the size of the 4a third feature maps to obtain 4a fourth feature maps; The second customized module is used to extract features from 4a fourth feature maps to obtain 16a fifth feature maps; The second downsampling layer is used to compress the size of the 16a fifth feature maps to obtain 16a sixth feature maps; The third customized module is used to extract features from 16a sixth feature maps to obtain 64a seventh feature maps; The third downsampling layer is used to compress the size of 64a seventh feature maps to obtain 64a eighth feature maps; The fourth customized module is used to extract features from 64a eighth feature maps to obtain 128a ninth feature maps; A global average pooling layer is used to compress the size of 128a ninth feature maps to obtain 128a tenth feature maps; The second normalization layer is used to normalize the 128a tenth feature maps to obtain 128a first-category feature maps.

6. The method according to claim 5, characterized in that The target customized module includes at least one of a first customized module, a second customized module, a third customized module and a fourth customized module; The target customized module has a third normalization layer, a second convolutional layer, a fourth normalization layer, a third convolutional layer, a GELU activation function, a fourth convolutional layer and a fifth normalization layer connected in sequence.

7. The method according to claim 4, characterized in that The Transformer encoder includes a multi-head self-attention mechanism and a feedforward neural network. The Transformer encoder is used to process 128a first-category feature maps to obtain second-category feature maps, including: The multi-head self-attention mechanism is used to compress 128a first-category feature maps to obtain 16a first compressed feature maps; Using the feedforward neural network to planarize and reorganize the 16a first compressed feature maps to obtain a first reorganized feature map; Use the feedforward neural network to mark the positions of b feature elements in the first reorganized feature map to obtain b position codes; The b position codes and the first reorganized feature map are combined into a second type feature map.

8. The method according to claim 4, characterized in that The Transformer decoder includes a multi-head self-attention mechanism and a multi-head cross-attention mechanism. The Transformer decoder is used to solve the second type of feature map to obtain p first feature vectors, including: A multi-head self-attention mechanism is used to solve b Q vectors and b K vectors corresponding to b feature elements in the second type of feature graph; For each Q vector of the b characteristic elements, solve the b first correlation values ​​corresponding to the b K vectors one by one to obtain b*b first correlation values; The b*b first correlation values ​​are encoded according to b positions to form p first eigenvectors.

9. The method according to claim 4, characterized in that The feed-forward fully connected layer includes a first linear layer and a first softmax layer, The p first feature vectors are processed by the feedforward fully connected layer to obtain n target detection frames and n target confidences in the first reduction furnace image, including: Using the first linear layer to compress the p first eigenvectors to obtain p first single point values, and combining the p first single point values ​​into a second eigenvector; The first softmax layer is used to map the second feature vector into n target detection boxes and n target confidences in the first reduction furnace image.

10. The method according to claim 1, characterized in that The target detection model includes a temperature detection model, the detection result includes a plurality of different first detection sequences, and the target furnace operating condition includes a target temperature value; The target detection model is used to detect the first reduction furnace image to obtain the detection result, including: Using a temperature detection model to detect the first reduction furnace image and first furnace information to obtain a plurality of different first detection sequences, wherein the first furnace information includes at least one of a first current value trend, a first voltage value trend, and a first raw material flow rate trend of the target reduction furnace before a first moment; Determining the target furnace operating condition of the target reduction furnace at the second moment according to the detection result includes: According to a plurality of different first detection sequences, a target temperature value of the target reduction furnace at a second moment is determined.

11. The method according to claim 10, characterized in that The first detection sequence includes a predicted time, a predicted silicon rod temperature value, and a predicted ambient temperature value, and the target temperature value includes at least one of a target silicon rod temperature value and a target ambient temperature value; The step of determining the target temperature value of the target reduction furnace at the second moment according to a plurality of different first detection sequences includes: Determine at least two detection sequences with higher first confidences among the multiple different first detection sequences as at least two second detection sequences; The predicted silicon rod temperature value corresponding to the predicted time closest to the first moment in at least two second detection sequences is determined as the target silicon rod temperature value, and / or the predicted ambient temperature value corresponding to the predicted time closest to the first moment in at least two second detection sequences is determined as the target ambient temperature value.

12. The method according to claim 10, characterized in that The temperature detection model includes a sparse autoencoder structure, which includes a first input layer, a first hidden layer and a first output layer connected in sequence; the first input layer includes a convolution substructure, the first hidden layer includes a sparse substructure, and the first output layer includes a decoding substructure; The temperature detection model is used to detect the first reduction furnace image and the first furnace information to obtain a plurality of different first detection sequences, including: The convolution substructure is used to transform the first reduction furnace in-furnace image and the first furnace in-furnace information into a plurality of third-category feature maps; Using the sparse substructure to perform sparsity constraints on multiple third-category feature maps to obtain multiple fourth-category feature maps; The decoding substructure is used to decode multiple fourth-category feature maps to obtain multiple different first detection sequences.

13. The method according to claim 1, characterized in that The target detection model includes a segmentation model, the detection result includes a first pixel point position covered by the silicon rod in the first reduction furnace image and a second pixel point position on its edge, and the target furnace working condition includes a target silicon rod diameter; The target detection model is used to detect the image inside the first reduction furnace to obtain the detection results, including: The first reduction furnace image is detected using a segmentation model to obtain first pixel positions covered by silicon rods and second pixel positions at its edge in the first reduction furnace image; Determining the target furnace operating condition of the target reduction furnace at the second moment according to the detection result includes: The target silicon rod diameter of the target reduction furnace at the second moment is determined according to the first pixel point positions covered by the silicon rods and the second pixel point positions on the edge of the first reduction furnace in the image inside the first reduction furnace.

14. The method according to claim 13, characterized in that Determining the target silicon rod diameter of the target reduction furnace at the second moment according to the first pixel point position covered by the silicon rod in the first reduction furnace image and the second pixel point position of the edge thereof, comprising: Determine the pixel diameter of the silicon rod in the first reduction furnace image according to the first pixel position covered by the silicon rod and the second pixel position of the edge thereof in the first reduction furnace image; When the type of the silicon rod is the first type, determining a target silicon rod diameter of the silicon rod according to a first mapping relationship and the pixel diameter, wherein the first mapping relationship includes a one-to-one mapping relationship between a plurality of preset pixel diameters and a plurality of preset silicon rod diameters; Alternatively, when the type of the silicon rod is the second type, the target actual spacing is determined according to a second mapping relationship and the pixel diameter, and the target silicon rod diameter of the silicon rod is determined according to the target actual spacing, wherein the second mapping relationship includes a one-to-one mapping relationship between multiple preset pixel diameters and multiple preset actual spacings.

15. The method according to claim 13, characterized in that The segmentation model includes an image encoder, a hint encoder, and a mask decoder; The segmentation model is used to detect the first reduction furnace image to obtain the first pixel position covered by the silicon rod in the first reduction furnace image and the second pixel position of the edge thereof, including: Processing the first reduction furnace interior image using the image encoder to obtain a first image embedding; Encoding the first type of information using a hint encoder to obtain a first embedding vector, wherein the first type of information includes at least one of a background point, a mask, and a bounding box; A mask decoder is used to predict and segment the mask of the first image embedding and the first embedding vector, so as to obtain the first pixel point position covered by the silicon rod in the first reduction furnace image and the second pixel point position of the edge thereof.

16. The method according to claim 15, characterized in that The image encoder includes a connected mask encoder and a visual transformer, and the visual transformer includes a second CNN feature extraction layer of a CSP-Darknet structure; The image encoder is used to process the first reduction furnace image to obtain a first image embedding, including: The mask code is used to cover part of the information of the first reduction furnace interior image to obtain the second reduction furnace interior image; The second CNN feature extraction layer is used to extract features from the image in the second reduction furnace to obtain a fifth type of feature map; The visual Transformer is used to reconstruct the features in the fifth type of feature map to obtain a first image embedding.

17. The method according to claim 15, characterized in that The segmentation model further includes a verification module, and the verification module is connected to the image encoder and the mask decoder respectively; After the mask decoder is used to embed the first image and the first embedding vector to predict the segmentation mask and obtain the first pixel point position covered by the silicon rod in the first reduction furnace image and the second pixel point position of the edge thereof, the method further includes: The verification module is used to convert the first reduction furnace interior image into the HSV space, and a binary demarcation point is determined in the HSV space; Based on the verification module, the image inside the first reduction furnace is binarized using the binarization demarcation point to obtain a binarized image; Based on the verification module, an edge extraction operator is used to extract the edge of the binary image to obtain edge features; The verification module is used to embed the first image and fuse the edge features, and the first rule is used to retain the second edge features with higher confidence, so as to obtain the third pixel point position covered by the silicon rod in the first reduction furnace image and the fourth pixel point position of the edge thereof; The third pixel position covered by the silicon rod in the first reduction furnace image and the fourth pixel position on its edge are respectively used as the first pixel position covered by the silicon rod in the first reduction furnace image and the second pixel position on its edge.

18. A detection model training method, characterized in that: Using the target detection model according to any one of claims 1 to 17; the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model and a segmentation model, the method includes: In a case where the target detection model includes a reduction furnace base detection model, a first training set is obtained, wherein the first training set includes a plurality of first training samples, and the first training samples include historical reduction furnace in-furnace images and their corresponding historical detection frames; Using a plurality of first training samples to train a first preset model to obtain a reduction furnace base detection model; and / or, In the case where the target detection model includes a temperature detection model, a second training set is obtained, wherein the second training set includes a plurality of second training samples, wherein the second training samples include a historical reduction furnace image, second furnace information, and a plurality of different historical sequences, wherein the second furnace information includes at least one of a second current value trend, a second voltage value trend, and a second raw material flow rate trend of the reduction furnace before a third moment, and the historical sequence includes historical time, historical silicon rod temperature values, and historical ambient temperature values; Using a plurality of second training samples to train the second preset model to obtain a temperature detection model; And / or, in the case where the target detection model includes a segmentation model, a third training set and a fourth training set are obtained, wherein the third training set includes a target image and its corresponding target category label, and the fourth training set includes a plurality of fourth training samples, and the third training sample includes an image inside a historical reduction furnace, and a fifth pixel point position covered in the image inside the historical reduction furnace and a sixth pixel point position on its edge; Using a plurality of third training samples to train the third preset model to obtain an initial segmentation model; The initial segmentation model is trained using a plurality of fourth training samples to obtain a segmentation model.

19. A device for determining the working condition in a reduction furnace, characterized in that: include: A first acquisition module is used to acquire an image of a first reduction furnace in a target reduction furnace at a first moment; A first detection module, connected to the first acquisition module, for detecting the first reduction furnace image using a target detection model to obtain a detection result, wherein the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model, and a segmentation model; A first determination module is connected to the first detection module, and is used to determine a target furnace operating condition of the target reduction furnace at a second moment based on the detection result, wherein the target furnace operating condition includes at least one of a target atomization value, a target temperature value, and a target silicon rod diameter, and the target furnace operating condition corresponds to the target detection model one by one, and the first moment is earlier than the second moment.

20. The device according to claim 19, characterized in that The target detection model includes a reduction furnace base detection model, the detection result includes n target detection frames and n target confidences in the first reduction furnace in-furnace image, the target detection frames and target confidences correspond one to one, and n is a positive integer; the target furnace working condition includes a target atomization value; The first detection module is specifically used for: Using the reduction furnace base detection model to detect the first reduction furnace interior image, and obtaining n target detection frames and n target confidences in the first reduction furnace interior image; The first determining module is specifically configured to: According to the n target detection frames and the n target confidences, a target atomization value of the target reduction furnace at a second moment is determined.

21. The device according to claim 19, characterized in that The target detection model includes a temperature detection model, the detection result includes a plurality of different first detection sequences, and the target furnace operating condition includes a target temperature value; The first detection module is specifically used for: Using a temperature detection model to detect the first reduction furnace image and first furnace information to obtain a plurality of different first detection sequences, wherein the first furnace information includes at least one of a first current value trend, a first voltage value trend, and a first raw material flow rate trend of the target reduction furnace before a first moment; The first determining module is specifically configured to: According to a plurality of different first detection sequences, a target temperature value of the target reduction furnace at a second moment is determined.

22. The device according to claim 19, characterized in that The target detection model includes a segmentation model, the detection result includes a first pixel point position covered by the silicon rod in the first reduction furnace image and a second pixel point position on its edge, and the target furnace working condition includes a target silicon rod diameter; The first detection module is specifically used for: The first reduction furnace image is detected using a segmentation model to obtain first pixel positions covered by silicon rods and second pixel positions at its edge in the first reduction furnace image; The first determining module is specifically configured to: The target silicon rod diameter of the target reduction furnace at the second moment is determined according to the first pixel point positions covered by the silicon rods and the second pixel point positions on the edge of the first reduction furnace in the image inside the first reduction furnace.

23. A detection model training device, characterized in that: The target detection model according to any one of claims 1 to 17 is adopted; the target detection model includes at least one of a reduction furnace base detection model, a temperature detection model and a segmentation model, and the device includes: A second acquisition module is used to acquire a first training set when the target detection model includes a reduction furnace base detection model, wherein the first training set includes a plurality of first training samples, and the first training samples include historical reduction furnace in-furnace images and their corresponding historical detection frames; A first training module, connected to the second acquisition module, is used to train the first preset model using a plurality of first training samples to obtain a reduction furnace base detection model; and / or, A third acquisition module is used to acquire a second training set when the target detection model includes a temperature detection model, the second training set includes a plurality of second training samples, the second training samples include a historical reduction furnace image, second furnace information and a plurality of different historical sequences, the second furnace information includes at least one of a second current value trend, a second voltage value trend and a second raw material flow rate trend of the reduction furnace before a third moment, and the historical sequence includes a historical time, a historical silicon rod temperature value and a historical ambient temperature value; A second training module, connected to the third acquisition module, is used to train the second preset model using a plurality of second training samples to obtain a temperature detection model; and / or, a fourth acquisition module, for acquiring, when the target detection model includes a segmentation model, a third training set and a fourth training set, wherein the third training set includes a target image and its corresponding target category label, and the fourth training set includes a plurality of fourth training samples, and the third training samples include an image in a historical reduction furnace, and a fifth pixel position covered in the image in the historical reduction furnace and a sixth pixel position on its edge; A third training module, connected to the fourth acquisition module, for training the third preset model with a plurality of third training samples to obtain an initial segmentation model; The fourth training module is used to connect with the third training module and to train the initial segmentation model using a plurality of fourth training samples to obtain a segmentation model.

Images