A method and system for auxiliary process of molten iron temperature measurement based on deep learning

By constructing a deep learning model to determine the appearance of temperature measurers in the video, wearing protective devices, postures and distances, the problem of inaccurate judgment of the temperature measurement process in the casting process of the cast iron factory is solved, and safer and more accurate measurement of the iron temperature is achieved. Reduced product failure rate.

CN113901886BActive Publication Date: 2024-11-08WUHAN UNIV OF TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111085954.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-16
Publication Date
2024-11-08
Estimated Expiration
2041-09-16

AI Technical Summary

Technical Problem

In the prior art, the temperature measurement process judgment method in the casting process of the casting iron plant is not accurate and comprehensive enough, which may lead to casualties of operators or defects in casting products.

Method used

Using the auxiliary method of molten iron temperature measurement process based on deep learning, four deep learning models are constructed to determine whether the temperature measuring personnel appear in the video, whether they wear protective devices correctly, whether the temperature measurement posture is qualified, and the distance between the body part and the molten iron furnace. Use the camera to obtain video information in real time for judgment and issue alarm information.

Benefits of technology

It improves the accuracy and safety of the temperature measurement process, avoiding operator casualties and unqualified casting products caused by incorrect measurements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113901886B_ABST
    Figure CN113901886B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for assisting the molten iron temperature measurement process based on deep learning, which can monitor in real time whether the operator's position is safe, whether the protection is in place, and whether the temperature measurement action is qualified, and issue an alarm sound prompt when it is unsafe or the action is unqualified, thereby avoiding risks. It can effectively assist the temperature measurement operator to safely and correctly measure the molten iron temperature, thereby effectively avoiding operator casualties caused by incorrect measurement and the unqualified rate of cast products caused by inaccurate temperature measurement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of intersection between deep learning and molten iron temperature measurement, and in particular to a molten iron temperature measurement process auxiliary method and system based on deep learning. Background Art

[0002] In the prior art, before the casting process of the iron foundry is carried out, the operator needs to hold a temperature gun, open the cover of the pouring molten iron furnace, and move the temperature gun in the correct posture to a certain distance from the surface of the molten iron to measure the temperature of the molten iron. Only when the temperature is qualified (within a certain error of 1500℃) can the subsequent processes be carried out.

[0003] If the temperature measurement is not carried out in time, the construction period may be delayed. If the operator's posture is improper, such as the elbows and other parts of the body are too close to the molten iron surface, or if the protective equipment is not worn correctly, it may cause serious casualties such as burns. In addition, if the operator's posture is improper or the handheld temperature gun is too close or too far away from the molten iron, the measured molten iron temperature will not match the actual molten iron temperature, resulting in defects in the cast product. If the actual molten iron temperature is too high, it is easy to cause defects such as thermal cracking, shrinkage holes, local oxidation, and unqualified dimensions; if the actual molten iron temperature is too low, it is easy to cause defects such as insufficient casting, cold shut, slag inclusion, sand inclusion, etc. Summary of the invention

[0004] The present invention proposes a deep learning-based auxiliary method and system for molten iron temperature measurement process, which is used to solve or at least partially solve the problem that the temperature measurement process judgment method in the prior art is not accurate and comprehensive enough.

[0005] In order to solve the above technical problems, the first aspect of the present invention provides a molten iron temperature measurement process auxiliary method based on deep learning, comprising:

[0006] Four deep learning models are pre-built, among which the first deep learning model is used to determine whether the temperature measurement personnel appear in the video image, the second deep learning model is used to determine whether the temperature measurement personnel are correctly wearing protective equipment of the specified color, the third deep learning model is used to determine whether the temperature measurement posture of the temperature measurement personnel is qualified, and the fourth deep learning model is used to predict the distance between the body part of the temperature measurement personnel and the molten iron furnace;

[0007] A large number of videos of temperature measurement actions of personnel measuring the temperature of molten iron in the cast iron pouring process are obtained. The images in the temperature measurement action videos include features corresponding to qualified temperature measurement processes and features corresponding to unqualified temperature measurement processes. Among them, the features corresponding to qualified temperature measurement processes include that the temperature measurement personnel appear in the video, correctly wear the specified color protective clothing, have a correct temperature measurement posture, and are within a safe distance. The features of unqualified temperature measurement processes include that the temperature measurement personnel do not appear in the video, do not correctly wear the specified color protective clothing, have an incorrect temperature measurement posture, and are not within a safe distance. After annotating the images according to the characteristics of each deep learning model, a corresponding training data set is constructed, and each deep learning model is trained using the corresponding training data set;

[0008] Before measuring the temperature of molten iron, a camera is used to obtain the temperature measurement video information of the current temperature measurement personnel in real time, and the trained first deep learning training model is used to determine whether the temperature measurement personnel appears. If they do not appear during the current temperature measurement period, an alarm message is issued. If they appear during the current temperature measurement period, the image of the video information is judged by the second deep learning model, the third deep learning model and the fourth deep learning model. If the judgment result is that the temperature measurement personnel does not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, or the temperature measurement personnel is not within the safe distance, an alarm message is issued.

[0009] In one embodiment, before issuing the alarm information, the method further includes:

[0010] Save the temperature measurement action video, judgment results and time data.

[0011] In one embodiment, the first deep learning model, the second deep learning model and the third deep learning model are classification models, including convolutional layers, pooling layers, batch normalization layers, residual network blocks, fully connected layers and softmax layers, wherein the convolutional layers, pooling layers, batch normalization layers and residual network blocks are used for feature extraction, the fully connected layers are used to output probability values ​​of two categories, and the softmax layer is used to output the final predicted classification results.

[0012] In one embodiment, when the first deep learning model, the second deep learning model, and the third deep learning model are trained using the corresponding training data sets, the loss function adopts the cross entropy loss function, and is trained using an Adam optimizer with momentum and a strategy in which the learning rate decays exponentially with the number of training times.

[0013] In one embodiment, the fourth deep learning model includes a convolutional layer, a pooling layer, a batch normalization layer, a residual network block, two fully connected layers and a ReLU activation function. The convolutional layer, the pooling layer, the batch normalization layer, and the residual network block are used to extract relevant features of the distance between the furnace body contour and the hand contour of the temperature measurement personnel. The fully connected layer is used to output the regressed distance, and the ReLU activation function of the fully connected layer is used to ensure that the output distance value is a value greater than zero.

[0014] In one embodiment, when the fourth deep learning model is trained using the corresponding training data set, the loss function uses a mean square error function.

[0015] Based on the same inventive concept, the second aspect of the present invention provides a molten iron temperature measurement process auxiliary system based on deep learning, comprising:

[0016] A model building module pre-builds four deep learning models, wherein the first deep learning model is used to determine whether the temperature measuring personnel appear in the video image, the second deep learning model is used to determine whether the temperature measuring personnel correctly wears the protective device of the specified color, the third deep learning model is used to determine whether the temperature measuring personnel's temperature measuring posture is qualified, and the fourth deep learning model is used to predict the distance between the temperature measuring personnel's body parts and the molten iron furnace;

[0017] The model training module obtains a large number of temperature measurement action videos of molten iron temperature measurement personnel in the cast iron pouring process. The images in the temperature measurement action videos include features corresponding to qualified temperature measurement processes and features corresponding to unqualified temperature measurement processes. Among them, the features corresponding to qualified temperature measurement processes include that the temperature measurement personnel appear in the video, correctly wear the specified color protective clothing, have a correct temperature measurement posture, and are within a safe distance. The features of unqualified temperature measurement processes include that the temperature measurement personnel do not appear in the video, do not correctly wear the specified color protective clothing, have an incorrect temperature measurement posture, and are not within a safe distance. After annotating the images according to the characteristics of each deep learning model, a corresponding training data set is constructed, and each deep learning model is trained using the corresponding training data set;

[0018] The temperature measurement process auxiliary module is used to use the camera to obtain the current temperature measurement video information of the temperature measurement personnel in real time before measuring the temperature of the molten iron, and use the trained first deep learning training model to determine whether the temperature measurement personnel appears. If they do not appear during the current temperature measurement period, an alarm message is issued. If they appear during the current temperature measurement period, the image of the video information is judged by the second deep learning model, the third deep learning model and the fourth deep learning model. If the judgment result is that the temperature measurement personnel does not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, or the temperature measurement personnel is not within the safe distance, an alarm message is issued.

[0019] The above one or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects:

[0020] The present invention provides a method for assisting the molten iron temperature measurement process based on deep learning, pre-constructing four deep learning models, and constructing corresponding training data sets to train the four deep learning models, the first deep learning model is used to determine whether the temperature measurement personnel appear in the video image (i.e., whether the temperature measurement personnel appear in the specified position), the second deep learning model is used to determine whether the temperature measurement personnel correctly wear the protective device of the specified color, the third deep learning model is used to determine whether the temperature measurement posture of the temperature measurement personnel is qualified, and the fourth deep learning model is used to predict the distance between the body part of the temperature measurement personnel and the molten iron furnace (if the distance is less than the threshold, it means that it is within the safe distance); it can improve the accuracy of the measurement method judgment, and effectively assist the temperature measurement personnel of the casting process to measure the molten iron temperature in a timely, safe and correct manner. In the actual production process, it can avoid operator casualties caused by incorrect measurement, and effectively reduce the unqualified rate of cast products caused by inaccurate temperature measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flow chart of a molten iron temperature measurement process auxiliary method based on deep learning in an embodiment of the present invention;

[0023] Figure 2 is a training flow chart of the first deep learning model in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the correct temperature measurement action, protective device and safe distance from the furnace body in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention provides a molten iron temperature measurement process auxiliary method based on deep learning, which is used to improve the problem of inaccurate measurement method existing in the method in the prior art.

[0026] The main ideas of the present invention are as follows:

[0027] Four deep learning models are pre-built, and corresponding training data sets are constructed to train the four deep learning models. The first deep learning model is used to determine whether the temperature measurement personnel appears in the video image (that is, whether the temperature measurement personnel appears at the designated location). The second deep learning model is used to determine whether the temperature measurement personnel correctly wears the protective device of the specified color. The third deep learning model is used to determine whether the temperature measurement posture of the temperature measurement personnel is qualified. The fourth deep learning model is used to predict the distance between the body part of the temperature measurement personnel and the molten iron furnace (if the distance is greater than the threshold, it means it is within a safe distance). Before measuring the temperature of molten iron, the camera is used to obtain the current temperature measurement video information of the temperature measurement personnel in real time, and the trained first deep learning training model is used to determine whether the temperature measurement personnel appear. If the judgment result is that the temperature measurement personnel do not appear during this temperature measurement period, an alarm message is issued, otherwise the second deep learning model, the third deep learning model and the fourth deep learning model are loaded to judge the image of the video information. If the judgment result is that the temperature measurement personnel do not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, and the temperature measurement personnel are not within the safe distance, an alarm message is issued, which can solve the problem that the judgment method of the iron temperature measurement process is not accurate and comprehensive enough, and effectively assist the temperature measurement personnel of the casting process to measure the molten iron temperature in a timely, safe and correct manner. In the actual production process, it can avoid operator casualties caused by incorrect measurement, and effectively reduce the unqualified rate of cast products caused by inaccurate temperature measurement.

[0028] The key points of the present invention include:

[0029] (1) The process auxiliary method provided is composed of four parallel network models that respectively complete different prediction functions, which can make the method more functional;

[0030] (2) The feature extraction layers of the four deep learning network models are similar. The residual network block is added to ensure that the generalization performance is stable while the fitting ability becomes stronger as the network layers become deeper. At the same time, the batch normalization layer is added to make the training process more stable and more accurate.

[0031] (3) The gradient update method uses the Adam optimizer and adds a strategy in which the learning rate gradually decreases during the training process, which makes it easier for the network model to converge to the global optimum and provide more accurate predictions.

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Embodiment 1

[0034] The embodiment of the present invention provides a molten iron temperature measurement process auxiliary method based on deep learning, comprising:

[0035] Four deep learning models are pre-built, among which the first deep learning model is used to determine whether the temperature measurement personnel appear in the video image, the second deep learning model is used to determine whether the temperature measurement personnel are correctly wearing protective equipment of the specified color, the third deep learning model is used to determine whether the temperature measurement posture of the temperature measurement personnel is qualified, and the fourth deep learning model is used to predict the distance between the body part of the temperature measurement personnel and the molten iron furnace;

[0036] A large number of videos of temperature measurement actions of personnel measuring the temperature of molten iron in the cast iron pouring process are obtained. The images in the temperature measurement action videos include features corresponding to qualified temperature measurement processes and features corresponding to unqualified temperature measurement processes. Among them, the features corresponding to qualified temperature measurement processes include that the temperature measurement personnel appear in the video, correctly wear the specified color protective clothing, have a correct temperature measurement posture, and are within a safe distance. The features of unqualified temperature measurement processes include that the temperature measurement personnel do not appear in the video, do not correctly wear the specified color protective clothing, have an incorrect temperature measurement posture, and are not within a safe distance. After annotating the images according to the characteristics of each deep learning model, a corresponding training data set is constructed, and each deep learning model is trained using the corresponding training data set;

[0037] Before measuring the temperature of molten iron, a camera is used to obtain the temperature measurement video information of the current temperature measurement personnel in real time, and the trained first deep learning training model is used to determine whether the temperature measurement personnel appears. If they do not appear during the current temperature measurement period, an alarm message is issued. If they appear during the current temperature measurement period, the image of the video information is judged by the second deep learning model, the third deep learning model and the fourth deep learning model. If the judgment result is that the temperature measurement personnel does not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, or the temperature measurement personnel is not within the safe distance, an alarm message is issued.

[0038] In the specific implementation process, a large number of videos of temperature measurement actions of personnel measuring the temperature of molten iron in the cast iron pouring process are firstly taken with a camera. The camera for shooting needs to be fixed at a pre-set position to obtain a clear and complete temperature measurement action picture, with the field of view including the furnace outline and the complete temperature measurement station. The pixel should not be less than the set value (such as 256*256).

[0039] Four deep learning models with the same input and different outputs are trained using the captured temperature measurement action video. (1) The first deep learning model is used to determine whether the temperature measurement process is carried out in a timely manner. The judgment method is to use deep learning model 1 (the first deep learning model) to classify and determine in real time whether the temperature measurement personnel appear in the video image. If the temperature measurement personnel are not detected in the video image within the specified temperature measurement time period, it means that the temperature measurement action is not carried out in a timely manner, and an alarm is issued; (2) Deep learning model 2 (the second deep learning model) is used to determine whether the temperature measurement personnel are wearing the specified protective equipment correctly. Specifically, the model gives a classification prediction based on the color and texture of the clothes and hats it extracts, whether they are wearing correctly or not. If they are not wearing correctly, a prompt is given to protect the staff; (3) Deep learning model 3 (the third deep learning model) is used to determine whether the temperature measurement posture of the temperature measurement personnel is qualified. Specifically, the model automatically extracts the angle features of the contour lines of the temperature measurement personnel's arms, legs, etc. to determine whether the posture is correct. (4) Deep learning model 4 (the fourth deep learning model) regresses and predicts the shortest distance between the furnace and the person based on the relevant features such as the contour lines of the furnace and the person in the picture.

[0040] See also Figure 1 , which is a flow chart of a molten iron temperature measurement process auxiliary method based on deep learning in an embodiment of the present invention.

[0041] Among them, model 1 is responsible for classifying and predicting whether the temperature measurement personnel appear in the input image, and the output results are 0 (not appearing) and 1 (appearing in the video); only when the prediction result of deep learning model 1 is 1, that is, someone appears, the other three models will work. The second model is responsible for classifying and predicting whether the temperature measurement personnel are wearing the specified color protective device correctly, and the output results are 0 (not wearing the specified color protective device correctly) and 1 (wearing correctly); the third model is responsible for classifying and predicting whether the temperature measurement posture of the temperature measurement personnel is qualified, and the output results are 0 (the actual temperature measurement posture does not meet the action annotation as described in claim 2) and 1 (the actual temperature measurement posture is qualified); the fourth model is responsible for regression prediction of the shortest distance between the outline of the temperature measurement personnel close to the furnace body and the furnace body, and outputs a value in meters (should not be greater than the safety distance 0.3). The normal operation of this model requires that the camera position and the furnace body position remain relatively fixed during the training phase and the actual use phase.

[0042] In one embodiment, before issuing the alarm information, the method further includes:

[0043] Save the temperature measurement action video, judgment results and time data.

[0044] During the specific implementation process, the temperature measurement action video, analysis results, time and other data will be saved for subsequent review.

[0045] In one embodiment, the first deep learning model, the second deep learning model and the third deep learning model are classification models, including convolutional layers, pooling layers, batch normalization layers, residual network blocks, fully connected layers and softmax layers, wherein the convolutional layers, pooling layers, batch normalization layers and residual network blocks are used for feature extraction, the fully connected layers are used to output probability values ​​of two categories, and the softmax layer is used to output the final predicted classification results.

[0046] In one embodiment, when the first deep learning model, the second deep learning model, and the third deep learning model are trained using the corresponding training data sets, the loss function adopts the cross entropy loss function, and is trained using an Adam optimizer with momentum and a strategy in which the learning rate decays exponentially with the number of training times.

[0047] Specifically, based on the pytorch framework, a deep convolution classification neural network can be built by using convolution layers, pooling layers, and fully connected layers alternately as the basic framework. In addition, the introduction of residual network blocks and batch normalization layers can effectively improve the fitting ability and generalization ability of the deep network and effectively improve the prediction accuracy. Each network model input is a picture with a pixel value of 3*256*256. The previous convolution layer, pooling layer, and repeated residual network blocks are used to extract features in the input picture that are useful for the final prediction. For example, in the model predicting whether the operator appears, the network will automatically extract the outline and color features of the person; the model predicting whether the operator wears the specified protective device correctly will automatically extract the color and skin texture features of the clothes; the model predicting whether the posture of the temperature measurement personnel is correct will automatically extract the contour angle features of the temperature measurement personnel's arms, legs, etc. These extracted features will pass through the second-to-last fully connected layer and then output the probability values ​​of the two categories, and then a softmax layer will output the final prediction classification results. The error function uses the cross entropy loss function, which is better in classification prediction. By training the convolutional neural network with the Adam optimizer with momentum and the strategy of exponential decay of the learning rate with the number of training times, faster convergence speed and higher accuracy can be achieved.

[0048] See also Figure 2 , which is a training flowchart of the first deep learning model in an embodiment of the present invention (the training process of the other three models is similar to that of model 1).

[0049] A certain amount of video is shot, including the correct temperature measurement process (the temperature measurement personnel appear in the video, wear the specified color protective clothing correctly, are at a safe distance, and have the correct temperature measurement posture) and some unqualified temperature measurement processes (such as not appearing in the video, not wearing the protective clothing correctly, being at a dangerous distance from the furnace, and having an unqualified temperature measurement posture). The real labels corresponding to the images are manually added to the video frame by frame (for example, if the temperature measurement personnel appears, the label is 1, and if not, it is 0; if the temperature measurement personnel wears the specified color protective clothing, the label is 1, and if it is not worn correctly, it is 0). The images are processed into a certain pixel size of 3*256*256 and the corresponding correct labels as a data set and randomly sorted and divided into three parts of 60%, 20%, and 20%, respectively, as a training data set, a cross-validation data set, and a test data set.

[0050] See also Figure 3 , which is a schematic diagram of the correct temperature measurement action, protective device and safe distance from the furnace body in an embodiment of the present invention.

[0051] The temperature measuring personnel must wear designated blue protective work clothes and a yellow safety helmet and stand at a designated position 0.5 meters away from the molten iron furnace, facing the furnace body. After opening the furnace cover, bend down and hold the temperature gun to the designated position 0.3 meters away from the molten iron surface. The elbows and other parts of the body must not be too close to the furnace body or the molten iron surface. Specifically, the safety distance should be no less than 0.3 meters.

[0052] When building the network, multiple sets of hyperparameters are set, such as the number of convolutional layers, the size of the convolution kernel, etc. The training data set is input into the deep model for training to obtain the model parameters with the highest detection accuracy corresponding to each set of hyperparameters. Then, the cross-validation set is used to input the trained models, and the model with the highest detection accuracy is selected as the final model, and its accuracy is verified using the test data set.

[0053] In one embodiment, the fourth deep learning model includes a convolutional layer, a pooling layer, a batch normalization layer, a residual network block, two fully connected layers and a ReLU activation function. The convolutional layer, the pooling layer, the batch normalization layer, and the residual network block are used to extract relevant features of the distance between the furnace body contour and the hand contour of the temperature measurement personnel. The fully connected layer is used to output the regressed distance, and the ReLU activation function of the fully connected layer is used to ensure that the output distance value is a value greater than zero.

[0054] In one embodiment, when the fourth deep learning model is trained using the corresponding training data set, the loss function uses a mean square error function.

[0055] Specifically, the fourth model is responsible for regressing and predicting the shortest distance between the outline of the temperature measurement personnel close to the furnace body and the furnace body, and outputting a value in meters (which should be no less than the safety distance of 0.3). The normal operation of this model requires that the camera position and the furnace position remain relatively fixed during the training phase and the actual use phase.

[0056] Model 4 implements regression prediction of the minimum distance value between the furnace body and the temperature measurement personnel's body. Its input is the same as the first three classification models. The first few layers used to extract image features are similar to the classification models, including (convolution, pooling, batch normalization layer and residual network block). These layers can automatically extract the relevant features of the distance between the furnace body contour and the temperature measurement personnel's hand contour. After extracting the features, the processing of the last few layers of the network is different. The model adds two fully connected layers to the features extracted by the convolution layer, and then uses the ReLU (rectified linear unit) activation function to ensure that the output distance value is a value greater than zero. The loss function uses the MSE (mean square error) function, which can effectively evaluate the deviation between the distance prediction value given by the model and the actual distance value for back propagation. The label when predicting the distance is the actual distance value. When the camera position and the furnace position are relatively fixed during the training stage and the actual use stage, the actual distance label can be obtained by manually calculating the pixel points on the picture and comparing them with the actual distance. Other aspects such as setting multiple sets of hyperparameter models, training with training sets, cross-validation set comparison and selection, and testing results with test sets are similar to the first three classification models.

[0057] In the specific implementation process, after the model is trained, the model and parameters are saved and imported into the industrial computer connected to the shooting camera. The performance of the industrial computer determines the number of image frames that can be processed in real time per unit time. A human-computer interactive graphical interface program is written to view camera images, replay the execution of the temperature measurement process within a period of time, set the start and stop of the temperature measurement auxiliary method, set the temperature measurement process time, etc., and execute it on the industrial computer.

[0058] When the temperature measurement process is enabled, the camera takes a real-time video of the temperature measurement operator's actions and transmits it to the industrial computer to import four deep learning models. First, model 1 predicts whether the temperature measurement operator appears in the video image: if not, it is determined whether it exceeds the scheduled temperature measurement process execution period. If it exceeds, it is determined that the temperature is not measured in time, and the system issues an alarm to remind the operator to measure the temperature in time. If it does not exceed, it continues to wait until the temperature measurement operator appears, and ignores the output results of the other three models before that. When model 1 detects that the temperature measurement operator appears in the video image, models 2, 3, and 4 respectively predict whether the temperature measurement operator is wearing the specified color protective device correctly, whether the temperature measurement action is qualified, and whether the temperature measurement operator is within a safe distance from the furnace. If there is danger or the temperature measurement action is unqualified, an alarm sound or light effect will be given to remind the operator.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] It can assist the temperature measurement personnel in the casting process to measure the molten iron temperature in a timely, safe and correct manner, which can effectively avoid casualties of operators caused by incorrect measurement, and can effectively reduce the unqualified rate of casting products caused by inaccurate temperature measurement.

[0061] Embodiment 2

[0062] Based on the same inventive concept, this embodiment provides a molten iron temperature measurement process auxiliary system based on deep learning, which includes:

[0063] A model building module pre-builds four deep learning models, wherein the first deep learning model is used to determine whether the temperature measuring personnel appear in the video image, the second deep learning model is used to determine whether the temperature measuring personnel correctly wears the protective device of the specified color, the third deep learning model is used to determine whether the temperature measuring personnel's temperature measuring posture is qualified, and the fourth deep learning model is used to predict the distance between the temperature measuring personnel's body parts and the molten iron furnace;

[0064] The model training module obtains a large number of temperature measurement action videos of molten iron temperature measurement personnel in the cast iron pouring process. The images in the temperature measurement action videos include features corresponding to qualified temperature measurement processes and features corresponding to unqualified temperature measurement processes. Among them, the features corresponding to qualified temperature measurement processes include that the temperature measurement personnel appear in the video, correctly wear the specified color protective clothing, have a correct temperature measurement posture, and are within a safe distance. The features of unqualified temperature measurement processes include that the temperature measurement personnel do not appear in the video, do not correctly wear the specified color protective clothing, have an incorrect temperature measurement posture, and are not within a safe distance. After annotating the images according to the characteristics of each deep learning model, a corresponding training data set is constructed, and each deep learning model is trained using the corresponding training data set;

[0065] The temperature measurement process auxiliary module is used to use the camera to obtain the current temperature measurement video information of the temperature measurement personnel in real time before measuring the temperature of the molten iron, and use the trained first deep learning training model to determine whether the temperature measurement personnel appears. If they do not appear during the current temperature measurement period, an alarm message is issued. If they appear during the current temperature measurement period, the image of the video information is judged by the second deep learning model, the third deep learning model and the fourth deep learning model. If the judgment result is that the temperature measurement personnel does not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, or the temperature measurement personnel is not within the safe distance, an alarm message is issued.

[0066] During the specific implementation process, an industrial camera that can clearly record the image of personnel, a computer connected to it, and a corresponding voice alarm are correctly placed at the temperature measurement process station. The trained deep learning model and a special human-computer interaction operation program for the temperature measurement process are run on the industrial computer connected to the camera. The entire auxiliary system switch is turned on through the operation program, and the time period for the temperature measurement process to be executed is set according to the actual situation. When the time period for the temperature measurement process to be executed is reached, the camera will start real-time shooting and transmit the video information to the industrial computer. Deep learning model 1 will analyze whether the operator appears in the video through color and texture. If not, it will continue to wait until the preset time period is exceeded or the temperature measurement personnel appear. If the preset time is exceeded, the system will determine that the temperature measurement process has not been carried out in time and remind the operator to measure the temperature in time; if the operator appears in time, the other three deep learning models will use image features to determine whether the temperature measurement personnel's protective measures and temperature measurement posture are qualified, and whether the distance from the furnace body is within the danger limit. If it is unqualified, it may cause danger or the measured temperature is inaccurate. The industrial computer issues a command to make the voice alarm emit a striking alarm to remind the operator to perform the temperature measurement operation correctly.

[0067] Since the system introduced in the second embodiment of the present invention is a system used to implement the auxiliary method for the molten iron temperature measurement process based on deep learning in the first embodiment of the present invention, the technical personnel in the field can understand the specific structure and deformation of the system based on the method introduced in the first embodiment of the present invention, so it is not repeated here. All systems adopted by the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assisting molten iron temperature measurement process based on deep learning, characterized in that: include: Four deep learning models are pre-built, among which the first deep learning model is used to determine whether the temperature measurement personnel appear in the video image, the second deep learning model is used to determine whether the temperature measurement personnel are correctly wearing protective equipment of the specified color, the third deep learning model is used to determine whether the temperature measurement posture of the temperature measurement personnel is qualified, and the fourth deep learning model is used to predict the distance between the body part of the temperature measurement personnel and the molten iron furnace; A large number of videos of temperature measurement actions of personnel measuring the temperature of molten iron in the cast iron pouring process are obtained. The images in the temperature measurement action videos include features corresponding to qualified temperature measurement processes and features corresponding to unqualified temperature measurement processes. Among them, the features corresponding to qualified temperature measurement processes include that the temperature measurement personnel appear in the video, correctly wear the specified color protective clothing, have a correct temperature measurement posture, and are within a safe distance. The features of unqualified temperature measurement processes include that the temperature measurement personnel do not appear in the video, do not correctly wear the specified color protective clothing, have an incorrect temperature measurement posture, and are not within a safe distance. After annotating the images according to the characteristics of each deep learning model, a corresponding training data set is constructed, and each deep learning model is trained using the corresponding training data set; Before measuring the temperature of molten iron, a camera is used to obtain the temperature measurement video information of the current temperature measurement personnel in real time, and the trained first deep learning training model is used to determine whether the temperature measurement personnel appears. If they do not appear during the current temperature measurement period, an alarm message is issued. If they appear during the current temperature measurement period, the image of the video information is judged by the second deep learning model, the third deep learning model and the fourth deep learning model. If the judgment result is that the temperature measurement personnel does not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, or the temperature measurement personnel is not within the safe distance, an alarm message is issued.

2. The molten iron temperature measurement process auxiliary method according to claim 1, characterized in that: Before issuing the alarm information, the method further includes: Save the temperature measurement action video, judgment results and time data.

3. The molten iron temperature measurement process auxiliary method according to claim 1, characterized in that: The first deep learning model, the second deep learning model and the third deep learning model are classification models, including convolutional layers, pooling layers, batch normalization layers, residual network blocks, fully connected layers and softmax layers, among which the convolutional layers, pooling layers, batch normalization layers and residual network blocks are used for feature extraction, the fully connected layers are used to output probability values ​​of two categories, and the softmax layer is used to output the final predicted classification results.

4. The molten iron temperature measurement process auxiliary method according to claim 3, characterized in that: When the first deep learning model, the second deep learning model and the third deep learning model are trained using the corresponding training data sets, the loss function adopts the cross entropy loss function, and is trained through an Adam optimizer with impulse and a strategy in which the learning rate decays exponentially with the number of training times.

5. The molten iron temperature measurement process auxiliary method according to claim 1, characterized in that: The fourth deep learning model includes a convolutional layer, a pooling layer, a batch normalization layer, a residual network block, two fully connected layers and a ReLU activation function. The convolutional layer, the pooling layer, the batch normalization layer, and the residual network block are used to extract relevant features of the distance between the furnace contour and the temperature measurement personnel's hand contour. The fully connected layer is used to output the regressed distance. The ReLU activation function of the fully connected layer is used to ensure that the output distance value is a value greater than zero.

6. The auxiliary method for molten iron temperature measurement process as described in claim 5, characterized in that: When the fourth deep learning model is trained using the corresponding training data set, the loss function uses a mean square error function.

7. A molten iron temperature measurement process auxiliary system based on deep learning, characterized in that: include: A model building module pre-builds four deep learning models, wherein the first deep learning model is used to determine whether the temperature measuring personnel appear in the video image, the second deep learning model is used to determine whether the temperature measuring personnel correctly wears the protective device of the specified color, the third deep learning model is used to determine whether the temperature measuring personnel's temperature measuring posture is qualified, and the fourth deep learning model is used to predict the distance between the temperature measuring personnel's body parts and the molten iron furnace; The model training module obtains a large number of temperature measurement action videos of molten iron temperature measurement personnel in the cast iron pouring process. The images in the temperature measurement action videos include features corresponding to qualified temperature measurement processes and features corresponding to unqualified temperature measurement processes. Among them, the features corresponding to qualified temperature measurement processes include that the temperature measurement personnel appear in the video, correctly wear the specified color protective clothing, have a correct temperature measurement posture, and are within a safe distance. The features of unqualified temperature measurement processes include that the temperature measurement personnel do not appear in the video, do not correctly wear the specified color protective clothing, have an incorrect temperature measurement posture, and are not within a safe distance. After annotating the images according to the characteristics of each deep learning model, a corresponding training data set is constructed, and each deep learning model is trained using the corresponding training data set; The temperature measurement process auxiliary module is used to use the camera to obtain the current temperature measurement video information of the temperature measurement personnel in real time before measuring the temperature of the molten iron, and use the trained first deep learning training model to determine whether the temperature measurement personnel appears. If they do not appear during the current temperature measurement period, an alarm message is issued. If they appear during the current temperature measurement period, the image of the video information is judged by the second deep learning model, the third deep learning model and the fourth deep learning model. If the judgment result is that the temperature measurement personnel does not correctly wear the protective device of the specified color, the temperature measurement action is unqualified, or the temperature measurement personnel is not within the safe distance, an alarm message is issued.