Steel ladle hanger unhooking AI identification system and method

Abstract

The invention provides a steel ladle hanger unhooking AI identification system which comprises a data acquisition module, a data processing module and a detection early warning module. The data acquisition module comprises a plurality of cameras mounted on a travelling crane, and the data processing module is used for processing steel ladle static image data and training a recognition model through an AI algorithm; and the detection early warning module receives the image signal and gives an alarm. The invention further provides a steel ladle hanger unhooking AI identification method. The method comprises the steps of data collection, identification model establishment, identification model optimization, real-time detection and the like. According to the method, the identification model is established for the image of the connection of the steel ladle hanger and the hook through the deep learning algorithm, and the collected steel ladle hanger image is deeply analyzed, so that various states of the hanger, such as normal hanging, partial loosening and abnormal deviation, can be accurately identified. Therefore, early warning is given in advance, and the safety and accuracy of unhooking operation of the steel ladle hanging lug are ensured.

Description

Technical Field

[0001] This invention relates to the field of metallurgical production, and in particular to an AI recognition system and method for identifying the detachment of ladle lugs. Background Technology

[0002] In the modern industrial system, the metallurgical industry, as a fundamental pillar industry, plays a crucial role in the stable operation of the entire economy through its continuous production process. Within the metallurgical production process, the ladle plays a critical role. As the core equipment for transferring high-temperature molten metal, it is like a key node in the human circulatory system, bearing the responsibility of transporting molten steel at high temperatures. The safety of its handling is directly linked to the lives of production personnel and the safety of expensive equipment.

[0003] The reliable connection between the lugs and hooks is undoubtedly of paramount importance in ladle handling operations. During handling, the ladle bears immense weight and complex stresses from the high-temperature environment, demanding a robust and reliable connection between the lugs and hooks. Every handling operation is a test of this connection's reliability; any unplanned disengagement could have catastrophic consequences. Unplanned disengagement can trigger a series of disastrous chain reactions. First, the ladle will instantly lose its balance, causing it to overturn. Once overturned, the ladle filled with molten steel will spill out like raging lava. Molten steel has extremely high temperature and fluidity; wherever it reaches, equipment will be instantly melted, causing irreparable damage. Even more seriously, nearby production personnel will have no time to escape and are highly likely to be burned or even killed by the molten steel. Furthermore, molten steel leakage can also trigger secondary accidents such as fires and explosions, causing enormous damage to the entire production workshop and even the surrounding area, resulting in incalculable economic losses and social impact.

[0004] To address this, various monitoring devices have been developed to ensure the connection status of the ladle lugs and hooks, thereby promptly detecting faults and eliminating safety hazards before they detach. For example, Chinese invention patent application number CN202011600150.8 discloses a method and system for identifying ladle detachment during casting crane hoisting. The method includes receiving point cloud data collected by lidar; receiving image data collected by a camera; preprocessing the point cloud data to remove noise and impurities, obtaining preprocessed point cloud data; performing three-dimensional reconstruction based on the preprocessed point cloud data and image data to obtain fused point cloud data, establishing a three-dimensional model of the ladle and hook; inputting the fused point cloud data into a pre-trained ladle detachment identification model to obtain the positions of the hook and lugs in real time; calculating the distance between the hook and lugs based on their positions, and then determining whether the hook is engaging or disengaging; if the distance exceeds a threshold, an early warning is triggered.

[0005] While the aforementioned patents and similar monitoring methods can serve a monitoring purpose, this traditional image detection method has a high error rate and may produce errors when monitoring ladle hooks in different shapes, environments, and states. Therefore, it is not ideal for practical use. To address this, this invention aims to introduce AI recognition technology into the field of ladle hook detachment monitoring, thereby achieving more accurate identification and monitoring. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an AI-based system and method for recognizing the detachment of ladle lugs. This system solves the problem that existing technologies may have errors in monitoring ladle lugs under different shapes, environments, and conditions, resulting in insufficient recognition accuracy in actual use.

[0007] To achieve the above objectives, in a first aspect, the present invention proposes an AI recognition system for steel ladle lug unhooking, which is used in conjunction with the steel ladle transport vehicle setup and includes a data acquisition module, a data processing module, and a detection and early warning module that are electrically connected to each other; The data acquisition module includes several cameras installed on the vehicle, and the cameras are positioned to face the left and right sides of the ladle hanging lugs. The data processing module first receives and processes the static image data of the ladle pre-collected by the data acquisition module, and trains a recognition model through AI algorithm; then, it extracts feature data from the real-time acquired ladle image data through deep learning algorithm, and combines it with the recognition model to determine whether the ladle hanging lug is currently in a normal hanging state or loose, offset or other state. The detection and early warning module receives signals from the data processing module. When the data processing module outputs that the ladle lug is in an abnormal state, it issues a warning to the staff.

[0008] Furthermore, the camera includes a high-resolution industrial camera and an infrared camera.

[0009] Furthermore, the data processing module also includes an image preprocessing unit, a feature extraction unit, and a state recognition unit; The image preprocessing unit performs preliminary optimization on the raw images captured by the camera, using algorithms such as noise reduction, grayscale conversion, and filtering to remove interference noise and redundant information from the images and improve image quality. The feature extraction unit uses deep learning algorithms to mine key features in the image, extracting feature information such as the shape, position, and size of the ladle lugs, and transforming the original image into a feature vector that can be analyzed. The state recognition unit, based on the extracted feature vector, inputs the recognition model to determine whether the ladle lug is currently in a normal suspension state or an abnormal state such as looseness or displacement.

[0010] Secondly, this invention also proposes an AI-based method for recognizing the detachment of a ladle hook, comprising the following steps: S1. Data Collection: First, image data of the ladle hanger under different working conditions, lighting conditions and angles is collected by the camera and transmitted to the data processing module for training the recognition model; then, in actual work, real-time image data of the ladle hanger is collected by the camera and transmitted to the data processing module to identify the actual working status of the ladle hanger. S2. Recognition Model Establishment: Replace the RepNCSPELAN4 module in the YOLO algorithm backbone network with the C3-ECA module, replace PANet in the neck network with BiFPN, and replace the original CIoU loss function with SIoU loss function to obtain the improved YOLO algorithm model. S3. Recognition Model Optimization: The improved YOLO algorithm model is trained on the training set, the model parameters are adjusted using the optimization algorithm, and the model is optimized based on the validation set results. S4. Real-time detection: Deploy a trained recognition model to detect the status of the ladle lugs in real time, assess the safety status of the ladle lugs based on the detection results, and issue an alarm when an anomaly occurs.

[0011] Furthermore, in step S1, at least 2000 images of the ladle lugs in both normal and unhooked states are collected and labeled using LabelMe software to construct the LL-dataECAt dataset. The training samples are expanded using enhancement techniques, including image flipping and brightness adjustment, to improve the dataset quality, and the dataset is divided into a training set, a test set, and a validation set in an 8:1:1 ratio.

[0012] Furthermore, step S2 specifically includes the following sub-steps: S2.1 Model Setup: In the PyCharm IDE development environment, the RepNCSPELAN4 module in the YOLOV9 backbone network was replaced with the C3-ECA module that integrates the ECA attention mechanism. The C3 module reduces the amount of computation and preserves the integrity of information by segmenting and reorganizing the input feature map in the channel dimension; the ECA attention mechanism automatically learns the weights of each feature channel, enhances the attention to important channels, and suppresses irrelevant channels. In the ECA operation, the shallow and mid-layer output features are first subjected to global flat pooling, transforming the features from H×W×C into a 1×1×C one-dimensional vector. Then, the size of the adaptive one-dimensional convolution kernel k is calculated based on the number of channels in the feature map, as shown in the following formula: Where k is the size of the adaptive convolution kernel, and c is the number of channels. b is a constant of 2, and b is a constant of 1. S2.2 Neck Network Improvement: The original PANet is replaced by a Bidirectional Feature Pyramid Network (BiFPN). BiFPN is based on PAN structure optimization, removing single input nodes, adding aggregation paths, and weighted fusion of input features. This enables the algorithm to more efficiently fuse information at different scales, enhances the feature extraction capability of the target, and thus improves the target detection performance. The calculation formula is as follows: In the formula, I i O represents the input feature, and W represents the output feature. i W j These are the learnable weights; S2.3 Loss Function Replacement: The SDIoU loss function is introduced to replace CIoU, making the model pay more attention to small targets, considering the vector angle between bounding box regressions, and replacing the aspect ratio with the difference in width and height, thereby accelerating convergence and improving recognition accuracy. The expression for SIoU is as follows: In the formula, 1-IoU is the conventional bounding box loss. This represents the positional error between the target bounding box and the ground truth bounding box. The difference in shape between the predicted bounding box and the ground truth bounding box; Calculation of angle loss: In the formula, C h This represents the height difference between the center points of the ground truth bounding box and the predicted bounding box. This is the distance between the center points of the ground truth bounding box and the predicted bounding box; Distance loss calculation: In the formula, and The center of the true bounding box; represents the displacement difference of the target bounding box; e is the exponent that handles a large range of deviations. and The center of the predicted bounding box; and These are the width and height of the frame, respectively; Shape loss calculation: In the formula, To control the focus on shape loss.

[0013] Furthermore, in step S3, the learning strategy selected is a cosine annealing decay strategy, dynamically adjusting the learning rate according to the law of the cosine function. The initial learning rate is set to 0.0001, the batch size is set to 8, the Adam optimizer is used, and training is conducted for 300 epochs using a freeze-thaw training method. The backbone network is frozen for the first 50 epochs, and then unfrozen training is performed after the first 50 epochs.

[0014] Compared with the prior art, the present invention has the following beneficial effects: In this invention, a deep learning algorithm is used to establish a recognition model for images of ladle lugs and hooks. Deep analysis of the acquired ladle lug images accurately identifies various lug states, such as normal suspension, partial loosening, and abnormal displacement. This allows for early warning, ensuring the safety and accuracy of ladle lug unhooking operations.

[0015] In addition, this invention can not only significantly improve the level of safety detection of ladle lug unhooking, reduce manual operation links and lower labor costs, but also effectively avoid safety accidents caused by human operation errors, greatly reduce operational risks, and promote the steel smelting industry towards intelligence and efficiency. Attached Figure Description

[0016] Figure 1 This is a system block diagram according to an embodiment of the present invention.

[0017] Figure 2 This is an improved network structure diagram in the identification model establishment step of an embodiment of the present invention.

[0018] Figure 3 This is a structural diagram of the C3-ECA module in the model building step of an embodiment of the present invention. Figure 4 This is a diagram of the BiFPN structure in the model building step of an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Example: like Figure 1 As shown, in the first aspect, the present invention proposes an AI recognition system for steel ladle lug unhooking, which is used in conjunction with the steel ladle transport vehicle setup, including a data acquisition module, a data processing module, and a detection and early warning module that are electrically connected to each other; The data acquisition module includes several cameras installed on the vehicle, and the cameras are positioned to face the left and right sides of the ladle hanging lugs. The data processing module first receives and processes the static image data of the ladle pre-collected by the data acquisition module, and trains a recognition model through AI algorithm; then, it extracts feature data from the real-time acquired ladle image data through deep learning algorithm, and combines it with the recognition model to determine whether the ladle hanging lug is currently in a normal hanging state or loose, offset or other state. The detection and early warning module receives signals from the data processing module. When the data processing module outputs that the ladle lug is in an abnormal state, it issues a warning to the staff. The warning includes sound, light, and text information on the display.

[0021] The system comprises both high-resolution industrial cameras and infrared cameras, each with its own specific function while complementing each other. The high-resolution industrial camera possesses superior image capture capabilities, enabling it to clearly capture the detailed texture, geometry, and surface condition of the ladle lugs in complex industrial environments, providing precise visual information for the system. The infrared camera utilizes infrared thermal imaging technology to overcome interference from light, smoke, and dust, sensing the thermal radiation distribution of the ladle lugs and acquiring information about their temperature field changes. In the high-temperature environment of steel smelting, the infrared camera can effectively detect potential changes in material properties caused by temperature variations, such as areas of concentrated thermal stress. Through the combination of these two cameras, the system can acquire high-definition images of the ladle lugs in a comprehensive, real-time manner. This includes not only intuitive visual images but also crucial thermal data, providing a solid data foundation for the subsequent intelligent analysis and accurate judgment of the AI-based ladle lug unhooking system, ensuring the safety and reliability of the entire unhooking process.

[0022] Specifically, the data processing module also includes an image preprocessing unit, a feature extraction unit, and a state recognition unit: The image preprocessing unit performs preliminary optimization on the raw images captured by the camera, using algorithms such as noise reduction, grayscale conversion, and filtering to remove interference noise and redundant information from the images, improve image quality, and provide a clear and accurate data foundation for subsequent processing.

[0023] The feature extraction unit uses deep learning algorithms to mine key features in the image, extracting feature information such as the shape, position, and size of the ladle lugs, and transforming the original image into a feature vector that can be analyzed.

[0024] Based on the extracted feature vectors, the status recognition unit inputs them into the recognition model. Through complex logical operations and pattern matching using AI algorithms, it accurately determines whether the ladle lug is currently in a normal hanging, loose, offset, or other abnormal state. This provides a reliable basis for subsequent system decisions and ensures the safety and accuracy of the ladle lug unhooking operation.

[0025] like Figure 2-4 As shown in the figure, this invention also proposes an AI-based method for recognizing the detachment of ladle hooks, comprising the following steps: I. Data Collection: First, image data of the ladle hanger under different working conditions, lighting conditions and angles is collected by a camera and transmitted to the data processing module for training the recognition model; then, in actual work, real-time image data of the ladle hanger is collected by a camera and transmitted to the data processing module to identify the actual working status of the ladle hanger.

[0026] Specifically, at least 2000 images of the ladle lugs in both normal and unhooked states were collected and labeled using LabelMe software to construct the LL-dataECAt dataset. Enhancement techniques, including image flipping and brightness adjustment, were used to expand the training samples and improve the dataset quality. The dataset was then divided into training, testing, and validation sets in an 8:1:1 ratio.

[0027] II. Model Establishment: The RepNCSPELAN4 module in the YOLO algorithm backbone network is replaced with the C3-ECA module, and PANet in the neck network is replaced with BiFPN. The SIoU loss function is used instead of the original CIoU loss function to obtain the improved YOLO algorithm model. The specific steps are as follows: (1) Model building: Based on the PyCharm IDE development environment, the RepNCSPELAN4 module in the YOLOV9 backbone network was replaced with the C3-ECA module that integrates the ECA attention mechanism. The C3 module reduces the amount of computation and retains the integrity of information by segmenting and reorganizing the input feature map in the channel dimension; the ECA attention mechanism automatically learns the weight of each feature channel, enhances the attention to important channels, and suppresses irrelevant channels. In the ECA operation, the shallow and mid-layer output features are first subjected to global flat pooling, transforming the features from H×W×C into a 1×1×C one-dimensional vector. Then, the size of the adaptive one-dimensional convolution kernel k is calculated based on the number of channels in the feature map, as shown in the following formula: Where k is the size of the adaptive convolution kernel, and c is the number of channels. b is a constant of 2, and b is a constant of 1. (2) Improved neck network: The original PANet is replaced by a bidirectional feature pyramid network (BiFPN). BiFPN is based on the PAN structure optimization, removes single input nodes, adds aggregation paths and performs weighted fusion of input features, so that the algorithm can more efficiently fuse information at different scales, enhance the feature extraction capability of the target, and thus improve the target detection performance. The calculation formula is as follows: In the formula, I i O represents the input feature, and W represents the output feature. i W j These are the learnable weights; (3) Loss function replacement: The SDIoU loss function is introduced to replace CIoU, making the model pay more attention to small targets, considering the vector angle between bounding box regressions, and replacing the aspect ratio with the difference in width and height, thereby accelerating convergence and improving recognition accuracy. The expression of its SIoU is as follows: In the formula, 1-IoU is the conventional bounding box loss. This represents the positional error between the target bounding box and the ground truth bounding box. The difference in shape between the predicted bounding box and the ground truth bounding box; Calculation of angle loss: In the formula, C h This represents the height difference between the center points of the ground truth bounding box and the predicted bounding box. This is the distance between the center points of the ground truth bounding box and the predicted bounding box; Distance loss calculation: In the formula, and The center of the true bounding box; represents the displacement difference of the target bounding box; e is the exponent that handles a large range of deviations. and The center of the predicted bounding box; and These are the width and height of the frame, respectively; Shape loss calculation: IoU calculation: In the formula, To control the focus on shape loss .

[0028] III. Recognition Model Optimization: The improved YOLO algorithm model is trained on the training set, and the model parameters are adjusted using optimization algorithms. The model is then optimized based on the validation set results.

[0029] Specifically, the learning strategy chosen is cosine annealing decay, dynamically adjusting the learning rate based on the pattern of the cosine function. The initial learning rate is set to 0.0001, the batch size is set to 8, the Adam optimizer is used, and training is conducted for 300 epochs using a freeze-thaw training method. The backbone network is frozen for the first 50 epochs, and then unfrozen training is implemented after the first 50 epochs.

[0030] IV. Real-time detection: Deploy the trained recognition model to detect the status of the ladle lugs in real time, assess the safety status of the ladle lugs based on the detection results, and issue an alarm when abnormalities occur.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A steel ladle lug unhooking AI recognition system, used in conjunction with a steel ladle transport crane, characterized in that: It includes a data acquisition module, a data processing module, and a detection and early warning module that are electrically connected to each other; The data acquisition module includes several cameras installed on the vehicle, and the cameras are positioned to face the left and right sides of the ladle hanging lugs. The data processing module first receives and processes the static image data of the ladle pre-collected by the data acquisition module, and trains a recognition model through AI algorithm; then, it extracts feature data from the real-time acquired ladle image data through deep learning algorithm, and combines it with the recognition model to determine whether the ladle hanging lug is currently in a normal hanging state or loose, offset or other state. The detection and early warning module receives signals from the data processing module. When the data processing module outputs that the ladle lug is in an abnormal state, it issues a warning to the staff.

2. The ladle lug unhooking AI recognition system as described in claim 1, characterized in that: The cameras include high-resolution industrial cameras and infrared cameras.

3. The ladle lug unhooking AI recognition system as described in claim 1, characterized in that: The data processing module further includes an image preprocessing unit, a feature extraction unit, and a state recognition unit; The image preprocessing unit performs preliminary optimization on the raw images captured by the camera, using algorithms such as noise reduction, grayscale conversion, and filtering to remove interference noise and redundant information from the images and improve image quality. The feature extraction unit uses deep learning algorithms to mine key features in the image, extracting feature information such as the shape, position, and size of the ladle lugs, and transforming the original image into a feature vector that can be analyzed. The state recognition unit, based on the extracted feature vector, inputs the recognition model to determine whether the ladle lug is currently in a normal suspension state or an abnormal state such as looseness or displacement.

4. A ladle lug unhooking AI recognition method for the system as described in claim 1, characterized in that, Includes the following steps: S1. Data Collection: First, image data of the ladle hanger under different working conditions, lighting conditions and angles is collected by the camera and transmitted to the data processing module for training the recognition model; then, in actual work, real-time image data of the ladle hanger is collected by the camera and transmitted to the data processing module to identify the actual working status of the ladle hanger. S2. Recognition Model Establishment: Replace the RepNCSPELAN4 module in the YOLO algorithm backbone network with the C3-ECA module, replace PANet in the neck network with BiFPN, and replace the original CIoU loss function with SIoU loss function to obtain the improved YOLO algorithm model. S3. Recognition Model Optimization: The improved YOLO algorithm model is trained on the training set, the model parameters are adjusted using the optimization algorithm, and the model is optimized based on the validation set results. S4. Real-time detection: Deploy a trained recognition model to detect the status of the ladle lugs in real time, assess the safety status of the ladle lugs based on the detection results, and issue an alarm when an anomaly occurs.

5. The ladle lug unhooking AI recognition system and method as described in claim 4, characterized in that: In step S1, at least 2000 images of the ladle lugs in both normal and unhooked states are collected and labeled using LabelMe software to construct the LL-dataECAt dataset. The training samples are expanded using enhancement techniques, including image flipping and brightness adjustment, to improve the dataset quality. The dataset is then divided into training, testing, and validation sets in an 8:1:1 ratio.

6. The ladle lug unhooking AI recognition system and method as described in claim 4, characterized in that, Step S2 specifically includes the following sub-steps: S2.1 Model Setup: In the PyCharm IDE development environment, the RepNCSPELAN4 module in the YOLOV9 backbone network was replaced with the C3-ECA module that integrates the ECA attention mechanism. The C3 module reduces the amount of computation and preserves the integrity of information by segmenting and reorganizing the input feature map in the channel dimension; the ECA attention mechanism automatically learns the weights of each feature channel, enhances the attention to important channels, and suppresses irrelevant channels. In the ECA operation, the shallow and mid-layer output features are first subjected to global flat pooling, transforming the features from H×W×C into a 1×1×C one-dimensional vector. Then, the size of the adaptive one-dimensional convolution kernel k is calculated based on the number of channels in the feature map, as shown in the following formula: Where k is the size of the adaptive convolution kernel, and c is the number of channels. b is a constant of 2, and b is a constant of 1. S2.2 Neck Network Improvement: The original PANet is replaced by a Bidirectional Feature Pyramid Network (BiFPN). BiFPN is based on PAN structure optimization, removing single input nodes, adding aggregation paths, and weighted fusion of input features. This enables the algorithm to more efficiently fuse information at different scales, enhances the feature extraction capability of the target, and thus improves the target detection performance. The calculation formula is as follows: In the formula, I i O represents the input feature, and W represents the output feature. i W j These are the learnable weights; S2.3 Loss Function Replacement: The SDIoU loss function is introduced to replace CIoU, making the model pay more attention to small targets, considering the vector angle between bounding box regressions, and replacing the aspect ratio with the difference in width and height, thereby accelerating convergence and improving recognition accuracy. The expression for SIoU is as follows: In the formula, 1-IoU is the conventional bounding box loss. This represents the positional error between the target bounding box and the ground truth bounding box. The difference in shape between the predicted bounding box and the ground truth bounding box; Calculation of angle loss: In the formula, C h This represents the height difference between the center points of the ground truth bounding box and the predicted bounding box. This is the distance between the center points of the ground truth bounding box and the predicted bounding box; Distance loss calculation: In the formula, and The center of the true bounding box; represents the displacement difference of the target bounding box; e is the exponent that handles a large range of deviations. and The center of the predicted bounding box; and These are the width and height of the frame, respectively; Shape loss calculation: In the formula, To control the focus on shape loss.

7. The ladle lug unhooking AI recognition system and method as described in claim 4, characterized in that, In step S3, the learning strategy selected is cosine annealing decay strategy, and the learning rate is dynamically adjusted according to the law of cosine function. The initial learning rate is set to 0.0001, the batch size is set to 8, the Adam optimizer is used, and the training is carried out for 300 rounds. The freeze-thaw training method is used, with the backbone network frozen for the first 50 rounds and unfrozen training after the first 50 rounds.

Images