A steel plant material taking behavior recognition method and system based on target detection

Abstract

The application provides a steel plant material taking behavior recognition method and system based on target detection, and relates to the technical field of computer vision, which comprises the following steps: collecting historical steelmaking site videos; processing the historical steelmaking site videos through a mask algorithm to generate interest images; establishing a target detection model with YOLOv5 as the architecture, wherein the target detection model comprises a backbone network module, a neck module and a detection head module, and the backbone network module and the neck module both contain CBAM attention units; inputting the interest images into the target detection model for training until the loss function value of the target detection model is less than a preset loss function value; obtaining a to-be-recognized steelmaking site video; inputting the to-be-recognized steelmaking site video into the trained target detection model to output a detection result; and outputting a steel plant material taking behavior recognition result according to the detection result. The application can effectively prevent and reduce safety accidents in the workplace and protect the life safety of workers.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for recognizing steel plant material handling behavior based on target detection. Background Technology

[0002] Object detection is a computer vision technique used to identify the category and location (such as bounding boxes) of target objects in images or videos. Typical algorithms include YOLO and Faster R-CNN. In steel mills, material handling refers to the process of measuring the composition of molten steel in a converter after it has reached a certain stage of refining. At this time, the converter opening faces outwards, and workers use a long material handling rod to reach into the molten steel, collect a certain amount, and retrieve it. After the steel cools, it can be tapped off and used to test its composition. Steel mill material handling behavior recognition based on object detection utilizes image recognition techniques from artificial intelligence to intelligently analyze videos from the steelmaking site, thereby identifying whether the crucial action of "material handling" has occurred.

[0003] The development of the steel industry is closely related to a country's industrial level. With the introduction of "Made in China 2025," China's steel industry has entered the era of intelligent manufacturing and urgently needs to undergo intelligent transformation and upgrading. In steelmaking, converter steelmaking is the first process and the starting point for turning iron into steel. During this process, there is a material handling operation that workers must perform, which has a significant impact on the quality of the produced steel. By measuring a portion of the extracted molten steel, the composition of the molten steel in the converter can be examined, thereby judging the quality of the produced steel and determining the condition for tapping a batch of steel. Therefore, identifying the operational behavior of the material handling operation is of great significance.

[0004] However, existing methods for recognizing material handling behavior in steel mills rely on manual features (such as LBP, SIFT, and HOG) and traditional machine learning models (such as SVM). While robust to localized information, these methods struggle to identify complex or forged behaviors. Deep learning methods, although improving recognition capabilities, are computationally intensive and slow inference, making them unsuitable for scenarios with high real-time requirements. Especially in high-load environments like steel mills, the high complexity of behavior recognition models hinders deployment, increasing computational resource burden, reducing recognition accuracy, and failing to meet real-time performance demands. Summary of the Invention

[0005] To address the shortcomings of existing steel mill material handling behavior recognition methods, which rely on manual feature recognition (such as LBP, SIFT, HOG, etc.) and traditional machine learning models, making it difficult to identify complex or forged behaviors, computationally intensive, and slow inference, thus unsuitable for scenarios with high real-time requirements, this invention provides a steel mill material handling behavior recognition method and system based on object detection. This system addresses the challenges of high-load environments like steel mills where the complexity of behavior recognition models hinders deployment, increases computational resource burden, reduces recognition accuracy, and fails to meet real-time requirements.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect:

[0008] This invention provides a method for recognizing steel mill material handling behavior based on target detection, comprising:

[0009] S1: Collect historical steelmaking site videos;

[0010] S2: Using a masking algorithm, historical steelmaking videos are processed to generate images of interest;

[0011] S3: Based on YOLOv5 architecture, an object detection model is established. The object detection model includes: backbone network module, neck module and detection head module. Both backbone network module and neck module contain CBAM attention units.

[0012] S4: Input the interest image into the object detection model for training until the loss function value of the object detection model is less than the preset loss function value;

[0013] S5: Obtain the video of the steelmaking site to be identified;

[0014] S6: Input the video of the steelmaking site to be identified into the trained target detection model and output the detection results;

[0015] S7: Based on the detection results, output the steel mill material handling behavior identification results.

[0016] The second aspect:

[0017] This invention provides a steel plant material handling behavior recognition system based on target detection, comprising:

[0018] processor;

[0019] The memory stores computer-readable instructions, which, when executed by the processor, implement the steel plant material handling behavior recognition method based on target detection as described in the first aspect.

[0020] Third aspect:

[0021] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steel plant material handling behavior recognition method based on target detection as described in the first aspect.

[0022] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0023] In this embodiment of the invention, a masking algorithm is used to process historical steelmaking site videos to extract regions of interest (ROIs) from the images, thereby reducing interference from other irrelevant targets on the model. Next, a target detection model is built using YOLOv5 as the architecture, and the ROI image is input into the target detection model for training until the loss function value of the target detection model is less than a preset loss function value, ensuring that the model can efficiently learn core features. Finally, the steelmaking site video to be identified is acquired and input into the trained target detection model, outputting the detection results to complete the identification of steel plant material handling behavior. Combined with a lightweight model architecture, this not only ensures recognition accuracy and robustness and reduces the burden of computational resources, but also improves processing efficiency and deployment flexibility, making it particularly suitable for industrial scenarios like steel plants with high real-time and resource requirements. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a steel plant material handling behavior recognition method based on target detection, provided in an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the structure of CABM attention provided in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of a steel plant material handling behavior recognition system based on target detection, provided as an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0029] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0030] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0031] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0032] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0033] Reference manual attached Figure 1 The diagram shows a flowchart of a steel plant material handling behavior recognition method based on target detection provided by an embodiment of the present invention.

[0034] This invention provides a method for recognizing steel mill material handling behavior based on target detection. This method can be implemented by a steel mill material handling behavior recognition device based on target detection, which can be a terminal or a server. The processing flow of the steel mill material handling behavior recognition method based on target detection may include the following steps:

[0035] S1: Collect historical steelmaking videos.

[0036] Among them, historical steelmaking site videos refer to steelmaking operations that have occurred in the past, which are the actual industrial production environment in which steelmaking processes are carried out in steel plants, and may include multiple visual targets such as workers, molten steel, and material handling equipment.

[0037] It should be noted that by collecting historical steelmaking videos, existing monitoring resources in the steel plant can be utilized, eliminating the need for additional surveillance and reducing data acquisition costs. Furthermore, these historical videos cover various working conditions and environmental changes, making the training data more diverse and richer, which helps improve the model's generalization ability.

[0038] S2: Using a masking algorithm, historical steelmaking videos are processed to generate images of interest.

[0039] Among them, the masking algorithm is an image processing method that constructs a "mask" region to retain only the parts of the image of interest while masking irrelevant background information. It is often used for tasks such as target extraction and image segmentation. The image of interest is a key region image extracted from the original video frame, containing only the parts directly related to the "material handling behavior," such as the worker, the material handling rod, and the molten steel.

[0040] It should be noted that by accurately extracting the core visual regions related to the material picking behavior through the masking algorithm, the focus and efficiency of subsequent model training are significantly improved. The generation of interest images can also reduce the workload of annotation, improve the efficiency of data utilization, and enhance the model's adaptability in real-world scenarios.

[0041] In one possible implementation, S2 specifically includes:

[0042] S201: Extract frames from historical steelmaking videos and save them as images.

[0043] Frame splitting refers to dividing a video into static images along the timeline, which facilitates subsequent image-level processing and analysis.

[0044] S202: The images are filtered using the pre-trained weighted object detection model YOLOv5 to identify worker images.

[0045] YOLOv5 refers to a mainstream object detection algorithm that has the advantages of high accuracy and real-time performance, and is suitable for detecting objects and their locations in images.

[0046] S203: Randomly flip and stretch the worker images to generate a worker image dataset.

[0047] Among them, random horizontal flipping and stretching are ways to artificially change the content of images in order to expand the dataset and improve the generalization ability of the model.

[0048] S204: Label the worker image dataset to obtain the label file corresponding to each image in the worker image dataset.

[0049] S205: Extract the center coordinates of each detection box in the label file and normalize each center coordinate:

[0050]

[0051] Where, x ij Let y represent the x-coordinate of the top-left corner of the j-th detection box in the i-th image. ijThis represents the ordinate of the top-left corner of the j-th bounding box in the i-th image, where norm represents the normalization operation, and w ij h represents the width of the j-th detection box in the i-th image. ij This represents the height of the j-th detection box in the i-th image. This represents the x-coordinate of the center c of the normalized detection box. The vertical coordinate of c represents the center of the normalized detection box.

[0052] Specifically, the detection box represents the position of the target in the image, usually expressed as the coordinates of the top left corner, width, and height.

[0053] S206: Perform K-means clustering on the normalized center coordinates:

[0054]

[0055] Where min represents taking the minimum value, μ k Let z represent the center coordinates of the k-th cluster, k = 1, 2, ..., K, where K represents the total number of clusters. ij S represents the cluster label of the center point of each detection box. k Let (cx, cy) represent the set of center points of the k-th cluster, and (cx, cy) represent the normalized coordinates of the center of the detection box.

[0056] Specifically, K-means clustering is an unsupervised learning method that divides a set of points into K categories to minimize the differences between points of the same category, and is used for spatial region classification.

[0057] S207: Generate a rectangular mask based on the clustering results, and extract the main regions of the image to obtain the image of interest.

[0058]

[0059] I masked (u,v)=I(u,v)⊙M(u,v)

[0060] Among them, M k (u,v) represents the mask coverage area of ​​the k-th cluster, where u and v both represent the coordinates of the center point of the k-th cluster, μ k,x w represents the x-coordinate of the k-th cluster center. k Let μ represent the width of the k-th cluster, W represent the image width, and μ represent the width of the cluster. k,y h represents the ordinate of the k-th cluster center. k H represents the height of the k-th cluster, and I represents the image height. masked (u,v) represents the image after region extraction, I(u,v) represents the image before region extraction, and M(u,v) represents the mask region M. k The union of (u,v), where ⊙ denotes element-wise multiplication.

[0061] It should be noted that using a pre-trained model to screen worker images, combined with data augmentation and precise annotation, improves the diversity and representativeness of the data. Normalization and K-means clustering make it easier for the model to learn spatial distribution patterns, while masking operations further compress irrelevant information, improving training efficiency and reducing computational costs. This approach is particularly suitable for complex environments like steel plants where targets are concentrated but limited in number, and is a crucial data preparation step for achieving a highly robust and real-time behavior recognition model.

[0062] In one possible implementation, the types marked in S204 specifically include: person, material handling rod, and molten steel.

[0063] Reference manual attached Figure 2 The diagram shows a schematic representation of the CABM attention structure provided in an embodiment of the present invention.

[0064] Figure 2 In this diagram, Channel Attention Module represents the channel attention mechanism, Spatial Attention Module represents the spatial attention mechanism, Input represents the input feature map, Average Pool represents average pooling, used to extract the global average information of the feature map, Max Pool represents max pooling, used to extract the most significant feature values ​​in the feature map, Shared MLP represents shared multilayer perceptron (fully connected neural network), used to perform non-linear transformations on the pooled features to extract channel relationships, CONV represents convolution operation, used to extract attention weights in the spatial dimension, and Output represents the output of the module, i.e. the feature map after being weighted by the attention mechanism.

[0065] Specifically, the spatial attention part performs average pooling and max pooling (along the channel dimension) on the input feature map to generate two 2D feature maps. These two maps are then concatenated to form a new feature map, which is then fed into a convolution operation (CONV). After the convolution result is processed by the Sigmoid activation function, a spatial attention map (each pixel has a weight) is generated. Finally, this attention map is used to perform pixel-wise weighting on the input feature map to obtain the spatial attention output.

[0066] It should be noted that the channel attention part performs global average pooling and max pooling (along the spatial dimension) on the input feature map respectively. Each operation will result in a channel-level description vector. Then, the two pooling results are fed into a shared MLP network (two FC layers) for non-linear transformation, and the MLP output is weighted and fused (⊕). Then, the attention weight of each channel is obtained by passing through the Sigmoid activation function. Finally, these weights are used to weight the original input feature map channel by channel, and the output is a feature map with channel attention.

[0067] S3: Based on YOLOv5 architecture, an object detection model is established. The object detection model includes: a backbone network module, a neck module, and a detection head module. Both the backbone network module and the neck module contain CBAM attention units.

[0068] YOLOv5 is an advanced single-stage object detection algorithm (You Only Look Once), characterized by its high detection speed and accuracy, making it suitable for real-time industrial vision applications. Object detection models are neural network structures used to identify the location (usually a bounding box) and category of objects in an image, and are widely used in scenarios such as face detection and vehicle recognition.

[0069] The backbone network module is responsible for extracting basic image features, such as edges and textures, and is often used to extract deep semantic information.

[0070] The neck module, located between the backbone and the detection head, fuses feature information from different scales to enhance the model's adaptability to targets of varying sizes (such as FPN and PAN structures). The detection head module outputs the final target category and location prediction results, typically including a classifier and a bounding box regressor.

[0071] Among them, CBAM attention unit refers to a mechanism that integrates channel attention and spatial attention, which can improve the model's ability to focus on key regions and information.

[0072] It should be noted that an efficient and intelligent object detection model was constructed by introducing the YOLOv5 architecture and the CBAM attention mechanism. YOLOv5, as a lightweight single-stage detection algorithm, has a simple structure and fast computation speed, making it highly suitable for applications like steel mills that have extremely high real-time requirements. The hierarchical structure of the backbone network and neck module effectively extracts and fuses multi-scale information from the image, enabling the model to recognize both overall actions and detailed features. The addition of the CBAM attention unit further enhances the model's ability to recognize key image regions (such as worker's hands, the end of the material handling rod, and molten steel spots), reducing attention to redundant areas and thus improving model accuracy and robustness.

[0073] In one possible implementation, the CBAM attention unit includes channel attention and spatial attention.

[0074] Channel attention is used to measure and adjust the importance of each channel (i.e., a certain type of semantic information in the feature map). For example, one channel may focus on detecting edges, while another channel focuses on color. Channel attention can enhance useful channels and suppress redundant or irrelevant channels.

[0075] Spatial attention focuses on adjusting the importance of different "locations" in an image, that is, enabling the model to automatically determine "where" in the image is most worthy of attention. For example, in an image, only the worker's hand and the material handling rod may be the key points of behavior, and spatial attention can help the model focus on these areas.

[0076] It's important to note that channel attention helps the model decide "what feature type to look at," while spatial attention helps the model determine "where to look." The two work together to improve the model's target recognition capabilities in complex images. Especially in steel mill scenarios, this mechanism allows the model to focus more on areas such as "the region where the material handling action occurs" rather than the background molten steel or equipment, thereby significantly improving detection accuracy.

[0077] In one possible implementation, channel attention is used to generate channel weights, and the generation of channel weights specifically includes:

[0078] S301: Calculate the global average pooling and max pooling of the feature map:

[0079]

[0080] F max =max p,g X(:;p,g)

[0081] Among them, F avg This represents the global average pooling result of the feature map. X represents the input feature map, p represents the pixel row index, and g represents the pixel column index. In X(:,:,p,g), the first ":" indicates that each input image is processed, and the second ":" indicates that each channel of the feature map is processed.

[0082] It should be noted that the dimensions of the input feature map are (b, c, p, g), where b is the number of input images and c is the number of channels in the feature map.

[0083] Specifically, global average pooling averages all pixels in each channel, compressing the spatial dimension while preserving global semantic information between channels. Max pooling extracts the maximum response value from each channel, retaining the strongest feature representation in the channel and emphasizing salient regions.

[0084] S302: The global average pooling result and the max pooling result are input into the fully connected layer, processed by the ReLU activation function, and the channel information of the feature map is output:

[0085]

[0086] in, The channel weights represent the results of global average pooling. W1 and W2 represent the channel weights of the max pooling result, respectively. W1 and W2 both represent the weight matrices of the fully connected layer, and ReLU represents the ReLU activation function.

[0087] Specifically, a fully connected layer is a layer in a neural network in which all input neurons are connected to all output neurons, and it is used to learn high-level nonlinear feature relationships.

[0088] S303: Based on the channel information, generate the fused channel weights for each channel using the Sigmoid activation function:

[0089]

[0090] X out =X⊙M c

[0091] Among them, M c X represents the fusion channel weight. out This represents the output feature map after channel attention weighting, and σ represents the Sigmoid activation function.

[0092] Specifically, the fused channel weights are the final channel importance scores generated by combining the channel weight information from average pooling and max pooling using the Sigmoid function.

[0093] It should be noted that channel-level statistical features are extracted through global average pooling and max pooling, then channel information is generated after passing through a fully connected layer and ReLU nonlinear transformation. Finally, the channel weights are fused using the sigmoid function, achieving "weighted attention" to each channel in the feature map. This method endows the neural network with the ability to learn and adjust the importance of different semantic features, so that the network no longer "treats" all channels equally, but focuses more on discriminative feature dimensions.

[0094] In one possible implementation, spatial attention is used to generate spatial weights, specifically including:

[0095] S304: Calculate the average pooling and max pooling of the channel dimension in the channel attention output feature map:

[0096]

[0097] F′ max =max c X out (:,c,:,:)

[0098] Among them, F′ avg F′ represents the average pooling result along the channel dimension. max This represents the max pooling result for the channel dimension, where c represents the c-th channel and C represents the total number of channels.

[0099] S305: Concatenate the average pooling result and the max pooling result along the channel dimension:

[0100] M s =W s *[F′ avg ||F′ max ]

[0101] Among them, M s W represents spatial weights. s This represents the weight matrix of the fully connected layer, * represents matrix multiplication, and || represents concatenation.

[0102] S306: Based on the splicing result, spatial weights are generated using the sigmoid activation function.

[0103] M s =σ(M s )

[0104] X final =X out ⊙M s

[0105] Among them, X final This represents the final output feature map.

[0106] It's important to note that by using channel-dimensional average pooling and max pooling operations, the model can extract the average and maximum responses of each spatial location in the image across all channels, thereby capturing both global and local salient features. The concatenation operation merges the two pooling results, preserving spatial information from different angles. The spatial weight map generated by the sigmoid function guides the model to focus on regions in the image related to the target behavior (such as a worker's hand, material handling actions, etc.), suppressing background interference. Compared to relying solely on channel attention, spatial attention can guide the model to focus on the location where the behavior occurs with finer granularity, thus improving recognition accuracy and localization precision.

[0107] S4: Input the interest image into the object detection model for training until the loss function value of the object detection model is less than the preset loss function value.

[0108] The loss function value measures the error between the model's predictions and the true labels; a smaller value indicates better model performance. Common loss functions include classification loss, regression loss, and confidence loss.

[0109] Those skilled in the art can set the value of the preset loss function according to the actual situation, and the present invention does not limit it.

[0110] It's worth noting that by inputting previously carefully processed "interest images" into the object detection model for training, we can ensure that the model focuses on core regions related to behavior, thereby significantly improving training efficiency and detection accuracy. The training process is guided by a loss function, ensuring that the model continuously optimizes in each learning round, constantly approaching the optimal parameter state. By setting a preset loss threshold, we can avoid overfitting, improve training efficiency, and achieve rational utilization of training resources.

[0111] In one possible implementation, the formula for calculating the loss function value is as follows:

[0112] L=λ box L box +λ cls L cls +λ obj L obj

[0113] Where L represents the loss function value, λ box L represents the weights of the bounding box regression loss. box λ represents the regression loss of the detection box. cls L represents the weights of the classification loss. cls Let λ represent the classification loss. obj L represents the weight of the confidence loss. obj This represents the confidence loss.

[0114] S5: Obtain the video of the steelmaking site to be identified.

[0115] S6: Input the video of the steelmaking site to be identified into the trained target detection model and output the detection results.

[0116] It should be noted that by inputting real-time or archived video footage from the site into the trained target detection model, the system gains the ability to automatically identify material handling activities and quickly output detection results for key objects (such as workers, material handling rods, and molten steel), greatly reducing the burden of manual analysis. Compared to traditional methods that rely on manual review of monitoring data, this method is not only more efficient but also produces more objective and consistent results. Because it uses an optimized model trained on images of interest, the detection process has strong generalization and adaptability, maintaining high accuracy even under different working conditions, lighting, or angles.

[0117] S7: Based on the detection results, output the steel mill material handling behavior identification results.

[0118] The detection result is the recognition information output by the target detection model, which usually includes the target location (box coordinates) in each frame of the image, the target category (such as worker, molten steel, material handling rod) and its confidence level.

[0119] The identification result is the final behavior judgment output, which indicates whether the "material picking" behavior has occurred in the video, and can be used as the basis for monitoring alarms, recording or analysis.

[0120] It should be noted that by performing logical analysis on the spatiotemporal relationships between multiple detection targets (such as whether a worker is close to molten steel or holds a material handling rod), the system can be transformed from "image-level recognition" to "behavioral-level recognition," which greatly improves the intelligence level and practical application value of the system.

[0121] In one possible implementation, S7 specifically refers to:

[0122] The post-processing module determines whether a material handling operation has occurred based on the class and location information in the detection results.

[0123]

[0124] Among them, C t D indicates whether a material handling operation has occurred at time t. t-n This indicates whether the category and position of the nth detection box meet the requirements, and N represents the total number of detection boxes. This represents the time of the (t+1)th material retrieving operation. The time for the t-th material handling operation is represented by ΔT, where ΔT represents the time threshold and Π represents the product sign.

[0125] It should be noted that the method of combining target detection and post-processing judgment is used to identify and judge the material handling action in the converter steelmaking process. A simple model is used instead of the complex model of behavior recognition algorithm, which reduces the burden of on-site computing resources.

[0126] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0127] In this embodiment of the invention, a masking algorithm is used to process historical steelmaking site videos to extract regions of interest (ROIs) from the images, thereby reducing interference from other irrelevant targets on the model. Next, a target detection model is built using YOLOv5 as the architecture, and the ROI image is input into the target detection model for training until the loss function value of the target detection model is less than a preset loss function value, ensuring that the model can efficiently learn core features. Finally, the steelmaking site video to be identified is acquired and input into the trained target detection model, outputting the detection results to complete the identification of steel plant material handling behavior. Combined with a lightweight model architecture, this not only ensures recognition accuracy and robustness and reduces the burden of computational resources, but also improves processing efficiency and deployment flexibility, making it particularly suitable for industrial scenarios like steel plants with high real-time and resource requirements.

[0128] Reference manual attached Figure 3 The diagram shows a structural schematic of a steel plant material handling behavior recognition system based on target detection provided by the present invention.

[0129] The present invention also provides a steel mill material handling behavior recognition system 20 based on target detection, applied to the above-mentioned steel mill material handling behavior recognition method based on target detection, including:

[0130] Processor 201.

[0131] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, they implement the steel plant material handling behavior recognition method based on target detection as described in the method embodiment.

[0132] The steel plant material handling behavior recognition system 20 based on target detection provided by the present invention can execute the above-mentioned steel plant material handling behavior recognition method based on target detection and achieve the same or similar technical effects. To avoid duplication, the present invention will not elaborate further.

[0133] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0134] In this embodiment of the invention, a masking algorithm is used to process historical steelmaking site videos to extract regions of interest (ROIs) from the images, thereby reducing interference from other irrelevant targets on the model. Next, a target detection model is built using YOLOv5 as the architecture, and the ROI image is input into the target detection model for training until the loss function value of the target detection model is less than a preset loss function value, ensuring that the model can efficiently learn core features. Finally, the steelmaking site video to be identified is acquired and input into the trained target detection model, outputting the detection results to complete the identification of steel plant material handling behavior. Combined with a lightweight model architecture, this not only ensures recognition accuracy and robustness and reduces the burden of computational resources, but also improves processing efficiency and deployment flexibility, making it particularly suitable for industrial scenarios like steel plants with high real-time and resource requirements.

[0135] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0136] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0137] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0138] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0139] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0140] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0146] If the functionality is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steel plant material handling behavior recognition method based on target detection as described in the method embodiment.

[0148] The present invention provides a computer-readable storage medium that can implement the steps and effects of the steel plant material handling behavior recognition method based on target detection in the above-described method embodiments. To avoid repetition, the present invention will not repeat the details.

[0149] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0150] In this embodiment of the invention, a masking algorithm is used to process historical steelmaking site videos to extract regions of interest (ROIs) from the images, thereby reducing interference from other irrelevant targets on the model. Next, a target detection model is built using YOLOv5 as the architecture, and the ROI image is input into the target detection model for training until the loss function value of the target detection model is less than a preset loss function value, ensuring that the model can efficiently learn core features. Finally, the steelmaking site video to be identified is acquired and input into the trained target detection model, outputting the detection results to complete the identification of steel plant material handling behavior. Combined with a lightweight model architecture, this not only ensures recognition accuracy and robustness and reduces the burden of computational resources, but also improves processing efficiency and deployment flexibility, making it particularly suitable for industrial scenarios like steel plants with high real-time and resource requirements.

[0151] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0152] The following points need to be explained:

[0153] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0154] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the present invention; that is, these drawings are not drawn to actual scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element, or there may be intermediate elements.

[0155] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0156] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A target detection based steel plant material handling behavior recognition method, characterized in that, include: S1: Collect historical steelmaking site videos; S2: The historical steelmaking site video is processed using a masking algorithm to generate an image of interest; S2 specifically includes: S201: The historical steelmaking site video is split into frames and saved as an image; S202: Use the pre-trained weighted object detection model YOLOv5 to filter images and determine worker images; S203: Randomly flip and stretch the worker images to generate a worker image dataset; S204: Label the worker image dataset to obtain the label file corresponding to each image in the worker image dataset; S205: Extract the center coordinates of each detection box in the label file, and normalize each of the center coordinates: ； in, Indicates the first i The first picture j The x-coordinate of the top left corner of each detection box. y ij Indicates the first i The first picture j The ordinate of the top left corner of each detection box. norm This indicates a normalization operation. w ij Indicates the first i The first picture j The width of each detection box, h ij Indicates the first i The first picture j The height of each detection box Indicates the center of the normalized detection box. c x-coordinate Indicates the center of the normalized detection box. c The ordinate; S206: Perform K-means clustering on the normalized center coordinates: ； Where min represents taking the minimum value. μ k Indicates the first k The center coordinates of the cluster , K Indicates the total number of clusters. z ij Cluster labels representing the center points of each detection frame, S k Indicates the first k The set of the center points of the cluster, ( cx , cy () represents the normalized coordinates of the center of the detection box; S207: Generate a rectangular mask based on the clustering results, and extract the main regions of the image to obtain the interest image: ； ； in, M k ( u , v ) indicates the first k The area covered by the cluster's mask. u and v All indicate the first k The coordinates of the cluster's center point μ k,x Indicates the first k The x-coordinate of each cluster center w k Indicates the first k Cluster width, W Indicates the image width. μ k,y Indicates the first k The ordinate of each cluster center h k Indicates the first k Cluster height, H Indicates the image height. I masked ( u , v () represents the image after the region has been extracted. I ( u , v () represents the image before the extraction region. M ( u , v ) indicates the mask area M k ( u , v The union of ) is represented by ⊙, which indicates element-wise multiplication. S3: Based on YOLOv5 architecture, establish an object detection model, which includes: a backbone network module, a neck module, and a detection head module. Both the backbone network module and the neck module contain CBAM attention units. S4: Input the interest image into the target detection model for training until the loss function value of the target detection model is less than the preset loss function value; S5: Obtain the video of the steelmaking site to be identified; S6: Input the steelmaking site video to be identified into the trained target detection model and output the detection result; S7: Based on the detection results, output the steel mill material handling behavior identification results.

2. The target detection based steel plant material taking behavior recognition method according to claim 1, characterized in that, The types marked in S204 specifically include: person, material handling rod, and molten steel.

3. The target detection based steel plant material handling behavior recognition method of claim 1, wherein, The CBAM attention unit includes channel attention and spatial attention.

4. The target detection based steel plant material taking behavior recognition method according to claim 3, characterized in that, The channel attention is used to generate channel weights, and the generation of channel weights specifically includes: S301: Calculate the global average pooling and max pooling of the feature maps: ； ； in, F avg This represents the result of global average pooling of the feature map. X Indicates the input feature map, p Represents the pixel row index, g Indicates the pixel column index. The first colon (:) indicates that each input image is processed, and the second colon (:) indicates that each channel of the feature map is processed. S302: Input the global average pooling result and the max pooling result into the fully connected layer, process them through the ReLU activation function, and output the channel information of the feature map: ； in, The channel weights represent the results of global average pooling. The channel weights represent the results of max pooling. W 1 and W 2 represents the weight matrix of the fully connected layer. ReLU express ReLU Activation function; S303: Based on the channel information, generate the fused channel weights for each channel using the Sigmoid activation function: ； in, M c Indicates the weight of the fusion channel. X out This represents the output feature map after channel attention weighting. σ This represents the Sigmoid activation function.

5. The target detection based steel plant material taking behavior recognition method according to claim 3, characterized in that, The spatial attention is used to generate spatial weights, and the generation of the spatial weights specifically includes: S304: Calculate the average pooling and max pooling of the channel dimension in the channel attention output feature map: ； in, This represents the average pooling result along the channel dimension. This represents the result of max pooling along the channel dimension. c Indicates the first c aisle, C Indicates the total number of channels. X out This represents the output feature map after channel attention weighting; S305: Concatenate the average pooling result and the max pooling result along the channel dimension: ； in, M s Indicates spatial weights, W s This represents the weight matrix of the fully connected layer. || represents matrix multiplication, and || represents concatenation. S306: Based on the splicing result, spatial weights are generated using the sigmoid activation function. ； wherein, X final denotes the final output feature map.

6. The target detection based steel plant material handling behavior recognition method of claim 1, wherein, The specific formula for calculating the loss function value is as follows: ； in, L This represents the value of the loss function. λ box The weights represent the regression loss of the detection boxes. L box This represents the regression loss of the detection box. λ cls The weights represent the classification loss. L cls Represents classification loss, λ obj The weights representing the confidence loss L obj This represents the confidence loss.

7. The target detection based steel plant material handling behavior recognition method of claim 1, wherein, Specifically, S7 is: The post-processing module determines whether a material handling operation has occurred based on the class and location information in the detection results. ； in, C t express t A marker indicating whether a material handling operation has occurred at any given time. D t-n Indicates the first t - n Whether the category and position of each detection box meet the requirements. N This indicates the total number of detection frames. Indicates the first t +1 material handling time Indicates the first t The time required for the next material handling operation. Indicates the time threshold. The product symbol is used to represent the product.

8. A target detection based material handling behavior recognition system for a steel plant, the system comprising: include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the steel plant material handling behavior recognition method based on target detection as described in any one of claims 1 to 7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the steel mill material handling behavior recognition method based on target detection as described in any one of claims 1 to 7.

Images