Industrial casting chilling block identification method based on improved YOLOv11 and double-flow network

Abstract

The invention discloses an industrial casting chilling block identification method based on improved YOLOv11 and a double-flow network. The method comprises the steps that S1, a multi-source data graph of an industrial casting chilling block is acquired; s2, performing target category and bounding box coordinate labeling on the multi-source data graph to obtain a labeled data graph; preprocessing the annotated data graph to obtain a preprocessed image; s3, dividing the preprocessed image according to a preset proportion to obtain a training set and a test set; s4, constructing an industrial casting chilling block identification model based on the improved YOLOv11 and the double-flow network; and S5, constructing a total loss function of self-supervised learning, carrying out model training and evaluation on the constructed industrial casting chilling block identification model according to the training set and the test set, and obtaining an optimal industrial casting chilling block identification model so as to realize detection and identification of the industrial casting chilling block according to the optimal industrial casting chilling block identification model. The problem that in the prior art, under the low-light environment, the detection performance is insufficient, shielding and overlapping exist, a large amount of labeled data are relied on, and consequently the industrial casting chilling block recognition precision is low is solved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method for identifying industrial casting chills based on an improved YOLOv11 and a two-stream network. Background Technology

[0002] With the rapid development of industrial automation technology, intelligent production lines have been widely used in many fields, especially in the casting industry. The casting process typically requires a large number of auxiliary parts, and chills are one important type. Chills are components used in the casting process to guide metal flow and improve casting accuracy. They usually have relatively simple geometric shapes and are gray or metallic in color. However, due to the gray or metallic surface characteristics of the chill material itself, and because chills often accumulate during the casting process, they are difficult to distinguish accurately in complex environments. Therefore, this places high demands on the automatic detection technology of chills.

[0003] Existing methods for inspecting chills primarily rely on traditional manual sorting and machine vision-based target detection. In manual sorting, workers must manually observe and sort each chill, which is not only time-consuming and labor-intensive but also susceptible to errors due to operator skill levels and environmental factors. As production scales up, this method clearly cannot meet the demands for efficiency, accuracy, and automation.

[0004] While machine vision-based detection methods have made significant progress in recent years, particularly with the development of deep learning technology leading to breakthroughs in many fields, several challenges remain in the detection of chills in industrial casting. First, chills are typically exposed to low light conditions and complex background interference in industrial production environments, making the performance of existing traditional target detection methods less than ideal. Second, the gray appearance of chills is similar to the surface color of the cast metal, and chills are often stacked in groups or obscured by other objects during casting, making target detection algorithms prone to false positives or false negatives in these situations. Finally, although some existing deep learning algorithms can address these challenges to some extent, they typically utilize only single-sensor information, such as RGB images, neglecting additional information like infrared and depth images. This results in less than ideal performance in complex backgrounds and situations with multiple overlapping targets.

[0005] Currently, YOLO-based target detection algorithms have achieved good results in various applications, with YOLOv11, as a representative, widely used in the field of target detection due to its high efficiency and accuracy. However, the YOLOv11 model still faces performance bottlenecks when dealing with complex backgrounds, low-light environments, and target occlusion, especially in the detection of specific industrial targets such as chilled iron. Traditional YOLOv11 has not been optimized for the specific characteristics of such targets. Therefore, how to improve the accuracy of chilled iron target detection while ensuring real-time performance, especially in complex environments such as low light, overlap, and occlusion, is a pressing issue that needs to be addressed in current technology.

[0006] In summary, the existing technology has the following drawbacks:

[0007] (1) Insufficient performance in low-light environments: Although YOLOv11 performs well on many standard image datasets, its detection accuracy is insufficient in industrial environments with low or uneven lighting. Cold iron targets are often in shadow areas or environments with insufficient light, and existing technologies have low accuracy in such scenarios.

[0008] (2) Occlusion and overlap issues: Cold irons in industrial foundries are usually stacked together and have strong occlusion and overlap with other objects. Existing target detection methods, especially YOLOv11, do not have specific handling for overlap and occlusion between objects, resulting in serious missed detections and false detections.

[0009] (3) Dependence on a large amount of labeled data: Existing deep learning methods usually rely on a large amount of high-quality labeled data for training. However, in industrial applications, obtaining labeled data is both time-consuming and labor-intensive, and in some special cases (such as large changes in cold iron targets, strong environmental interference, etc.), it is difficult to obtain enough labeled data. This high dependence on labeled data limits the widespread application of existing technologies in some practical scenarios. Summary of the Invention

[0010] This invention provides a method for identifying chills in industrial casting based on an improved YOLOv11 and a dual-flow network, in order to overcome the aforementioned technical problems.

[0011] To achieve the above objectives, the technical solution of the present invention is as follows:

[0012] A method for identifying industrial casting chills based on an improved YOLOv11 and a two-stream network specifically includes the following steps:

[0013] S1: Obtain multi-source data images of industrial casting chills; and the multi-source data images include RGB images, infrared images, and depth images, i.e., three-dimensional structural feature maps of industrial casting chills;

[0014] S2: Label the target category and bounding box coordinates of the multi-source data map to obtain a labeled data map; preprocess the labeled data map to obtain a preprocessed image;

[0015] S3: Divide the preprocessed images into training and testing sets according to a preset ratio;

[0016] S4: Construct an industrial casting chill identification model based on improved YOLOv11 and dual-flow network;

[0017] The industrial casting chill identification model includes an input layer, an RGB image stream network based on an improved YOLOv11 backbone network, a multispectral image stream network based on an improved YOLOv11 backbone network, a feature fusion module, and a detection and identification module.

[0018] The input layer is used to perform feature processing on the RGB image in the preprocessed image to obtain the output feature map, and transmit it to the RGB image stream network; and to perform feature processing on the infrared image and depth image in the preprocessed image, and transmit them to the multispectral image stream network.

[0019] An RGB image stream network is used to extract visible light feature maps from RGB images, and the visible light features include at least size features and pixel features;

[0020] A multispectral image stream network is used to extract spectral feature maps of industrial casting chills based on infrared and depth images;

[0021] The feature fusion module is used to fuse the visible light feature map and the spectral feature map to obtain the fused feature map of industrial casting chills;

[0022] The detection and identification module is used to detect and identify industrial casting chills based on the fused feature map;

[0023] S5: Construct a total loss function for self-supervised learning, and train and evaluate the constructed industrial casting chill identification model based on the training set and test set to obtain the optimal industrial casting chill identification model, so as to realize the detection and identification of industrial casting chills based on the optimal industrial casting chill identification model.

[0024] Furthermore, the input layer in S4 includes the GhostFocus module and the GhostConv module connected in sequence;

[0025] Feature processing: The GhostFocus module downsamples the input preprocessed image, and the GhostConv module performs a lightweight convolution operation on the output of the GhostFocus module to obtain the output feature map; the formula for obtaining the output feature map is as follows:

[0026] F out=GhostConv(Focus(I in ))

[0027] In the formula: F out Indicates the output feature map; I in This indicates a preprocessed image.

[0028] Furthermore, the improved YOLOv11 backbone network in S4 includes a depthwise separable convolutional module, an EVCBlock module, a downsampling module, and a weighted bidirectional feature pyramid network BiFPN connected in sequence.

[0029] The depthwise separable convolution module is used to perform convolution operations on the input data to obtain depthwise convolutional feature maps; and the formula for calculating the number of parameters of the depthwise separable convolution module is as follows:

[0030] Params depthwise =K×K×C in

[0031] Params pointwise =1×1×C in ×C out

[0032] In the formula: K represents the convolution kernel size; C in C out These represent the number of input channels and the number of output channels of a depthwise separable convolution, respectively; Params depthwise The parameters representing a depthwise convolution; Params pointwise The parameters represent the pointwise convolution;

[0033] The EVCBlock module is used to simultaneously extract local and global feature maps from deep convolutional feature maps, and to perform weighted fusion of the local and global feature maps to obtain enhanced feature maps.

[0034] And the formula for obtaining the enhanced feature map is:

[0035] F evc =αF local +(1-α)F global

[0036] In the formula: F evc F represents the enhanced feature map; α represents the learnable weight coefficients; local Represents a local feature map; F global Represents the global feature map;

[0037] The downsampling module is used to perform downsampling operations on the enhanced feature map to obtain the downsampled feature map;

[0038] The weighted bidirectional feature pyramid network BiFPN is used to perform multi-scale feature fusion on deep convolutional feature maps, augmented feature maps, and downsampled feature maps to obtain a weighted feature map.

[0039] And the formula for obtaining the weighted feature map is:

[0040]

[0041] In the formula: C out Indicates the output fused feature map; F i The input feature maps include depthwise convolutional feature maps, augmented feature maps, and downsampled feature maps; w i ∈ represents the learnable fusion weights; ∈ represents the design constant to avoid division by zero errors.

[0042] Furthermore, the RGB image stream network described in S4 includes a first feature extraction module and a CBAM attention module based on an improved YOLOv11 backbone network;

[0043] The first feature extraction module is used to extract the weighted feature map of the RGB image after the input layer feature processing based on the improved YOLOv11 backbone network;

[0044] The CBAM attention module is used to extract attention features from the output of the first feature extraction module based on channel attention and spatial attention mechanisms, so as to fuse and obtain visible light feature maps.

[0045] And the expression for the CBAM attention module is:

[0046]

[0047] In the formula: F' represents the visible light characteristic map; M c Indicates the channel attention mapping operation; M s This represents the spatial attention mapping operation; represents the element-wise multiplication symbol; F represents the output of the first feature extraction module.

[0048] Furthermore, the multispectral image stream network described in S4 includes a channel stitching module, a dual-channel GhostFocus module, a second feature extraction module based on the improved YOLOv11 backbone network, and a mid-course modal feature fusion module.

[0049] The channel stitching module is used to stitch the infrared image and the depth image after the input layer feature processing to obtain a dual-channel input feature map;

[0050] The dual-channel GhostFocus module is used to perform downsampling operations on the infrared image and depth image in the dual-channel input feature map to obtain the dual-channel sampled feature map;

[0051] The second feature extraction module is used to extract the weighted feature map of industrial casting chills from the dual-channel sampling feature map, namely the infrared spectral feature map and the depth spectral feature map, based on the improved YOLOv11 backbone network.

[0052] The mid-mode feature fusion module is used to fuse infrared spectral feature maps and depth spectral feature maps using the Early Fusion mechanism to obtain spectral feature maps.

[0053] And the formula for obtaining the spectral feature map is:

[0054] F fusion =αF ir +(1-α)F depth

[0055] In the formula: F fusion Represents spectral characteristic map; F ir Indicates infrared spectral characteristics; F depth Indicates depth spectral characteristics.

[0056] Furthermore, the expression for the feature fusion module described in S4 is:

[0057] F fused =α·F flow1 +β·F flow2

[0058] In the formula: F fused This indicates the acquisition of the fusion feature map of industrial casting chills; F flow1 This represents the visible light feature map output by the RGB image stream network; F flow2 The output of the multispectral image stream network is represented as the spectral feature map; α and β represent the learnable weight coefficients.

[0059] The detection and recognition module detects and recognizes industrial casting chills based on the fused feature map, and its expression is as follows:

[0060] B^=(x 0 ,y 0 ,w,h)

[0061] P class =softmax(W class ·F fused )

[0062] In the formula: B^ represents the predicted bounding box; x 0 ,y 0 The coordinates of the center of the bounding box are represented by ; w and h represent the width and height of the bounding box; P class W represents the probability distribution of categories for industrial casting chills; classThis indicates the category classification weight of industrial casting chills.

[0063] Furthermore, the method for obtaining the optimal industrial casting chill identification model in S5 includes the following steps:

[0064] S31: Train the constructed industrial casting chill identification model based on the training set to obtain the trained industrial casting chill identification model:

[0065] S32: Construct the total loss function for self-supervised learning, and evaluate the trained industrial casting chill identification model based on the test set to determine whether the output of the trained industrial casting chill identification model has converged.

[0066] If so, then the industrial casting chill identification model trained at this time is the optimal industrial casting chill identification model.

[0067] Otherwise, the Adam optimizer, based on the backpropagation algorithm, is used to adaptively adjust the hyperparameters of the trained industrial casting chill identification model, and step S31 is repeated.

[0068] Furthermore, the total loss function for self-supervised learning constructed in S32 is:

[0069] L reconstruction =E[||xx^|| 2 ]

[0070] L contrastive =E[||f(x)-f(x)] + )|| 2 ]

[0071] L total =L det +λ1L reconstruction +λ2L contrastive

[0072] In the formula: L total L represents the total loss function; det λ1 and λ2 represent the target detection loss of industrial casting chills; L represents the weighting coefficients of the adjustment loss term. reconstruction Represents the reconstruction loss; x represents the original multi-source data map; x^ represents the reconstructed multi-source data map; L contrastive f(x) represents the contrast loss; f(x) represents the feature representation of the original multi-source data map; f(x) + ) represents the augmented feature representation of the original multi-source data graph in the contrastive learning task; E represents the expected loss value.

[0073] This invention provides a method for identifying chills in industrial castings based on an improved YOLOv11 and a two-stream network, with the following advantages:

[0074] (1) Performance improvement in low light environment: By introducing multimodal data streams of infrared and depth images, this invention significantly improves the detection capability of cold iron targets in low light environment. Compared with the traditional YOLOv11 model that only relies on the single mode of RGB images, this invention utilizes the temperature information provided by infrared images and the spatial structural features enhanced by depth images to accurately identify cold iron targets in low light environment, avoiding the problem of missed detection when there is insufficient light. In addition, the introduction of depth images enables the model to better understand the three-dimensional structure of the target, further improving the accuracy of target detection.

[0075] (2) Solving the occlusion and overlap problem: Traditional YOLOv11 models often struggle to provide high-precision detection results when faced with target stacking and occlusion problems, especially when chill targets overlap with other objects; by introducing an improved YOLOv11 backbone network and CBAM attention module, an industrial casting chill recognition model based on improved YOLOv11 and dual-stream network is constructed, which can enhance the attention to the target area. In particular, when the target is partially occluded or overlapped, the model can improve the detection accuracy of these areas by relying on enhanced local feature responses; the channel attention mechanism and spatial attention mechanism of the CBAM module effectively improve the model's perception ability of targets at different scales, especially the detection ability of the model is significantly enhanced in small targets and complex backgrounds.

[0076] (3) Reduce reliance on large amounts of labeled data: In many industrial applications, the cost of acquiring labeled data is high, and the shape of cold iron targets is varied and the background is complex, making it difficult to obtain high-quality labeled data. This invention introduces self-supervised learning tasks, especially occlusion reconstruction tasks and contrast learning tasks, so that the model can learn effective feature representations from unlabeled data without relying on a large amount of labeled data. This self-supervised learning framework can significantly improve training efficiency, reduce reliance on manual labeling, and enhance the adaptability of the model, making it particularly suitable for industrial scenarios where labeled data is scarce. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 The flowchart shows the industrial casting chill identification method based on the improved YOLOv11 and dual-flow network of the present invention.

[0079] Figure 2This is an example diagram of the multi-source data collected in this embodiment;

[0080] Figure 3 This is a schematic diagram of the industrial casting chill identification model based on the improved YOLOv11 and two-stream network constructed in this embodiment.

[0081] Figure 4 This is a diagram showing the recognition effect of the optimal industrial casting chill recognition model in this embodiment. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] This embodiment provides a method for identifying chills in industrial castings based on an improved YOLOv11 and a two-stream network, such as... Figure 1 As shown, the specific steps include:

[0084] S1: Obtain multi-source data images of industrial casting chills; and the multi-source data images include RGB images, infrared images, and depth images, i.e., three-dimensional structural feature maps of industrial casting chills;

[0085] Specifically, such as Figure 2 To train and test the target detection model for industrial casting chills, a high-quality dataset suitable for chill recognition tasks was first collected and created. Specifically, multiple industrial cameras and sensors were used for data acquisition to ensure data diversity and meet the application needs of various industrial casting sites. The dataset includes RGB images, infrared images, and depth images. The specific acquisition equipment is as follows:

[0086] (1) RGB Images: Images of chilled iron were captured using a Canon EOS 5D Mark IV high-definition industrial camera with a resolution of 1920x1080 under different stacking methods and lighting conditions. This camera uses a CMOS sensor, which can provide clear image details and adapt to high dynamic range environments.

[0087] (2) Infrared image (IR): The FLIRA655sc infrared thermal imaging camera with a resolution of 640x480 was used to capture the heat distribution of cold iron at different temperatures, and it performed well, especially in low light conditions.

[0088] (3) Depth: An Intel RealSense D435 depth camera with a resolution of 640x480 was used to obtain the depth information of the chill in space and provide the three-dimensional structural features of the chill.

[0089] The multi-source data map in this embodiment includes 5000 images from different environments, angles, and stacking states, ensuring the diversity of the dataset and meeting various needs in actual industrial casting environments. Each image contains at least one type of chilled iron target, specifically including:

[0090] 1. Block-shaped Cold Iron

[0091] 2. Ring-shaped cold iron structure

[0092] 3. Strip-shaped cold iron structure

[0093] 4. Bolt-hole Cold Iron Structure

[0094] S2: Label the target category and bounding box coordinates of the multi-source data map to obtain a labeled data map; preprocess the labeled data map to obtain a preprocessed image;

[0095] Specifically, all collected multi-source data images need to undergo manual annotation and a series of preprocessing steps to ensure data consistency and prepare for subsequent model training. The annotation tool used is LabelImg, and the annotation data for each image is saved in Pascal VOC format. The annotation process includes:

[0096] S001: Bounding box annotation: The cold iron targets in each image are annotated with rectangular boxes. The annotation includes the target category (cold iron block, cold iron ring structure, cold iron strip structure, cold iron bolt structure) and the bounding box coordinates. These coordinate values ​​represent the center coordinates (x, y) of the bounding box as well as the width (w) and height (h), which will be consistent with the bounding box representation in subsequent model training.

[0097] S002: XML file storage: The category label information for each image is stored in XML file format. Each XML file contains the image file name, target category and its corresponding bounding box coordinates. This annotation information will be used for data reading during the training and validation phases.

[0098] To ensure data consistency and model training effectiveness, all collected multi-source data graphs in this embodiment undergo the following preprocessing operations:

[0099] S100: Image size adjustment: Adjust the size of all RGB images, infrared images and depth images to 640x640 pixels;

[0100] S200: Pixel Normalization: Normalizes the pixel values ​​of each image to the range [0,1] to ensure the uniformity of the input data. Its expression is:

[0101]

[0102] S300: Denoising: Gaussian filtering is used to denoise the infrared and depth images to reduce the impact of environmental noise on the training effect of the model.

[0103] S400: Data Augmentation: To increase data diversity and avoid overfitting the model to specific scenarios, this invention employs common image augmentation techniques such as rotation, scaling, flipping, and color dithering. These techniques are widely used in object detection tasks and are considered existing technologies. However, this embodiment applies these basic techniques to a specific industrial casting environment, especially under low light, occlusion, and complex background conditions, improving the model's robustness and generalization ability. Therefore, although these techniques themselves are existing technologies, they have significant innovative value and practical significance in the specific application scenario of this embodiment.

[0104] After data acquisition and preprocessing, the final dataset in this embodiment includes 5000 images covering four categories of cold iron targets. Each image is labeled with its category, and the labeling information includes the category of the cold iron target and its corresponding bounding box coordinates (x, y, y). min ,y min ,x max ,y max The preprocessed image data will be used for subsequent object detection model training and testing.

[0105] S3: Divide the preprocessed images into training and testing sets according to a preset ratio;

[0106] S4: Construct an industrial casting chill identification model based on improved YOLOv11 and dual-flow network;

[0107] like Figure 3 As shown, the industrial casting chill identification model includes an input layer, an RGB image stream network based on an improved YOLOv11 backbone network, a multispectral image stream network based on an improved YOLOv11 backbone network, a feature fusion module, and a detection and identification module.

[0108] The input layer is used to perform feature processing on the RGB image in the preprocessed image to obtain the output feature map, and transmit it to the RGB image stream network; and to perform feature processing on the infrared image and depth image in the preprocessed image, and transmit them to the multispectral image stream network.

[0109] Specifically, the input layer includes the GhostFocus module and the GhostConv module connected in sequence;

[0110] Feature processing: At the input end, the preprocessed input image is downsampled by 4×4 (i.e., 2×2 partitioning + stacking) using the GhostFocus module. Then, the feature representation is expanded using the GhostConv module, which performs a lightweight convolution operation on the output of the GhostFocus module to obtain the output feature map. Simultaneously, a linear transformation is used to generate redundant features and significantly reduce computation. The formula for obtaining the output feature map is as follows:

[0111] F out =GhostConv(Focus(I in ))

[0112] In the formula: F out The output feature map, whose size and number of channels are determined by the size of the input image, the GhostConv operation, and the size of the convolution kernel; I in Indicates a preprocessed image;

[0113] In a specific embodiment, the improved YOLOv11 backbone network includes a depthwise separable convolution module, an enhanced visual context block (EVCBlock) module, a downsampling module, and a weighted bidirectional feature pyramid network (BiFPN) connected in sequence.

[0114] The depthwise separable convolution module is used to perform convolution operations on the input data to obtain depthwise convolution feature maps. In this embodiment, depthwise separable convolution is used to replace traditional convolution to reduce the number of parameters. Its single-layer convolution consists of a depthwise convolution (channel-wise convolution) and a pointwise convolution (1×1 convolution).

[0115] Furthermore, the formula for calculating the number of parameters in a depthwise separable convolutional module is as follows:

[0116] Params depthwise =K×K×C in

[0117] Params pointtwise =1×1×C in ×C out

[0118] In the formula: K represents the convolution kernel size; C in C out These represent the number of input channels and the number of output channels of a depthwise separable convolution, respectively; Params depthwise The parameters representing a depthwise convolution; Params pointwise The parameters represent the pointwise convolution; however, the number of parameters in a depthwise separable convolution does not directly reflect the output feature map F. out However, its computation depends on the number of channels in the input feature map and the design of the convolution kernel.

[0119] The EVCBlock module is used to simultaneously extract local and global feature maps from deep convolutional feature maps, and to perform weighted fusion of the local and global feature maps to obtain enhanced feature maps.

[0120] This embodiment introduces the EVCBlock module into the backbone network. The EVCBlock module enhances feature representation by simultaneously extracting local details and global contextual information. Specifically, it first extracts local features using a 3×3 standard convolution, then performs global average pooling (GAP), followed by a 1×1 convolution to generate global features. Finally, the local and global features are fused using weighted averages to obtain the final enhanced features. The formula for obtaining the enhanced feature map is as follows:

[0121] F evc =αF local +(1-α)F global

[0122] In the formula: F evc F represents the enhanced feature map; α represents the learnable weight coefficients; local Represents a local feature map; F global This represents the global feature map; in this embodiment, the default initial value of α is 0.5. The EVC Block module is typically introduced at key locations in the backbone network, especially for effectively fusing small target features and global contextual features. The placement of the EVCBlock module aims to capture richer contextual information, thereby improving target detection accuracy. The weighted fusion operation in the EVCBlock module does not significantly increase the number of additional parameters. Since the fusion process is based on a linear combination of feature maps and does not involve additional convolution operations, it is similar to Params. depthwise and Params pointwise The computation is not directly affected. The introduction of the EVC Block module is more of a weighted fusion of existing features, without adding new convolutional layers or significantly increasing the computational cost, which is beneficial for enhancing the feature map F. evc(That is, the feature map output by the EVCBlock module), generally speaking, does not require additional downsampling operations because the computational cost has already been effectively reduced in the network through operations such as depthwise convolution and pointwise convolution, and the output feature map of the EVCBlock module has already been integrated into features suitable for subsequent networks. Therefore, the features after the EVCBlock module... Figure 1 Generally, the original size will be maintained unless there is a clear need for downsampling operations in subsequent layers.

[0123] The downsampling module is used to perform downsampling operations on the enhanced feature map to obtain the downsampled feature map;

[0124] Specifically, in the backbone network, in order to reduce the spatial size while ensuring information preservation, a depthwise separable convolution with a stride of 2 is used. This operation effectively reduces the spatial size of the feature map and maintains the richness of information.

[0125] In this embodiment, the improved backbone network output includes three feature layers of different scales, corresponding to:

[0126] P3 layer, i.e., depthwise convolutional feature map: size 80×80, number of channels 128;

[0127] P4 layer is the enhanced feature map: size 40×40, number of channels 256;

[0128] P5 layer is the downsampled feature map: size 20×20, number of channels 512.

[0129] The weighted bidirectional feature pyramid network BiFPN is used to perform multi-scale feature fusion on deep convolutional feature maps, augmented feature maps, and downsampled feature maps to obtain a weighted feature map.

[0130] Specifically, this embodiment employs an improved version of the Weighted Bidirectional Feature Pyramid Network (BiFPN) for multi-scale feature fusion. BiFPN introduces independent, learnable weights for each fusion path to dynamically adjust feature importance, thereby effectively fusing features at different scales. Compared to the traditional Feature Pyramid Network (FPN), BiFPN not only improves the flexibility of feature fusion but also optimizes the importance of each feature layer by learning weights, thus enhancing the network's ability to detect multi-scale targets. BiFPN receives multiple feature maps of different scales (e.g., P3, P4, P5 layers) from the backbone network and combines them through weighted fusion. These feature maps are obtained by downsampling in previous stages using convolution operations with a stride of 2, representing image information at different resolutions. BiFPN dynamically adjusts the importance of features at each scale by learning weights, thereby effectively fusing features at different scales. The weights on each fusion path are learnable, representing the contribution of each scale feature.

[0131] And the formula for obtaining the weighted feature map is:

[0132]

[0133] In the formula: C out Indicates the output fused feature map; F i The input feature maps include depthwise convolutional feature maps, augmented feature maps, and downsampled feature maps; w i Represents the learnable fusion weights; ∈ represents the design constant (10) to avoid division-by-zero errors. -4 ).

[0134] An RGB image stream network is used to extract visible light feature maps from RGB images, and the visible light features include at least size features and pixel features;

[0135] Specifically, the RGB image stream network in S4 includes a first feature extraction module and a CBAM attention module based on an improved YOLOv11 backbone network;

[0136] The first feature extraction module is used to extract the weighted feature map of the RGB image after the input layer feature processing based on the improved YOLOv11 backbone network; specifically, the RGB image stream network processes the standard visible light RGB image, and its input is a three-channel image of H×W×3 (640×640×3);

[0137] The CBAM attention module is used to extract attention features from the output of the first feature extraction module based on channel attention and spatial attention mechanisms, so as to fuse and obtain visible light feature maps.

[0138] To further optimize feature fusion, BiFPN introduces the CBAM (Convolutional Block Attention Module) attention mechanism in this embodiment. CBAM enhances the attention to important features by introducing channel attention and spatial attention. Specifically, CBAM calculates attention weights based on the channel and spatial dimensions of each feature map, thereby helping the network to better focus on important features during the fusion process.

[0139] And the expression for the CBAM attention module is:

[0140]

[0141] In the formula: F' represents the visible light characteristic map; M c Indicates the channel attention mapping operation; M s This represents the spatial attention mapping operation; Indicates the element-wise multiplication symbol; F represents the output of the first feature extraction module;

[0142] In this embodiment, the RGB image stream network further includes a detection head: a decoupled detection head is used to process the regression and classification of the visible light feature map separately, and the output format is:

[0143] Output = (B, s, s) × 3 × (cls + 5)

[0144] In this context, FocalLoss is used for classification prediction, CIoULoss is used for regression bounding boxes, cls represents the number of categories (the number of categories in the classification task), B represents the batch size, and S represents the spatial size of the input feature map.

[0145] The multispectral image stream network is used to extract spectral feature maps of industrial casting chills based on infrared and depth images. In this embodiment, the multispectral image stream network is responsible for processing image inputs composed of infrared (IR) and depth maps. Its input dimension is H×W×2, consistent with the RGB stream. The multispectral stream also uses the improved YOLOv11 backbone network, but specific adaptations are performed in the input processing and early feature processing parts. Specifically, the multispectral image stream network includes a channel stitching module, a dual-channel GhostFocus module, a second feature extraction module based on the improved YOLOv11 backbone network, and a mid-process modal feature fusion module.

[0146] The channel stitching module is used to stitch the infrared image and the depth image after the input layer feature processing to obtain a dual-channel input feature map;

[0147] Specifically, infrared and depth image preprocessing: The infrared and depth images are standardized to ensure the consistency of their input features. In this step, the two images are resized and pixel normalized so that they can be appropriately input into the network. The infrared and depth images are resized to 640x640 and the pixel values ​​are normalized to the range of [0,1] to ensure that data from different sensors can be processed in the same feature space. In order to make full use of the complementary information of infrared and depth images, the infrared image and the depth image are channel-stitched to form an H×W×2 dual-channel input.

[0148] The dual-channel GhostFocus module is used to perform downsampling operations on the infrared image and depth image in the dual-channel input feature map to obtain the dual-channel sampled feature map;

[0149] The second feature extraction module is used to extract the weighted feature map of industrial casting chills from the dual-channel sampling feature map, namely the infrared spectral feature map and the depth spectral feature map, based on the improved YOLOv11 backbone network.

[0150] The mid-mode feature fusion module is used to fuse infrared spectral feature maps and depth spectral feature maps using the Early Fusion mechanism to obtain spectral feature maps.

[0151] In this embodiment, due to the significant differences in characteristics between infrared and depth images, an Early Fusion mechanism is employed to fuse the feature information of the infrared and depth image streams. Unlike the RGB image stream network structure, in the mid-feature fusion stage, the streams involved are both infrared and depth image streams. After processing by their respective networks, the output features of these streams are fused. Therefore, the improved YOLOv11 backbone network used here employs different processing methods for each input modality (RGB, infrared, depth), while maintaining a consistent overall structure to ensure the network can handle multimodal data. Specifically, after the output of layer P3, a feature-weighted fusion method is introduced to obtain the spectral feature map.

[0152] And the formula for obtaining the spectral feature map is:

[0153] F fusion =αF ir +(1-α)F depth

[0154] In the formula: F fusion Represents spectral characteristic map; F ir Indicates infrared spectral characteristics; F depth α represents the depth spectral features, and α represents the learnable parameter, which is initialized to 0.5 and can be automatically optimized during training.

[0155] The feature fusion module is used to fuse the visible light feature map and the spectral feature map to obtain the fused feature map of industrial casting chills;

[0156] In this embodiment, the output fused feature map in the feature fusion module is fed into the YOLOv11 Neck structure to maintain a consistent feature alignment strategy. The goal of this module is to perform more refined fusion of features from different sources, providing more comprehensive information for subsequent object detection. Unlike the mid-modal feature fusion module, which is primarily performed after the P3 layer output and involves fusing features from infrared and depth images, this module fuses features from these two modalities using a specific weighting method after the input data has undergone appropriate modal processing. Its focus is on the joint processing of data from different sensors in the early stages, aiming to maximize the complementarity of multimodal data and enhance the model's adaptability to complex scenes such as low light and occlusion. The feature fusion module, on the other hand, operates in the intermediate stage after multi-layer feature map extraction, primarily fusing feature maps from different layers (e.g., P3, P4, P5). At this point, the network has extracted multi-level feature information. The feature fusion module further refines the combination of multi-scale features through the Neck structure. Unlike mid-process modal feature fusion, the main purpose of the feature fusion module is to enhance the spatial and semantic consistency of features at different levels, thereby providing richer global information for the final target detection. In this embodiment, through this fusion, the model can simultaneously utilize information from different sensors to enhance the ability to detect targets, especially in low-light or complex background scenes.

[0157] The expression for the feature fusion module is:

[0158] F fused =α·F flow1 +β·F flow2

[0159] In the formula: F fused This indicates the acquisition of the fusion feature map of industrial casting chills; F flow1 This represents the visible light feature map output by the RGB image stream network; F flow2 The output of the multispectral image stream network is the spectral feature map; α and β represent learnable weight coefficients. This fusion method can effectively combine features from two different streams, improving the ability to detect targets under different environmental conditions.

[0160] The detection and identification module is used to detect and identify industrial casting chills based on the fused feature map;

[0161] In this embodiment, during the detection head stage, the fused multimodal feature map is fed into the decoupled detection head for bounding box regression and target classification prediction. At this time, the model independently completes the classification (predicting the target category) and regression (predicting the target location) tasks. Detection results: The detection results are generated by the decoupled detection head and include the position of the bounding box (regression output) and the target classification information (classification output). These detection results do not directly refer to the output of the multispectral image stream network, but are the final target detection results output after multimodal information fusion.

[0162] In this embodiment, the detection head adopts a decoupled head design, which handles classification tasks, bounding box regression tasks, and target confidence prediction tasks respectively. Each task corresponds to a separate convolutional layer, all of which use the SiLU activation function.

[0163] And the loss functions for each task are as follows:

[0164] 1. Classification task: Focal Loss is used to enhance attention to difficult samples;

[0165] 2. Bounding box regression task: Using CIoULoss, specifically defined as:

[0166]

[0167] In the formula: ρ represents the distance between the center points; c represents the diagonal length; αv represents the shape consistency term; target confidence is usually used to represent the model's confidence in the existence of the target. Specifically, the expression for target confidence is:

[0168] P obj =σ(W obj ·F')

[0169] In the formula: W obj σ represents the weights for predicting target confidence; F' represents the feature map after weighted fusion of CBAM and BiFPN; σ represents the sigmoid activation function, used to output the probability value of target presence. In summary, the fully improved YOLOv11 backbone network significantly enhances feature representation capabilities and detection accuracy while maintaining lightweight efficiency, making it particularly suitable for small target and multimodal scene detection.

[0170] Specifically, the detection and identification module detects and identifies industrial casting chills based on the fused feature map, and its expression is as follows:

[0171] B^=(x 0 ,y 0 ,w,h)

[0172] P class =softmax(Wclass ·F fused )

[0173] In the formula: B^ represents the predicted bounding box; x 0 ,y 0 The coordinates of the center of the bounding box are represented by ; w and h represent the width and height of the bounding box; P class W represents the probability distribution of categories for industrial casting chills; class This represents the category classification weights for industrial casting chills. Ultimately, the model output includes the following components:

[0174] 1. Bounding Box Coordinates: The position of each target, including the center coordinates (x, y) and the width (w) and height (h) of the bounding box;

[0175] 2. Target Category: The category of each detected target (cold iron block, cold iron ring structure, cold iron strip structure, cold iron strip bolt structure);

[0176] 3. Confidence: The confidence value of each target box represents the model's confidence in classifying that target.

[0177] S5: Construct a total loss function for self-supervised learning, and train and evaluate the constructed industrial casting chill identification model based on the training set and test set to obtain the optimal industrial casting chill identification model, so as to realize the detection and identification of industrial casting chills based on the optimal industrial casting chill identification model.

[0178] In a specific embodiment, the method for obtaining the optimal industrial casting chill identification model includes the following steps:

[0179] S31: Train the constructed industrial casting chill identification model based on the training set to obtain the trained industrial casting chill identification model:

[0180] S32: Construct the total loss function for self-supervised learning, and evaluate the trained industrial casting chill identification model based on the test set to determine whether the output of the trained industrial casting chill identification model has converged.

[0181] If so, then the industrial casting chill identification model trained at this time is the optimal industrial casting chill identification model.

[0182] Otherwise, the Adam optimizer, based on the backpropagation algorithm, is used to adaptively adjust the hyperparameters of the trained industrial casting chill identification model, and step S31 is repeated.

[0183] Specifically, the total loss function for the constructed self-supervised learning is as follows:

[0184] In this embodiment, two self-supervised learning tasks were designed to train and obtain an industrial casting chill identification model that learns effective features from unlabeled data:

[0185] (1) Occlusion Reconstruction Task: A portion of the input image is randomly occluded (10% of the image is occluded). The model is required to reconstruct the occluded region based on the remaining image portion. This process uses a self-supervised learning model, which is not directly used for classification or regression tasks. Instead, it enhances the network's ability to understand global features of the image by learning to reconstruct the occluded region. Through this task, the model can learn more robust feature representations in the face of image occlusion, thereby improving its performance in subsequent classification or object detection tasks. This method does not conflict with traditional classification models but enhances the model's feature learning ability. Specifically, the classification and regression models still exist, but their performance is improved by the occlusion reconstruction task during training. The formula for calculating the reconstruction loss is:

[0186] L reconstruction =E[||xx^|| 2 ]

[0187] In the formula: x represents the original multi-source data map; x^ represents the reconstructed multi-source data map;

[0188] (2) Contrastive Learning Task: In the contrastive learning task, multiple augmented versions are generated for each image. Contrastive loss is used to drive the model to learn feature representations of more similar images and to distance feature representations of dissimilar images. Specifically, the loss function calculates the similarity between features of each pair of images and optimizes the model so that features of similar images are closer together and features of dissimilar images are more separated. The formula for calculating the contrastive loss is:

[0189] L contrastive =E[||f(x)-f(x)] + )|| 2 ]

[0190] In the formula: f(x) represents the feature representation of the original multi-source data map; f(x) + ) represents the augmented feature representation of the original multi-source data image in the contrastive learning task; in this way, the model not only learns how to distinguish different images during training, but also optimizes the representation of the feature space, making the features of similar images closer together, while the features of dissimilar images are further apart in the feature space.

[0191] (3) Joint training: In order to make full use of the advantages of the object detection task and the self-supervised learning task, this embodiment constructs a joint training framework. This framework combines the object detection loss of YOLOv11 with the loss of the self-supervised learning task (reconstruction loss and contrast loss) to improve the performance of the model through joint training.

[0192] The object detection part uses an improved YOLOv11 architecture to calculate the localization and classification errors of the bounding boxes. During training, the YOLOv11 detection head calculates the position and category of the cold iron target in the image, thereby generating the object detection loss (L). det This is the main loss term during training, ensuring the model can accurately detect cold iron targets in multispectral images. The loss for self-supervised learning includes reconstruction loss (L...). reconstruction ) and contrast loss (L contrastive Through these two losses, the model can learn stronger feature representations from unlabeled data. The reconstruction task helps the model recover target features even with image occlusion, while the contrastive learning task helps the model learn more robust feature representations through diverse augmented images. The final total loss function is a weighted combination of the object detection loss and the self-supervised learning loss, and the formula for calculating the total loss function is:

[0193] L total =L det +λ1L reconstruction +λ2L contrastive

[0194] In the formula: L det Let λ1 = 1 and λ2 = 1 represent the object detection loss. These are hyperparameters used to adjust the weights of the loss term (reconstruction loss and contrast loss contribute equally to the total loss). During training, the Adam optimizer is used to optimize the parameters and minimize the total loss function. For each training batch, the loss is calculated through forward propagation, and then the model parameters are updated through backpropagation. Training continues until the model's performance reaches its optimal level on the validation set.

[0195] This embodiment also includes model deployment and application.

[0196] Model Deployment: In this embodiment, a dual-stream network model, namely an RGB image stream network and a multispectral image stream network, is used. This model performs target detection by fusing RGB and multispectral image streams. The goal of deployment is to use this model in a production line monitoring system to achieve accurate detection and classification of chilled iron targets. The deployment hardware selected is the NVIDIA Jetson Xavier AGX as the edge computing device, equipped with an NVIDIA Volta architecture GPU, 32GB of RAM, and 512GB of SSD storage. This device supports GPU acceleration, enabling efficient execution of the dual-stream network model's inference tasks. The operating system used is Ubuntu 18.04 LTS, and computational acceleration utilizes CUDA 11.2 and cuDNN 8.0. During deployment, the model is optimized using TensorRT 7.2 to ensure inference speed. The deep learning frameworks used are TensorFlow 2.4.1 and Keras 2.4.3, and the Python version is Python 3.8.

[0197] The specific deployment process is as follows:

[0198] Use the `tf.saved_model.save()` method to save the trained, optimal industrial casting chill recognition model as a SavedModel format; use TensorRT's `trtexec` tool to convert the SavedModel format model into a TensorRT-optimized model to improve inference speed; upload the TensorRT-optimized model file and necessary dependencies to the NVIDIA Jetson Xavier AGX device; configure the network connection between the device and the industrial production line to ensure that the device can receive image data from sensors in real time and perform target detection.

[0199] After the model is deployed, object detection and classification will be performed according to the following process:

[0200] (1) Data acquisition: The production line monitoring system acquires 10 images per second through RGB, infrared and depth sensors and transmits them to the equipment for processing in real time.

[0201] (2) Data preprocessing: All acquired image data will undergo the following preprocessing steps to ensure that it can be successfully fed into the two-stream network model for inference in step two:

[0202] First, the images (RGB, infrared, and depth images) were resized to 640x640 pixels to meet the input requirements of the dual-stream network. Then, the pixel values ​​of each image were normalized to between 0 and 1 to ensure consistent processing of input data from different sensors. Finally, Gaussian filtering was applied to the infrared and depth images to reduce noise interference with model training and inference.

[0203] (3) Target detection: The preprocessed image data will be used for target detection through the optimal industrial casting chill recognition model. The model will perform the following processing on the image:

[0204] RGB Image Stream: This stream uses a modified YOLOv11 architecture, introducing a Spatial Attention (SAM) mechanism and a Convolutional Block Attention (CBAM) module to improve object detection accuracy. These modules allow the model to adaptively adjust its sensitivity to targets at different scales, enhancing target localization accuracy.

[0205] Multispectral image stream: This stream processes infrared and depth images and integrates features from both sensors through weighted fusion, thereby enhancing adaptability to complex environments such as low light and occlusion.

[0206] The output of the object detection model will include: bounding box coordinates: the center coordinates (x, y), width (w), and height (h) of the bounding box; object category: the predicted category of the cold iron object, classified as "cold iron block", "cold iron ring structure", "cold iron strip structure", or "cold iron with bolt structure"; confidence score: the confidence score value of the object detection result, indicating whether the bounding box is accurate.

[0207] (4) The target detection results will undergo the following processing steps:

[0208] Non-maximum suppression (NMS): Set the IOU threshold to 0.4 to remove target boxes with large overlap and keep only boxes with high confidence; Confidence filtering: Only retain target boxes with confidence values ​​higher than the set threshold (0.5), and other low confidence boxes will be filtered out.

[0209] (5) Cold Iron Target Classification: Based on the target bounding box output, the system classifies the detected cold iron targets. Cold iron targets are classified into the following four types:

[0210] 1. Block-shaped cold iron: Block-shaped or flat cold iron, commonly used for support or fixation. 2. Ring-shaped cold iron: Curved cold iron, commonly used for connecting or fixing structures. 3. Strip-shaped cold iron: Strip-shaped or bar-shaped cold iron, typically used for support or structural parts. 4. Bolt-hole cold iron: Cold iron with bolt holes, used for fixing or connecting other parts.

[0211] Figure 4 The final output example image of the model is shown. Through target detection and classification, the deployed optimal industrial casting chill identification model will automatically identify and label different types of chill targets.

[0212] To comprehensively evaluate the performance of the industrial casting chill identification method based on improved YOLOv11 and dual-stream network in this embodiment, this experiment conducted a detailed comparison between the method described in this embodiment and YOLOv11, YOLOv4, YOLOv5, and Faster R-CNN. All models were trained and tested on the same hardware and dataset. The specific experimental setup is as follows:

[0213] 1. Hardware Device: This embodiment uses an NVIDIA Jetson Xavier AGX as the inference hardware device, configured with 32GB RAM and 512GB SSD storage, and equipped with an NVIDIA Volta architecture GPU. This device supports GPU acceleration, ensuring efficient image processing and object detection tasks, and is suitable for real-time inference on industrial production lines.

[0214] 2. Dataset: The dataset contains 5,000 images collected at industrial foundry sites, covering four types of cold iron targets: block-shaped cold iron, ring-shaped cold iron, strip-shaped cold iron, and bolt-hole cold iron. These images include RGB images, infrared images, and depth images, covering various lighting conditions, complex backgrounds, target occlusion, and target overlap.

[0215] Image data acquisition was accomplished using the following devices:

[0216] (1) RGB image: Canon EOS 5D Mark IV industrial camera with a resolution of 1920x1080 was used;

[0217] (2) Infrared image: FLIRA655sc infrared thermal imaging camera with a resolution of 640x480 was used;

[0218] (3) Depth image: Using an Intel RealSense D435 depth camera with a resolution of 640x480;

[0219] Each image was manually labeled, including bounding box coordinates, target category, and category label. All images underwent standardization preprocessing before training. First, the image size was uniformly adjusted to 640x640 pixels. Then, the image pixel values ​​were normalized to the [0,1] range. Gaussian filtering was applied to the infrared and depth images to remove noise. Data augmentation techniques were then applied, including random rotation (angle range: -15° to 15°), random scaling (0.8x to 1.2x), horizontal flipping, and random cropping, to increase the diversity of the training data.

[0220] All models were trained using TensorFlow 2.4.1 with the Adam optimizer, an initial learning rate of 0.001, a batch size of 16, and 300 training epochs. The loss function used was:

[0221] (1) Focal Loss (used to handle class imbalanced classification tasks);

[0222] (2) CIoULoss (used for bounding box regression to improve positioning accuracy);

[0223] During training, an early stopping strategy is adopted: training is stopped when the validation set mAP fails to improve for 10 consecutive iterations.

[0224] 3. Comparison Results

[0225] The main evaluation metrics for the experimental results include mAP (mean accuracy), inference speed (FPS), accuracy, and inference time. Table 1 shows the comparison results between the method described in this embodiment and YOLOv11, YOLOv4, YOLOv5, and Faster R-CNN:

[0226] Table 1. Model Performance Comparison Results

[0227] Model Average accuracy Reasoning speed Accuracy This invention 92.3％ 63FPS 96.5％ YOLOv11 87.1％ 54FPS 92.8％ YOLOv4 85.7％ 46FPS 92.0％ YOLOv5 88.5％ 56FPS 93.8％ Faster R-CNN 82.6％ 39FPS 90.5％

[0228] The method described in this embodiment has significant advantages, especially in low-light conditions, target occlusion, and complex backgrounds. Firstly, compared to YOLOv11, the method described in this embodiment improves the mAP by 5.2 percentage points (from 87.1% to 92.3%), and the inference speed also reaches 63 FPS, significantly outperforming YOLOv4 and YOLOv5. This is because this invention introduces a multispectral image stream (fusing RGB, infrared, and depth images) and an improved network structure (Ghost Focus module and CBAM attention module) on top of YOLOv11. These innovations enable the model to extract features more accurately, especially in complex industrial environments, adapting to low-light conditions, occlusion, and multiple overlapping targets. Compared to YOLOv4 and YOLOv5, the method described in this embodiment is superior in inference speed while maintaining high detection accuracy. The mAP of YOLOv4 and YOLOv5 are 85.7% and 88.5%, respectively, significantly lower than the 92.3% of this invention. Furthermore, in object detection tasks, YOLOv4 achieves an inference speed of 46 FPS, and YOLOv5 achieves 56 FPS, both lower than the 63 FPS of this invention. This means that the method described in this embodiment can run in real-time on industrial production lines without affecting production efficiency. This is achieved by optimizing the network structure, reducing redundant computation, and utilizing the GPU acceleration of NVIDIA Jetson Xavier AGX. The method described in this embodiment is more suitable for real-time industrial inspection applications and can handle more complex environments. Compared to Faster R-CNN, although Faster R-CNN performs better in high-precision object detection, its inference speed is slower, providing only 39 FPS, which is not suitable for real-time detection tasks. Moreover, Faster R-CNN is insufficient in handling occluded and overlapping objects, with an mAP of 82.6%, resulting in lower detection accuracy in cases of stacked cold iron targets and occlusion.

[0229] In summary, compared with existing target detection models (especially YOLOv11, YOLOv4, YOLOv5, and Faster R-CNN), the method described in this embodiment not only demonstrates superior detection accuracy, but also exhibits stronger adaptability in low-light and target occlusion environments. Furthermore, it maintains high inference speed and low inference time, fully meeting the real-time requirements of industrial production lines. This method enables efficient and accurate detection and automatic classification of chilled iron targets in industrial foundries, solving problems such as low light, occlusion and overlap, and insufficient labeled data. It improves the automation level of industrial production lines, reduces manual intervention, and enhances sorting efficiency and production safety.

[0230] The beneficial effects of the method described in this embodiment are as follows:

[0231] (1) Performance improvement in low light environment: By introducing multimodal data streams of infrared and depth images, this invention significantly improves the detection capability of cold iron targets in low light environment. Compared with the traditional YOLOv11 model that only relies on the single mode of RGB images, this invention utilizes the temperature information provided by infrared images and the spatial structural features enhanced by depth images to accurately identify cold iron targets in low light environment, avoiding the problem of missed detection when there is insufficient light. In addition, the introduction of depth images enables the model to better understand the three-dimensional structure of the target, further improving the accuracy of target detection. (2) Solving the occlusion and overlap problem: Traditional YOLOv11 models often struggle to provide high-precision detection results when faced with target stacking and occlusion problems, especially when chill targets overlap with other objects; by introducing an improved YOLOv11 backbone network and CBAM attention module, an industrial casting chill recognition model based on improved YOLOv11 and dual-stream network is constructed, which can enhance the attention to the target area. In particular, when the target is partially occluded or overlapped, the model can improve the detection accuracy of these areas by relying on enhanced local feature responses; the channel attention mechanism and spatial attention mechanism of the CBAM module effectively improve the model's perception ability of targets at different scales, especially the detection ability of the model is significantly enhanced in small targets and complex backgrounds. (3) Reduced dependence on large amounts of labeled data: In many industrial applications, the cost of acquiring labeled data is high, and the shapes of chilled iron targets are varied and the backgrounds are complex, making it difficult to obtain high-quality labeled data. This invention introduces self-supervised learning tasks, especially occlusion reconstruction and contrast learning tasks, enabling the model to learn effective feature representations from unlabeled data without relying on a large amount of labeled data. This self-supervised learning framework can significantly improve training efficiency, reduce dependence on manual labeling, and enhance the adaptability of the model, making it particularly suitable for industrial scenarios where labeled data is scarce. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this invention, and not to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this invention.

Claims

1. A method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network, characterized in that, Specifically, the following steps are included: S1: Obtain multi-source data images of industrial casting chills; and the multi-source data images include RGB images, infrared images, and depth images, i.e., three-dimensional structural feature maps of industrial casting chills; S2: Label the target category and bounding box coordinates of the multi-source data map to obtain a labeled data map; preprocess the labeled data map to obtain a preprocessed image; S3: Divide the preprocessed images into training and test sets according to a preset ratio; S4: Construct an industrial casting chill identification model based on improved YOLOv11 and dual-flow network; The industrial casting chill identification model includes an input layer, an RGB image stream network based on an improved YOLOv11 backbone network, a multispectral image stream network based on an improved YOLOv11 backbone network, a feature fusion module, and a detection and identification module. The input layer is used to perform feature processing on the RGB image in the preprocessed image to obtain the output feature map, and transmit it to the RGB image stream network; and to perform feature processing on the infrared image and depth image in the preprocessed image, and transmit them to the multispectral image stream network. An RGB image stream network is used to extract visible light feature maps from RGB images, and the visible light features include at least size features and pixel features; A multispectral image stream network is used to extract spectral feature maps of industrial casting chills based on infrared and depth images; The feature fusion module is used to fuse the visible light feature map and the spectral feature map to obtain the fused feature map of industrial casting chills; The detection and identification module is used to detect and identify industrial casting chills based on the fused feature map; S5: Construct a total loss function for self-supervised learning, and train and evaluate the constructed industrial casting chill identification model based on the training set and test set to obtain the optimal industrial casting chill identification model, so as to realize the detection and identification of industrial casting chills based on the optimal industrial casting chill identification model.

2. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 1, characterized in that, The input layer described in S4 includes the GhostFocus module and the GhostConv module connected in sequence; Feature processing: The GhostFocus module downsamples the input preprocessed image, and the GhostConv module performs a lightweight convolution operation on the output of the GhostFocus module to obtain the output feature map; the formula for obtaining the output feature map is as follows: F out =GhostConv(Focus(I in )) In the formula: F out Indicates the output feature map; I in This indicates a preprocessed image.

3. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 2, characterized in that, The improved YOLOv11 backbone network in S4 includes a depthwise separable convolutional module, an EVCBlock module, a downsampling module, and a weighted bidirectional feature pyramid network BiFPN connected in sequence. The depthwise separable convolution module is used to perform convolution operations on the input data to obtain depthwise convolutional feature maps; and the formula for calculating the number of parameters of the depthwise separable convolution module is as follows: Params depthwise =K×K×C in Params pointwise =1×1×C in ×C out In the formula: K represents the convolution kernel size; C in C out These represent the number of input channels and the number of output channels of a depthwise separable convolution, respectively; Params depthwise The parameters representing a depthwise convolution; Params pointwise The parameters represent the pointwise convolution; The EVCBlock module is used to simultaneously extract local and global feature maps from deep convolutional feature maps, and to perform weighted fusion of the local and global feature maps to obtain enhanced feature maps. And the formula for obtaining the enhanced feature map is: F evc =αF local +(1-α)F global In the formula: F evc F represents the enhanced feature map; α represents the learnable weight coefficients; local Represents a local feature map; F global Represents the global feature map; The downsampling module is used to perform downsampling operations on the enhanced feature map to obtain the downsampled feature map; The weighted bidirectional feature pyramid network BiFPN is used to perform multi-scale feature fusion on deep convolutional feature maps, augmented feature maps, and downsampled feature maps to obtain a weighted feature map. And the formula for obtaining the weighted feature map is: In the formula: C out Indicates the output fused feature map; F i The input feature maps include depthwise convolutional feature maps, augmented feature maps, and downsampled feature maps; w i Represents the learnable fusion weights; ∈ represents a design constant to avoid division by zero errors.

4. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 3, characterized in that, The RGB image stream network described in S4 includes a first feature extraction module and a CBAM attention module based on an improved YOLOv11 backbone network. The first feature extraction module is used to extract the weighted feature map of the RGB image after the input layer feature processing based on the improved YOLOv11 backbone network; The CBAM attention module is used to extract attention features from the output of the first feature extraction module based on channel attention and spatial attention mechanisms, so as to fuse and obtain visible light feature maps. And the expression for the CBAM attention module is: In the formula: F' represents the visible light characteristic map; M c Indicates the channel attention mapping operation; M s This represents the spatial attention mapping operation; represents the element-wise multiplication symbol; F represents the output of the first feature extraction module.

5. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 4, characterized in that, The multispectral image stream network described in S4 includes a channel stitching module, a dual-channel GhostFocus module, a second feature extraction module based on an improved YOLOv11 backbone network, and a mid-course modal feature fusion module. The channel stitching module is used to stitch the infrared image and the depth image after the input layer feature processing to obtain a dual-channel input feature map; The dual-channel GhostFocus module is used to perform downsampling operations on the infrared image and depth image in the dual-channel input feature map to obtain the dual-channel sampled feature map. The second feature extraction module is used to extract the weighted feature map of industrial casting chills from the dual-channel sampling feature map, namely the infrared spectral feature map and the depth spectral feature map, based on the improved YOLOv11 backbone network. The mid-mode feature fusion module is used to fuse infrared spectral feature maps and depth spectral feature maps using the Early Fusion mechanism to obtain spectral feature maps. And the formula for obtaining the spectral feature map is: F fusion =αF ir +(1-α)F depth In the formula: F fusion Represents spectral characteristic map; F ir Indicates infrared spectral characteristics; F depth Indicates depth spectral characteristics.

6. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 5, characterized in that, The expression for the feature fusion module described in S4 is: F fused =α·F flow1 +β·F flow2 In the formula: F fused This indicates the acquisition of the fusion feature map of industrial casting chills; F flow1 This represents the visible light feature map output by the RGB image stream network; F flow2 The output of the multispectral image stream network is represented as the spectral feature map; α and β represent the learnable weight coefficients. The detection and identification module detects and identifies industrial casting chills based on the fused feature map, and its expression is B^=(x 0 ,y 0 ,w,h) P class =softmax(W class ·F fused ) In the formula: B^ represents the predicted bounding box; x 0 ,y 0 The coordinates of the center of the bounding box are represented by ; w and h represent the width and height of the bounding box; P class W represents the probability distribution of categories for industrial casting chills; class This indicates the category classification weight of industrial casting chills.

7. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 6, characterized in that, The method for obtaining the optimal industrial casting chill identification model in S5 includes the following steps: S31: Train the constructed industrial casting chill identification model based on the training set to obtain the trained industrial casting chill identification model: S32: Construct the total loss function for self-supervised learning, and evaluate the trained industrial casting chill identification model based on the test set to determine whether the output of the trained industrial casting chill identification model has converged. If so, then the industrial casting chill identification model trained at this time is the optimal industrial casting chill identification model. Otherwise, the Adam optimizer, based on the backpropagation algorithm, is used to adaptively adjust the hyperparameters of the trained industrial casting chill identification model, and step S31 is repeated.

8. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 7, characterized in that, The total loss function for self-supervised learning constructed in S32 is: L reconstruction =E[||x-x^|| 2 ] L contrastive =E[||f(x)-f(x + )|| 2 ] L total =L det +λ1L reconstruction +λ2L contrastive In the formula: L total L represents the total loss function; det λ1 and λ2 represent the target detection loss of industrial casting chills; L represents the weighting coefficients of the adjustment loss term. reconstruction Represents the reconstruction loss; x represents the original multi-source data map; x^ represents the reconstructed multi-source data map; L contrastive f(x) represents the contrast loss; f(x) represents the feature representation of the original multi-source data map; f(x) + ) represents the augmented feature representation of the original multi-source data graph in the contrastive learning task; E represents the expected loss value.

Images