Improved yolov11 and double-stream network-based industrial casting chill identification method

Abstract

The application discloses an industrial casting chill identification method based on an improved YOLOv11 and a double-flow network, comprising the following steps: S1, acquiring a multi-source data graph of an industrial casting chill; S2, performing target category and boundary box coordinate labeling on the multi-source data graph to acquire a labeled data graph; preprocessing the labeled data graph to acquire a preprocessed image; S3, dividing the preprocessed image according to a preset proportion to acquire a training set and a test set; S4, constructing an industrial casting chill identification model based on the improved YOLOv11 and the double-flow network; S5, constructing a total loss function of self-supervised learning, performing model training and evaluation on the constructed industrial casting chill identification model according to the training set and the test set, and acquiring an optimal industrial casting chill identification model, so as to realize detection and identification of the industrial casting chill according to the optimal industrial casting chill identification model. The application solves the problem of low industrial casting chill identification precision caused by insufficient detection performance in a low-light environment, occlusion and overlap, and dependence on a large amount of labeled data in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method for identifying industrial casting chills based on an improved YOLOv11 and a two-stream network. Background Technology

[0002] With the rapid development of industrial automation technology, intelligent production lines have been widely used in many fields, especially in the casting industry. The casting process typically requires a large number of auxiliary parts, and chills are one important type. Chills are components used in the casting process to guide metal flow and improve casting accuracy. They usually have relatively simple geometric shapes and are gray or metallic in color. However, due to the gray or metallic surface characteristics of the chill material itself, and because chills often accumulate during the casting process, they are difficult to distinguish accurately in complex environments. Therefore, this places high demands on the automatic detection technology of chills.

[0003] Existing methods for inspecting chills primarily rely on traditional manual sorting and machine vision-based target detection. In manual sorting, workers must manually observe and sort each chill, which is not only time-consuming and labor-intensive but also susceptible to errors due to operator skill levels and environmental factors. As production scales up, this method clearly cannot meet the demands for efficiency, accuracy, and automation.

[0004] While machine vision-based detection methods have made significant progress in recent years, particularly with the development of deep learning technology leading to breakthroughs in many fields, several challenges remain in the detection of chills in industrial casting. First, chills are typically exposed to low light conditions and complex background interference in industrial production environments, making the performance of existing traditional target detection methods less than ideal. Second, the gray appearance of chills is similar to the surface color of the cast metal, and chills are often stacked in groups or obscured by other objects during casting, making target detection algorithms prone to false positives or false negatives in these situations. Finally, although some existing deep learning algorithms can address these challenges to some extent, they typically utilize only single-sensor information, such as RGB images, neglecting additional information like infrared and depth images. This results in less than ideal performance in complex backgrounds and situations with multiple overlapping targets.

[0005] Currently, YOLO-based target detection algorithms have achieved good results in various applications, with YOLOv11, as a representative, widely used in the field of target detection due to its high efficiency and accuracy. However, the YOLOv11 model still faces performance bottlenecks when dealing with complex backgrounds, low-light environments, and target occlusion, especially in the detection of specific industrial targets such as chilled iron. Traditional YOLOv11 has not been optimized for the specific characteristics of such targets. Therefore, how to improve the accuracy of chilled iron target detection while ensuring real-time performance, especially in complex environments such as low light, overlap, and occlusion, is a pressing issue that needs to be addressed in current technology.

[0006] In summary, the existing technology has the following drawbacks: (1) Insufficient performance in low-light environments: Although YOLOv11 performs well on many standard image datasets, its detection accuracy is insufficient in industrial environments with low or uneven lighting. Cold iron targets are often in shadow areas or environments with insufficient light, and existing technologies have low accuracy in such scenarios.

[0007] (2) Occlusion and overlap issues: Cold irons in industrial foundries are usually stacked together and have strong occlusion and overlap with other objects. Existing target detection methods, especially YOLOv11, do not have specific handling for overlap and occlusion between objects, resulting in serious missed detections and false detections.

[0008] (3) Dependence on a large amount of labeled data: Existing deep learning methods usually rely on a large amount of high-quality labeled data for training. However, in industrial applications, obtaining labeled data is both time-consuming and labor-intensive, and in some special cases (such as large changes in cold iron targets, strong environmental interference, etc.), it is difficult to obtain enough labeled data. This high dependence on labeled data limits the widespread application of existing technologies in some practical scenarios. Summary of the Invention

[0009] This invention provides a method for identifying chills in industrial casting based on an improved YOLOv11 and a dual-flow network, in order to overcome the aforementioned technical problems.

[0010] To achieve the above objectives, the technical solution of the present invention is as follows: A method for identifying industrial casting chills based on an improved YOLOv11 and a two-flow network specifically includes the following steps: S1: Obtain multi-source data images of industrial casting chills; and the multi-source data images include RGB images, infrared images and depth images of industrial casting chills, where the depth image is a three-dimensional structural feature map; S2: Label the target category and bounding box coordinates of the multi-source data map to obtain a labeled data map; preprocess the labeled data map to obtain a preprocessed image; S3: Divide the preprocessed images into training and test sets according to a preset ratio; S4: Construct an industrial casting chill identification model based on improved YOLOv11 and dual-flow network; Furthermore, the industrial casting chill identification model includes an input layer, an RGB image stream network based on an improved YOLOv11 backbone network, a multispectral image stream network based on an improved YOLOv11 backbone network, a feature fusion module, and a detection and identification module. The input layer is used to perform feature processing on the RGB image in the preprocessed image to obtain the output feature map, and then transmit it to the RGB image stream network; it is also used to perform feature processing on the infrared image and depth image in the preprocessed image, and then transmit them to the multispectral image stream network. An RGB image stream network is used to extract visible light feature maps from RGB images, and the visible light features include at least size features and pixel features; A multispectral image stream network is used to extract spectral feature maps of industrial casting chills based on infrared and depth images; The feature fusion module is used to fuse the visible light feature map and the spectral feature map to obtain the fused feature map of industrial casting chills; The detection and identification module is used to detect and identify industrial casting chills based on the fused feature map; S5: Construct a total loss function for self-supervised learning, and train and evaluate the constructed industrial casting chill identification model based on the training set and test set to obtain the optimal industrial casting chill identification model, so as to realize the detection and identification of industrial casting chills based on the optimal industrial casting chill identification model.

[0011] Furthermore, the input layer in S4 includes sequentially connected... Modules and Module; Feature processing: through The module performs a downsampling operation on the input preprocessed image, through... Module pair The module's output undergoes a lightweight convolution operation to obtain an output feature map; and the formula for obtaining the output feature map is as follows:

[0012] In the formula: This represents the output feature map; This indicates a preprocessed image.

[0013] Furthermore, the improved YOLOv11 backbone network in S4 includes a depthwise separable convolutional module, an EVCBlock module, a downsampling module, and a weighted bidirectional feature pyramid network connected in sequence. ; The depthwise separable convolution module is used to perform convolution operations on the input data to obtain depthwise convolution feature maps; Furthermore, the formula for calculating the number of parameters in a depthwise separable convolutional module is as follows:

[0014]

[0015] In the formula: K represents the convolution kernel size; , These represent the number of input channels and the number of output channels of a depthwise separable convolution, respectively. The parameters represent the depthwise convolution; The parameters represent the pointwise convolution; The EVCBlock module is used to simultaneously extract local and global feature maps from deep convolutional feature maps, and to perform weighted fusion of the local and global feature maps to obtain enhanced feature maps. And the formula for obtaining the enhanced feature map is:

[0016] In the formula: Represents enhanced feature maps; Represents the learnable weight coefficients; Represents a local feature map; Represents the global feature map; The downsampling module is used to perform downsampling operations on the enhanced feature map to obtain the downsampled feature map; Weighted bidirectional feature pyramid network It is used to perform multi-scale feature fusion on deep convolutional feature maps, enhanced feature maps, and downsampled feature maps to obtain weighted feature maps; And the formula for obtaining the weighted feature map is:

[0017] In the formula: This indicates the output fused feature map; The input feature maps are represented by depthwise convolutional feature maps, augmented feature maps, and downsampled feature maps. Represents the learnable fusion weights; This represents a design constant to avoid division by zero errors.

[0018] Furthermore, the RGB image stream network described in S4 includes a first feature extraction module and a CBAM attention module based on an improved YOLOv11 backbone network; The first feature extraction module is used to extract the weighted feature map of the RGB image after the input layer feature processing based on the improved YOLOv11 backbone network; The CBAM attention module is used to extract attention features from the output of the first feature extraction module based on channel attention and spatial attention mechanisms, so as to fuse and obtain visible light feature maps. And the expression for the CBAM attention module is:

[0019] In the formula: Represents the visible light characteristic map; This indicates a channel attention mapping operation; This represents the spatial attention mapping operation; Indicates the element-wise multiplication symbol; This represents the output of the first feature extraction module.

[0020] Furthermore, the multispectral image stream network described in S4 includes a channel stitching module and a dual-channel... The module includes a second feature extraction module based on the improved YOLOv11 backbone network and a mid-process modal feature fusion module; The channel stitching module is used to stitch the infrared image and the depth image after the input layer feature processing to obtain a dual-channel input feature map; Dual Channel The module is used to perform downsampling operations on the infrared image and depth image in the dual-channel input feature map to obtain the dual-channel sampled feature map; The second feature extraction module is used to extract the weighted feature map of industrial casting chills from the dual-channel sampling feature map, namely the infrared spectral feature map and the depth spectral feature map, based on the improved YOLOv11 backbone network. The mid-modal feature fusion module is used to fuse infrared spectral feature maps and depth spectral feature maps using the Early Fusion mechanism to obtain spectral feature maps. And the formula for obtaining the spectral feature map is:

[0021] In the formula: Represents spectral characteristic maps; Indicates infrared spectral characteristics; Indicates depth spectral characteristics.

[0022] Furthermore, the expression for the feature fusion module described in S4 is:

[0023] In the formula: This represents the fusion feature map obtained from industrial casting chills; This represents the visible light feature map output by the RGB image stream network; This represents the output of a multispectral image stream network, namely the spectral feature map. Represents the learnable weight coefficients; The detection and recognition module detects and recognizes industrial casting chills based on the fused feature map, and its expression is as follows:

[0024]

[0025] In the formula: Represents the predicted bounding box; Indicates the center coordinates of the bounding box; Indicates the width and height of the bounding box; This represents the probability distribution of categories for industrial casting chills; This indicates the category classification weight of industrial casting chills.

[0026] Furthermore, the method for obtaining the optimal industrial casting chill identification model in S5 includes the following steps: S31: Train the constructed industrial casting chill identification model based on the training set to obtain the trained industrial casting chill identification model: S32: Construct the total loss function for self-supervised learning, and evaluate the trained industrial casting chill identification model based on the test set to determine whether the output of the trained industrial casting chill identification model has converged. If so, then the industrial casting chill identification model trained at this time is the optimal industrial casting chill identification model. Otherwise, the Adam optimizer, based on the backpropagation algorithm, is used to adaptively adjust the hyperparameters of the trained industrial casting chill identification model, and step S31 is repeated.

[0027] Furthermore, the total loss function for self-supervised learning constructed in S32 is:

[0028]

[0029]

[0030]

[0031] In the formula: Represents the total loss function; This indicates the target detection loss of industrial casting chills; 、 These represent the weighting coefficients of the adjustment loss term; Indicates the losses incurred during reconstruction; Represents the original multi-source data graph; This represents the reconstructed multi-source data graph; Indicates comparative loss; The feature representation of the original multi-source data graph; This represents the augmented feature representation of the original multi-source data graph in a contrastive learning task; This represents the expected loss value.

[0032] This invention provides a method for identifying chills in industrial castings based on an improved YOLOv11 and a two-stream network, with the following advantages: (1) Performance improvement in low light environment: By introducing multimodal data streams of infrared images and depth images, this invention significantly improves the detection capability of cold iron targets in low light environment. Compared with the traditional YOLOv11 model that only relies on the single mode of RGB images, this invention utilizes the temperature information provided by infrared images and the spatial structural features enhanced by depth images to accurately identify cold iron targets in low light environment, avoiding the problem of missed detection when there is insufficient light. In addition, the introduction of depth images enables the model to better understand the three-dimensional structure of the target, further improving the accuracy of target detection.

[0033] (2) Solving the occlusion and overlap problem: Traditional YOLOv11 models often fail to provide high-precision detection results when faced with target stacking and occlusion problems, especially when chill targets overlap with other objects; by introducing an improved YOLOv11 backbone network and CBAM attention module, an industrial casting chill recognition model based on improved YOLOv11 and dual-stream network is constructed, which can strengthen the attention to the target area. In particular, when the target part is occluded or overlapped, the model can improve the detection accuracy of these areas by relying on the enhanced local feature response; the channel attention mechanism and spatial attention mechanism of the CBAM module effectively improve the model's perception ability of targets of different scales, especially the detection ability of the model is significantly enhanced in small targets and complex backgrounds.

[0034] (3) Reduce dependence on a large amount of labeled data: In many industrial applications, the cost of obtaining labeled data is high, and the shape of cold iron targets is varied and the background is complex, making it difficult to obtain high-quality labeled data. This invention introduces self-supervised learning tasks, especially occlusion reconstruction tasks and contrast learning tasks, so that the model can learn effective feature representations from unlabeled data without relying on a large amount of labeled data. This self-supervised learning framework can significantly improve training efficiency, reduce dependence on manual labeling, and improve the adaptability of the model, making it particularly suitable for industrial scenarios where labeled data is scarce. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 The flowchart shows the industrial casting chill identification method based on the improved YOLOv11 and dual-flow network of the present invention. Figure 2 This is an example diagram of the multi-source data collected in this embodiment; Figure 3 This is a schematic diagram of the industrial casting chill identification model based on the improved YOLOv11 and two-stream network constructed in this embodiment. Figure 4 This is a diagram showing the recognition effect of the optimal industrial casting chill recognition model in this embodiment. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] This embodiment provides a method for identifying chills in industrial castings based on an improved YOLOv11 and a two-stream network, such as... Figure 1 As shown, the specific steps include: S1: Obtain multi-source data images of industrial casting chills; and the multi-source data images include RGB images, infrared images and depth images of industrial casting chills, where the depth image is a three-dimensional structural feature map; Specifically, such as Figure 2 To train and test the target detection model for industrial casting chills, a high-quality dataset suitable for chill recognition tasks was first collected and created. Specifically, multiple industrial cameras and sensors were used for data acquisition to ensure data diversity and meet the application needs of various industrial casting sites. The dataset includes RGB images, infrared images, and depth images. The specific acquisition equipment is as follows: (1) RGB images: Images of chilled iron were captured using a Canon EOS 5D Mark IV high-definition industrial camera with a resolution of 1920x1080 under different stacking methods and lighting conditions. This camera uses a CMOS sensor, which can provide clear image details and adapt to high dynamic range environments.

[0039] (2) Infrared image (IR): The FLIRA655sc infrared thermal imaging camera with a resolution of 640x480 was used to capture the heat distribution of cold iron at different temperatures, and it performed well, especially in low light conditions.

[0040] (3) Depth: Using an Intel RealSense D435 depth camera with a resolution of 640x480, the depth information of the chill in space is obtained, providing the three-dimensional structural features of the chill.

[0041] The multi-source data map in this embodiment includes 5000 images from different environments, angles, and stacking states, ensuring the diversity of the dataset and meeting various needs in actual industrial casting environments. Each image contains at least one type of chilled iron target, specifically including: 1. Block-shaped Cold Iron 2. Ring-shaped cold iron structure 3. Strip-shaped cold iron structure 4. Bolt-hole Cold Iron S2: Label the target category and bounding box coordinates of the multi-source data map to obtain a labeled data map; preprocess the labeled data map to obtain a preprocessed image; Specifically, all collected multi-source data maps need to undergo manual annotation and a series of preprocessing steps to ensure data consistency and prepare for subsequent model training. The annotation tool used is... The annotation data for each image is saved in Pascal VOC format, and the annotation process includes: S001: Bounding box annotation: The cold iron targets in each image are annotated with rectangular boxes. The annotation includes the target category (cold iron block, cold iron ring structure, cold iron strip structure, cold iron bolt structure) and the bounding box coordinates. These coordinate values ​​represent the center coordinates (x, y) of the bounding box as well as the width (w) and height (h), which will be consistent with the bounding box representation in subsequent model training.

[0042] S002: XML file storage: The category label information for each image is stored in XML file format. Each XML file contains the image file name, target category and its corresponding bounding box coordinates. This annotation information will be used for data reading during the training and validation phases.

[0043] To ensure data consistency and model training effectiveness, all collected multi-source data graphs in this embodiment undergo the following preprocessing operations: S100: Image size adjustment: Adjust the size of all RGB images, infrared images and depth images to 640x640 pixels; S200: Pixel Normalization: Normalizes the pixel values ​​of each image to the range [0,1] to ensure the uniformity of the input data. Its expression is:

[0044] S300: Denoising: Gaussian filtering is used to denoise the infrared and depth images to reduce the impact of environmental noise on the training effect of the model.

[0045] S400: Data Augmentation: To increase data diversity and avoid overfitting the model to specific scenarios, this invention employs common image augmentation techniques such as rotation, scaling, flipping, and color dithering. These techniques are widely used in object detection tasks and are considered existing technologies. However, this embodiment applies these basic techniques to a specific industrial casting environment, especially under low light, occlusion, and complex background conditions, improving the model's robustness and generalization ability. Therefore, although these techniques themselves are existing technologies, they have significant innovative value and practical significance in the specific application scenario of this embodiment.

[0046] After data acquisition and preprocessing, the final dataset in this embodiment includes 5000 images covering four categories of cold iron targets. Each image is labeled with its category, and the labeling information includes the category of the cold iron target and its corresponding bounding box coordinates. The preprocessed image data will be used for subsequent training and testing of the object detection model.

[0047] S3: Divide the preprocessed images into training and test sets according to a preset ratio; S4: Construct an industrial casting chill identification model based on improved YOLOv11 and dual-flow network; like Figure 3 As shown, the industrial casting chill identification model includes an input layer, an RGB image stream network based on an improved YOLOv11 backbone network, a multispectral image stream network based on an improved YOLOv11 backbone network, a feature fusion module, and a detection and identification module. The input layer is used to perform feature processing on the RGB image in the preprocessed image to obtain the output feature map, and transmit it to the RGB image stream network; and to perform feature processing on the infrared image and depth image in the preprocessed image, and transmit them to the multispectral image stream network. Specifically, the input layer includes sequentially connected... Modules and Module; Feature processing: At the input end, through The module performs a 4×4 downsampling operation (i.e., 2×2 partitioning + stacking) on ​​the input preprocessed image, and then... Module operation extends feature representation, that is, through Module pair The module's output undergoes lightweight convolution operations to obtain output feature maps. Simultaneously, a linear transformation is used to generate redundant features while significantly reducing computational cost. The formula for obtaining the output feature maps is as follows:

[0048] In the formula: The output feature map is defined by the size and number of channels of the input image. The operation and the size of the convolution kernel determine the operation; Indicates a preprocessed image; In a specific embodiment, the improved YOLOv11 backbone network includes a depthwise separable convolution module, an EVCBlock module (Enhanced Visual ContextBlock), a downsampling module, and a weighted bidirectional feature pyramid network connected in sequence. ; The depthwise separable convolution module is used to perform convolution operations on the input data to obtain depthwise convolution feature maps. In this embodiment, depthwise separable convolution is used to replace traditional convolution to reduce the number of parameters. Its single-layer convolution consists of a depthwise convolution (channel-wise convolution) and a pointwise convolution (1×1 convolution). Furthermore, the formula for calculating the number of parameters in a depthwise separable convolutional module is as follows:

[0049]

[0050] In the formula: K represents the convolution kernel size; , These represent the number of input channels and the number of output channels of a depthwise separable convolution, respectively. The parameters represent the depthwise convolution; The parameters represent the pointwise convolution; however, the number of parameters in depthwise separable convolution does not directly reflect the output feature map. However, its computation depends on the number of channels in the input feature map and the design of the convolution kernel.

[0051] The EVCBlock module is used to simultaneously extract local and global feature maps from deep convolutional feature maps, and to perform weighted fusion of the local and global feature maps to obtain enhanced feature maps. This embodiment introduces the EVCBlock module into the backbone network. The EVCBlock module enhances feature representation by simultaneously extracting local details and global contextual information. Specifically, it first extracts local features using a 3×3 standard convolution, then performs global average pooling (GAP), followed by a 1×1 convolution to generate global features. Finally, the local and global features are weighted and fused to obtain the final enhanced features. The formula for obtaining the enhanced feature map is as follows:

[0052] In the formula: Represents enhanced feature maps; Represents the learnable weight coefficients; Represents a local feature map; Represents a global feature map; in this embodiment The default initial value is 0.5. This EVC Block module is typically introduced at key locations in the backbone network, especially for effectively fusing small target features and global contextual features. The placement of the EVCBlock module aims to capture richer contextual information, thereby improving object detection accuracy. The weighted fusion operation in the EVCBlock module does not significantly increase the number of additional parameters. Since the fusion process is based on a linear combination of feature maps and does not involve additional convolution operations, it is compatible with... and The computation is not directly affected; the introduction of the EVC Block module is more of a weighted fusion of existing features, without adding new convolutional layers or significantly increasing computational cost, thus enhancing the feature map. (That is, the feature map output by the EVCBlock module). Generally speaking, no additional downsampling operation is needed because the computational cost has already been effectively reduced in the network through operations such as depthwise convolution and pointwise convolution, and the output feature map of the EVCBlock module has already been integrated into features suitable for subsequent networks. Therefore, the features after the EVCBlock module... Figure 1 Generally, the original size will be maintained unless there is a clear need for downsampling operations in subsequent layers.

[0053] The downsampling module is used to perform downsampling operations on the enhanced feature map to obtain the downsampled feature map; Specifically, in the backbone network, in order to reduce the spatial size while ensuring information preservation, a depthwise separable convolution with a stride of 2 is used. This operation effectively reduces the spatial size of the feature map and maintains the richness of information. In this embodiment, the improved backbone network output includes three feature layers of different scales, corresponding to: P3 layer, i.e., depthwise convolutional feature map: size 80×80, number of channels 128; P4 layer is the enhanced feature map: size 40×40, number of channels 256; P5 layer is the downsampled feature map: size 20×20, number of channels 512.

[0054] Weighted bidirectional feature pyramid network It is used to perform multi-scale feature fusion on deep convolutional feature maps, augmented feature maps, and downsampled feature maps to obtain weighted feature maps; Specifically, this embodiment employs an improved version of the weighted bidirectional feature pyramid network (…). Multi-scale feature fusion is performed. By introducing independent learnable weights for each fusion path, dynamic adjustment of feature importance is achieved, thereby effectively fusing features at different scales. Compared with the traditional Feature Pyramid Network (FPN), This not only improves the flexibility of feature fusion, but also enhances the network's ability to detect multi-scale targets by learning weights to optimize the importance of each feature layer. It receives multiple feature maps of different scales (e.g., P3, P4, P5 layers) from the backbone network and combines them through weighted fusion. These feature maps are obtained by downsampling in the previous stage through a convolution operation with a stride of 2, representing image information at different resolutions. By learning weights to dynamically adjust the importance of features at each scale, features from different scales can be effectively fused. The weights on each fusion path are learnable and represent the contribution of each scale feature. And the formula for obtaining the weighted feature map is:

[0055] In the formula: This indicates the output fused feature map; The input feature maps are represented by depthwise convolutional feature maps, augmented feature maps, and downsampled feature maps. Represents the learnable fusion weights; This represents the design constant for avoiding division by zero errors. ).

[0056] An RGB image stream network is used to extract visible light feature maps from RGB images, and the visible light features include at least size features and pixel features; Specifically, the RGB image stream network in S4 includes a first feature extraction module and a CBAM attention module based on an improved YOLOv11 backbone network; The first feature extraction module is used to extract a weighted feature map of the RGB image after input layer feature processing based on an improved YOLOv11 backbone network; specifically, the RGB image stream network processes standard visible light RGB images, and its input is... Three-channel images; The CBAM attention module is used to extract attention features from the output of the first feature extraction module based on channel attention and spatial attention mechanisms, so as to fuse and obtain visible light feature maps. This embodiment aims to further optimize feature fusion. The Convolutional Block Attention Module (CBAM) attention mechanism is introduced. CBAM enhances the attention to important features by introducing channel attention and spatial attention. Specifically, CBAM calculates attention weights based on the channel and spatial dimensions of each feature map, thereby helping the network to better focus on important features during the fusion process. And the expression for the CBAM attention module is:

[0057] In the formula: Represents the visible light characteristic map; This indicates a channel attention mapping operation; This represents the spatial attention mapping operation; Indicates the element-wise multiplication symbol; This represents the output of the first feature extraction module; In this embodiment, the RGB image stream network further includes a detection head: a decoupled detection head is used to process the regression and classification of the visible light feature map separately, and the output format is:

[0058] In this study, Focal Loss was used for classification prediction, and regression boxes were calculated using... , Indicates the number of categories (the number of categories for the classification task). Indicates the batch size. This indicates the spatial dimensions of the input feature map.

[0059] A multispectral image stream network is used to extract spectral feature maps of industrial casting chills based on infrared and depth images. In this embodiment, the multispectral image stream network is responsible for processing the image input consisting of an infrared (IR) image and a depth image, with an input dimension of [missing information]. Consistent with the RGB stream, the multispectral stream also employs the improved YOLOv11 backbone network, but with specific adaptations in the input processing and early feature processing sections; specifically, the multispectral image stream network includes a channel stitching module and a dual-channel... The module includes a second feature extraction module based on the improved YOLOv11 backbone network and a mid-process modal feature fusion module; The channel stitching module is used to stitch the infrared image and the depth image after the input layer feature processing to obtain a dual-channel input feature map; Specifically, infrared and depth image preprocessing involves standardizing both images to ensure consistency in their input features. This step resizes and normalizes the pixels to ensure they can be appropriately input into the network. The infrared and depth images are resized to 640x640 pixels, and pixel values ​​are normalized to the range [0,1] to ensure data from different sensors can be processed in the same feature space. To fully utilize the complementary information of the infrared and depth images, they are concatenated channel by channel to form... Dual-channel input; Dual Channel The module is used to perform downsampling operations on the infrared image and depth image in the dual-channel input feature map to obtain the dual-channel sampled feature map; The second feature extraction module is used to extract the weighted feature map of industrial casting chills from the dual-channel sampling feature map, namely the infrared spectral feature map and the depth spectral feature map, based on the improved YOLOv11 backbone network. The mid-mode feature fusion module is used to fuse infrared spectral feature maps and depth spectral feature maps using the Early Fusion mechanism to obtain spectral feature maps. In this embodiment, due to the significant differences in characteristics between infrared and depth images, an Early Fusion mechanism is employed to fuse the feature information of the infrared and depth image streams. Unlike the RGB image stream network structure, in the mid-feature fusion stage, the streams involved are both infrared and depth image streams. After processing by their respective networks, the output features of these streams are fused. Therefore, the improved YOLOv11 backbone network used here employs different processing methods for each input modality (RGB, infrared, depth), while maintaining a consistent overall structure to ensure the network can handle multimodal data. Specifically, after the output of layer P3, a feature-weighted fusion method is introduced to obtain the spectral feature map. And the formula for obtaining the spectral feature map is:

[0060] In the formula: Represents spectral characteristic maps; Indicates infrared spectral characteristics; Indicates depth spectral characteristics, This represents a learnable parameter, initialized to 0.5 and automatically optimized during training. The feature fusion module is used to fuse the visible light feature map and the spectral feature map to obtain the fused feature map of industrial casting chills; In this embodiment, the output fused feature map in the feature fusion module is fed into the YOLOv11 Neck structure to maintain a consistent feature alignment strategy. The goal of this module is to perform more refined fusion of features from different sources, providing more comprehensive information for subsequent object detection. Unlike the mid-modal feature fusion module, which is primarily performed after the P3 layer output and involves fusing features from infrared and depth images, this module fuses features from these two modalities using a specific weighting method after the input data has undergone appropriate modal processing. Its focus is on the joint processing of data from different sensors in the early stages, aiming to maximize the complementarity of multimodal data and enhance the model's adaptability to complex scenes such as low light and occlusion. The feature fusion module, on the other hand, operates in the intermediate stage after multi-layer feature map extraction, primarily fusing feature maps from different layers (e.g., P3, P4, P5). At this point, the network has extracted multi-level feature information. The feature fusion module further refines the combination of multi-scale features through the Neck structure. Unlike mid-process modal feature fusion, the main purpose of the feature fusion module is to enhance the spatial and semantic consistency of features at different levels, thereby providing richer global information for the final target detection. In this embodiment, through this fusion, the model can simultaneously utilize information from different sensors to enhance the ability to detect targets, especially in low-light or complex background scenes. The expression for the feature fusion module is:

[0061] In the formula: This represents the fusion feature map obtained from industrial casting chills; This represents the visible light feature map output by the RGB image stream network; This represents the output of a multispectral image stream network, namely the spectral feature map. The weight coefficients represent learnable values. This fusion method can effectively combine features from two different streams, improving the ability to detect targets under different environmental conditions. The detection and identification module is used to detect and identify industrial casting chills based on the fused feature map; In this embodiment, during the detection head stage, the fused multimodal feature map is fed into the decoupled detection head for bounding box regression and target classification prediction. At this time, the model independently completes the classification (predicting the target category) and regression (predicting the target location) tasks. Detection results: The detection results are generated by the decoupled detection head and include the position of the bounding box (regression output) and the target classification information (classification output). These detection results do not directly refer to the output of the multispectral image stream network, but are the final target detection results output after multimodal information fusion.

[0062] In this embodiment, the detection head adopts a decoupled head design, which handles classification tasks, bounding box regression tasks, and target confidence prediction tasks respectively. Each task corresponds to a separate convolutional layer, all of which use the SiLU activation function. And the loss functions for each task are as follows: 1. Classification task: Focal Loss is used to enhance attention to difficult samples; 2. Bounding box regression task: using Specifically defined as:

[0063] In the formula: Indicates the distance from the center point; Indicates the length of the diagonal; This represents the shape consistency term; target confidence is typically used to express the model's confidence in the existence of the target. Specifically, the expression for target confidence is:

[0064] In the formula: Indicates the weight of the target confidence prediction; Indicates that after CBAM and The feature map after weighted fusion; express The activation function is used to output the probability value of the target's presence. In summary, the fully improved YOLOv11 backbone network significantly enhances feature representation capabilities and detection accuracy while maintaining lightweight efficiency, making it particularly suitable for small target and multimodal scene detection.

[0065] Specifically, the detection and identification module detects and identifies industrial casting chills based on the fused feature map, and its expression is as follows:

[0066]

[0067] In the formula: Represents the predicted bounding box; Indicates the center coordinates of the bounding box; Indicates the width and height of the bounding box; This represents the probability distribution of categories for industrial casting chills; This represents the category classification weights for industrial casting chills. Ultimately, the model output includes the following components: 1. Bounding Box Coordinates: The position of each target, including the center coordinates (x, y) and the width (w) and height (h) of the bounding box; 2. Target Category: The category of each detected target (cold iron block, cold iron ring structure, cold iron strip structure, cold iron strip bolt structure). 3. Confidence: The confidence value of each target box represents the model's confidence in classifying that target.

[0068] S5: Construct a total loss function for self-supervised learning, and train and evaluate the constructed industrial casting chill identification model based on the training set and test set to obtain the optimal industrial casting chill identification model, so as to realize the detection and identification of industrial casting chills based on the optimal industrial casting chill identification model. In a specific embodiment, the method for obtaining the optimal industrial casting chill identification model includes the following steps: S31: Train the constructed industrial casting chill identification model based on the training set to obtain the trained industrial casting chill identification model: S32: Construct the total loss function for self-supervised learning, and evaluate the trained industrial casting chill identification model based on the test set to determine whether the output of the trained industrial casting chill identification model has converged. If so, then the industrial casting chill identification model trained at this time is the optimal industrial casting chill identification model. Otherwise, the Adam optimizer, based on the backpropagation algorithm, is used to adaptively adjust the hyperparameters of the trained industrial casting chill identification model, and step S31 is repeated. Specifically, the total loss function for the constructed self-supervised learning is as follows: In this embodiment, two self-supervised learning tasks were designed to train and obtain an industrial casting chill identification model that learns effective features from unlabeled data: (1) Occlusion Reconstruction Task: A portion of the input image is randomly occluded (10% of the image is occluded). The model is required to reconstruct the occluded region based on the remaining image portion. This process uses a self-supervised learning model, which is not directly used for classification or regression tasks, but rather enhances the network's ability to understand global features of the image by learning to reconstruct the occluded region. Through this task, the model can learn more robust feature representations when faced with image occlusion, thereby improving its performance in subsequent classification or object detection tasks. This method does not conflict with traditional classification models, but rather enhances the model's feature learning ability. Specifically, the classification and regression models still exist, but their performance is improved by the occlusion reconstruction task during training, and the formula for calculating the reconstruction loss is:

[0069] In the formula: Represents the original multi-source data graph; This represents the reconstructed multi-source data graph; (2) Contrastive Learning Task: In the contrastive learning task, multiple augmented versions are generated for each image. Contrastive loss is used to drive the model to learn feature representations of more similar images and to distance the feature representations of dissimilar images. Specifically, the loss function calculates the similarity between features of each pair of images and optimizes the model, making the features of similar images closer together and the features of dissimilar images more separated. The formula for calculating the contrastive loss is:

[0070] In the formula: The feature representation of the original multi-source data graph; This represents the augmented feature representation of the original multi-source data image in the contrastive learning task. In this way, the model not only learns how to distinguish different images during training, but also optimizes the representation of the feature space, making the features of similar images closer together, while the features of dissimilar images are further apart in the feature space.

[0071] (3) Joint training: In order to make full use of the advantages of the object detection task and the self-supervised learning task, this embodiment constructs a joint training framework. This framework combines the object detection loss of YOLOv11 with the loss of the self-supervised learning task (reconstruction loss and contrast loss) to improve the performance of the model through joint training.

[0072] The object detection part uses an improved YOLOv11 architecture to calculate the localization and classification errors of the bounding boxes. During training, the YOLOv11 detection head calculates the location and category of the cold iron target in the image, thereby generating the object detection loss (…). This is the primary loss term during training, ensuring the model can accurately detect cold iron targets in multispectral images. The loss for self-supervised learning includes reconstruction loss. And comparative loss Through these two losses, the model can learn stronger feature representations from unlabeled data. The reconstruction task helps the model recover target features even with image occlusion, while the contrastive learning task helps the model learn more robust feature representations through diverse augmented images. The final total loss function is a weighted combination of the object detection loss and the self-supervised learning loss, and the formula for calculating the total loss function is:

[0073] In the formula: Indicates the target detection loss. and These are hyperparameters used to adjust the weights of the loss term (reconstruction loss and contrastive loss contribute equally to the total loss). During training, the Adam optimizer is used to optimize the parameters and minimize the total loss function. For each training batch, the loss is calculated via forward propagation, and then the model parameters are updated via backpropagation. Training continues until the model's performance is optimal on the validation set.

[0074] This embodiment also includes model deployment and application. Model Deployment: In this embodiment, a dual-stream network model, namely an RGB image stream network and a multispectral image stream network, is used. This model performs target detection by fusing RGB and multispectral image streams. The goal of deployment is to use this model in a production line monitoring system to achieve accurate detection and classification of chilled iron targets. The deployment hardware selected is the NVIDIA Jetson Xavier AGX as the edge computing device, equipped with an NVIDIA Volta architecture GPU, 32GB of RAM, and 512GB of SSD storage. This device supports GPU acceleration, enabling efficient execution of the dual-stream network model's inference tasks. The operating system used is Ubuntu 18.04 LTS, and computational acceleration utilizes CUDA 11.2 and cuDNN 8.0. During deployment, the model is optimized using TensorRT 7.2 to ensure inference speed. The deep learning frameworks used are TensorFlow 2.4.1 and Keras 2.4.3, and the Python version is Python 3.8.

[0075] The specific deployment process is as follows: Use the `tf.saved_model.save()` method to save the trained, optimal industrial casting chill recognition model as a SavedModel format; use TensorRT's `trtexec` tool to convert the SavedModel format model into a TensorRT-optimized model to improve inference speed; upload the TensorRT-optimized model file and necessary dependencies to the NVIDIA Jetson Xavier AGX device; configure the network connection between the device and the industrial production line to ensure that the device can receive image data from sensors in real time and perform target detection.

[0076] After the model is deployed, object detection and classification will be performed according to the following process: (1) Data acquisition: The production line monitoring system acquires 10 images per second through RGB, infrared and depth sensors and transmits them to the equipment for processing in real time.

[0077] (2) Data preprocessing: All acquired image data will undergo the following preprocessing steps to ensure that it can be successfully fed into the two-stream network model for inference in step two: First, the images (RGB, infrared, and depth images) were resized to 640x640 pixels to meet the input requirements of the dual-stream network. Then, the pixel values ​​of each image were normalized to between 0 and 1 to ensure consistent processing of input data from different sensors. Finally, Gaussian filtering was applied to the infrared and depth images to reduce noise interference with model training and inference.

[0078] (3) Target detection: The preprocessed image data will be used for target detection through the optimal industrial casting chill recognition model. The model will perform the following processing on the image: RGB Image Stream: This stream uses a modified YOLOv11 architecture, introducing a Spatial Attention (SAM) mechanism and a Convolutional Block Attention (CBAM) module to improve object detection accuracy. These modules allow the model to adaptively adjust its sensitivity to objects at different scales, enhancing object localization accuracy.

[0079] Multispectral image stream: This stream processes infrared and depth images and integrates features from both sensors through weighted fusion, thereby enhancing adaptability to complex environments such as low light and occlusion.

[0080] The output of the object detection model will include: bounding box coordinates: the center coordinates (x, y), width (w), and height (h) of the bounding box; object category: the predicted category of the cold iron object, classified as "cold iron block", "cold iron ring structure", "cold iron strip structure", or "cold iron with bolt structure"; confidence score: the confidence score value of the object detection result, indicating whether the bounding box is accurate.

[0081] (4) The target detection results will undergo the following processing steps: Non-maximum suppression (NMS): Set the IOU threshold to 0.4 to remove target boxes with large overlap and keep only boxes with high confidence; Confidence filtering: Only retain target boxes with confidence values ​​higher than the set threshold (0.5), and other low confidence boxes will be filtered out.

[0082] (5) Cold Iron Target Classification: Based on the target bounding box output, the system classifies the detected cold iron targets. Cold iron targets are classified into the following four types: 1. Block-shaped cold iron: Block-shaped or flat cold iron, commonly used for support or fixation. 2. Ring-shaped cold iron: Curved cold iron, commonly used for connecting or fixing structures. 3. Strip-shaped cold iron: Strip-shaped or bar-shaped cold iron, typically used for support or structural parts. 4. Bolt-hole cold iron: Cold iron with bolt holes, used to fix or connect other parts.

[0083] Figure 4 The final output example image of the model is shown. Through target detection and classification, the deployed optimal industrial casting chill identification model will automatically identify and label different types of chill targets.

[0084] To comprehensively evaluate the performance of the industrial casting chill identification method based on improved YOLOv11 and dual-stream network in this embodiment, this experiment conducted a detailed comparison between the method described in this embodiment and YOLOv11, YOLOv4, YOLOv5, and Faster R-CNN. All models were trained and tested on the same hardware and dataset. The specific experimental setup is as follows: 1. Hardware Device: This embodiment uses an NVIDIA Jetson Xavier AGX as the inference hardware device, configured with 32GB RAM and 512GB SSD storage, and equipped with an NVIDIA Volta architecture GPU. This device supports GPU acceleration, ensuring efficient image processing and object detection tasks, and is suitable for real-time inference on industrial production lines.

[0085] 2. Dataset: The dataset contains 5,000 images collected at industrial foundry sites, covering four types of cold iron targets: block-shaped cold iron, ring-shaped cold iron, strip-shaped cold iron, and bolt-hole cold iron. These images include RGB images, infrared images, and depth images, covering various lighting conditions, complex backgrounds, target occlusion, and target overlap.

[0086] Image data acquisition was accomplished using the following devices: (1) RGB image: Canon EOS 5D Mark IV industrial camera with a resolution of 1920x1080 was used; (2) Infrared image: FLIRA655sc infrared thermal imaging camera with a resolution of 640x480 was used; (3) Depth image: Intel RealSense D435 depth camera with a resolution of 640x480 was used; Each image was manually labeled, including bounding box coordinates, target category, and category label. All images underwent standardization preprocessing before training. First, the image size was uniformly adjusted to 640x640 pixels. Then, the image pixel values ​​were normalized to the [0, 1] range. Gaussian filtering was applied to the infrared and depth images to remove noise. Data augmentation techniques were then applied, including random rotation (angle range: -15° to 15°), random scaling (0.8x to 1.2x), horizontal flipping, and random cropping, to increase the diversity of the training data.

[0087] All models were trained using TensorFlow 2.4.1 with the Adam optimizer, an initial learning rate of 0.001, a batch size of 16, and 300 training epochs. The loss function used was: (1) Focal Loss (used to handle class imbalanced classification tasks). (2) (Used for bounding box regression to improve positioning accuracy); During training, an early stopping strategy is used when the validation set... Stop training if there is no improvement after 10 consecutive iterations. 3. Comparison Results The main evaluation indicators of the experimental results include (Average precision), inference speed (FPS), accuracy, and inference time. The following are the comparison results between the method described in this embodiment and YOLOv11, YOLOv4, YOLOv5, and Faster R-CNN, as shown in Table 1: Table 1. Model Performance Comparison Results

[0088] The method described in this embodiment has significant advantages, especially in low light, target occlusion, and complex backgrounds. Firstly, compared to YOLOv11, the method described in this embodiment... The accuracy improved by 5.2 percentage points (from 87.1% to 92.3%), and the inference speed also reached 63 FPS, significantly outperforming YOLOv4 and YOLOv5. This is because this invention introduces a multispectral image stream (fusing RGB, infrared, and depth images) and an improved network structure (Ghost Focus module and CBAM attention module) on top of YOLOv11. These innovations enable the model to extract features more accurately, especially in complex industrial environments, adapting to low light, occlusion, and multi-target overlap. Compared to YOLOv4 and YOLOv5, the method described in this embodiment is superior in inference speed while maintaining high detection accuracy. YOLOv4 and YOLOv5... The accuracy rates were 85.7% and 88.5% respectively, significantly lower than the 92.3% of this invention. Furthermore, in object detection tasks, YOLOv4's inference speed was 46 FPS, and YOLOv5's was 56 FPS, both lower than the 63 FPS of this invention. This means that the method described in this embodiment can run in real-time on industrial production lines without affecting production efficiency. This is achieved by optimizing the network structure, reducing redundant computation, and utilizing the GPU acceleration of NVIDIA Jetson Xavier AGX. The method described in this embodiment is more suitable for real-time industrial inspection applications and can handle more complex environments. Compared to Faster R-CNN, although Faster R-CNN performs better in high-precision object detection, its inference speed is slower, providing only 39 FPS, which is not suitable for real-time detection tasks. Moreover, Faster R-CNN is insufficient in handling occluded and overlapping objects. The accuracy was 82.6%, which is relatively low when cold iron targets are stacked or obscured.

[0089] In summary, compared to existing target detection models (especially YOLOv11, YOLOv4, YOLOv5, and Faster R-CNN), the method described in this embodiment not only demonstrates superior detection accuracy but also exhibits stronger adaptability in low-light and target occlusion environments. Furthermore, it maintains high inference speed and low inference time, fully meeting the real-time requirements of industrial production lines. This method enables efficient and accurate detection and automatic classification of chilled iron targets in industrial foundries, solving problems such as low light, occlusion and overlap, and insufficient labeled data. It improves the automation level of industrial production lines, reduces manual intervention, and enhances sorting efficiency and production safety.

[0090] The beneficial effects of the method described in this embodiment are as follows: (1) Performance improvement in low light environment: By introducing multimodal data streams of infrared images and depth images, this invention significantly improves the detection capability of cold iron targets in low light environment. Compared with the traditional YOLOv11 model that only relies on the single mode of RGB images, this invention utilizes the temperature information provided by infrared images and the spatial structural features enhanced by depth images to accurately identify cold iron targets in low light environment, avoiding the problem of missed detection when there is insufficient light. In addition, the introduction of depth images enables the model to better understand the three-dimensional structure of the target, further improving the accuracy of target detection. (2) Solving the occlusion and overlap problem: Traditional YOLOv11 models often fail to provide high-precision detection results when faced with target stacking and occlusion problems, especially when chill targets overlap with other objects; by introducing an improved YOLOv11 backbone network and CBAM attention module, an industrial casting chill recognition model based on improved YOLOv11 and dual-stream network is constructed, which can strengthen the attention to the target area. In particular, when the target part is occluded or overlapped, the model can improve the detection accuracy of these areas by relying on the enhanced local feature response; the channel attention mechanism and spatial attention mechanism of the CBAM module effectively improve the model's perception ability of targets of different scales, especially the detection ability of the model is significantly enhanced in small targets and complex backgrounds. (3) Reduce reliance on large amounts of labeled data: In many industrial applications, the cost of acquiring labeled data is high, and the shape of cold iron targets is varied and the background is complex, making it difficult to obtain high-quality labeled data. This invention introduces self-supervised learning tasks, especially occlusion reconstruction and contrast learning tasks, enabling the model to learn effective feature representations from unlabeled data without relying on a large amount of labeled data. This self-supervised learning framework can significantly improve training efficiency, reduce reliance on manual labeling, and enhance the adaptability of the model, making it particularly suitable for industrial scenarios where labeled data is scarce. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this invention and are not intended to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this invention.

Claims

1. A method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network, characterized in that, Specifically, the following steps are included: S1: Obtain multi-source data images of industrial casting chills; and the multi-source data images include RGB images, infrared images and depth images of industrial casting chills, where the depth image is a three-dimensional structural feature map; S2: Label the target category and bounding box coordinates of the multi-source data map to obtain a labeled data map; preprocess the labeled data map to obtain a preprocessed image; S3: Divide the preprocessed images into training and test sets according to a preset ratio; S4: Construct an industrial casting chill identification model based on improved YOLOv11 and dual-flow network; Furthermore, the industrial casting chill identification model includes an input layer, an RGB image stream network based on an improved YOLOv11 backbone network, a multispectral image stream network based on an improved YOLOv11 backbone network, a feature fusion module, and a detection and identification module. The input layer is used to perform feature processing on the RGB image in the preprocessed image to obtain the output feature map, and then transmit it to the RGB image stream network; it is also used to perform feature processing on the infrared image and depth image in the preprocessed image, and then transmit them to the multispectral image stream network. The multispectral image stream network includes a channel stitching module and a dual-channel... The module includes a second feature extraction module based on the improved YOLOv11 backbone network and a mid-process modal feature fusion module; The channel stitching module is used to stitch the infrared image and the depth image after the input layer feature processing to obtain a dual-channel input feature map; Dual Channel The module is used to perform downsampling operations on the infrared image and depth image in the dual-channel input feature map to obtain the dual-channel sampled feature map; The second feature extraction module is used to extract the weighted feature map of industrial casting chills from the dual-channel sampling feature map, namely the infrared spectral feature map and the depth spectral feature map, based on the improved YOLOv11 backbone network. The mid-mode feature fusion module is used to fuse infrared spectral feature maps and depth spectral feature maps using the Early Fusion mechanism to obtain spectral feature maps. And the formula for obtaining the spectral feature map is: In the formula: Represents spectral characteristic maps; Indicates infrared spectral characteristics; Indicates depth spectral characteristics; Represents the learnable weight coefficients; An RGB image stream network is used to extract visible light feature maps from RGB images, and the visible light features include at least size features and pixel features; A multispectral image stream network is used to extract spectral feature maps of industrial casting chills based on infrared and depth images; The feature fusion module is used to fuse the visible light feature map and the spectral feature map to obtain the fused feature map of industrial casting chills; The detection and identification module is used to detect and identify industrial casting chills based on the fused feature map; S5: Construct a total loss function for self-supervised learning, and train and evaluate the constructed industrial casting chill identification model based on the training set and test set to obtain the optimal industrial casting chill identification model, so as to realize the detection and identification of industrial casting chills based on the optimal industrial casting chill identification model. The method for obtaining the optimal industrial casting chill identification model includes the following steps: S31: Train the constructed industrial casting chill identification model based on the training set to obtain the trained industrial casting chill identification model: S32: Construct the total loss function for self-supervised learning, and evaluate the trained industrial casting chill identification model based on the test set to determine whether the output of the trained industrial casting chill identification model has converged. If so, then the industrial casting chill identification model trained at this time is the optimal industrial casting chill identification model. Otherwise, the Adam optimizer, based on the backpropagation algorithm, is used to adaptively adjust the hyperparameters of the trained industrial casting chill identification model, and step S31 is repeated.

2. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 1, characterized in that, The input layer described in S4 includes sequentially connected... Modules and Module; Feature processing: through The module performs a downsampling operation on the input preprocessed image, through... Module pair The module's output undergoes a lightweight convolution operation to obtain an output feature map; and the formula for obtaining the output feature map is: In the formula: This represents the output feature map; This indicates a preprocessed image.

3. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 2, characterized in that, The improved YOLOv11 backbone network in S4 includes a series of sequentially connected deep separable convolutional modules, EVCBlock modules, downsampling modules, and a weighted bidirectional feature pyramid network. ; The depthwise separable convolution module is used to perform convolution operations on the input data to obtain depthwise convolution feature maps; Furthermore, the formula for calculating the number of parameters in a depthwise separable convolutional module is: In the formula: K represents the convolution kernel size; , These represent the number of input channels and the number of output channels of a depthwise separable convolution, respectively. The parameters represent the depthwise convolution; The parameters represent the pointwise convolution; The EVCBlock module is used to simultaneously extract local and global feature maps from deep convolutional feature maps, and to perform weighted fusion of the local and global feature maps to obtain enhanced feature maps. Furthermore, the formula for obtaining the enhanced feature map is: In the formula: Represents enhanced feature maps; Represents the learnable weight coefficients; Represents a local feature map; Represents the global feature map; The downsampling module is used to perform downsampling operations on the enhanced feature map to obtain the downsampled feature map; Weighted bidirectional feature pyramid network It is used to perform multi-scale feature fusion on deep convolutional feature maps, enhanced feature maps, and downsampled feature maps to obtain weighted feature maps; The formula for obtaining the weighted feature map is: In the formula: This indicates the output fused feature map; The input feature maps are represented by depthwise convolutional feature maps, augmented feature maps, and downsampled feature maps. Represents the learnable fusion weights; This represents a design constant to avoid division by zero errors.

4. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 3, characterized in that, The RGB image stream network described in S4 includes a first feature extraction module and a CBAM attention module based on an improved YOLOv11 backbone network. The first feature extraction module is used to extract the weighted feature map of the RGB image after the input layer feature processing based on the improved YOLOv11 backbone network; The CBAM attention module is used to extract attention features from the output of the first feature extraction module based on channel attention and spatial attention mechanisms, so as to fuse and obtain visible light feature maps. And the expression for the CBAM attention module is: In the formula: Represents the visible light characteristic map; This indicates a channel attention mapping operation; This represents the spatial attention mapping operation; Indicates the element-wise multiplication symbol; This represents the output of the first feature extraction module.

5. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 4, characterized in that, The expression for the feature fusion module described in S4 is: In the formula: This represents the fusion feature map obtained from industrial casting chills; This represents the visible light feature map output by the RGB image stream network; This represents the output of a multispectral image stream network, namely the spectral feature map. Represents the learnable weight coefficients; The detection and identification module detects and identifies industrial casting chills based on the fused feature map, and its expression is as follows: In the formula: Represents the predicted bounding box; Indicates the center coordinates of the bounding box; Indicates the width and height of the bounding box; This represents the probability distribution of categories for industrial casting chills; This indicates the category classification weight of industrial casting chills.

6. The method for identifying industrial casting chills based on improved YOLOv11 and a two-stream network according to claim 5, characterized in that, The total loss function for self-supervised learning constructed in S32 is: In the formula: Represents the total loss function; This indicates the target detection loss of industrial casting chills; 、 These represent the weighting coefficients of the adjustment loss term; Indicates the losses incurred during reconstruction; Represents the original multi-source data graph; This represents the reconstructed multi-source data graph; Indicates comparative loss; The feature representation of the original multi-source data graph; This represents the augmented feature representation of the original multi-source data graph in the contrastive learning task; This represents the expected loss value.

Images