A cast sprue segmentation method based on texture decoupling and structure perception segmentation network

Abstract

This invention discloses a method for segmenting casting gatings and risers based on a texture decoupling and structure-aware segmentation network. The method includes: acquiring and preprocessing casting images; constructing a texture decoupling and structure-aware segmentation network, which includes a background texture extraction model and a gating and riser segmentation model; extracting casting background texture features from the gating and riser segmentation dataset based on the trained background texture extraction model; training the gating and riser segmentation model based on the casting background texture features and the gating and riser segmentation dataset; the gating and riser segmentation model including a global feature extraction model, a segmentation decoder, and an output module; and performing actual gating and riser identification based on the trained gating and riser segmentation model to obtain the gating and riser segmentation result. The model in this invention can be deployed in industrial settings to process casting images captured by cameras in real time, outputting accurate gating and riser location and morphology information to guide automated robots in precise gating and riser removal operations.

Description

Technical Field

[0001] This invention relates to the field of casting processing technology, and in particular to a method for segmenting casting gating and riser systems based on texture decoupling and structure-aware segmentation networks. Background Technology

[0002] Casting is a widely used metal forming technology in modern industrial production. After casting is completed, auxiliary forming structures, such as risers and gating gates, often remain. These residual risers and gating gates must be removed through subsequent cutting and grinding processes to ensure the final dimensional accuracy and surface quality of the casting. Traditionally, riser and gating gate removal has relied mainly on manual operation, which is inefficient, labor-intensive, and difficult to guarantee consistent removal quality. To improve production automation, some industrial sectors have attempted to introduce 3D scanning equipment to assist in guiding riser and gating gate removal. However, 3D scanning equipment is expensive to purchase, has complex data processing, and a slow scanning speed, making it difficult to meet the stringent requirements of high efficiency and low cost for large-scale automated production lines.

[0003] Currently, machine vision technology based on two-dimensional (2D) images shows great potential in industrial inspection. However, existing 2D image processing methods face significant technical challenges in the automated identification and segmentation of casting risers and gatings. The surface of the casting body typically exhibits highly complex random textures, irregular roughness, and complex shadows formed by casting processes and lighting conditions. These background features are highly misleading to the texture and structural features of the risers and gatings themselves. For example, the unevenness of the casting surface may produce shadows similar to the edges of the risers and gatings, or local rough textures may be misidentified as riser and gating structures. Existing 2D image segmentation methods often struggle to effectively distinguish these complex background textures from foreground target structures. Traditional image processing methods (such as thresholding and edge detection) are not robust enough when faced with varying lighting conditions, low-contrast areas, and risers and gatings of different shapes, and are prone to missegmentation or omission. Even when using general-purpose deep learning segmentation networks, the models often overemphasize texture features rather than true structural information in images. This leads to misidentification of background noise as foreground in complex textured environments, or foreground being overwhelmed by background texture, thus reducing segmentation accuracy and reliability. Effectively decoupling background texture from images containing complex texture information and accurately perceiving and highlighting target structures such as risers and gating gates is a crucial problem that urgently needs to be solved to achieve high-precision automated detection.

[0004] Therefore, industrial production urgently needs a method that can effectively overcome the above challenges and achieve high-precision, robust, automated segmentation of casting gatings and risers by relying solely on conventional 2D image input, so as to significantly improve casting processing efficiency, reduce production costs, and enhance product consistency. Summary of the Invention

[0005] This invention provides a casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network. This method overcomes the shortcomings of existing casting gating and riser segmentation methods based on two-dimensional images, which suffer from low accuracy and poor robustness in complex backgrounds. These methods struggle to effectively distinguish between the complex textures, rough surfaces, and lighting and shadows of the casting body and the structural features of foreground targets such as gating and risers. Consequently, these methods are susceptible to interference in practical industrial applications, leading to missegmentation or omissions, which limits the efficiency and quality of automated gating and riser removal.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A method for segmenting casting gating and risers based on texture decoupling and structure-aware segmentation networks, comprising the following steps: S1. Acquire casting images and perform preprocessing to obtain a processed casting image dataset, wherein the processed casting image dataset includes a background texture dataset and a gating and riser segmentation dataset; S2. Construct a texture decoupling and structure-aware segmentation network, which includes a background texture extraction model and a gating and riser segmentation model; S3. Perform self-supervised learning pre-training on the background texture extraction model to obtain the trained background texture extraction model, and extract the casting background texture features of the gating and riser segmentation dataset based on the trained background texture extraction model; S4. The gating and riser segmentation model is trained based on the background texture features of the casting and the gating and riser segmentation dataset to obtain the trained gating and riser segmentation model; The riser segmentation model includes a global feature extraction model, a segmentation decoder, and an output module; The global feature extraction model is used to extract global features from the gating and riser segmentation dataset, and uses the inverse spatial attention mechanism to decouple the global features from the casting background texture and gating and riser features in combination with the casting background texture features to generate a decoupled feature map. The segmentation decoder is used to perform feature fusion based on the decoupled feature map to generate a fused feature map; The output module is used to generate a gating and riser segmentation result based on the fused feature map; S5. Based on the trained gating and riser segmentation model, perform actual gating and riser identification to obtain gating and riser segmentation results.

[0007] Furthermore, the background texture extraction model includes a background texture encoder and a background texture decoder; The background texture encoder includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a fifth convolutional block that are structurally identical and connected in sequence. During the self-supervised learning pre-training process of the background texture extraction model, the background texture encoder performs downsampling operations on the input data sequentially through the first convolutional block, the second convolutional block, the third convolutional block, and the fourth convolutional block to learn the casting background texture features in the input data; the fifth convolutional block is used to further extract the casting background texture features in the data transmitted by the fourth convolutional block; The background texture decoder is used to reconstruct the casting background texture features learned by the background texture encoder into an image with the same resolution as the input data. Furthermore, the specific steps for the trained background texture extraction model to extract the casting background texture features from the riser and gating segmentation dataset include: The casting background texture features of the gating and riser segmentation dataset are extracted by the second convolutional block to obtain the first casting background texture features, which are then transmitted to the global feature extraction model. The casting background texture features of the riser and gating segment dataset are extracted by the third convolutional block to obtain the second casting background texture features, which are then transmitted to the global feature extraction model. The casting background texture features of the gating and riser segmentation dataset are extracted by the fourth convolutional block to obtain the third casting background texture features, which are then transmitted to the global feature extraction model.

[0008] Furthermore, the global feature extraction model includes a global feature encoder and a feature decoupling module; The global feature encoder is built based on a pre-trained ResNet-50 network architecture, including a first residual stage, a second residual stage, a third residual stage, and a fourth residual stage. The feature decoupling module includes a first feature decoupling unit, a second feature decoupling unit, and a third feature decoupling unit; The specific steps for the global feature encoder to extract global features from the riser segmentation dataset include: The first feature map is obtained by extracting global features from the gating and riser segmentation dataset through the first residual stage and then transmitting it to the segmentation decoder. The second residual stage extracts global features from the gating and riser segmentation dataset to obtain a second feature map, which is then transmitted to the first feature decoupling unit. The third residual stage extracts global features from the riser segmentation dataset to obtain a third feature map, which is then transmitted to the second feature decoupling unit. The fourth residual stage extracts global features from the gating and riser segmentation dataset to obtain a fourth feature map, which is then transmitted to the third feature decoupling unit.

[0009] Furthermore, the feature decoupling module utilizes an inverse spatial attention mechanism and combines the casting background texture features to decouple the global features from the casting background texture and gating / riser features, generating a decoupled feature map including: The first feature decoupling unit uses the inverse spatial attention mechanism and combines the first casting background texture features to decouple the casting background texture and gating features of the second feature map, generating the first decoupled feature map, and transmitting it to the segmentation decoder; The second feature decoupling unit uses the inverse spatial attention mechanism and combines the second casting background texture feature to decouple the casting background texture and gating features of the third feature map, generating the second decoupled feature map, and transmitting it to the segmentation decoder; The third feature decoupling unit uses the inverse spatial attention mechanism and combines the third casting background texture feature to decouple the casting background texture and gating features of the fourth feature map, generating the third decoupled feature map, and transmitting it to the segmentation decoder.

[0010] Furthermore, the specific steps for each feature decoupling unit in the first, second, and third feature decoupling units to decouple the casting background texture and gating features from the corresponding feature map transmitted by the global feature encoder using the inverse spatial attention mechanism and in combination with the casting background texture features include: The corresponding feature map transmitted by the global feature encoder and casting background texture features Concatenation along the channel dimension is represented as follows:

[0011] Through size The first convolutional layer, the first batch normalization layer, the first ReLU activation function, and the size are The second convolutional layer and Sigmoid Activation function on concatenated features The process is repeated sequentially to generate a single-channel similarity map. The calculation formula is as follows:

[0012] Using similarity graphs right Perform element-wise weighting to generate the decoupled feature map. The calculation formula is as follows:

[0013] in, This indicates element-wise multiplication.

[0014] Furthermore, the segmentation decoder includes a first segmentation module, a second segmentation module, a third segmentation module, and a fourth segmentation module; The segmentation decoder performs feature fusion based on the decoupled feature map, and the specific steps for generating the fused feature map include: The fourth segmentation module receives the third decoupled feature map and sequentially passes it through a segmentation module with a size of [size missing]. The third convolutional layer adjusts the number of channels in the decoupled feature map and After the upsampling operation, a first fused feature map is generated and transmitted to the third segmentation module; The third segmentation module receives the second decoupled feature map and sequentially processes it through a segmentation module of size [size missing]. The fourth convolutional layer adjusts the number of channels in the second decoupled feature map and After the upsampling operation, the feature map is fused with the first fusion feature map element by element to generate the second fusion feature map, which is then transmitted to the second segmentation module. The second segmentation module receives the first decoupled feature map and sequentially passes it through a segmentation module with a size of [size missing]. The fifth convolutional layer adjusts the number of channels in the first decoupled feature map and After the upsampling operation, the feature map is fused with the second fusion feature map element by element to generate the third fusion feature map, which is then transmitted to the first segmentation module. The first segmentation module receives the first feature map and sequentially passes it through a segmentation module with a size of [size missing]. The sixth convolutional layer adjusts the number of channels in the first feature map and performs... After the upsampling operation, the fourth fusion feature map is generated by adding it element by element with the third fusion feature map and then transmitting it to the output module.

[0015] Furthermore, the specific steps by which the output module generates the gating and riser segmentation result based on the fused feature map include: The fourth fused feature map is processed sequentially by a 3×3 seventh convolutional layer, a 1×1 eighth convolutional layer, and a Sigmoid activation function to generate the gating and riser segmentation result.

[0016] Furthermore, during the self-supervised learning pre-training process of the background texture extraction model, the MAE loss function is used to measure the pixel difference between the reconstructed image and the input data. The calculation formula for the MAE loss function is as follows:

[0017] in, This represents the total number of pixels in an image block. Input data at pixels The value, To reconstruct the image at the pixel level The value of .

[0018] Furthermore, during the training of the gating and riser segmentation model based on the casting background texture features and the gating and riser segmentation dataset, a total segmentation loss is set. for:

[0019] in, The binary cross-entropy loss function; The Dice loss function; The weighting coefficients are set. The binary cross-entropy loss function is expressed as:

[0020] in The total number of pixels in the image. The true label is either 0 or 1; The probability of a gating and riser predicted by the gating and riser segmentation model; The Dice loss function is expressed as: .

[0021] Beneficial effects: This invention learns the background texture features of castings through a constructed background texture extraction model, extracts global features from the gating and riser segmentation dataset through a global feature extraction model, and uses an inverse spatial attention mechanism to decouple the casting background texture from the gating and riser features by combining the global features with the casting background texture features, generating a decoupled feature map; the decoupled feature map is then fused by a segmentation decoder to generate a fused feature map, and the gating and riser segmentation result is generated based on the fused feature map; this invention can effectively suppress the interference of complex background textures of castings (such as casting marks, rough surfaces, complex shadows) on target recognition, allowing the gating and riser segmentation model to focus on the structural features of the gating and riser itself. This significantly improves the segmentation accuracy and robustness to changes in ambient lighting in real industrial scenarios, especially when background textures and targets are easily confused, and effectively reduces the false alarm rate. Meanwhile, the independent pre-training of the background texture extraction model enables it to fully understand the general rules of "normal background". Combined with the effective use of this background prior knowledge by the gating and riser segmentation model during the training process, the entire gating and riser segmentation model shows stronger generalization ability for gating and risers of different casting types, shapes and lighting conditions, reducing the dependence on a large amount of specific scene data. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network in this invention. Figure 2 This is a flowchart of the model used in the casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network in this invention. Figure 3 The image shows a casting including risers and gating gates, collected in an embodiment of the present invention. Figure 4 This is a diagram showing the result of gating and riser segmentation in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] This embodiment provides a method for segmenting casting gating and riser systems based on texture decoupling and structure-aware segmentation networks, such as... Figure 1 and Figure 2 As shown, the specific steps include: S1. Acquire casting images and perform preprocessing to obtain a processed casting image dataset, wherein the processed casting image dataset includes a background texture dataset and a gating and riser segmentation dataset; Specifically, to support the training and validation of the various neural network models proposed in this invention, the construction of a high-precision image dataset is the primary and crucial step, including: Hardware Deployment and Image Acquisition: In this embodiment, a unit with a resolution of [resolution value missing] is configured. A high-resolution color CCD industrial camera with a 25mm focal length lens is used, supplemented by a uniformly illuminating LED surface light source to illuminate the casting inspection area. The camera connects to an industrial computer via a GigE Vision interface. During image acquisition, the camera exposure time is set to 10 milliseconds and the gain to 0dB to obtain clear and unsaturated images of the casting.

[0026] Specifically, this embodiment relies solely on conventional 2D industrial cameras as image input, eliminating the need for expensive and complex 3D scanning equipment. This significantly reduces the hardware cost of system construction, simplifies integration complexity, and makes it easier to deploy and promote on existing production lines.

[0027] Constructing a background texture dataset: 50,000 images of pure casting surfaces, excluding any gating gates or other defects, were systematically collected from an actual casting production line. These images cover the material characteristics of the S45C steel castings, surface wear conditions, and typical lighting variations. Background textures were randomly and non-overlappingly cropped from each original image. Image patches of pixels are used to obtain a background texture dataset, which is used for pre-training of the background texture extraction model.

[0028] Constructing and labeling the initial gating and riser segmentation dataset: 5,000 images of castings containing gating and risers were collected. These images cover various gating and riser morphologies (including cylindrical, conical, and plate-shaped gating and risers) on 8 different casting models, ensuring data diversity. Figure 3 For data examples, pixel-level annotations were performed on all casting images containing risers and gating gates using the professional image annotation tool Labelme. The annotation results generated a binary segmentation mask, in which the pixel values ​​of the riser and gating gate area were precisely set to 1 (foreground), and the pixel values ​​of the remaining background areas were set to 0 (background).

[0029] To improve the robustness and generalization ability of the riser and gating segmentation model, this embodiment performs the following data augmentation operations on the riser and gating segmentation dataset, including: Random rotation processing: within the angle range The image is rotated randomly inside; Perform random horizontal and vertical flipping on the image; Random scaling: in the scaling factor The image is randomly scaled internally; Random brightness adjustment: in intensity factor The brightness of the image is randomly adjusted internally. Random contrast adjustment: in intensity factor The contrast of the image is randomly adjusted internally. Add Gaussian noise: The noise mean is 0, and the standard deviation is [missing value]. Random noise is added to the image; Random cropping: The image is randomly cropped, with the cropped area covering 80% to 100% of the original image area. All of the above transformations are applied simultaneously to the original gating and riser segmentation image and its corresponding segmentation mask.

[0030] S2. Construct a texture decoupling and structure-aware segmentation network, which includes a background texture extraction model and a gating and riser segmentation model; S3. Perform self-supervised learning pre-training on the background texture extraction model based on the background texture dataset to obtain the trained background texture extraction model, and extract the casting background texture features of the gating and riser segmentation dataset based on the trained background texture extraction model; Specifically, this embodiment constructs an autoencoder that specifically learns the background texture pattern of castings. Its training results will provide immutable background feature priors for the subsequent gating and riser segmentation model.

[0031] In a specific embodiment, the background texture extraction model includes a background texture encoder (BTE) and a background texture decoder; The background texture encoder includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a fifth convolutional block that are structurally identical and connected in sequence. Specifically, the first, second, third, fourth, and fifth convolutional blocks each include a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function.

[0032] Specifically, during the self-supervised learning pre-training process of the background texture extraction model, the background texture encoder performs downsampling operations on the input data sequentially through the first convolutional block, the second convolutional block, the third convolutional block, and the fourth convolutional block to learn the casting background texture features in the input data; the fifth convolutional block is used to further extract the casting background texture features in the data transmitted by the fourth convolutional block, improving the abstraction and expressive power of the features for use by the subsequent background texture decoder; Specifically, the first, second, third, and fourth convolutional blocks use convolution with a stride of 2 for downsampling, which halves the feature map size layer by layer, and the fifth convolutional block has a stride of 1. The number of channels in the first, second, third, fourth, and fifth convolutional blocks is configured as follows: 3 input channels, 32 output channels after passing through the first convolutional block, and 64, 128, 256, and 512 channels respectively.

[0033] The background texture decoder is used to reconstruct the casting background texture features learned by the background texture encoder into an image with the same resolution as the input data.

[0034] Specifically, the background texture decoder consists of five sequentially connected transposed convolutional blocks, each containing a... A transposed convolutional layer (stride 2), a batch normalization layer, and a ReLU activation function are used for upsampling layer by layer, ultimately restoring the input feature map to be identical to the input image. Pixel resolution.

[0035] Specifically, during the self-supervised learning pre-training process of the background texture extraction model, the background texture encoder is further trained to learn the resolution difference between the reconstructed image and the input data through the set MAE loss function, so as to optimize the parameters of the background texture encoder. In a specific embodiment, during the self-supervised learning pre-training process of the background texture extraction model, a pixel-level L1 loss (Mean Absolute Error, MAE) function is used to accurately measure the pixel difference between the reconstructed image and the original background image. The MAE loss function calculation formula is as follows:

[0036] in, This represents the total number of pixels in an image block. Input data at pixels The value, To reconstruct the image at the pixel level The value of .

[0037] Specifically, during the self-supervised learning pre-training process of the background texture extraction model, the Adam optimizer is used, and the learning rate is fixed at 1%. The batch size was set to 64, and the training epochs were 200. After training, BTE was able to reconstruct the pure background texture image with an average pixel error of less than 5, indicating that it had successfully learned the intrinsic distribution of the casting background texture. In the subsequent end-to-end network training of the gating and riser segmentation model, all parameters of BTE will be frozen, and it will only be used as a background feature extractor. In the subsequent end-to-end network training, the reconstruction results of the background texture decoder will no longer be directly used for subsequent segmentation tasks. Instead, the output features of the second, third, and fourth convolutional blocks in BTE will be used as representatives of the background texture for the background feature input of the subsequent feature decoupling module.

[0038] In a specific embodiment, the specific steps for the trained background texture extraction model to extract the casting background texture features of the riser and gating segmentation dataset include: The casting background texture features of the gating and riser segmentation dataset are extracted by the second convolutional block to obtain the first casting background texture features, which are then transmitted to the global feature extraction model. The casting background texture features of the riser and gating segment dataset are extracted by the third convolutional block to obtain the second casting background texture features, which are then transmitted to the global feature extraction model. The casting background texture features of the gating and riser segmentation dataset are extracted by the fourth convolutional block to obtain the third casting background texture features, which are then transmitted to the global feature extraction model.

[0039] Specifically, the output features of the second, third, and fourth convolutional blocks in the BTE part are selected as representative background textures for use as background feature inputs in the subsequent feature decoupling module. This is usually because these layers contain rich background texture information at different scales that is most effective for the decoupling task. The first convolutional block contains too low-level edge and local textures, while the fifth convolutional block contains too abstract and highly semantic features. Therefore, selecting the middle second, third, and fourth convolutional blocks can provide more representative and discriminative background texture features while maintaining sufficient scale information, so as to achieve the best feature decoupling effect.

[0040] Specifically, the Background Texture Encoder (BTE) is a convolutional neural network specifically designed to learn and encode patterns of casting background textures. Its design goal is to deeply understand the inherent visual features of casting surfaces (e.g., casting marks, grinding textures, material grains, etc.) under different lighting conditions. BTE is self-supervised pre-trained on a large-scale, diverse range of pure casting background images (without any gating points or defects), enabling it to capture the intrinsic distribution and statistical properties of casting background textures with high fidelity. Throughout the end-to-end training of the Texture Decoupling and Structure-Aware Segmentation Network (TDS-Net), all parameters of BTE are frozen, acting as an immutable background feature extractor to ensure that its understanding of the background is not obscured by foreground object information.

[0041] S4. The gating and riser segmentation model is trained based on the background texture features of the casting and the gating and riser segmentation dataset to obtain the trained gating and riser segmentation model; The riser segmentation model includes a global feature extraction model, a segmentation decoder (SD), and an output module; The global feature extraction model is used to extract global features from the gating and riser segmentation dataset, and uses the inverse spatial attention mechanism to decouple the global features from the casting background texture and gating and riser features in combination with the casting background texture features to generate a decoupled feature map. The segmentation decoder is used to perform feature fusion based on the decoupled feature map to generate a fused feature map; The output module is used to generate a gating and riser segmentation result based on the fused feature map; S5. Based on the trained gating and riser segmentation model, perform actual gating and riser identification to obtain the gating and riser segmentation results, such as... Figure 4 As shown. The core of this invention lies in treating the gating system and riser as a geometric target with independent structural features superimposed on the inherent background texture of the casting. By specifically learning and suppressing the complex texture information of the casting body, the shape and boundary of target structures such as the gating system and riser can be effectively highlighted, achieving accurate identification and segmentation of the gating system and riser against complex backgrounds.

[0042] In a specific embodiment, the global feature extraction model includes a global structure encoder (GSE) and a feature decoupling module (FDM). The global feature encoder is built based on a pre-trained ResNet-50 network architecture, including a first residual stage (conv2), a second residual stage (conv3), a third residual stage (conv3), and a fourth residual stage (conv5). Specifically, in this embodiment, a ResNet-50 pre-trained on the ImageNet dataset is selected as the backbone network of the global feature encoder. This network has powerful feature extraction capabilities, and its main function is to extract multi-scale, multi-semantic-level global features from the input complete casting image. These features contain all visual information in the image, including gating and rising gates, background textures, and the intricate mixture of features between the two. The feature maps output by GSE at different depths represent different levels of abstraction of the image, from edges and textures to high-level semantic information.

[0043] The feature decoupling module includes a first feature decoupling unit, a second feature decoupling unit, and a third feature decoupling unit; Specifically, the feature decoupling module is responsible for precisely "stripping" the background texture features from the mixed features extracted by GSE, thereby highlighting foreground structures such as gating gates and risers.

[0044] The specific steps for the global feature encoder to extract global features from the riser segmentation dataset include: The first residual stage extracts global features from the gating and riser segmentation dataset to obtain a first feature map (containing edge and texture features), which is then transmitted to the segmentation decoder. The second residual stage extracts global features from the riser segmentation dataset to obtain a second feature map (containing local shape features), which is then transmitted to the first feature decoupling unit. The third residual stage extracts global features from the gating and riser segmentation dataset to obtain a third feature map (containing object component features), which is then transmitted to the second feature decoupling unit. The fourth residual stage extracts global features from the riser segmentation dataset to obtain a fourth feature map (containing high-level semantic features), which is then transmitted to the third feature decoupling unit.

[0045] Specifically, this embodiment selects the feature maps of ResNet-50's conv2, conv3, conv4, and conv5 layers as subsequent inputs because these layers represent feature information at different scales, from shallower low-level features (such as edges and textures) to deeper high-level semantic features. This multi-scale feature output can provide rich, hierarchical information for subsequent feature decoupling modules and segmentation decoders, helping the model to simultaneously capture the fine structural details of the gating system and overall semantic information, thereby improving segmentation accuracy and robustness. While the conv1 layer contains excessive detail noise, deeper layers provide more abstract and compressed information. The conv2, conv3, conv4, and conv5 layers are considered to be a good multi-scale feature combination for image segmentation tasks.

[0046] In a specific embodiment, the feature decoupling module utilizes an inverse spatial attention mechanism and combines the casting background texture features to decouple the global features from the casting background texture and gating features, generating a decoupled feature map including: The first feature decoupling unit uses the inverse spatial attention mechanism and combines the first casting background texture features to decouple the casting background texture and gating features of the second feature map, generating the first decoupled feature map, and transmitting it to the segmentation decoder; The second feature decoupling unit uses the inverse spatial attention mechanism and combines the second casting background texture feature to decouple the casting background texture and gating features of the third feature map, generating the second decoupled feature map, and transmitting it to the segmentation decoder; The third feature decoupling unit uses the inverse spatial attention mechanism and combines the third casting background texture feature to decouple the casting background texture and gating features of the fourth feature map, generating the third decoupled feature map, and transmitting it to the segmentation decoder.

[0047] In a specific embodiment, the specific steps of each feature decoupling unit in the first, second, and third feature decoupling units to decouple the casting background texture and gating features from the corresponding feature map transmitted by the global feature encoder using the inverse spatial attention mechanism and in combination with the casting background texture features include: The corresponding feature map transmitted by the global feature encoder and casting background texture features Concatenation along the channel dimension is represented as follows:

[0048] Through size The first convolutional layer, the first batch normalization layer, the first ReLU activation function, and the size are The second convolutional layer and Sigmoid Activation function on concatenated features The process is repeated sequentially to generate a single-channel similarity map. The calculation formula is as follows:

[0049] Specifically, similarity graph Each pixel value precisely quantizes the corresponding spatial location. Features and The degree of similarity of features, its range is strictly within... between.

[0050] Specifically, if Size and If there is a mismatch, precise adjustments are made using bilinear interpolation.

[0051] Using similarity graphs right Perform element-wise weighting to generate the decoupled feature map. The calculation formula is as follows:

[0052] in, This indicates element-wise multiplication, an operation that precisely multiplies regions that are highly similar to the background texture (i.e., The characteristics of regions with values ​​close to 1 The middle is suppressed (because) (close to 0), while the area that differs greatly from the background texture and corresponds to the foreground target's gating point (i.e. Features of regions with values ​​close to 0 are fully preserved (because) (Approaching 1), thus achieving suppression of background texture and highlighting of foreground structure.

[0053] Specifically, in this embodiment, the feature decoupling module is integrated after multiple feature output layers of GSE. Its key function is to accurately remove or suppress interference information caused by the casting background texture from the global features extracted by GSE, thereby purifying and strengthening the structural features of the gating and riser. FDM achieves feature decoupling by introducing a reverse spatial attention mechanism. Specifically, FDM compares the similarity between global features from GSE (containing mixed background and foreground information) and pure background texture features from frozen BTE. For regions that are spatially highly similar to the background texture (e.g., flat casting surfaces), FDM calculates an attention weight with a high activation value (close to 1); for regions that are significantly different from the background texture (e.g., the gating and riser body or its boundary), FDM calculates an attention weight with a low activation value (close to 0). Subsequently, FDM utilizes... Element-wise weighting of the global features of the GSE effectively suppresses features similar to the background texture (multiplied by a weight close to 0), while preserving features of foreground objects dissimilar to the background texture (multiplied by a weight close to 1). This decoupling process is performed in parallel at different feature scales of the GSE, ensuring the purification of multi-scale features. The feature map after FDM processing has minimized background interference and greatly highlighted the structural information of the gating system, becoming an ideal input for subsequent segmentation.

[0054] In a specific embodiment, the segmentation decoder includes a first segmentation module (FPN-2), a second segmentation module (FPN-3), a third segmentation module (FPN-4), and a fourth segmentation module (FPN-5). Specifically, the segmentation decoder adopts a feature pyramid network (FPN) structure, which performs feature fusion and upsampling through top-down paths and lateral connections.

[0055] The segmentation decoder performs feature fusion based on the decoupled feature map, and the specific steps for generating the fused feature map include: The fourth segmentation module receives the third decoupled feature map and sequentially passes it through a segmentation module with a size of [size missing]. The third convolutional layer adjusts the number of channels in the decoupled feature map and After the upsampling operation, a first fused feature map is generated and transmitted to the third segmentation module; The third segmentation module receives the second decoupled feature map and sequentially processes it through a segmentation module of size [size missing]. The fourth convolutional layer adjusts the number of channels in the second decoupled feature map and After the upsampling operation, the feature map is fused with the first fusion feature map element by element to generate the second fusion feature map, which is then transmitted to the second segmentation module. The second segmentation module receives the first decoupled feature map and sequentially passes it through a segmentation module with a size of [size missing]. The fifth convolutional layer adjusts the number of channels in the first decoupled feature map and After the upsampling operation, the feature map is fused with the second fusion feature map element by element to generate the third fusion feature map, which is then transmitted to the first segmentation module. The first segmentation module receives the first feature map and sequentially passes it through a segmentation module with a size of [size missing]. The sixth convolutional layer adjusts the number of channels in the first feature map and performs... After the upsampling operation, the fourth fusion feature map is generated by adding it element by element with the third fusion feature map and then transmitting it to the output module.

[0056] Specifically, the segmentation decoder is responsible for fusing and upsampling the multi-scale feature maps decoupled and cleaned by FDM, gradually restoring them to the resolution of the original image, and finally generating a pixel-level binary segmentation mask. This mask accurately identifies the location and shape of the riser in the image. SD employs a multi-scale feature fusion architecture, effectively fusing features from different levels of GSE and FDM to ensure that the segmentation result captures both the macroscopic semantic information of the target and finely depicts its microscopic boundary details. Because the input features it receives have effectively removed background texture interference, SD can focus more on restoring the precise boundaries and details of the riser, thereby improving the accuracy and clarity of the final segmentation result.

[0057] In a specific embodiment, the specific steps of the output module in generating the gating and riser segmentation result based on the fused feature map include: The fourth fused feature map is processed sequentially by a 3×3 seventh convolutional layer, a 1×1 eighth convolutional layer, and a Sigmoid activation function to generate the gating and riser segmentation result.

[0058] Generate a single-channel binary segmentation mask with the same size as the processed casting image.

[0059] Specifically, during the end-to-end training of the gating and riser segmentation model, the weights of the pre-trained BTE remain frozen and do not participate in gradient updates. Only the parameters of GSE, FDM, and SD are optimized end-to-end. The Adam optimizer is used, with an initial learning rate fixed at a certain value. The batch size is set to 16, the total training duration is 100 epochs, and the learning rate scheduling strategy uses cosine annealing, adjusting the learning rate from... linear decay to During training, model performance is evaluated every 5 epochs on an independent validation set. Key metrics monitored include the Dice coefficient and the Intersection over Union (IoU). The model with the highest Dice coefficient on the validation set is saved as the optimal model.

[0060] In a specific embodiment, during the training of the gating and riser segmentation model based on the casting background texture features and the gating and riser segmentation dataset, a total segmentation loss is set. for:

[0061] in, The binary cross-entropy loss function; The Dice loss function; The weighting coefficients are set. In this embodiment, the weighting coefficients for both BCE loss and Dice loss are fixed at 0.5.

[0062] The binary cross-entropy loss (BCE) function is expressed as follows:

[0063] in The total number of pixels in the image. The true label is either 0 or 1; The probability of a gating and riser predicted by the gating and riser segmentation model; The Dice loss function is expressed as: .

[0064] Specifically, after the training process is completed, this embodiment conducts a comprehensive and in-depth evaluation of the actual performance of the riser and gating segmentation model, and iteratively optimizes it based on the evaluation results to ensure that the model achieves optimal performance in real industrial scenarios, including: (1) Performance Evaluation and Comparative Analysis: In this embodiment, a strictly independent test set is used to conduct the final performance evaluation of the trained riser and gating segmentation model. The test set contains 1,000 casting images with similar data distribution to the training set but which were not used in the model training, along with their corresponding precise annotations. The evaluation process focuses on the calculation of the following key quantitative indicators: average Dice coefficient, average intersection-over-union ratio, precision, and recall. These indicators can objectively reflect the accuracy of the riser and gating segmentation model in pixel-level segmentation tasks, its ability to cover foreground targets, and its robustness in complex backgrounds. To fully verify the superiority of the method proposed in this embodiment, rigorous comparative experiments were conducted with several mainstream image segmentation models based on the same test set. These comparative models included: U-Net model based on ResNet-50 backbone, DeepLabV3+ model, PSPNet model, and FCN model. The experimental results are shown in Table 1 below: Table 1

[0065] As can be seen from the comparative data above, the texture decoupling and structure-aware segmentation network (TDS-Net) proposed in this embodiment significantly outperforms existing mainstream image segmentation models in all key performance indicators. Specifically, TDS-Net achieves an average Dice coefficient of 0.925, which is approximately 4.5, 5.0, 3.0, and 2.0 percentage points higher than U-Net, FCN, DeepLabV3+, and PSPNet, respectively. In terms of average IoU, TDS-Net reaches 0.864, also showing a significant improvement. Simultaneously, its precision and recall remain at leading levels, indicating that TDS-Net can accurately identify foreground targets while effectively avoiding misidentifying background as foreground. These improvements fully demonstrate that by introducing a background texture encoder for prior learning of the casting background texture and combining it with a feature decoupling module to effectively separate mixed features, TDS-Net can extract purer and more structurally-oriented gating and riser features from complex backgrounds, thereby significantly improving segmentation accuracy and robustness to complex working conditions. Especially in challenging scenarios such as complex surface textures and uneven lighting on castings, TDS-Net demonstrates stronger anti-interference capabilities and boundary recognition accuracy, which is fully reflected in the quantitative indicators, proving the innovation and effectiveness of the method proposed in this embodiment in solving the problem of casting gating and riser segmentation.

[0066] (2) Model Optimization: Based on the above performance evaluation results, the model parameters and strategies are finely adjusted to further improve model performance and meet the needs of practical applications. If the evaluation data shows that the model misses some small or special-shaped gating gates, the weight ratio of Dice loss and BCE loss in the loss function will be adjusted first, or the relevant parameters of attention calculation in the feature decoupling module will be fine-tuned to enhance the model's ability to perceive small foreground targets. To meet the stringent real-time requirements of inference speed in industrial settings, this embodiment implements two mainstream lightweighting techniques for the final model: model pruning and model quantization. Model pruning reduces the number of model parameters and computational complexity by identifying and removing redundant connections or unimportant neurons with weight values ​​below a certain threshold. Model quantization converts the floating-point weights and activation values ​​of the model into low-bit integer representations, such as 8-bit integers. This not only significantly reduces the model size but also utilizes dedicated hardware (such as GPU TensorCore) to accelerate inference calculations, further improving operating efficiency and ensuring that deployment efficiency is considered while meeting accuracy requirements.

[0067] Specifically, in this embodiment, the optimized TDS-Net model will be deployed to an industrial computer on the production line. This computer is equipped with an Intel Xeon E3-1505M v5 processor and an NVIDIA Quadro P1000 GPU. As a professional graphics processing unit, the NVIDIA Quadro P1000 GPU provides ample parallel computing power, ensuring that the model can achieve low-latency, high-throughput inference operations in the industrial setting.

[0068] Specifically, with the aforementioned hardware configuration, the TDS-Net model proposed in this embodiment exhibits superior real-time processing capabilities and segmentation accuracy. The inference time for a single 1920×1080 pixel casting image is precisely controlled within 30 milliseconds, far below the typical real-time requirements of industrial production lines (usually in the hundreds of milliseconds range). On independent test sets, the model achieves an average Dice coefficient of 0.925 and an average IoU of 0.864. These quantitative indicators not only verify the model's high accuracy but, more importantly, demonstrate that its performance fully meets the stringent requirements of the foundry industry for real-time defect detection and high-precision positioning of risers and gatings, providing strong technical support for intelligent manufacturing.

[0069] This embodiment employs Python programming language for software integration and development. It leverages the deployment APIs provided by the PyTorch deep learning framework and NVIDIA's TensorRT runtime library, optimized for deep learning inference, to efficiently encapsulate the trained model into a callable service. Furthermore, it integrates the OpenCV library for image reading, necessary preprocessing operations, and visualization of the final segmentation results. This integration scheme fully utilizes the inference acceleration advantages of TensorRT, ensuring efficient deployment.

[0070] In practice, on the production line, the real-time processing of casting images follows a precise automated process: an industrial camera captures casting images with a resolution of 1920×1080 pixels in real time as the raw input for model inference. The captured raw casting images are first precisely scaled to the required input size of 1024×768 pixels; then, channel order adjustment (converting from BGR format to RGB format) and normalization are performed, that is, the pixel values ​​are linearly scaled to the range of [0,1] to match the input specifications during model training; the preprocessed image is fed into the deployed TDS-Net model in real time; the TDS-Net model performs forward inference calculations under GPU acceleration and outputs a single-channel probability map representing the gating and riser region. Each pixel value in the map represents the probability that it belongs to the gating and riser. The probability map output by the model is binarized through a fixed threshold (set to 0.5 in this embodiment) to generate the final binary segmentation mask. This mask clearly identifies the position and exact shape of the gating and riser in the casting image with pixel-level precision.

[0071] Specifically, the texture decoupling and structure awareness ideas proposed in this invention are universal. The method proposed in this invention is not only applicable to casting riser segmentation, but can also be extended to other industrial image segmentation tasks with complex backgrounds but obvious target structural features, such as metal surface defect detection, weld recognition, foreign object detection, etc., and has broad application potential and commercial value. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for segmenting casting gatings and risers based on texture decoupling and structure-aware segmentation networks, characterized in that, The specific steps include: S1. Acquire casting images and perform preprocessing to obtain a processed casting image dataset, wherein the processed casting image dataset includes a background texture dataset and a gating and riser segmentation dataset; S2. Construct a texture decoupling and structure-aware segmentation network, which includes a background texture extraction model and a gating and riser segmentation model; S3. Perform self-supervised learning pre-training on the background texture extraction model to obtain the trained background texture extraction model, and extract the casting background texture features of the gating and riser segmentation dataset based on the trained background texture extraction model; S4. The gating and riser segmentation model is trained based on the background texture features of the casting and the gating and riser segmentation dataset to obtain the trained gating and riser segmentation model; The riser segmentation model includes a global feature extraction model, a segmentation decoder, and an output module; The global feature extraction model is used to extract global features from the gating and riser segmentation dataset, and uses the inverse spatial attention mechanism to decouple the global features from the casting background texture features in combination with the casting background texture features to generate a decoupled feature map. The global feature extraction model includes a global feature encoder and a feature decoupling module; The global feature encoder is built based on a pre-trained ResNet-50 network architecture, including a first residual stage, a second residual stage, a third residual stage, and a fourth residual stage. The feature decoupling module includes a first feature decoupling unit, a second feature decoupling unit, and a third feature decoupling unit; The specific steps for the global feature encoder to extract global features from the riser segmentation dataset include: The first feature map is obtained by extracting global features from the gating and riser segmentation dataset through the first residual stage and then transmitting it to the segmentation decoder. The second residual stage extracts global features from the gating and riser segmentation dataset to obtain a second feature map, which is then transmitted to the first feature decoupling unit. The third residual stage extracts global features from the riser segmentation dataset to obtain a third feature map, which is then transmitted to the second feature decoupling unit. The fourth residual stage extracts global features from the gating and riser segmentation dataset to obtain a fourth feature map, which is then transmitted to the third feature decoupling unit. The feature decoupling module utilizes an inverse spatial attention mechanism and combines the casting background texture features to decouple the global features from the casting background texture and gating / riser features, generating a decoupled feature map including: The first feature decoupling unit uses the inverse spatial attention mechanism and combines the first casting background texture features to decouple the casting background texture and gating features of the second feature map, generating the first decoupled feature map, and transmitting it to the segmentation decoder; The second feature decoupling unit uses the inverse spatial attention mechanism and combines the second casting background texture features to decouple the casting background texture and gating features of the third feature map, generating the second decoupled feature map, and transmitting it to the segmentation decoder; The third feature decoupling unit uses the inverse spatial attention mechanism and combines the third casting background texture feature to decouple the casting background texture and gating features of the fourth feature map, generating the third decoupled feature map, and transmitting it to the segmentation decoder. The segmentation decoder is used to perform feature fusion based on the decoupled feature map to generate a fused feature map; The output module is used to generate a gating and riser segmentation result based on the fused feature map; S5. Based on the trained gating and riser segmentation model, perform actual gating and riser identification to obtain gating and riser segmentation results.

2. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 1, characterized in that, The background texture extraction model includes a background texture encoder and a background texture decoder; The background texture encoder includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a fifth convolutional block that are structurally identical and connected in sequence. During the self-supervised learning pre-training process of the background texture extraction model, the background texture encoder performs downsampling operations on the input data sequentially through the first convolutional block, the second convolutional block, the third convolutional block, and the fourth convolutional block in order to learn the casting background texture features in the input data. The fifth convolutional block is used to further extract the casting background texture features from the data transmitted by the fourth convolutional block; The background texture decoder is used to reconstruct the casting background texture features learned by the background texture encoder into an image with the same resolution as the input data.

3. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 2, characterized in that, The specific steps for the trained background texture extraction model to extract the casting background texture features from the riser and gating segmentation dataset include: The casting background texture features of the gating and riser segmentation dataset are extracted by the second convolutional block to obtain the first casting background texture features, which are then transmitted to the global feature extraction model. The casting background texture features of the riser and gating segment dataset are extracted by the third convolutional block to obtain the second casting background texture features, which are then transmitted to the global feature extraction model. The casting background texture features of the gating and riser segmentation dataset are extracted by the fourth convolutional block to obtain the third casting background texture features, which are then transmitted to the global feature extraction model.

4. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 1, characterized in that, The specific steps of each feature decoupling unit in the first, second, and third feature decoupling units to decouple the casting background texture and gating features from the corresponding feature map transmitted by the global feature encoder using the inverse spatial attention mechanism and in combination with the casting background texture features include: The corresponding feature map transmitted by the global feature encoder and casting background texture features Concatenation along the channel dimension is represented as follows: Through size The first convolutional layer, the first batch normalization layer, the first ReLU activation function, and the size are The second convolutional layer and Sigmoid Activation function on concatenated features The process is repeated sequentially to generate a single-channel similarity map. The calculation formula is as follows: Using similarity graphs right Perform element-wise weighting to generate the decoupled feature map. The calculation formula is as follows: in, This indicates element-wise multiplication.

5. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 4, characterized in that, The segmentation decoder includes a first segmentation module, a second segmentation module, a third segmentation module, and a fourth segmentation module; The segmentation decoder performs feature fusion based on the decoupled feature map, and the specific steps for generating the fused feature map include: The fourth segmentation module receives the third decoupled feature map and sequentially passes it through a segmentation module with a size of [size missing]. The third convolutional layer adjusts the number of channels in the decoupled feature map and After the upsampling operation, a first fused feature map is generated and transmitted to the third segmentation module; The third segmentation module receives the second decoupled feature map and sequentially processes it through a segmentation module of size [size missing]. The fourth convolutional layer adjusts the number of channels in the second decoupled feature map and After the upsampling operation, the feature map is fused with the first fusion feature map element by element to generate the second fusion feature map, which is then transmitted to the second segmentation module. The second segmentation module receives the first decoupled feature map and sequentially passes it through a segmentation module with a size of [size missing]. The fifth convolutional layer adjusts the number of channels in the first decoupled feature map and After the upsampling operation, the feature map is fused with the second fusion feature map element by element to generate the third fusion feature map, which is then transmitted to the first segmentation module. The first segmentation module receives the first feature map and sequentially passes it through a segmentation module with a size of [size missing]. The sixth convolutional layer adjusts the number of channels in the first feature map and performs... After the upsampling operation, the fourth fusion feature map is generated by adding it element by element with the third fusion feature map and then transmitting it to the output module.

6. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 5, characterized in that, The specific steps by which the output module generates the gating and riser segmentation result based on the fused feature map include: The fourth fused feature map is processed sequentially by a 3×3 seventh convolutional layer, a 1×1 eighth convolutional layer, and a Sigmoid activation function to generate the gating and riser segmentation result.

7. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 2, characterized in that, During the self-supervised learning pre-training process of the background texture extraction model, the MAE loss function is used to measure the pixel difference between the reconstructed image and the input data. The calculation formula of the MAE loss function is as follows: in, This represents the total number of pixels in an image block. Input data at pixels The value, To reconstruct the image at the pixel level The value of .

8. The casting gating and riser segmentation method based on texture decoupling and structure-aware segmentation network according to claim 1, characterized in that, During the training of the gating and riser segmentation model based on the casting background texture features and the gating and riser segmentation dataset, a total segmentation loss is set. for: in, The binary cross-entropy loss function; The Dice loss function; The weighting coefficients are set. The binary cross-entropy loss function is expressed as: in The total number of pixels in the image. The true label is either 0 or 1; The probability of a gating and riser predicted by the gating and riser segmentation model; The Dice loss function is expressed as: 。

Images