Turbine blade CT image segmentation and point detection method and system

Abstract

This invention provides a method and system for turbine blade CT image segmentation and point detection, relating to the field of image processing. The method includes: S1, acquiring industrial CT images of turbine blades and constructing a dataset; S2, performing semi-automatic segmentation and annotation, as well as key point annotation; S3, performing image flipping, rotation, and contrast adjustment operations for data enhancement; S4, constructing an image segmentation and point detection model; S5, designing a loss function to train a multi-task learning model; and S6, model training, testing, and output. The system includes an image acquisition module, an image annotation module, a segmentation and point detection model construction module, and a segmentation and point detection module. This invention uses multi-task learning to simultaneously segment and detect points in industrial CT images of turbine blades, achieving automated and high-precision detection of turbine blade wall thickness parameters, and providing a reliable digital solution for quality control and process optimization of key components of aero-engines.

Description

Technical Field

[0001] This invention relates to the field of industrial CT image processing, and specifically to a method and system for CT image segmentation and point detection of turbine blades. Background Technology

[0002] Turbine blades, as core components of aero-engines and gas turbines, directly affect the efficiency, safety, and service life of the power plant. Because turbine blades operate under extreme conditions of high temperature, high pressure, and high-speed rotation, their manufacturing quality must be strictly controlled. Any minute defect can lead to blade failure or even serious accidents. Therefore, high-precision and high-efficiency quality inspection of turbine blades is a crucial step in ensuring the reliability of aero-engines.

[0003] Computed tomography (CT) technology, as an advanced non-destructive testing method, can clearly present internal defects such as porosity and cracks, as well as geometric features, in three-dimensional imaging without damaging the blade structure. It overcomes the shortcomings of traditional testing methods such as ultrasonic or X-ray inspection in terms of accuracy and comprehensiveness. It can display the internal and external structure of the object being inspected and accurately locate and measure errors in size, shape, and contour. Turbine blades are complex structural components made of high density and multiple materials, and their CT images often face various problems such as strong artifacts, low contrast, noise, and uneven grayscale caused by beam hardening. These factors significantly increase the difficulty of image segmentation, thus affecting the accuracy and precision of segmentation. Furthermore, measuring the wall thickness at specific locations on turbine blades is of great significance; therefore, achieving high-precision segmentation and accurate location of measurement points is particularly important. CT image analysis methods based on multi-task deep learning can simultaneously achieve high-precision segmentation of the blade region and automatic detection of key points. This not only improves inspection efficiency but also reduces computational redundancy through a shared feature extraction network, providing more reliable technical support for turbine blade quality assessment.

[0004] In the industrial inspection of turbine blades, wall thickness measurement is a crucial step in ensuring their performance and safety. Currently, the main method relies on manual calibration of measurement points layer by layer on CT images, which suffers from low efficiency and poor repeatability. With the development of intelligent manufacturing, the industry urgently needs a technology that can automatically and accurately segment the blade area and locate measurement points. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the present invention aims to provide a method and system for turbine blade CT image segmentation and point detection. This method uses multi-task learning to simultaneously segment and detect points in industrial CT images of turbine blades, achieving automated and high-precision detection of turbine blade wall thickness parameters. It provides a reliable digital solution for quality control and process optimization of key components of aero-engines, and strongly supports the construction needs of intelligent manufacturing production lines.

[0006] Specifically, the present invention provides a method for turbine blade CT image segmentation and point detection, which includes the following steps:

[0007] S1. Acquire industrial CT images of turbine blades: Use industrial CT to scan turbine blades of the same type and obtain two-dimensional images of the turbine blades through reconstruction algorithms;

[0008] S2. Image annotation: Semi-automatic segmentation and annotation of samples, as well as annotation of key points;

[0009] S3. Enhance data by flipping, rotating, or adjusting contrast;

[0010] S4. Construct a turbine blade segmentation and point detection model based on multi-task deep learning. The segmentation task outputs m channels, corresponding to the m segmentation categories of the turbine blade; the point detection outputs n channels, where n is the number of key points, and each channel represents the probability of each key point. The model includes a skip-connected encoder and decoder. The encoder uses a combination of convolution and transformer for feature extraction, and improves the transformer by modulating attention weights with high-frequency features.

[0011] S5. Construct a target loss function based on a combination of cross-entropy loss function and Euclidean distance loss function. :

[0012] ;

[0013] in, Let cross-entropy be the loss function. For keypoint loss function, Let be the probability that the pixel predicted by the model belongs to the i-th class. The desired coordinates of the target point. The actual coordinates of the target point;

[0014] S6. Model Training, Testing and Output: Put the training set samples into the constructed multi-task deep learning model. After the model training is completed, save the optimal model and test the test set images. Use the trained model to output the segmented images and point coordinates.

[0015] Preferably, in S4, the turbine blade segmentation and point detection model based on multi-task deep learning;

[0016] The encoder uses a hybrid convolutional transformer as its backbone network. First, convolutional layers are used to extract image features. After obtaining the image features, batch normalization layers are used to normalize the image features. The ReLU function is used to perform a nonlinear transformation on the convolutional data to increase the nonlinear expressive power and feature extraction capability of the CNN model. Subsequently, max pooling layers and three stages of residual blocks are used for feature extraction. The residual blocks can extract local features and alleviate gradient vanishing through residual connections. Finally, the modified transformer obtains encoded image blocks from the CNN features, which are used as output feature sequences for global feature extraction.

[0017] The decoder reshapes the feature sequence output by the transformer into a spatial feature map, upsamples it step by step through transposed convolution, and makes skip connections with the feature maps of the corresponding stages of the encoder. The skip connections pass the low-level detailed features retained in the encoder to the decoder, supplementing the high-level semantic information and promoting feature fusion, which helps to restore the spatial structure of the image, improves decoding accuracy and reconstruction quality, and maps the number of channels to m+n channels through convolution, where the number of target categories is m and the number of detection points is n.

[0018] Preferably, the improved transformer in the encoder of S4 includes a layer normalization layer, a multi-head attention mechanism layer, a fully connected layer, and a residual layer. The transformer, based on the image features extracted by the convolutional layers and combined with positional information, further extracts global information of the image through a self-attention layer.

[0019] Specifically, combining location information involves adding trainable location codes of the same size to image feature blocks to increase location information; the layer normalization layer is used to normalize the hidden layers in the neural network to a standard normal distribution; the multi-head attention mechanism layer is used to learn global features based on image features carrying location codes; the fully connected layer is used to perform nonlinear transformation on the learning results of the multi-head attention mechanism layer; and the residual layer is used to introduce a residual mechanism between the multi-head attention mechanism layer and the fully connected layer.

[0020] Preferably, in step S4, to address the blurred edges in turbine blade CT images, the transformer is improved by modulating attention weights with high-frequency features, enabling the model to focus on the edge regions. This specifically includes the following sub-steps:

[0021] Step 1: First, define the high-frequency feature map:

[0022] ;

[0023] in, It is the spatial form of the input features; Sobel is the edge detection operator. ;

[0024] Step 2: Combine the global modeling capabilities of the Transformer with the local edge extraction capabilities of the Sobel operator;

[0025] Step 3: Modulate the attention weights using high-frequency features. The improved attention output based on high-frequency edge feature modulation is as follows:

[0026] ;

[0027] Where Q (Query) represents the association request proposed by the current image region during information interaction, used to retrieve relevant image regions in the feature space; K (Key) represents the feature encoding of each image region, used for similarity matching with Query in the attention mechanism to measure the relevance between regions; V (Value) represents the actual semantic information carried by each image region, which is weighted and aggregated according to its relevance after calculating attention weights to update the feature representation; H represents edge features; where , The edge features H are mapped to the same dimension as the Query, which is used to modulate the Q vector. The edge features H are mapped to the same dimension as the Key, which is used to modulate the K vector.

[0028] Preferably, S1 obtains data of different height layers of the same type of turbine blade through industrial CT scanning, and then obtains a two-dimensional image of the turbine blade through a three-dimensional reconstruction algorithm;

[0029] S2 specifically includes the following sub-steps:

[0030] S21. Load the image to be labeled into the SAM model, provide the initial interaction signal, use the pre-trained SAM model to generate candidate segmentation masks in real time, add positive and negative points to adjust the bounding box for inaccurately segmented regions to dynamically optimize the mask boundary, and finally output the labeled data in JSON format.

[0031] S22. Use the labelme tool to calibrate measurement points on the same image and save it as a JSON file.

[0032] Preferably, the method of flipping, rotating, or adjusting the contrast in S3 is as follows:

[0033] Flipping is divided into horizontal flipping and vertical flipping. Horizontal flipping is specifically as follows:

[0034] ;

[0035] Flip vertically as follows:

[0036] ;

[0037] The original image is I, with dimensions W*H and pixel coordinates of... Pixel intensity ;

[0038] The rotation specifically refers to the transformation of coordinates when the image center is used as the rotation center and the counterclockwise rotation angle is θ. The relationship with the original coordinates (x, y) is as follows:

[0039] (y- sin + ;

[0040] (y- cos ;

[0041] in, The coordinates of the image center;

[0042] Contrast adjustment specifically involves performing an affine transformation on all pixels:

[0043] ;

[0044] in, Enhance contrast, 0< Reduce contrast at times. Control the overall brightness offset.

[0045] Preferably, in step S6, the image data, corresponding mask image, and key points are loaded into the model; after obtaining the labeled data for the segmentation task, a high-precision mask image is loaded for model training. The specific steps are as follows:

[0046] Step 1: Generate a high-resolution grayscale image: Assume the target resolution is... The number of pixels in the image is increased by interpolation, and the image is oversampled to 4N×4N, where N>H and N>W;

[0047] Step 2, Downsampling stage: The oversampled data... Image downsampling to .

[0048] Preferably, in step S6, the training set is fed into the model for training. After obtaining the optimal weights, the trained model is quantitatively evaluated. The Iou value is used to evaluate the segmentation accuracy, and the key point localization error is used to evaluate the detection accuracy. The average Iou value is ≥0.90, and the average localization error is ≤3 pixels.

[0049] The formula for calculating the Iou value is:

[0050] ;

[0051] In this case, the predicted segmentation result is A, and the true label is B.

[0052] Preferably, the keypoint loss function in S5 is calculated through the following steps:

[0053] Applying spatial softmax to the keypoint output channels yields a probability map. :

[0054] ;

[0055] in, This is the raw value of the j-th channel in the model point detection task, representing the location of the j-th keypoint. The value;

[0056] The desired coordinates of the target point are obtained by weighted summation with the coordinate grid. ;

[0057] = ;

[0058] = ;

[0059] The coordinate grid uses normalized coordinates:

[0060] ;

[0061] Calculate the true coordinates of the target point Desired coordinates of the target point The Euclidean distance is used as the keypoint loss function.

[0062] On the other hand, the present invention provides a detection system for a turbine blade CT image segmentation and point detection method, comprising: an image acquisition module, an image annotation module, a segmentation and point detection model construction module, and a segmentation and point detection module;

[0063] The image acquisition module is used to acquire industrial CT images of turbine blades; the image annotation module is used to perform semi-automatic segmentation and annotation of key points on the acquired industrial CT images of turbine blades.

[0064] The segmentation and point detection model building module is used to build a neural network model including an encoder and a decoder as a segmentation and point detection model. After the model is built, it is trained and tested. The encoder includes a hybrid network structure of convolution and improved Transformer. The decoder is upsampled through transposed convolution and fused with the encoder feature map through skip connections to reconstruct the spatial feature map.

[0065] The segmentation and point detection module is used to output segmented images and point coordinates using the trained segmentation and point detection model.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] (1) This invention provides a method for CT image segmentation and point detection of turbine blades. Compared with traditional manual measurement methods, this method solves the technical bottlenecks such as low measurement efficiency and accumulation of subjective errors, realizes automated and high-precision detection of turbine blade wall thickness parameters, provides a reliable digital solution for quality control and process optimization of key components of aero-engines, and strongly supports the construction needs of intelligent manufacturing production lines.

[0068] (2) This invention provides an intelligent detection method for turbine blade CT images based on a multi-task deep learning model. It achieves integrated parallel output of image segmentation and key point detection through a network architecture that combines convolution and an improved transformer. Convolution cannot model long-distance dependencies well, while the global self-attention mechanism of the Transformer can effectively acquire global information, but it is insufficient in acquiring low-level detailed information, which leads to limitations in localization ability. Therefore, the combination of convolution and the improved transformer can effectively extract features.

[0069] (3) The method of the present invention adopts a hybrid optimization strategy that combines cross-entropy loss and Euclidean distance loss. Through the feature sharing mechanism, the two tasks promote each other, which significantly improves the segmentation accuracy of complex blade boundaries and the positioning accuracy of key measurement points.

[0070] (4) To address the unique issues of metal artifact interference and boundary blurring in turbine blade CT images, this invention utilizes high-frequency feature modulation of attention weights to encourage the model to focus on edge regions. This approach effectively focuses on edges, uses high-frequency feature modulation of attention weights to specifically improve the edge blurring problem in turbine blade CT images, and enhances the model's adaptability to different imaging conditions, making it perform better in handling various complex imaging situations. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0072] Figure 2 This is a flowchart illustrating the process of a turbine blade CT image segmentation and point detection method according to the present invention.

[0073] Figure 3 This is a network model structure diagram of the multi-task learning model of the present invention;

[0074] Figure 4 This refers to the Transformer Layer structure in the multi-task learning model of this invention;

[0075] Figure 5 For the data annotation display in this embodiment of the invention, the left image shows the segmentation data annotation, and the right image shows the point data annotation;

[0076] Figure 6 This is a demonstration of the evaluation effect of the multi-task model in an embodiment of the present invention;

[0077] Figure 7 This refers to the loss variation during the training process of the multi-task model in this embodiment of the invention;

[0078] Figure 8 This is a schematic block diagram of the detection system of the present invention;

[0079] Figure 9 This is a schematic diagram of the physical structure of the electronic device in an embodiment of the present invention. Detailed Implementation

[0080] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0081] Specifically, this invention provides a method for turbine blade CT image segmentation and point detection, such as... Figure 1 As shown, it includes the following steps:

[0082] S1. Acquiring Industrial CT Images of Turbine Blades: Turbine blades of the same type are scanned using industrial CT scanners, and two-dimensional images of the turbine blades are obtained through reconstruction algorithms. In S1, data of different height layers of the same type of turbine blade are obtained through industrial CT scanning, and then two-dimensional images of the turbine blades are obtained through three-dimensional reconstruction algorithms.

[0083] S2. Image annotation: Semi-automatic segmentation and annotation of samples, specifically including the following sub-steps:

[0084] S21. Load the image to be labeled into the pre-trained SAM model, provide the initial interaction signal, the SAM model generates candidate segmentation masks in real time, and adds positive and negative points to adjust the bounding box to dynamically optimize the mask boundary for inaccurately segmented regions. Finally, output the labeled data in JSON format.

[0085] S22. Use the labelme tool to calibrate measurement points on the same image and save it as a JSON file.

[0086] S3. Data augmentation using flipping, rotation, or contrast adjustment; flipping is divided into horizontal flipping and vertical flipping, with horizontal flipping specifically as follows:

[0087] ;

[0088] Flip vertically as follows:

[0089] ;

[0090] The original image is I, with dimensions W*H and pixel coordinates of... Pixel intensity .

[0091] The rotation is specifically as follows: At θ, the transformed coordinates The relationship with the original coordinates (x, y) is as follows:

[0092] (y- sin + ;

[0093] (y- cos ;

[0094] in, These are the coordinates of the image center.

[0095] Contrast adjustment specifically involves performing an affine transformation on all pixels:

[0096] ;

[0097] in, Enhance contrast, 0< Reduce contrast at times. Control the overall brightness offset.

[0098] S4. Construct a turbine blade segmentation and point detection model based on multi-task deep learning. The segmentation task outputs m channels, corresponding to the m segmentation categories of the turbine blade; the point detection outputs n channels, where n is the number of key points, and each channel represents the probability of each key point. The model includes a skip-connected encoder and decoder. The encoder uses a combination of convolution and transformer for feature extraction, and the transformer is improved by modulating attention weights with high-frequency features.

[0099] like Figure 3 and Figure 4 As shown, in the multi-task learning model of turbine blade segmentation and point detection model, the model includes a skip-connected encoder and decoder. The encoder combines convolution and transformer. First, convolutional layers are used to extract image features. After obtaining the image features, batch normalization layers are used to normalize the image features.

[0100] The ReLU function is used to perform a non-linear transformation on the convolutional data, thereby increasing the non-linear expressive power and feature extraction capability of the CNN model.

[0101] After features are extracted by convolutional layers, residual blocks are used to extract richer and more abstract local features layer by layer, enhancing the network's feature representation capabilities. At the same time, residual connections alleviate gradient vanishing, maintain feature stability, and provide a better local feature foundation for subsequent Transformer modules, so as to better model global information.

[0102] The transformer consists of a layer normalization (LN) layer, a multi-head attention mechanism layer, a fully connected layer, and a residual layer. Based on the image features extracted by the convolutional layers, combined with location information, a self-attention layer further extracts global information from the image: in this step, trainable location codes of the same size are added to the image feature blocks to increase location information. The layer normalization layer normalizes the hidden layers in the neural network to a standard normal distribution; based on the image features carrying location codes, the multi-head attention mechanism layer learns global features; the fully connected layer performs a non-linear transformation on the learning results of the multi-head attention mechanism layer; and the residual layer introduces a residual mechanism between the multi-head attention mechanism layer and the fully connected layer.

[0103] In one embodiment, the specific structure of the network includes:

[0104] The encoder uses a hybrid convolutional transformer as the backbone network. Initial features are extracted through convolution, followed by batch normalization and ReLU activation, then max pooling layers, and finally three stages of residual blocks for feature extraction. The extracted features are then passed through the improved transformer layers. The improved transformer obtains encoded image blocks from the CNN features, which are used as output feature sequences for global feature extraction.

[0105] The decoder reshapes the improved transformer output feature sequence into a spatial feature map, progressively upsampling it through transposed convolutions and then performing skip connections with the corresponding feature maps from the encoder. The purpose of these skip connections is to pass low-level detail features retained in the encoder to the decoder, supplementing high-level semantic information, promoting feature fusion, helping to recover the image's spatial structure, and improving decoding accuracy and reconstruction quality. Convolution maps the number of channels to m+n channels, where m are the number of target categories and n are the number of detection points.

[0106] To address the issue of blurred edges in turbine blade CT images, the transformer is improved by modulating attention weights with high-frequency features, enabling the model to focus on edge regions. This involves the following sub-steps:

[0107] Step 1: First, define the high-frequency feature map:

[0108] ;

[0109] in, It is the spatial form of the input features; Sobel is the edge detection operator. .

[0110] Step 2: Combine the global modeling capabilities of Transformer with the local edge extraction capabilities of Sobel operator; By combining the global modeling capabilities of Transformer with the local edge extraction capabilities of Sobel operator, the model's accuracy in segmenting complex geometric structures and detecting key points can be significantly improved.

[0111] Step 3: Modulate the attention weights using high-frequency features. The improved attention output based on high-frequency edge feature modulation is as follows:

[0112] ;

[0113] Where Q (Query) represents the association request proposed by the current image region during information interaction, used to retrieve relevant image regions in the feature space; K (Key) represents the feature encoding of each image region, used for similarity matching with Q in the attention mechanism to measure the relevance between regions; V (Value) represents the actual semantic information carried by each image region, which is weighted and summed according to its relevance after calculating the attention weights to update the feature representation; H represents edge features; where , The edge features H are mapped to the same dimension as Q to modulate the Q vector. The edge feature H is mapped to the same dimension as K, which is used to modulate the K vector.

[0114] In the specific calculation process, to optimize computation, complete edge features are calculated only in the first Transformer layer, and subsequent layers reuse and update these features. This improvement allows the model to significantly enhance the detection accuracy of blurred edges while maintaining its ability to recognize the main structure.

[0115] S5. Construct a target loss function based on a combination of the cross-entropy loss function and the Euclidean distance loss function. The specific loss function is as follows:

[0116] ;

[0117] in, Let cross-entropy be the loss function. For keypoint loss function, Let be the probability that the pixel predicted by the model belongs to the i-th class. The desired coordinates of the target point. The actual coordinates of the target point.

[0118] Preferably, the keypoint loss function in S5 is calculated through the following steps:

[0119] Applying spatial softmax to the keypoint output channels yields a probability map. :

[0120] ;

[0121] in, This is the raw value of the j-th channel in the model point detection task, representing the location of the j-th keypoint. The value of .

[0122] The desired coordinates of the target point are obtained by weighted summation with the coordinate grid. ;

[0123] = ;

[0124] = ;

[0125] The coordinate grid uses normalized coordinates:

[0126] ;

[0127] Calculate the desired coordinates with actual coordinates The Euclidean distance is used as the keypoint loss function.

[0128] S6. Model Training and Testing: The training set samples are fed into the constructed multi-task deep learning model. After training, the optimal model is saved, and the model is tested on the test set images. The trained model is then used to output segmented images and point coordinates.

[0129] Preferably, in step S6, the image data, corresponding mask image, and keypoint locations are loaded into the model. After obtaining the labeled data for the segmentation task, a high-precision mask image is loaded for model training. When loading the data and corresponding mask image into the model, the quality of the mask image is improved through oversampling anti-aliasing. Rendering is performed on a canvas with a higher resolution than the target image, and more sampling point pixel values ​​are calculated through interpolation. The image is then scaled down to the target resolution to smooth out jagged edges. The specific steps are as follows:

[0130] Step 1: Generate a high-resolution grayscale image: Assume the target resolution is... The number of pixels in the image is increased by interpolation, and the image is oversampled to 4N×4N, where N>H and N>W.

[0131] This step increases the number of pixels in the image through interpolation and other methods, allowing the image to display more details at higher resolutions. Common interpolation methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. The choice of interpolation algorithm affects the image quality and detail. Bilinear and bicubic interpolation can generate smoother images, but they are relatively computationally expensive; nearest neighbor interpolation is simple to calculate, but it may cause a mosaic effect in the image.

[0132] Step 2, Downsampling stage: The oversampled data... Image downsampling to This step downsamples the oversampled image, recalculating pixel values ​​using a downsampling algorithm to smooth the image and reduce jagged edges. Common downsampling methods include simple averaging, weighted averaging, and Gaussian filtering downsampling. Oversampling anti-aliasing effectively reduces jagged edges and improves image quality.

[0133] Preferably, in step S6, the training set is fed into the model for training. After obtaining the optimal weights, the trained model is quantitatively evaluated. The Iou value is used to evaluate the segmentation accuracy, and the key point localization error is used to evaluate the detection accuracy. The average Iou value is ≥0.90, and the average localization error is ≤3 pixels.

[0134] The formula for calculating the Iou value is:

[0135] ;

[0136] In this case, the predicted segmentation result is A, and the true label is B. Specific Implementation

[0138] like Figure 2 As shown, this embodiment of the invention provides an industrial CT turbine blade segmentation and point detection method based on multi-task learning.

[0139] S1. Data Acquisition: Industrial CT scanning equipment was used to acquire tomographic images of turbine blades. A total of 96 images of the same model but different layers were acquired.

[0140] S2. Data annotation: An interactive semi-automatic image annotation tool based on the SAM model is used for sample segmentation and annotation, and the annotation data is output in JSON format. In addition, labelme is used for point calibration, with a point count of 6.

[0141] S3. Data Augmentation Processing: Perform multi-dimensional data augmentation on industrial CT images, including flipping, rotating, and contrast adjustment.

[0142] Spatial transformation: random horizontal / vertical flip (probability 0.5), rotation (angle range ±180°).

[0143] Flipping can be divided into horizontal flipping and vertical flipping. Horizontal flipping is as follows:

[0144] ;

[0145] Flip vertically as follows:

[0146] ;

[0147] The original image is I, with dimensions W*H and pixel coordinates of... Pixel intensity The probability of flipping is 0.5.

[0148] Image center coordinates are Rotate the image center by ±180°, and then transform the coordinates. The relationship with the original coordinates (x, y) is as follows:

[0149] (y- sin + ;

[0150] (y- cos ;

[0151] Contrast adjustment involves performing an affine transformation on all pixels:

[0152] ;

[0153] in, Enhance contrast, 0< Reduce contrast at times. Control the overall brightness offset, among which It ranges from 0.8 to 1.2.

[0154] S4. Network Model Construction: Construct a deep learning-based image segmentation and point detection model, which includes an encoder and a decoder.

[0155] The encoder uses a hybrid convolutional transformer as the backbone network. It extracts initial features through 7×7 convolutions (64 convolutional kernels, stride 2), and after batch normalization and ReLU activation, it downsamples through 3×3 max pooling (stride 2). Then, it passes through three stages of residual blocks. Its output is processed by patch embedding and position encoding as input to the transformer. The transformer obtains encoded image blocks from the feature maps of the CNN for global feature extraction.

[0156] The decoder reshapes the feature sequence output by the improved Transformer into a spatial feature map, upsamples it step by step through transposed convolution, and makes skip connections with the feature maps of the corresponding stages of the encoder. Finally, it maps the number of channels to the sum of the number of target categories (1 in this embodiment) and the number of detection points (6 in this embodiment) through 1×1 convolution.

[0157] S5. The objective function employs a combination of cross-entropy loss and Euclidean distance loss:

[0158] + ;

[0159] in, The cross-entropy loss function is used to calculate the difference between the model's predicted probability and the true label. The keypoint loss function is obtained by applying spatial softmax to the keypoint output channel to obtain a probability map, which is then weighted and summed with the coordinate grid to obtain the expected coordinates. The Euclidean distance between the expected coordinates and the true coordinates is calculated. λ is the balancing weight, which is 1.0 in this embodiment.

[0160] S6. Model Training and Evaluation: When loading images and corresponding mask images into the model, oversampling anti-aliasing is used to improve the accuracy of mask image edges. JSON data is loaded, and the accuracy of edges can be improved by upsampling and then downsampling the JSON data.

[0161] The trained model is quantitatively evaluated using Iou values ​​to assess segmentation accuracy (average Iou value ≥ 0.90) and keypoint localization errors to assess detection accuracy (average localization error ≤ 3 pixels). Figure 5The data annotation in this embodiment has been tested. The left image shows the segmentation data annotation, and the right image shows the point data annotation. Figure 6 The evaluation results of the multi-task model are shown; Figure 7 The loss variation during the training process of the multi-task model was tested.

[0162] The technical solution of this invention realizes automated and accurate segmentation and point detection of CT images of turbine blades, which significantly improves detection efficiency and accuracy, and provides reliable technical support for intelligent manufacturing and quality control of turbine blades.

[0163] On the other hand, the present invention provides a detection system for turbine blade CT image segmentation and point detection methods, such as... Figure 8 As shown, it includes an image acquisition module 1, an image annotation module 2, a segmentation and point detection model construction module 3, and a segmentation and point detection module 4.

[0164] Image acquisition module 1 is used to acquire industrial CT images of turbine blades; image annotation module 2 is used to perform semi-automatic segmentation and annotation of key points on the acquired industrial CT images of turbine blades.

[0165] The segmentation and point detection model building module 3 is used to build a neural network model including an encoder and a decoder as a segmentation and point detection model. After the model is built, it is trained and tested. The encoder includes a hybrid network structure of convolution and improved Transformer. The decoder is upsampled through transposed convolution and fused with the encoder feature map through skip connections to reconstruct the spatial feature map.

[0166] The segmentation and point detection module 4 is used to output segmented images and point coordinates using the trained segmentation and point detection model.

[0167] Figure 9 A schematic diagram of the physical structure of an electronic device is shown, such as... Figure 9 As shown, the electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory to execute the following steps:

[0168] Receive industrial CT images of turbine blades.

[0169] Based on the received image data, the multi-task learning model constructed using the aforementioned method for constructing turbine blade CT image segmentation and point detection models outputs segmented images and point coordinates.

[0170] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0171] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the following steps:

[0172] Receive industrial CT images of turbine blades; based on the received image data, output segmented images and point coordinates using the multi-task learning model constructed by the above-mentioned method for constructing turbine blade CT image segmentation and point detection models.

[0173] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0174] Receive industrial CT images of turbine blades; based on the received image data, output segmented images and point coordinates using the multi-task learning model constructed by the above-mentioned method for constructing turbine blade CT image segmentation and point detection models.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A turbine blade CT image segmentation and point detection method, characterized in that: It includes the following steps: S1. Acquire industrial CT images of turbine blades: Use industrial CT to scan turbine blades of the same type and obtain two-dimensional images of the turbine blades through reconstruction algorithms; S2. Image annotation: Semi-automatic segmentation and annotation of samples, as well as annotation of key points; S3. Enhance data by flipping, rotating, or adjusting contrast; S4. Construct a turbine blade segmentation and point detection model based on multi-task deep learning. The segmentation task outputs m channels, corresponding to the m segmentation categories of the turbine blade; the point detection outputs n channels, where n is the number of key points, and each channel represents the probability of each key point. The model includes a skip-connected encoder and decoder. The encoder uses a combination of convolution and transformer for feature extraction, and improves the transformer by modulating attention weights with high-frequency features. S5、constructing a target loss function based on the combination of the cross-entropy loss function and the Euclidean distance loss function : ； in, Let cross-entropy be the loss function. For keypoint loss function, Let be the probability that the pixel predicted by the model belongs to the i-th class. The desired coordinates of the target point. The actual coordinates of the target point; S6. Model Training, Testing and Output: Put the training set samples into the constructed multi-task deep learning model. After the model training is completed, save the optimal model and test the test set images. Use the trained model to output the segmented images and point coordinates.

2. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: In the turbine blade segmentation and point detection model based on multi-task deep learning in S4; The encoder uses a hybrid convolutional transformer as its backbone network. First, convolutional layers are used to extract image features. After obtaining the image features, batch normalization layers are used to normalize the image features. The ReLU function is used to perform a nonlinear transformation on the convolutional data to increase the nonlinear expressive power and feature extraction capability of the CNN model. Subsequently, max pooling layers and three stages of residual blocks are used for feature extraction. The residual blocks can extract local features and alleviate gradient vanishing through residual connections. Finally, the modified transformer obtains encoded image blocks from the CNN features, which are used as output feature sequences for global feature extraction. The decoder reshapes the feature sequence output by the transformer into a spatial feature map, upsamples it step by step through transposed convolution, and makes skip connections with the feature maps of the corresponding stages of the encoder. The skip connections pass the low-level detailed features retained in the encoder to the decoder, supplementing the high-level semantic information and promoting feature fusion, which helps to restore the spatial structure of the image, improves decoding accuracy and reconstruction quality, and maps the number of channels to m+n channels through convolution, where the number of target categories is m and the number of detection points is n.

3. The method for turbine blade CT image segmentation and point detection according to claim 2, characterized in that: The improved transformer in the S4 encoder includes a layer normalization layer, a multi-head attention mechanism layer, a fully connected layer, and a residual layer. The transformer combines image features extracted from the convolutional layers with location information and further extracts global information from the image through a self-attention layer. Specifically, combining location information involves adding trainable location codes of the same size to image feature blocks to increase location information; the layer normalization layer is used to normalize the hidden layers in the neural network to a standard normal distribution; the multi-head attention mechanism layer is used to learn global features based on image features carrying location codes; the fully connected layer is used to perform nonlinear transformation on the learning results of the multi-head attention mechanism layer; and the residual layer is used to introduce a residual mechanism between the multi-head attention mechanism layer and the fully connected layer.

4. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: In S4, to address the blurred edges in turbine blade CT images, the transformer is improved by modulating attention weights with high-frequency features, enabling the model to focus on edge regions. This involves the following sub-steps: Step 1: First, define the high-frequency feature map: ； in, It is the spatial form of the input features; Sobel is the edge detection operator. ; Step 2: Combine the global modeling capabilities of the Transformer with the local edge extraction capabilities of the Sobel operator; Step 3: Modulate the attention weights using high-frequency features. The improved attention output based on high-frequency edge feature modulation is as follows: ； Where Q (Query) represents the association request proposed by the current image region during information interaction, used to retrieve relevant image regions in the feature space; K (Key) represents the feature encoding of each image region, used for similarity matching with Query in the attention mechanism to measure the relevance between regions; V (Value) represents the actual semantic information carried by each image region, which is weighted and aggregated according to its relevance after calculating attention weights to update the feature representation; H represents edge features; where , The edge features H are mapped to the same dimension as the Query, which is used to modulate the Q vector. The edge features H are mapped to the same dimension as the Key, which is used to modulate the K vector.

5. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: In S1, data of different height layers of the same type of turbine blade are obtained through industrial CT scanning, and then two-dimensional images of the turbine blade are obtained through a three-dimensional reconstruction algorithm. S2 specifically includes the following sub-steps: S21. Load the image to be labeled into the SAM model, provide the initial interaction signal, use the pre-trained SAM model to generate candidate segmentation masks in real time, add positive and negative points to adjust the bounding box for inaccurately segmented regions to dynamically optimize the mask boundary, and finally output the labeled data in JSON format. S22. Use the labelme tool to calibrate measurement points on the same image and save it as a JSON file.

6. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: The specific ways to flip, rotate, or adjust contrast in S3 are as follows: Flipping is divided into horizontal flipping and vertical flipping. Horizontal flipping is specifically as follows: ； Flip vertically as follows: ； The original image is I, with dimensions W*H and pixel coordinates of... Pixel intensity ; The rotation specifically refers to the transformation of coordinates when the image center is used as the rotation center and the counterclockwise rotation angle is θ. The relationship with the original coordinates (x, y) is as follows: (and- )without + ; (and- )cos ; in, The coordinates of the image center; Contrast adjustment specifically involves performing an affine transformation on all pixels: ； in, Enhance contrast, 0< Reduce contrast at times. Control the overall brightness offset.

7. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: In S6, image data, corresponding mask images, and key points are loaded into the model. After obtaining the labeled data for the segmentation task, a high-precision mask image is loaded for model training. The specific steps are as follows: Step 1: Generate a high-resolution grayscale image: Assume the target resolution is... The number of pixels in the image is increased by interpolation, and the image is oversampled to 4N×4N, where N>H and N>W; Step 2, Downsampling stage: The oversampled data... Image downsampling to .

8. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: S6 feeds the training set into the model for training. After obtaining the optimal weights, it performs a quantitative evaluation of the trained model, using the Iou value to evaluate the segmentation accuracy and the key point localization error to evaluate the detection accuracy; the average Iou value is ≥0.90 and the average localization error is ≤3 pixels. The formula for calculating the Iou value is: ； In this case, the predicted segmentation result is A, and the true label is B.

9. The method for turbine blade CT image segmentation and point detection according to claim 1, characterized in that: The keypoint loss function in S5 is calculated through the following steps: Applying spatial softmax to the keypoint output channels yields a probability map. , ； in, This is the raw value of the j-th channel in the model point detection task, representing the location of the j-th keypoint. The value; The desired coordinates of the target point are obtained by weighted summation with the coordinate grid. ; = ； = ； The coordinate grid uses normalized coordinates: ； Calculate the true coordinates of the target point Desired coordinates of the target point The Euclidean distance is used as the keypoint loss function.

10. A detection system for the turbine blade CT image segmentation and point detection method according to claim 1, characterized in that: It includes: Image acquisition module, image annotation module, segmentation and point detection model construction module, and segmentation and point detection module; The image acquisition module is used to acquire industrial CT images of turbine blades; The image annotation module is used to perform semi-automatic segmentation and annotation of key points on the acquired industrial CT images of turbine blades; The segmentation and point detection model building module is used to build a neural network model including an encoder and a decoder as a segmentation and point detection model. After the model is built, it is trained and tested. The encoder includes a hybrid network structure of convolution and improved Transformer. The decoder is upsampled through transposed convolution and fused with the encoder feature map through skip connections to reconstruct the spatial feature map. The segmentation and point detection module is used to output segmented images and point coordinates using the trained segmentation and point detection model.

Images