X-Ray image segmentation method based on feature enhancement and related equipment
By constructing a feature-enhanced X-Ray image segmentation method and utilizing multi-level feature fusion and feature enhancement modules, the problem of inaccurate damage identification in composite structures is solved, and accurate damage identification and structural safety evaluation are achieved.
Patent Information
- Application Number
- CN202510644078.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-19
- Publication Date
- 2025-10-03
Smart Images
Figure CN120747154A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an X-ray image segmentation method based on feature enhancement and related equipment. Background Art
[0002] Composite materials, with their superior properties such as light weight, high strength, and design flexibility, are increasingly being used in engineering structures. Due to their excellent mechanical properties, composites can be found in a wide range of engineering structures and products and are often used as primary structural components in high-end technologies such as aircraft, automobiles, and spacecraft. However, composite materials face the challenges of complex chemical and physical environments during service. The occurrence and accumulation of damage can lead to catastrophic failure, posing a serious challenge to structural safety. Therefore, to achieve damage tolerance and ensure structural safety, it is essential to accurately identify the type, location, size, and shape of damage.
[0003] Structural health monitoring technology based on guided wave has been widely used for rapid damage detection in metal and composite structures. To excite and receive ultrasonic guided wave signals, low-cost piezoelectric (PZT) transducers are increasingly being used in the field of structural health monitoring, due to their advantages of low price, small size, and low power consumption. For structural damage identification, due to the complex multimodal and dispersion characteristics of guided wave propagation, practical applications require accurate acquisition of guided wave propagation information in the inspected structure. In addition, accurate and reliable analysis and processing of the received guided wave signals are crucial for guided wave-based damage shape identification methods. Traditional damage identification methods often only focus on locating the damage position, ignoring the damage size and shape, which directly affect the safety assessment and design of composite structures. That is, the damage identification methods in the existing technology are not accurate enough in the safety assessment and design of composite structures, and have limitations. Summary of the Invention
[0004] In view of this, the present invention provides an X-Ray image segmentation method and related equipment based on feature enhancement, which are used to solve the problem that the damage identification method in the existing technology is not accurate enough and has limitations in the safety evaluation and design of composite materials structures.
[0005] In a first aspect, an embodiment of the present invention provides an X-ray image segmentation method based on feature enhancement, the method comprising:
[0006] Constructing a data set that includes at least X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and dividing the data set into a training set and a test set, training a composite material plate damage image segmentation model using the training set, and testing the trained composite material plate damage image segmentation model using the test set, to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features;
[0007] An X-ray image to be segmented is obtained, and the X-ray image to be segmented is input into the target segmentation model to obtain a segmentation result.
[0008] Optionally, the step of training the composite material plate damage image segmentation model using the training set and testing the trained composite material plate damage image segmentation model using the test set to obtain a target segmentation model that meets preset conditions includes:
[0009] Performing a first preprocessing on the images in the training set to obtain a target training set, wherein the first preprocessing includes at least one of a cropping process, a horizontal flipping process, and a normalization process;
[0010] Performing a second preprocessing on the images in the test set to obtain a target test set, wherein the second preprocessing includes at least one of scaling, cropping, and normalization;
[0011] Three composite material plate damage image segmentation models using different loss functions are trained using the training set to obtain three trained composite material plate damage image segmentation models;
[0012] The three trained composite material plate damage image segmentation models are tested using the test set, and the composite material plate damage image segmentation model when the damage function no longer decreases is used as the target segmentation model to obtain three target segmentation models.
[0013] Optionally, the step of inputting the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result includes:
[0014] Inputting the X-Ray image to be segmented into the feature extraction module of the target segmentation model to obtain a shallow feature map and deep features of different levels of the X-Ray image to be segmented;
[0015] Inputting the shallow feature map into the CIM module of the target segmentation model to obtain shallow feature data combining channel attention and spatial attention;
[0016] Inputting the deep features into the CFM module of the target segmentation model to fuse the deep features of different levels to obtain deep fused feature data;
[0017] The shallow feature data and the deep fusion feature data are input into the SAM Ham module of the target segmentation model to fuse the shallow feature data and the deep fusion feature data to obtain target feature data, and a segmentation result is obtained according to the target feature data.
[0018] Optionally, the CIM module includes a channel attention unit and a spatial attention unit, and the step of inputting the shallow feature map into the CIM module of the target segmentation model to obtain shallow feature data combining channel attention and spatial attention includes:
[0019] Inputting the shallow feature map into the channel attention unit of the CIM module in the target segmentation model, calculating the global average pooling data of the shallow feature map through the attention unit, and converting each shallow feature in the shallow feature map into a vector representation according to the global average pooling data, inputting the vector representation into an activation function for activation to obtain a first channel weight, and performing weighted processing on each channel of the shallow feature map based on the first channel weight to obtain a target shallow feature map;
[0020] Inputting the target shallow feature map into the spatial attention unit of the CIM module in the target segmentation model, and downsampling the target shallow feature map in the channel dimension and the spatial dimension by the spatial attention unit to obtain downsampled data;
[0021] Performing global average pooling on the downsampled data in the channel dimension to obtain a channel weight vector;
[0022] Performing a convolution operation on the downsampled data in the spatial dimension to obtain a weight value for each position in the target shallow feature map, thereby generating a spatial weight map;
[0023] Multiplying the channel weight vector and the spatial weight map to obtain a spatial attention weight map;
[0024] The spatial attention weight map is used to perform a weighted operation on the target shallow feature map to obtain shallow feature data that combines channel attention and spatial attention.
[0025] Optionally, the step of inputting the deep features into the CFM module of the target segmentation model to fuse the deep features of different levels to obtain deep fused feature data includes:
[0026] Inputting the deep features into the CFM module of the target segmentation model, upsampling the highest-level feature map in the deep features through the CFM module to obtain first upsampled data corresponding to the second highest-level feature map in the deep features;
[0027] Transferring the first up-sampled data through a convolution unit to obtain a transfer result;
[0028] Performing a Hadamard product calculation on the transferred result and the second highest level feature map in the deep feature layer to obtain a first target feature map;
[0029] Splicing the first target feature map and the transferred result to obtain a first splicing result, and smoothing the splicing result using a convolution unit to obtain a fused feature map;
[0030] Upsampling the fused feature map, the highest-level feature map in the deep features, and the second-highest-level feature map in the deep features to obtain second upsampled data corresponding to the third-highest-level feature map in the deep features;
[0031] Smoothing the first sampled data corresponding to the highest-level feature map in the deep features in the second upsampled data, the second sampled data corresponding to the second highest-level feature map in the deep features in the second upsampled data, and the third highest-level feature map in the deep features to obtain smoothed data;
[0032] Perform Hadamard product calculation on the smoothed data to obtain the second target feature map;
[0033] The second target feature map is spliced with the third sampling data corresponding to the fused feature map in the smoothed second up-sampled data to obtain deep fused feature data.
[0034] Optionally, the step of inputting the shallow feature data and the deep fusion feature data into the SAM Ham module of the target segmentation model to fuse the shallow feature data and the deep fusion feature data to obtain target feature data includes:
[0035] Inputting the deep fusion feature data into the SAM Ham module of the target segmentation model, and performing dimensionality reduction on the deep fusion feature data through the SAMHam module to obtain a first dimensionality reduction feature map and a target dimensionality reduction feature map;
[0036] The shallow feature data is input into the SAM Ham module of the target segmentation model, the shallow feature data is reduced in dimension by the SAM Ham module to obtain an initial reduced-dimensionality feature map, and the second reduced-dimensionality feature map is interpolated to the same size as the deep fusion feature data to generate a second reduced-dimensionality feature map; a Softmax function is applied to the channel dimension of the second reduced-dimensionality feature map, and any channel of the second reduced-dimensionality feature map is selected as an attention map to obtain a target feature map; a Hadamard product is calculated between the first reduced-dimensionality feature map and the target feature map to assign different weights to different pixels to obtain a feature map to be cropped; an adaptive pooling operation is performed to reduce feature shift and center crop the feature map to be cropped to obtain a feature map;
[0037] Then, the correlation between the feature map and each pixel in the first dimensionality reduction feature map is established through the inner product to obtain a related attention map; the related attention map is multiplied by the target dimensionality reduction feature map to obtain a product feature map; the product feature map is fed to the graph convolution layer to obtain a graph convolution feature map, and the inner product of the related attention map and the graph convolution feature map is calculated, and the graph domain features are reconstructed into the original structural features according to the inner product to obtain a reconstructed feature map; the reconstructed feature map is adjusted to the same channel size as the feature map through a convolution layer with a kernel size of 1×1 to obtain a target feature map; the target feature map is combined with the deep fusion feature data to obtain target feature data.
[0038] Optionally, the step of obtaining a segmentation result according to the target feature data includes:
[0039] Obtain a confidence distribution map of the target segmentation model, integrate the target feature data in combination with a threshold, and obtain a segmentation result.
[0040] On the other hand, the present application provides an X-Ray image segmentation system based on feature enhancement, the system comprising:
[0041] A training module is used to construct a data set containing at least X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and divide the data set into a training set and a test set, train a composite material plate damage image segmentation model using the training set, and test the trained composite material plate damage image segmentation model using the test set to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features;
[0042] The segmentation module is used to obtain the X-Ray image to be segmented and input the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result.
[0043] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the feature enhancement-based X-Ray image segmentation method as described above is implemented.
[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the feature enhancement-based X-Ray image segmentation method as described above.
[0045] The technical solution of an embodiment of the present invention constructs a dataset containing at least X-ray images of slight wire interference, severe wire interference, and no wire interference, and divides the dataset into a training set and a test set. A composite material plate damage image segmentation model is trained using the training set, and the trained composite material plate damage image segmentation model is tested using the test set to obtain a target segmentation model that meets preset conditions. The composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow and deep features. The X-ray image to be segmented is obtained and input into the target segmentation model to obtain a segmentation result. The CFM module and the CIM module implement jump connections between the encoding network and the decoding network. Therefore, the deep learning model can effectively accommodate target parts of different shapes and sizes, thereby more accurately segmenting the target image. For example, applying the deep learning network model to the field of X-ray scanning can solve the problem that existing deep learning models cannot effectively and accurately segment target X-ray images. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0047] in:
[0048] Figure 1Schematic diagram of a flow chart of an X-ray image segmentation method based on feature enhancement in an embodiment of the present invention;
[0049] Figure 2 Schematic diagram of a flow chart of an X-ray image segmentation method based on feature enhancement in an embodiment of the present invention;
[0050] Figure 3 4 is a logical framework diagram of an X-Ray image segmentation method based on feature enhancement in an embodiment of the present invention;
[0051] Figure 4 This is a flowchart of a model training method for X-Ray image segmentation based on feature enhancement in an embodiment of the present invention;
[0052] Figure 5 Schematic diagram of the structure of an X-ray image segmentation system based on feature enhancement in an embodiment of the present invention;
[0053] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0054] Figure 7 It is a structural diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION
[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0056] like Figure 1-Figure 3 As shown, an X-ray image segmentation method based on feature enhancement according to an embodiment of the present invention specifically includes the following steps:
[0057] S110, constructing a data set that at least includes X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and dividing the data set into a training set and a test set, training a composite material plate damage image segmentation model using the training set, and testing the trained composite material plate damage image segmentation model using the test set, to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features;
[0058] S120 , obtaining an X-Ray image to be segmented, and inputting the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result.
[0059] By constructing a data set that at least includes X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and dividing the data set into a training set and a test set, the composite material plate damage image segmentation model is trained by the training set, and the trained composite material plate damage image segmentation model is tested by the test set, so as to obtain a target segmentation model that meets the preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusion of shallow features and deep features; obtaining the X-Ray image to be segmented, and inputting the X-Ray image to be segmented into the target segmentation model to obtain the segmentation result. The jump connection between the encoding network and the decoding network is realized by the CFM module and the CIM module. Therefore, the deep learning model can effectively be compatible with target parts of different shapes and sizes, so that the target image can be segmented more accurately. For example, applying the deep learning network model to the field of X-Ray scanning can solve the problem that the existing deep learning model cannot effectively and accurately segment the target X-Ray image.
[0060] In a possible embodiment, the step of training the composite material plate damage image segmentation model using the training set, and testing the trained composite material plate damage image segmentation model using the test set to obtain a target segmentation model that meets preset conditions includes:
[0061] Performing a first preprocessing on the images in the training set to obtain a target training set, wherein the first preprocessing includes at least one of a cropping process, a horizontal flipping process, and a normalization process;
[0062] Performing a second preprocessing on the images in the test set to obtain a target test set, wherein the second preprocessing includes at least one of scaling, cropping, and normalization;
[0063] Three composite material plate damage image segmentation models using different loss functions are trained using the training set to obtain three trained composite material plate damage image segmentation models;
[0064] The three trained composite material plate damage image segmentation models are tested using the test set, and the composite material plate damage image segmentation model when the damage function no longer decreases is used as the target segmentation model to obtain three target segmentation models.
[0065] For example, X-ray images are collected from the database and divided into a training set and a test set in an 8:2 ratio.
[0066] The images in the training set were randomly cropped to 224×224, and images were randomly sampled at different locations on the image to ensure that the discriminative features appeared. The images were horizontally flipped with a probability of 0.5 to reduce the model's dependence on the location of the features. The mean and variance of the images in the dataset were then calculated and used to normalize the image data. This operation expanded the size of the training set and improved the generalization ability of the trained model. The images in the test set were resized to 256×256 and then cropped to 224×224 using a center crop method to ensure that the recognition images contained sufficient discriminative features. The images were then normalized.
[0067] Exemplarily, data enhancement may be performed on the data set, and the data enhancement scheme is a geometric enhancement scheme.
[0068] In some embodiments, the geometric enhancement scheme includes random inversion in the X and Y directions, random rotation in the range of [-30, 30], and random shift in the range of [-5, 5] in the X and Y directions.
[0069] For example, Figure 4 As shown, three different loss functions are used for training, verification and testing to obtain three trained composite material plate damage image segmentation models; the three different loss functions include the mean square error function MSE, the binary cross entropy loss function BCEWithLogitsLoss, and the intelligent loss function LossNet.
[0070] The mean square error function is a commonly used regression loss function. The calculation method is to find the sum of the squares of the distances between the predicted value and the true value. The calculation formula is:
[0071]
[0072] Among them, y i is the true value, is the predicted value, and n is the number of samples.
[0073] The binary cross entropy loss function is often used for binary classification problems, and the calculation formula is:
[0074] Loss={l1,...,l N},l n =-[y n ·log(σ(x n ))+(1-y n )·log(1-σ(x n ))]
[0075] Among them, assuming there are N batches, each batch predicts n labels, σ(x n ) is the sigmoid function, which can map x to the interval (0, 1).
[0076] The intelligent loss function includes weighted IoU Loss and BCE Loss, as well as a network called LossNet to further optimize the segmentation details and structure.
[0077]
[0078] Among them, IoU Loss and BCE Loss are widely used in segmentation tasks and have been proven to be effective. LossNet uses an ImgeNet pre-trained classification network (such as ResNet) to extract multi-scale features of predictions and actual values, and calculates the feature difference between them as the loss function
[0079]
[0080] Loss L m Through pixel-level Euclidean distance supervision, the model can generate comprehensive supervision at the feature level, where low-level feature maps contain rich boundary information, while high-level feature maps depict location information.
[0081] In a possible implementation, the step of inputting the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result includes:
[0082] Inputting the X-Ray image to be segmented into the feature extraction module of the target segmentation model to obtain a shallow feature map and deep features of different levels of the X-Ray image to be segmented;
[0083] Inputting the shallow feature map into the CIM module of the target segmentation model to obtain shallow feature data combining channel attention and spatial attention;
[0084] Inputting the deep features into the CFM module of the target segmentation model to fuse the deep features of different levels to obtain deep fused feature data;
[0085] The shallow feature data and the deep fusion feature data are input into the SAM Ham module of the target segmentation model to fuse the shallow feature data and the deep fusion feature data to obtain target feature data, and a segmentation result is obtained according to the target feature data.
[0086] Exemplarily, the CIM module includes a channel attention unit and a spatial attention unit, and the step of inputting the shallow feature map into the CIM module of the target segmentation model to obtain shallow feature data combining channel attention and spatial attention includes:
[0087] Inputting the shallow feature map into the channel attention unit of the CIM module in the target segmentation model, calculating the global average pooling data of the shallow feature map through the attention unit, and converting each shallow feature in the shallow feature map into a vector representation according to the global average pooling data, inputting the vector representation into an activation function for activation to obtain a first channel weight, and performing weighted processing on each channel of the shallow feature map based on the first channel weight to obtain a target shallow feature map;
[0088] Inputting the target shallow feature map into the spatial attention unit of the CIM module in the target segmentation model, and downsampling the target shallow feature map in the channel dimension and the spatial dimension by the spatial attention unit to obtain downsampled data;
[0089] Performing global average pooling on the downsampled data in the channel dimension to obtain a channel weight vector;
[0090] Performing a convolution operation on the downsampled data in the spatial dimension to obtain a weight value for each position in the target shallow feature map, thereby generating a spatial weight map;
[0091] Multiplying the channel weight vector and the spatial weight map to obtain a spatial attention weight map;
[0092] The spatial attention weight map is used to perform a weighted operation on the target shallow feature map to obtain shallow feature data that combines channel attention and spatial attention.
[0093] For example, the backbone network is responsible for converting the input image into a high-level feature vector and consists of multiple convolutional layers. Each convolutional layer includes a convolution operation, batch normalization, and a rectified linear unit (ReLU) activation function. The number of convolution kernels in the first few layers gradually increases, shrinking and abstracting the input image to obtain higher-level features. The specific formula is as follows:
[0094] H i =ReLU(BatchNorm(W i *H i-1 +b i ))
[0095] Among them, H i-1 Represents the output feature map of the previous layer, Wi Represents the weight parameter of the current convolutional layer, b i Represents the bias parameter, BatchNorm represents the batch normalization operation, ReLU represents the rectified linear unit activation function, and * represents the convolution operation.
[0096] In some embodiments, the backbone network includes a CIM module and a CFM module, wherein the CIM module is composed of a channel attention operation Att c (·) and the spatial attention operation Att s (·) composition, this operation can be expressed as:
[0097] T2=Att s (Att c (X1))
[0098] Channel attention operation Att c (·) can be written as:
[0099] Att c (x)=σ(H1(P max (x))+H2(P avg (x)))(·)x
[0100] Among them, x is the input tensor, σ(·) is the Softmax function, P max (x) and P avg (x) represents the adaptive maximum pooling function and the adaptive average pooling function, H i (·),i∈{1,2} shared parameters, consists of a convolutional layer with a kernel size of 1×1 to reduce the channel dimension by a factor of 16, followed by a ReLU layer and another 1×1 convolutional layer to restore the original channel dimension.
[0101] Spatial attention operation Att s (·) can be written as:
[0102] Att s (x)=σ(G(Concat(R max (x),R avg (x))))(·)x
[0103] Among them, R max (x) and R avg (x) denotes the maximum and average values obtained along the channel dimension, respectively. G(·) denotes a 7×7 convolutional layer with padding set to 3.
[0104] In a possible implementation, the step of inputting the deep features into the CFM module of the target segmentation model to fuse the deep features of different levels to obtain deep fused feature data includes:
[0105] Inputting the deep features into the CFM module of the target segmentation model, upsampling the highest-level feature map in the deep features through the CFM module to obtain first upsampled data corresponding to the second highest-level feature map in the deep features;
[0106] Transferring the first up-sampled data through a convolution unit to obtain a transfer result;
[0107] Performing a Hadamard product calculation on the transferred result and the second highest level feature map in the deep feature layer to obtain a first target feature map;
[0108] Splicing the first target feature map and the transferred result to obtain a first splicing result, and smoothing the splicing result using a convolution unit to obtain a fused feature map;
[0109] Upsampling the fused feature map, the highest-level feature map in the deep features, and the second-highest-level feature map in the deep features to obtain second upsampled data corresponding to the third-highest-level feature map in the deep features;
[0110] Smoothing the first sampled data corresponding to the highest-level feature map in the deep features in the second upsampled data, the second sampled data corresponding to the second highest-level feature map in the deep features in the second upsampled data, and the third highest-level feature map in the deep features to obtain smoothed data;
[0111] Perform Hadamard product calculation on the smoothed data to obtain the second target feature map;
[0112] The second target feature map is spliced with the third sampling data corresponding to the fused feature map in the smoothed second up-sampled data to obtain deep fused feature data.
[0113] For example, the highest level feature map X'4 is upsampled to the same size as X'3, and the result is then passed through two convolutional units F1(·) and F2(·) to obtain and Finally, the convolution unit F3(·) is used to smooth the connected features to obtain the fused feature map This process can be expressed as:
[0114] X 34 =F3(Concat(F1(X'4)(·)X'3,F2(X'4)))
[0115] Among them, (·) represents the Hadamard product, and Concat(·) is the concatenation operation along the channel dimension.
[0116] For example, the feature maps X'4, X'3, X 34 Upsample to the same size as X'2 and smooth it using convolution units F4(·), F5(·) and F6(·). Then, multiply the smoothed X'4, X'3 with X'2 and compare the result with the upsampled and smoothed X'2. 34 Finally, the connected feature map is sent to two convolution units (i.e., F7(·) and F8(·)) for dimensionality reduction, and the obtained That is the output of the CFM module. This process can be expressed as:
[0117] T1=F8(F7(Concat(F4(X'4)(·)F5(X'3)(·)X'2,F6(X 34 )))).
[0118] In a possible implementation, the step of inputting the shallow feature data and the deep fusion feature data into the SAM Ham module of the target segmentation model to fuse the shallow feature data and the deep fusion feature data to obtain target feature data includes:
[0119] Inputting the deep fusion feature data into the SAM Ham module of the target segmentation model, and performing dimensionality reduction on the deep fusion feature data through the SAMHam module to obtain a first dimensionality reduction feature map and a target dimensionality reduction feature map;
[0120] The shallow feature data is input into the SAM Ham module of the target segmentation model, the shallow feature data is reduced in dimension by the SAM Ham module to obtain an initial reduced-dimensionality feature map, and the second reduced-dimensionality feature map is interpolated to the same size as the deep fusion feature data to generate a second reduced-dimensionality feature map; a Softmax function is applied to the channel dimension of the second reduced-dimensionality feature map, and any channel of the second reduced-dimensionality feature map is selected as an attention map to obtain a target feature map; a Hadamard product is calculated between the first reduced-dimensionality feature map and the target feature map to assign different weights to different pixels to obtain a feature map to be cropped; an adaptive pooling operation is performed to reduce feature shift and center crop the feature map to be cropped to obtain a feature map;
[0121] Then, the correlation between the feature map and each pixel in the first dimensionality reduction feature map is established through the inner product to obtain a related attention map; the related attention map is multiplied by the target dimensionality reduction feature map to obtain a product feature map; the product feature map is fed to the graph convolution layer to obtain a graph convolution feature map, and the inner product of the related attention map and the graph convolution feature map is calculated, and the graph domain features are reconstructed into the original structural features according to the inner product to obtain a reconstructed feature map; the reconstructed feature map is adjusted to the same channel size as the feature map through a convolution layer with a kernel size of 1×1 to obtain a target feature map; the target feature map is combined with the deep fusion feature data to obtain target feature data.
[0122] Exemplarily, the feature fusion network SAM_Ham fuses the shallow feature data and the deep fusion feature data, and first performs a hamburger operation on the deep features, wherein the hamburger structure is a special convolutional neural network module that can be used to improve the network's receptive field and expression capabilities. The hamburger structure usually consists of two branches: the input features are processed by two branches. The left branch adopts an ordinary convolutional neural network structure, while the right branch uses group convolution to extract features. Specifically, the right branch divides the input features into several groups, then performs a convolution operation on each group, and finally splices the output results. This group convolution method can reduce the number of parameters and the amount of calculation, while also helping to improve the network's receptive field and expression capabilities. The entire hamburger structure can be expressed as the following formula:
[0123] Y=Concatenate([Conv(X),Conv g (X)])
[0124] Among them, X represents the input feature, Conv represents the normal convolution operation, Conv g Represents the grouped convolution operation, Concatenate represents the concatenation operation, and Y represents the output feature.
[0125] In some embodiments, the simplest form of the hamburger model resembles a hamburger, that is, it consists of two linear transformations with a matrix decomposition model in the middle, which are sequentially denoted as Lower Bread, Ham, and Upper Bread, where Ham corresponds to the numerical method M for solving the matrix decomposition model. In order to better integrate deep features and thousand-layer features, and reasonably aggregate the characteristics of deep features and shallow features for global context modeling, the Hamburger model is introduced, which uses matrix decomposition to model global spatial information, using a combination of SAM+Hamburger.
[0126] In some instances, this paper only performs the Hamburger operation on deep features. This is because the encoder of the improved UNet in this paper adopts a convolution and downsampling architecture, which makes the shallow features contain too much underlying information, which in turn affects the semantic segmentation performance. Moreover, processing the shallow features may bring too much computational load.
[0127] In some embodiments, the backbone network in the SAM_Ham module is a similarity aggregation module that fuses the feature map T1 containing high-level semantic information obtained from the CFM module and the feature map T2 with rich appearance details (such as texture, shape, color, etc.) obtained from the CIM module through a self-attention mechanism. First, two linear mapping functions W are applied to T1. θ (·)and Perform dimensionality reduction to obtain feature maps and Among them, the linear mapping process uses a convolution operation with a kernel size of 1×1. This process can be expressed as:
[0128]
[0129] For T2, use the convolution unit W g (·) Reduce the channel dimension to 32 and interpolate it to the same size as T1. After that, apply the Softmax function on the channel dimension and select the second channel as the attention map, and get Next, the Hadamard product between K and T'2 is calculated to assign different weights to different pixels and increase the weight of edge pixels. After that, an adaptive pooling operation is used to reduce the feature shift and perform center cropping to obtain the feature map V∈R 4 ×4×16 . This process can be expressed as:
[0130] V=AP(K(·)F(W g (T)))
[0131] where AP(·) represents the pooling and cropping operations.
[0132] The correlation between each pixel in V and K is then established through the inner product, which can be expressed as:
[0133]
[0134] in represents the inner product operation, V T is the transpose of V, and f is the related attention map.
[0135] After obtaining the relevant attention map f, it is multiplied with the feature map Q, and the obtained feature map is fed into the graph convolution layer GCN(·) to obtain G∈R 4×4×16 , and then calculate the inner product of f and G to reconstruct the graph domain features into the original structural features. This process can be expressed as:
[0136]
[0137] The reconstructed feature map Y' is passed through a convolutional layer W with a kernel size of 1×1. z (·) is adjusted to the same channel size as Y, and then combined with feature T1 to obtain the final output This process can be expressed as:
[0138] Z=T1+W z (Y').
[0139] In a possible implementation, the step of obtaining a segmentation result according to the target feature data includes:
[0140] Obtain a confidence distribution map of the target segmentation model, integrate the target feature data in combination with a threshold, and obtain a segmentation result.
[0141] Exemplarily, a distribution map is obtained based on the prediction results of the three neural network models, and then its confidence distribution map is calculated based on the obtained distribution map and the prediction model. A threshold is set to retain reliable segmentation points and remove unreliable points. The segmentation results of the three neural network models are calculated in sequence and merged to obtain the final distribution map.
[0142] Among them, three different loss functions are selected and three neural network model diagrams are obtained; for each neural network model, the image of the test set is input into the model to obtain the confidence distribution diagram of each pixel; finally, for each pixel, the confidence values of the three models are compared, and the final outlining result is determined according to the set threshold.
[0143] The method for generating the confidence distribution map depends on the specific task and model. Common methods include: Softmax function output: If the model is a classification model, the Softmax function can be used to output the probability of each pixel belonging to each class. For each pixel, the probability value can be plotted as a confidence distribution map. Sigmoid function output: If the model is a binary classification model, the Sigmoid function can be used to output the probability of each pixel belonging to the positive class. For each pixel, the probability value can be used as the value of the confidence distribution map. Specific layer output: For specific tasks, such as semantic segmentation or object detection, features can be extracted from specific layers in the network to form the confidence distribution map. The output of these specific layers typically retains more spatial information and helps generate a more accurate distribution map. 4. Monte Carlo sampling: For tasks that require modeling uncertainty, such as reinforcement learning or uncertain image segmentation, Monte Carlo sampling can be used to estimate the confidence distribution. By running the model multiple times and counting the output of each pixel under different sampling conditions, a confidence distribution map can be generated.
[0144] Among them, there are some commonly used methods to select a suitable threshold and combine the confidence distribution maps of multiple models to reasonably set the threshold: fixed threshold method: simply select a fixed threshold to determine the outlining result; adaptive threshold method: adaptively select the threshold according to the confidence distribution of each pixel. The threshold can be dynamically determined by statistically analyzing the mean, variance and other information of the confidence distribution, or using some adaptive algorithms (such as the OTSU algorithm). Multi-model weighted method: For each pixel, the confidence distribution of multiple models is combined for weighted averaging. Logical rule method: Use some logical rules to integrate the outputs of multiple models. For example, a voting method can be used to vote on the judgment results of multiple models, and determine the final outlining result based on the voting results.
[0145] In some of the embodiments, when the damage has similar grayscale and texture features to the surrounding background, the deep learning-based loss automatic segmentation tool described in the present application utilizes a three-deep learning model integration method to fully exploit the similar grayscale and texture features of the damage and the surrounding background, thereby improving the segmentation accuracy.
[0146] The training, verification and testing unit uses three different loss functions for training, verification and testing to obtain three different neural network models; the integrated delineation unit is used to integrate the delineation results based on the confidence distribution maps of the three neural network models and combine them with the threshold to obtain the final segmentation map of the composite material plate damage image.
[0147] In one possible implementation, Figure 5As shown, the present application provides an X-Ray image segmentation system based on feature enhancement, the system comprising:
[0148] A training module 201 is configured to construct a data set comprising at least X-ray images of slight wire interference, X-ray images of severe wire interference, and X-ray images of no wire interference, and to divide the data set into a training set and a test set, to train a composite material plate damage image segmentation model using the training set, and to test the trained composite material plate damage image segmentation model using the test set, to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features;
[0149] The segmentation module 202 is used to obtain an X-Ray image to be segmented, and input the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result.
[0150] In one possible implementation, Figure 6 As shown, an embodiment of the present application provides a terminal device 300, including: a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements: constructing a data set comprising at least X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and dividing the data set into a training set and a test set, training a composite material plate damage image segmentation model through the training set, and testing the trained composite material plate damage image segmentation model through the test set to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features; obtaining an X-Ray image to be segmented, and inputting the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result.
[0151] In one possible implementation, Figure 7As shown, an embodiment of the present application provides a computer-readable storage medium 400, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, it implements: constructing a data set that at least includes X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and dividing the data set into a training set and a test set, training a composite material plate damage image segmentation model through the training set, and testing the trained composite material plate damage image segmentation model through the test set to obtain a target segmentation model that meets preset conditions, the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features; obtaining an X-Ray image to be segmented, and inputting the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result.
[0152] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.
[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0154] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.
[0155] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.
[0156] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0157] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0158] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0159] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.
[0160] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. A feature-enhanced X-ray image segmentation method, characterized in that: include: Constructing a data set that includes at least X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and dividing the data set into a training set and a test set, training a composite material plate damage image segmentation model using the training set, and testing the trained composite material plate damage image segmentation model using the test set, to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features; An X-ray image to be segmented is obtained, and the X-ray image to be segmented is input into the target segmentation model to obtain a segmentation result.
2. The X-ray image segmentation method based on feature enhancement according to claim 1, wherein: The step of training the composite material plate damage image segmentation model using the training set and testing the trained composite material plate damage image segmentation model using the test set to obtain a target segmentation model that meets preset conditions includes: Performing a first preprocessing on the images in the training set to obtain a target training set, wherein the first preprocessing includes at least one of a cropping process, a horizontal flipping process, and a normalization process; Performing a second preprocessing on the images in the test set to obtain a target test set, wherein the second preprocessing includes at least one of scaling, cropping, and normalization; Three composite material plate damage image segmentation models using different loss functions are trained using the training set to obtain three trained composite material plate damage image segmentation models; The three trained composite material plate damage image segmentation models are tested using the test set, and the composite material plate damage image segmentation model when the damage function no longer decreases is used as the target segmentation model to obtain three target segmentation models.
3. The X-ray image segmentation method based on feature enhancement according to claim 1, wherein: The step of inputting the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result includes: Inputting the X-Ray image to be segmented into the feature extraction module of the target segmentation model to obtain a shallow feature map and deep features of different levels of the X-Ray image to be segmented; Inputting the shallow feature map into the CIM module of the target segmentation model to obtain shallow feature data combining channel attention and spatial attention; Inputting the deep features into the CFM module of the target segmentation model to fuse the deep features of different levels to obtain deep fused feature data; The shallow feature data and the deep fusion feature data are input into the SAM Ham module of the target segmentation model to fuse the shallow feature data and the deep fusion feature data to obtain target feature data, and a segmentation result is obtained according to the target feature data.
4. The X-ray image segmentation method based on feature enhancement according to claim 3, wherein: The CIM module includes a channel attention unit and a spatial attention unit. The step of inputting the shallow feature map into the CIM module of the target segmentation model to obtain shallow feature data combining channel attention and spatial attention includes: Inputting the shallow feature map into the channel attention unit of the CIM module in the target segmentation model, calculating the global average pooling data of the shallow feature map through the attention unit, and converting each shallow feature in the shallow feature map into a vector representation according to the global average pooling data, inputting the vector representation into an activation function for activation to obtain a first channel weight, and performing weighted processing on each channel of the shallow feature map based on the first channel weight to obtain a target shallow feature map; Inputting the target shallow feature map into the spatial attention unit of the CIM module in the target segmentation model, and downsampling the target shallow feature map in the channel dimension and the spatial dimension by the spatial attention unit to obtain downsampled data; Performing global average pooling on the downsampled data in the channel dimension to obtain a channel weight vector; Performing a convolution operation on the downsampled data in the spatial dimension to obtain a weight value for each position in the target shallow feature map, thereby generating a spatial weight map; Multiplying the channel weight vector and the spatial weight map to obtain a spatial attention weight map; The spatial attention weight map is used to perform a weighted operation on the target shallow feature map to obtain shallow feature data that combines channel attention and spatial attention.
5. The X-ray image segmentation method based on feature enhancement according to claim 3, wherein: The step of inputting the deep features into the CFM module of the target segmentation model to fuse the deep features of different levels to obtain deep fusion feature data includes: Inputting the deep features into the CFM module of the target segmentation model, upsampling the highest-level feature map in the deep features through the CFM module to obtain first upsampled data corresponding to the second highest-level feature map in the deep features; Transferring the first up-sampled data through a convolution unit to obtain a transfer result; Performing a Hadamard product calculation on the transferred result and the second highest level feature map in the deep feature layer to obtain a first target feature map; Splicing the first target feature map and the transferred result to obtain a first splicing result, and smoothing the splicing result using a convolution unit to obtain a fused feature map; Upsampling the fused feature map, the highest-level feature map in the deep features, and the second-highest-level feature map in the deep features to obtain second upsampled data corresponding to the third-highest-level feature map in the deep features; Smoothing the first sampled data corresponding to the highest-level feature map in the deep features in the second upsampled data, the second sampled data corresponding to the second highest-level feature map in the deep features in the second upsampled data, and the third highest-level feature map in the deep features to obtain smoothed data; Perform Hadamard product calculation on the smoothed data to obtain the second target feature map; The second target feature map is concatenated with third sampling data corresponding to the fused feature map in the smoothed second up-sampled data to obtain deep fused feature data.
6. The X-ray image segmentation method based on feature enhancement according to claim 3, wherein: The step of inputting the shallow feature data and the deep fusion feature data into the SAM Ham module of the target segmentation model to fuse the shallow feature data and the deep fusion feature data to obtain target feature data includes: Inputting the deep fusion feature data into the SAM Ham module of the target segmentation model, and performing dimensionality reduction on the deep fusion feature data through the SAM Ham module to obtain a first dimensionality reduction feature map and a target dimensionality reduction feature map; The shallow feature data is input into the SAM Ham module of the target segmentation model, the shallow feature data is reduced in dimension by the SAM Ham module to obtain an initial reduced-dimensionality feature map, and the second reduced-dimensionality feature map is interpolated to the same size as the deep fusion feature data to generate a second reduced-dimensionality feature map; a Softmax function is applied to the channel dimension of the second reduced-dimensionality feature map, and any channel of the second reduced-dimensionality feature map is selected as an attention map to obtain a target feature map; a Hadamard product is calculated between the first reduced-dimensionality feature map and the target feature map to assign different weights to different pixels to obtain a feature map to be cropped; an adaptive pooling operation is performed to reduce feature shift and center crop the feature map to be cropped to obtain a feature map; Then, the correlation between the feature map and each pixel in the first dimensionality reduction feature map is established through the inner product to obtain a related attention map; the related attention map is multiplied by the target dimensionality reduction feature map to obtain a product feature map; the product feature map is fed to the graph convolution layer to obtain a graph convolution feature map, and the inner product of the related attention map and the graph convolution feature map is calculated, and the graph domain features are reconstructed into the original structural features according to the inner product to obtain a reconstructed feature map; the reconstructed feature map is adjusted to the same channel size as the feature map through a convolution layer with a kernel size of 1×1 to obtain a target feature map; the target feature map is combined with the deep fusion feature data to obtain target feature data.
7. The X-ray image segmentation method based on feature enhancement according to claim 3, wherein: The step of obtaining a segmentation result according to the target feature data comprises: Obtain a confidence distribution map of the target segmentation model, integrate the target feature data in combination with a threshold, and obtain a segmentation result.
8. An X-ray image segmentation system based on feature enhancement, characterized in that: The system comprises: A training module is used to construct a data set containing at least X-Ray images of slight wire interference, X-Ray images of severe wire interference, and X-Ray images of no wire interference, and divide the data set into a training set and a test set, train a composite material plate damage image segmentation model using the training set, and test the trained composite material plate damage image segmentation model using the test set to obtain a target segmentation model that meets preset conditions, wherein the composite material plate damage image segmentation model includes a CFM module for multi-level feature fusion, a CIM module for feature enhancement, and a SAM_Ham module for fusing shallow features with deep features; The segmentation module is used to obtain the X-Ray image to be segmented and input the X-Ray image to be segmented into the target segmentation model to obtain a segmentation result.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the X-Ray image segmentation method based on feature enhancement according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the X-ray image segmentation method based on feature enhancement according to any one of claims 1 to 7 is implemented.