A method and system for breast ultrasound image segmentation using cross-graph fusion networks

Abstract

This invention discloses a method and system for breast ultrasound image segmentation using a cross-image fusion network, relating to the field of image segmentation technology. The method includes: performing weak and strong transforms on the ultrasound image to obtain weakly transformed and strongly transformed images; performing discrete cosine transform on the features to obtain transformed image features; performing high-pass and low-pass filtering; processing the features after low-pass filtering to obtain weakly transformed and strongly transformed images; fusing the weakly transformed and strongly transformed images to obtain a weak-to-strong transform fusion image and a strong-to-weak transform fusion image; randomly mixing the high-pass filtered features and performing inverse discrete cosine transform with the features of the weak-to-strong transform fusion image and the strong-to-weak transform fusion image to obtain strong-weak inverse discrete cosine transform features; and inputting these features into a pre-established decoder to output the segmentation results of the weakly transformed image and the strongly transformed image, thereby obtaining a better segmentation result.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, specifically to a method and system for breast ultrasound image segmentation using cross-image fusion networks. Background Technology

[0002] Ultrasound image segmentation helps doctors accurately locate lesion areas, determine the benign or malignant nature of tumors, and facilitates diagnosis and treatment planning. However, due to the limitations of medical equipment and the characteristics of biological tissues, ultrasound images are often affected by various types of noise, such as artifacts and speckle noise. Furthermore, ultrasound images typically have low contrast, with subtle gray-level differences between different tissues, making it difficult to distinguish the edges and shapes of lesions. Currently, although convolutional neural networks play an important role in medical image segmentation, ultrasound image segmentation still faces many challenges. First, the tissue structures in medical images are highly complex, including organs, blood vessels, and tumors, which vary greatly in location and size, making it challenging to construct prior shape models. Second, the presence of various types of noise and artifacts in the images, such as speckle noise and gamma noise, leads to blurred lesion boundaries. In addition, the number of pixels between different tissues and lesions in medical images is often unbalanced, negatively impacting image quality and segmentation results. Summary of the Invention

[0003] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for breast ultrasound image segmentation using cross-graph fusion networks.

[0004] Firstly, the objective of this invention can be achieved through the following technical solution: a breast ultrasound image segmentation method using cross-image fusion networks, the method comprising the following steps:

[0005] The system receives ultrasound images, performs weak and strong transforms on them to obtain weakly transformed and strongly transformed images, and then performs discrete cosine transform on the features of the weakly and strongly transformed images to obtain transformed image features. High-pass and low-pass filters are then applied to the transformed image features, and further processing is performed based on the low-pass filtered features to obtain weakly transformed and strongly transformed images.

[0006] The images processed by weak transformation and those processed by strong transformation are fused to obtain weak-to-strong transformation fusion image and strong-to-weak transformation fusion image. The features after high-pass filtering are randomly mixed and then subjected to inverse discrete cosine transform with the features of the weak-to-strong transformation fusion image and the strong-to-weak transformation fusion image to obtain strong-weak inverse discrete cosine transform features.

[0007] The strong and weak discrete cosine inverse transform features are input into a pre-established decoder, and the outputs are the segmentation results of the weak transform image and the strong transform image.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the ultrasound image is obtained based on an ultrasound image training set, the ultrasound image training set containing ultrasound images. and its corresponding segmentation labels , , This represents the number of ultrasound images in the training set; for each ultrasound image... and its corresponding segmentation labels Weak transformation is performed to obtain the weakly transformed image. and their corresponding tags Meanwhile, for each ultrasound image and its corresponding segmentation labels Perform a strong transform to obtain the transformed strong transform image. and their corresponding tags The weak transformation includes image rotation, scaling, translation, and reversal; the strong transformation is an image masking operation.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the image masking operation includes: constructing an image of the same size as the ultrasound image. ,image All pixel values ​​in the image are initialized to 0. Multiple rectangular and triangular random regions are randomly added to the image, and preset pixel values ​​are set within each random region; then, another ultrasound image is selected. and its corresponding segmentation labels , ; with ultrasound images and its corresponding segmentation labels Merge:

[0010] .

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: a feature extractor based on a U-Net encoder for the features of the weakly transformed image and the strongly transformed image. The extracted data includes: images with weak transformations. With strong transform image Feed into feature extractor Obtain the corresponding features and ,in These represent the number of channels, height, and width of the feature, respectively. Represents a real matrix;

[0012] right , Each channel undergoes a discrete cosine transform independently, and the corresponding transform characteristics are as follows: , The results were obtained by using a high-pass filter and a low-pass filter, respectively. , Their respective high-frequency components and low-frequency components, , They are High-frequency components and low-frequency components; , They are High-frequency components and low-frequency components;

[0013] For low-frequency components , By using convolution operations to reduce the number of feature channels and flatten the pixels, the shape of the features becomes... ,in This represents the reduced number of feature channels. This indicates the number of pixels after flattening; the shape of the feature becomes... , Indicates the number of neurons in the output layer of a multilayer perceptron; low-frequency component. , The features after each operation are represented as follows , .

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: processing the features based on the low-pass filtering to obtain a weakly transformed image and a strongly transformed image, wherein the weakly transformed image includes:

[0015] Will Projected onto the vertex set Edge set is Weak Transformation Diagram In the graph, the parameters are respectively from , express; Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows:

[0016]

[0017] Where / indicates element division, This indicates a search for the 2-norm. The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation:

[0018]

[0019] Used to represent No. Column data, express No. Column data;

[0020] Through calculation Weak transform graphs are calculated using the similarity between mid-vertex features. adjacency matrix

[0021]

[0022] in, This represents the matrix transpose; then, convolution is used as shown in the diagram below to calculate the updated features:

[0023]

[0024] in, This represents the features after graph convolution. This represents a non-linear activation function.

[0025] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: a processing procedure for the strong transform graph, comprising:

[0026] Will Projected onto the vertex set Edge set is Strong Transformation Diagram In the graph, the parameters are respectively from , express. Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows:

[0027]

[0028] in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation:

[0029]

[0030] Used to represent No. Column data, express No. Column data;

[0031] Through calculation Calculate the similarity between mid-vertex features in the graph Adjacency matrix:

[0032]

[0033] The updated features are calculated using convolution as shown in the image below:

[0034]

[0035] in, This represents the features of the graph after convolution;

[0036] right , After layer normalization and linearization, the shape of the feature becomes The following characteristics were obtained:

[0037]

[0038] in Representation layer normalization, and Represents the learnable parameters of a linear transformation operation.

[0039] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the processing of the weak-to-strong transformation fusion graph includes:

[0040] Calculate the weak-to-strong transformation fusion graph adjacency matrix ,in This represents the softmax function, used to calculate the strong-to-weak transformation fusion graph. adjacency matrix .calculate Graph convolution features: , Represent the learnable parameters of graph convolution, and calculate Graph convolution features: , The learnable parameters of graph convolution are represented.

[0041] Weak-to-strong transformation fusion graph In the middle, the vertex set is Edge set is The parameters of the graph are respectively determined by , express, Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows:

[0042]

[0043] in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation:

[0044]

[0045] Used to represent No. Column data, express No. Column data.

[0046] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: a processing procedure for the strong-to-weak transform fusion graph, comprising:

[0047] Strong-to-weak transformation fusion graph In the middle, the vertex set is Edge set is The parameters of the graph are respectively determined by... , express. Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex can be calculated as follows:

[0048]

[0049] in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation can be calculated using the following equation:

[0050]

[0051] Used to represent No. Column data, express No. Column data;

[0052] To each , After performing layer normalization and linear transformation, the shape of the features becomes... Then change the shape Then, a convolution operation is used to transform it into a feature matrix. ,Right now .

[0053] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: randomly mixing the high-pass filtered features and performing an inverse discrete cosine transform.

[0054]

[0055] in, Representing weakly transformed images Discrete cosine inverse transform features of graph convolution features Represents a strongly transformed image Discrete cosine inverse transform features of graph convolution features This indicates that the discrete inverse cosine transform is performed independently on each channel of the feature. , Represents a random number between 0 and 1;

[0056] The strong and weak discrete cosine inverse transform features are input into a pre-established decoder, which outputs segmentation results for the weak transform image and the strong transform image, including:

[0057] Features , They are sent to the decoder respectively. Segmentation results of the weakly transformed image. Segmentation results with strongly transformed images Calculate the segmentation loss to optimize the entire network:

[0058]

[0059] in, This represents the number of pixels in the image. Indicates the number of categories in the segmentation. express The The class of 1 pixel, express The The class of 1 pixel, express The The class of 1 pixel, express The The class of 1 pixel.

[0060] Secondly, in order to achieve the above objectives, the present invention discloses a breast ultrasound image segmentation system based on a cross-graph fusion network, comprising:

[0061] The image processing module is used to receive ultrasound images, perform weak and strong transforms on the ultrasound images to obtain weakly transformed images and strongly transformed images, and perform discrete cosine transform on the features of the weakly transformed images and strongly transformed images to obtain transformed image features; then, high-pass filtering and low-pass filtering are performed on the transformed image features, and the features after low-pass filtering are processed to obtain weakly transformed processed images and strongly transformed processed images.

[0062] The image fusion module is used to fuse images processed by weak transformation and images processed by strong transformation to obtain weak-to-strong transformation fusion image and strong-to-weak transformation fusion image. The features after high-pass filtering are randomly mixed and then subjected to inverse discrete cosine transform with the features of the weak-to-strong transformation fusion image and the strong-to-weak transformation fusion image to obtain strong and weak inverse discrete cosine transform features.

[0063] The image segmentation module is used to input strong and weak discrete cosine inverse transform features into a pre-established decoder and output the segmentation results of the weak transform image and the strong transform image.

[0064] The beneficial effects of this invention are:

[0065] This invention improves the diversity of samples and segmentation labels by performing weak and strong transformations on breast ultrasound images; it enhances the robustness of the method to various types of noise in ultrasound images by performing high-pass and low-pass filtering on the transformed image features; and it improves the ability of the method to segment complex tissue structures in ultrasound images by constructing weak transformation maps, strong transformation maps, weak-to-strong transformation fusion maps, and strong-to-weak transformation fusion maps. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0068] Figure 2 This is a schematic diagram of the ultrasound image segmentation network of the cross-graph fusion network of the present invention;

[0069] Figure 3 This is a schematic diagram comparing the ultrasound image segmentation method of the cross-image fusion network of the present invention with existing segmentation methods;

[0070] Figure 4 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Example 1:

[0073] like Figure 1 As shown, a breast ultrasound image segmentation method using cross-graph fusion networks includes the following steps:

[0074] S101: Receive ultrasound images, perform weak and strong transformations on the ultrasound images to obtain weakly transformed and strongly transformed images, and perform discrete cosine transform on the features of the weakly transformed and strongly transformed images to obtain transformed image features; perform high-pass filtering and low-pass filtering on the transformed image features, and process the features based on the low-pass filtered features to obtain weakly transformed and strongly transformed images.

[0075] The ultrasound images are obtained based on an ultrasound image training set, which contains ultrasound images. and its corresponding segmentation labels, , , This represents the number of ultrasound images in the training set; for each ultrasound image... and its corresponding segmentation labels Weak transformation is performed to obtain the weakly transformed image. and their corresponding tags Meanwhile, for each ultrasound image and its corresponding segmentation labels Perform a strong transform to obtain the transformed strong transform image. and their corresponding tags The weak transformation includes image rotation, scaling, translation, and reversal; the strong transformation is an image masking operation.

[0076] The image masking operation includes: constructing an image of the same size as the ultrasound image. , All pixel values ​​are initialized to 0. Multiple rectangular and triangular random regions are randomly added to the image, and preset pixel values ​​are set within each random region; then, another ultrasound image is selected. and its corresponding segmentation labels , ; with ultrasound images and its corresponding segmentation labels Merge:

[0077] .

[0078] The features of the weakly transformed and strongly transformed images are extracted using the feature extractor of the U-Net encoder. The extracted data includes: images with weak transformations. With strong transform image Feed into feature extractor Obtain the corresponding features and ,in These represent the number of channels, height, and width of the feature, respectively. Represents a real matrix;

[0079] right , Each channel undergoes a discrete cosine transform independently, and the corresponding transform characteristics are as follows: , The results were obtained by using a high-pass filter and a low-pass filter, respectively. , Their respective high-frequency components and low-frequency components, , They are High-frequency components and low-frequency components; , They are High-frequency components and low-frequency components;

[0080] For low-frequency components , By using convolution operations to reduce the number of feature channels and flatten the pixels, the shape of the features becomes... ,in This represents the reduced number of feature channels. This indicates the number of pixels after flattening; the shape of the feature becomes... , Indicates the number of neurons in the output layer of a multilayer perceptron; low-frequency component. , The features after each operation are represented as follows , .

[0081] The process based on the low-pass filtered features yields a weakly transformed image and a strongly transformed image. The weakly transformed image includes:

[0082] Will Projected onto the vertex set Edge set is Weak Transformation Diagram In the graph, the parameters are respectively from , express; Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows:

[0083]

[0084] Where / indicates element division, This indicates a search for the 2-norm. The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation:

[0085]

[0086] Used to represent No. Column data, express No. Column data;

[0087] Through calculation Weak transform graphs are calculated using the similarity between mid-vertex features. adjacency matrix

[0088]

[0089] in, This represents the matrix transpose; then, convolution is used as shown in the diagram below to calculate the updated features:

[0090]

[0091] in, This represents the features after graph convolution. This represents a non-linear activation function.

[0092] The processing of the strong transform graph includes:

[0093] Will Projected onto the vertex set Edge set is Strong Transformation Diagram In the graph, the parameters are respectively from , express. Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows:

[0094]

[0095] in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation:

[0096]

[0097] Used to represent No. Column data, express No. Column data;

[0098] Through calculation Calculate the similarity between mid-vertex features in the graph Adjacency matrix:

[0099]

[0100] The updated features are calculated using convolution as shown in the image below:

[0101]

[0102] in, This represents the features of the graph after convolution;

[0103] right , After layer normalization and linearization, the shape of the feature becomes The following characteristics were obtained:

[0104]

[0105] in Representation layer normalization, and Represents the learnable parameters of a linear transformation operation.

[0106] S102: Fuse the weakly transformed image and the strongly transformed image to obtain a weak-to-strong transformed fusion image and a strong-to-weak transformed fusion image. Randomly mix the high-pass filtered features and perform inverse discrete cosine transform on the features of the weak-to-strong transformed fusion image and the strong-to-weak transformed fusion image to obtain inverse discrete cosine transform features.

[0107] The processing of the weak-to-strong transformation fusion graph includes:

[0108] Calculate the weak-to-strong transformation fusion graph adjacency matrix ,in This represents the softmax function, used to calculate the strong-to-weak transformation fusion graph. adjacency matrix .calculate Graph convolution features: , Represent the learnable parameters of graph convolution, and calculate Graph convolution features: , The learnable parameters of graph convolution are represented.

[0109] Weak-to-strong transformation fusion graph In the middle, the vertex set is Edge set is The parameters of the graph are respectively determined by , express, Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows:

[0110]

[0111] in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation:

[0112]

[0113] Used to represent No. Column data, express No. Column data.

[0114] The processing of the strong-to-weak transform fusion graph includes:

[0115] Strong-to-weak transformation fusion graph In the middle, the vertex set is Edge set is The parameters of the graph are respectively determined by... , express. Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex can be calculated as follows:

[0116]

[0117] in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation can be calculated using the following equation:

[0118]

[0119] Used to represent No. Column data, express No. Column data;

[0120] To each , After performing layer normalization and linear transformation, the shape of the features becomes... Then change the shape Then, a convolution operation is used to transform it into a feature matrix. ,Right now .

[0121] The high-pass filtered features are randomly mixed and then subjected to inverse discrete cosine transform:

[0122]

[0123] in, Representing weakly transformed images Discrete cosine inverse transform features of graph convolution features Represents a strongly transformed image Discrete cosine inverse transform features of graph convolution features This indicates that the discrete inverse cosine transform is performed independently on each channel of the feature. , Represents a random number between 0 and 1;

[0124] S103: Input the strong and weak discrete cosine inverse transform features into a pre-established decoder, and output the segmentation results of the weak transform image and the strong transform image.

[0125] The strong and weak discrete cosine inverse transform features are input into a pre-established decoder, which outputs segmentation results for the weak transform image and the strong transform image, including:

[0126] Features , They are sent to the decoder respectively. Segmentation results of the weakly transformed image. Segmentation results with strongly transformed images Calculate the segmentation loss to optimize the entire network:

[0127]

[0128] in, This represents the number of pixels in the image. Indicates the number of categories in the segmentation. express The The class of 1 pixel, express The The class of 1 pixel, express The The class of 1 pixel, express The The class of 1 pixel.

[0129] Specifically, the present invention will be further illustrated below through embodiments:

[0130] First, the ultrasound data used in this invention were acquired from patients with breast cancer. The BUSI dataset was provided online by Al Dhabyani et al. It consists of 780 ultrasound images in PNG format with corresponding segmentation labels. The average size of the images in this dataset is 500px × 500px. The dataset includes three categories: benign, malignant, and normal. The benign category contains 487 images of breast tumors, while the malignant category contains 210 images. In this study, only the malignant and benign categories are considered because benign and malignant ultrasound images involve tumor region segmentation, while normal category images do not contain any tumor regions. The aforementioned 597 images were randomly divided into 5 folds for cross-validation.

[0131] This invention uses the PyTorch deep learning framework, version 1.13.0. All training and validation processes were performed on an NVIDIA GeForce RTX 3090 graphics card equipped with 24GB of VRAM. During training, the neural network uses a mini-batch method to read data, with a batch size of 2. The optimizer uses the Stochastic Gradient Descent (SGD) method, with an initial learning rate of 0.01, optimizer momentum of 0.9, and optimizer regularization coefficient of 0.0001. The learning rate adjustment strategy uses the Poly method.

[0132] The experimental results of this invention are as follows:

[0133] To quantitatively evaluate the performance of the method proposed in this invention, two different evaluation metrics were selected to assess the performance of the neural network on the breast tumor segmentation task: the Dice similarity coefficient and the Intersection over Union (IoU). The formula for calculating the Dice similarity coefficient is as follows:

[0134]

[0135] in, It represents the number of true positive segmentation pixels. It represents the number of segmented pixels that produce false positives. This represents the number of false negative segmentation pixels. The intersection-over-union (IoU) ratio is calculated using the following formula:

[0136]

[0137] The present invention compares the segmentation results with existing methods of the past two years, such as TBConvL-Net, MFAR-Net, SIB-UNet, SCA-InceptionUNeXt, and MFAN, as shown in Table 1. Table 1 compares the segmentation results of the present invention with typical existing methods.

[0138] Table 1 Comparison with typical existing methods

[0139]

[0140] As shown in Table 1, compared with other existing methods, the proposed method outperforms them in terms of segmentation accuracy, both in terms of the Dyess similarity coefficient and the intersection-over-union (IoU) ratio. Compared with the best-performing TBConvL-Net method, the proposed method improves the IoU ratio by 0.77, reaching 92.74%; the proposed method also improves the Dyess similarity coefficient by 1.02, reaching 96.74%.

[0141] Reference Figure 3 As can be seen, the method of this invention is very close to the label, while other methods still have many erroneous segmentation cases. For example, the segmented regions of TBConvL-Net, SIB-UNet, EH-former, SCA-InceptionUNeXt, and MSMO methods, represented by columns (b)-(f) in the first row, are all smaller than the label region. For example, the segmented regions of TBConvL-Net, MFAR-Net, SIB-UNet, SCA-InceptionUNeXt, and MFAN methods, represented by columns (b)-(f) in the second row, are all larger than the label region for smaller tumor segmentation regions.

[0142] To verify the effectiveness of each part of the invention, ablation experiments were conducted. The baseline models were UNet and segmentation loss using only weak image transformations. ; sequentially add the image weak transform and strong transform combined , , , and .

[0143] Table 2 Ablation Test Table

[0144]

[0145] As can be seen from the ablation test above, compared to no addition... , , , and Both the Dessian similarity coefficient and the crossover-union ratio are gradually increasing.

[0146] like Figure 2 As shown, weakly transformed image Strong transform image Segmentation is fed into the feature extractor Obtain the corresponding features and , The features of the weakly transformed graph are obtained by sequentially performing discrete cosine transform, low-pass filter, convolution operation, and weak transform graph. ; The features of the strong transform image are obtained by sequentially performing discrete cosine transform, low-pass filter, convolution operation, and strong transform image convolution. ; , The graph is fused through layer normalization and linear transformation layers, and weak-to-strong transformation layers. (softmax function), layer normalization and linear transformation layer, high-pass filtering, and inverse discrete cosine transform to obtain inverse discrete cosine transform features. ; , After undergoing layer normalization and linear transformation layers, and strong-to-weak transformation fusion graphs, (softmax function), layer normalization and linear transformation layer, high-pass filtering, and inverse discrete cosine transform to obtain inverse discrete cosine transform features. Features , They are sent to the decoder respectively. Segmentation results of the weakly transformed image. Segmentation results with strongly transformed images The segmentation loss is calculated to optimize the entire network.

[0147] like Figure 3 As shown in the figure, two rows represent two typical cases. Column (a) shows the segmentation results of the method of the present invention (red). The method of the present invention is very close to the label (green). Other methods still have many incorrect segmentation cases. For example, columns (b)-(f) in the first row show that the segmented regions of TBConvL-Net, SIB-UNet, EH-former, SCA-InceptionUNeXt, and MSMO methods are all smaller than the label region. For example, columns (b)-(f) in the second row show that TBConvL-Net, MFAR-Net, SIB-UNet, SCA-InceptionUNeXt, and MFAN methods all have segmented regions larger than the label region for smaller tumors.

[0148] Example 2: To achieve the above objective, such as Figure 4As shown, based on Embodiment 1, this invention discloses a breast ultrasound image segmentation system using a cross-graph fusion network, comprising:

[0149] Image processing module 11 is used to receive ultrasound images, perform weak and strong transformations on the ultrasound images to obtain weakly transformed images and strongly transformed images, and perform discrete cosine transform on the features of the weakly transformed images and strongly transformed images to obtain transformed image features; perform high-pass filtering and low-pass filtering on the transformed image features, and process the features after low-pass filtering to obtain weakly transformed processed images and strongly transformed processed images.

[0150] Image fusion module 12 is used to fuse weakly transformed images and strongly transformed images to obtain weak-to-strong transformed fusion images and strong-to-weak transformed fusion images. It randomly mixes the features after high-pass filtering and performs inverse discrete cosine transform with the features of the weak-to-strong transformed fusion images and strong-to-weak transformed fusion images to obtain strong and weak inverse discrete cosine transform features.

[0151] Image segmentation module 13 is used to input strong and weak discrete cosine inverse transform features into a pre-established decoder and output the segmentation results of the weak transform image and the strong transform image.

[0152] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0153] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0154] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0155] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A method for breast ultrasound image segmentation using cross-graph fusion networks, characterized in that, The method includes the following steps: The system receives ultrasound images, performs weak and strong transforms on them to obtain weakly transformed and strongly transformed images, and then performs discrete cosine transform on the features of the weakly and strongly transformed images to obtain transformed image features. High-pass and low-pass filters are then applied to the transformed image features, and further processing is performed based on the low-pass filtered features to obtain weakly transformed and strongly transformed images. The images processed by weak transformation and those processed by strong transformation are fused to obtain weak-to-strong transformation fusion image and strong-to-weak transformation fusion image. The features after high-pass filtering are randomly mixed and then subjected to inverse discrete cosine transform with the features of the weak-to-strong transformation fusion image and the strong-to-weak transformation fusion image to obtain strong-weak inverse discrete cosine transform features. The processing of the weak-to-strong transformation fusion graph includes: Calculate the weak-to-strong transformation fusion graph adjacency matrix ,in This represents the softmax function. This represents the strong-to-weak transformation feature query component. This represents the characteristic bond components of the weak-to-strong transformation. Represent the eigenvalue components of the weak-to-strong transform; calculate the strong-to-weak transform fusion graph. adjacency matrix ; This represents the feature query component for the weak-to-strong transformation. This represents the characteristic bond components of the strong-to-weak transformation. Represent the eigenvalue components of the strong-to-weak transform; calculate the weak transform features after graph convolution. Graph convolution features: , Represent the learnable parameters of graph convolution, and compute the strong transform features after graph convolution. Graph convolution features: , Represents the learnable parameters of graph convolution; Weak-to-strong transformation fusion graph In the middle, the vertex set is Edge set is The parameters of the graph are respectively from , express, Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows: in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation: Used to represent No. Column data, express No. Column data; The processing of the strong-to-weak transform fusion graph includes: Strong-to-weak transformation fusion graph In the middle, the vertex set is Edge set is The parameters of the graph are respectively from , express; Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows: in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation: Used to represent No. Column data, express No. Column data; To each , After layer normalization and linear transformation layer processing, the shape of the feature becomes Then change the shape Then, a convolution operation is used to transform it into a feature matrix. ,Right now ; The strong and weak discrete cosine inverse transform features are input into a pre-established decoder, and the outputs are the segmentation results of the weak transform image and the strong transform image.

2. The breast ultrasound image segmentation method using cross-graph fusion networks according to claim 1, characterized in that, The ultrasound images are obtained based on an ultrasound image training set, which contains ultrasound images. and its corresponding segmentation labels , , This represents the number of ultrasound images in the training set; for each ultrasound image... and its corresponding segmentation labels Weak transformation is performed to obtain the weakly transformed image. and their corresponding tags Meanwhile, for each ultrasound image and its corresponding segmentation labels Perform a strong transform to obtain the transformed strong transform image. and their corresponding tags The weak transformation includes image rotation, scaling, translation, and reversal; the strong transformation is an image masking operation.

3. The breast ultrasound image segmentation method using cross-graph fusion networks according to claim 2, characterized in that, The image masking operation includes: constructing an image of the same size as the ultrasound image. ,image All pixel values ​​in the image are initialized to 0. Multiple rectangular and triangular random regions are randomly added to the image, and preset pixel values ​​are set within each random region; then, another ultrasound image is selected. and its corresponding segmentation labels , ; with ultrasound images and its corresponding segmentation labels Merge: 。 4. The breast ultrasound image segmentation method using cross-graph fusion networks according to claim 1, characterized in that, The features of the weakly transformed and strongly transformed images are extracted using the feature extractor of the U-Net encoder. The extracted data includes: images with weak transformations. With strong transform image Feed into feature extractor Obtain the corresponding features and ,in These represent the number of channels, height, and width of the feature, respectively. Represents a real matrix; right , Each channel undergoes a discrete cosine transform independently, and the corresponding transform characteristics are as follows: , The results were obtained by using a high-pass filter and a low-pass filter, respectively. , Their respective high-frequency components and low-frequency components, , They are High-frequency components and low-frequency components; , They are High-frequency components and low-frequency components; For low-frequency components , By using convolution operations to reduce the number of feature channels and flatten the pixels, the shape of the features becomes... ,in This represents the reduced number of feature channels. This indicates the number of pixels after flattening; the shape of the feature becomes... , Indicates the number of neurons in the output layer of a multilayer perceptron; low-frequency component. , The features after each operation are represented as follows , .

5. The breast ultrasound image segmentation method using cross-graph fusion networks according to claim 4, characterized in that, The low-pass filtered features are processed to obtain a weakly transformed image and a strongly transformed image. The processing steps for the weakly transformed image include: Will Projected onto the vertex set Edge set is Weak transform processing of images In the graph, the parameters are respectively from , express; Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows: Where / indicates element division, This indicates a search for the 2-norm. The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation: Used to represent No. Column data, express No. Column data; Through calculation The similarity between mid-vertex features is used to calculate the weak transform processing of the image. adjacency matrix in, This represents the matrix transpose; then, convolution is used as shown in the diagram below to calculate the updated features: in, This represents the features after graph convolution. This represents a non-linear activation function.

6. The breast ultrasound image segmentation method using cross-graph fusion networks according to claim 5, characterized in that, The process of processing the image with strong transform includes: Will Projected onto the vertex set Edge set is Strong transform processing of images In the graph, the parameters are respectively from , express; Used to represent No. Column data, corresponding to the chart The There are vertices, and the characteristics of each vertex are calculated as follows: in, The diagram shows... The Features of each vertex express No. Column data, express No. Column data, It is an eigenvector arrive The soft allocation is calculated using the following equation: Used to represent No. Column data, express No. Column data; Through calculation Calculate the similarity between mid-vertex features in the graph Adjacency matrix: The updated features are calculated using convolution as shown in the image below: in, This represents the features of the graph after convolution; right , After layer normalization and linear transformation layer processing, the shape of the feature becomes The following characteristics were obtained: in Representation layer normalization, and Represents the learnable parameters of a linear transformation operation.

7. The breast ultrasound image segmentation method using cross-graph fusion networks according to claim 6, characterized in that, The high-pass filtered features are randomly mixed and then subjected to inverse discrete cosine transform: in, Representing weakly transformed images Discrete cosine inverse transform features of graph convolution features Represents a strongly transformed image Discrete cosine inverse transform features of graph convolution features This indicates that the discrete inverse cosine transform is performed independently on each channel of the feature. , Represents a random number between 0 and 1; The strong and weak discrete cosine inverse transform features are input into a pre-established decoder, which outputs segmentation results for the weak transform image and the strong transform image, including: Features , They are sent to the decoder respectively. Segmentation results of the weakly transformed image. Segmentation results with strongly transformed images Calculate the segmentation loss to optimize the entire network: in, This represents the number of pixels in the image. Indicates the number of categories in the segmentation. express The The class of 1 pixel, express The The class of 1 pixel, express The The class of 1 pixel, express The The class of 1 pixel.

8. A breast ultrasound image segmentation system using cross-graph fusion networks, employing the breast ultrasound image segmentation method using cross-graph fusion networks as described in any one of claims 1 to 7, characterized in that... include: The image processing module is used to receive ultrasound images, perform weak and strong transforms on the ultrasound images to obtain weakly transformed images and strongly transformed images, and perform discrete cosine transform on the features of the weakly transformed images and strongly transformed images to obtain transformed image features; then, high-pass filtering and low-pass filtering are performed on the transformed image features, and the features after low-pass filtering are processed to obtain weakly transformed processed images and strongly transformed processed images. The image fusion module is used to fuse images processed by weak transformation and images processed by strong transformation to obtain weak-to-strong transformation fusion image and strong-to-weak transformation fusion image. The features after high-pass filtering are randomly mixed and then subjected to inverse discrete cosine transform with the features of the weak-to-strong transformation fusion image and the strong-to-weak transformation fusion image to obtain strong and weak inverse discrete cosine transform features. The image segmentation module is used to input strong and weak discrete cosine inverse transform features into a pre-established decoder and output the segmentation results of the weak transform image and the strong transform image.

Images