Breast cancer focus classification method, medium and terminal

Abstract

The invention is suitable for the technical field of intelligent diagnosis, and relates to a breast cancer lesion classification method, medium and terminal, and the method comprises the steps: S10, carrying out the continuous scanning at multiple angles through an ultrasonic probe, obtaining a two-dimensional image, and carrying out the three-dimensional reconstruction; s20, on the basis of the three-dimensional ultrasonic image, performing three-dimensional segmentation by adopting a 3D-Unet network architecture; s30, inputting the processed data into a U-point cloud model to obtain point cloud data; s40, processing the point cloud data by using a point cloud network, and extracting point cloud features; s50, processing the dynamic ultrasound contrast data by using 3D-CNN, and extracting dynamic contrast features; and S60, fusing the point cloud features and the dynamic contrast features, and classifying the lesions according to the fused features. The method is simple in process, convenient to operate and high in ultrasonic image analysis accuracy, and the breast cancer focus classification accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent diagnosis, and in particular relates to a breast cancer lesion classification method, medium and terminal. Background Art

[0002] In the process of breast cancer lesion classification, the existing technology has some technical shortcomings when processing three-dimensional ultrasound images and dynamic ultrasound contrast data. Traditional image processing methods such as threshold segmentation and edge detection are simple and easy to use, but they are easily affected by noise and uneven background when processing complex medical images, resulting in inaccurate segmentation results. This inaccurate segmentation directly affects the subsequent feature extraction and analysis, making the identification of the target area less accurate. Existing dynamic ultrasound contrast data feature extraction methods mainly rely on two-dimensional image sequences and lack comprehensive consideration of temporal and spatial features, which leads to incomplete dynamic feature information extracted and cannot fully describe the dynamic changes of tissues, thereby affecting the accurate classification of lesions. Poor feature fusion effect is also a shortcoming of the existing technology. Traditional feature fusion methods usually use simple weighted average or feature splicing, and fail to fully consider the correlation and complementarity between different features, resulting in redundant or lost feature vector information after fusion. This unsatisfactory feature fusion effect further affects the final lesion classification accuracy. The poor generalization ability of the classification model is also a problem that cannot be ignored. Existing classification models are mostly based on traditional machine learning methods, such as support vector machines (SVM) and random forests. These methods have poor generalization ability when processing high-dimensional and complex medical image features and are prone to overfitting, resulting in decreased classification performance on new data.

[0003] The patent with announcement number CN104720850B provides an ultrasound contrast imaging method and a method for detecting and developing contrast images. The method for detecting contrast images includes: obtaining tissue signals and contrast signals during ultrasound contrast imaging; preprocessing tissue signals and contrast signals respectively; comparing the preprocessed tissue signals and contrast signals; segmenting the image formed after the comparison process into contrast agent action areas, tissue residue action areas, and noise action areas by means of threshold segmentation; marking the corresponding positions on the contrast image as contrast agent areas, tissue residue areas, and noise areas. The image processing method in the above patent is threshold segmentation, which has questionable accuracy and has the same drawbacks as the prior art.

[0004] Therefore, how to improve the accuracy of ultrasound image analysis to improve the accuracy of lesion classification is an urgent problem to be solved by personnel in this technical field. Summary of the invention

[0005] In view of the deficiencies of the prior art, the purpose of the present invention is to provide a breast cancer lesion classification method to solve the problem of low accuracy in breast cancer lesion classification in the prior art; in addition, the present invention also provides a breast cancer lesion classification medium and terminal.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for classifying breast cancer lesions, comprising the following steps:

[0008] S10, continuously scanning at multiple angles with an ultrasonic probe to obtain a two-dimensional image and perform three-dimensional reconstruction;

[0009] S20, based on the three-dimensional ultrasound image, the 3D-Unet network architecture is used for three-dimensional segmentation;

[0010] S30, inputting the processed data into the U-point cloud model to obtain point cloud data;

[0011] S40, using a point cloud network to process point cloud data and extract point cloud features;

[0012] S50, using 3D-CNN to process dynamic ultrasound contrast data and extract dynamic contrast features;

[0013] S60, fusing the point cloud features and the dynamic angiography features, and classifying the lesions according to the fused features.

[0014] Furthermore, in step S10, the three-dimensional reconstruction process includes image registration, stitching and interpolation steps, and by injecting microbubble contrast agents, an image sequence of dynamic tissue changes is acquired in real time.

[0015] Furthermore, in step S30, the U-point cloud model includes an input layer, a U-type network module, a point cloud module and an output layer.

[0016] Furthermore, the U-shaped network module includes an analysis path and a synthesis path, each of which contains four resolution levels. In the analysis path, each layer includes two 3×3×3 convolutional layers, each convolutional layer is followed by a ReLU activation function, followed by a 2×2×2 maximum pooling layer with a pooling step of 2, which is used to extract high-level features.

[0017] Furthermore, in the synthesis path, each layer is first a 2×2×2 upconvolution layer with a stride of 2, which is used to restore the spatial resolution of the feature map. The upconvolution layer is followed by two 3×3×3 convolution layers, each of which is followed by a ReLU activation function. After each upsampling, the obtained feature image is concatenated with the corresponding downsampled feature image. The final softmax layer converts the output image into a segmentation result of the same size as the input image, and the segmentation result is a three-dimensional probability map, in which each voxel is assigned a probability distribution.

[0018] Furthermore, based on the output probability distribution result, a threshold selection function is provided for segmenting the lesion location, the data whose probability exceeds the threshold is recorded as 1, and the data whose probability does not exceed the threshold is recorded as 0, and the three-dimensional position of the image of the data recorded as 1 is saved and used as the input data of the point cloud module.

[0019] Furthermore, the specific steps of step S40 are as follows:

[0020] S401, the point cloud network receives an unordered point set as input, each point is represented by its three-dimensional coordinates;

[0021] S402, the point cloud network aligns the input point cloud data through an input transformation network, the input transformation network includes several convolutional layers and fully connected layers, and finally outputs a 3×3 transformation matrix for performing affine transformation on the coordinates of the input point cloud;

[0022] S403, the point cloud data output by the transformation matrix enters a shared multi-layer perceptron, which includes several convolution layers, and each point undergoes the same convolution operation to extract local features;

[0023] S404, the point cloud network uses a feature transformation network to align the extracted local features. The feature transformation network includes several convolutional layers and fully connected layers, and finally outputs a 64×64 transformation matrix for transforming the feature vector;

[0024] S405, the point cloud network uses a maximum pooling layer to extract a global feature vector from the local features of all points;

[0025] S406, the point cloud network passes the extracted global feature vector through the fully connected layer of the classification branch, and finally outputs the probability of each category.

[0026] Furthermore, the specific steps of step S50 are as follows:

[0027] S501, input data: dynamic ultrasound contrast imaging data is input into 3D-CNN in the form of three-dimensional tensor;

[0028] S502, sending the data of the input layer to the convolution layer for calculation;

[0029] S503, processing the data received by the convolution layer through an activation function to introduce nonlinearity and improve the expression ability of the model;

[0030] S504: Send the data to the pooling layer to reduce the spatial dimension of the feature map, reduce the amount of calculation, and retain important features;

[0031] S505, sending the data obtained in the pooling layer to the fully connected layer, flattening the output feature map of the previous layer into a one-dimensional vector, and mapping the features to a vector of fixed length through a fully connected operation;

[0032] S506, output data: The output result is used for classification tasks. The output of the fully connected layer is converted into a probability distribution through the Softmax function to represent the probability of each category.

[0033] In a second aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0034] In a third aspect, the present invention further provides an electronic terminal, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the method as described above.

[0035] Compared with the prior art, the breast cancer lesion classification method, medium and terminal provided by the present invention have at least the following beneficial effects:

[0036] In the process of breast cancer lesion classification, the prior art has some technical shortcomings when processing three-dimensional ultrasound images and dynamic ultrasound contrast data, resulting in low accuracy of ultrasound image analysis, which in turn leads to low accuracy of breast cancer lesion classification. The present invention uses 3D-Unet network architecture to perform high-precision three-dimensional segmentation of automatic volume ultrasound data, accurately identify the target area, effectively reduce the influence of noise and uneven background, and provide a high-quality basis for subsequent point cloud data generation; extract point cloud data from the segmentation result, and describe the geometric shape and boundary information of the target area in detail; use PointNet or other point cloud analysis models to extract geometric features of point cloud data, such as the distance, angle, curvature, etc. between points, and effectively describe the shape and spatial distribution of the target area; use 3D-CNN to process dynamic ultrasound contrast data, extract time and space features, comprehensively describe the dynamic changes of tissues, and extract rich blood flow and tissue information through multi-frame image sequences, which makes up for the shortcomings of existing methods in dynamic feature extraction; through feature level fusion, weighted fusion or neural network fusion and other methods, effectively combine point cloud features and dynamic contrast features to generate comprehensive feature vectors, ensure rich and accurate information, optimize the effect of feature fusion, and provide more reliable data support for subsequent classification. By integrating the above-mentioned key technologies, the present invention solves the deficiencies of the prior art in terms of image segmentation accuracy, dynamic feature extraction, feature fusion effect and classification model generalization ability, achieves accurate classification of lesions, improves the accuracy of ultrasound image analysis and the comprehensiveness of clinical diagnosis, and greatly improves the accuracy of breast cancer lesion classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the scheme of the present invention, a brief introduction is given below to the figures required for use in the description of the embodiments. Obviously, the figures described below are some embodiments of the present invention. For ordinary technicians in this field, other figures can be obtained based on these figures without paying any creative work.

[0038] Figure 1 A flow chart of a method for classifying breast cancer lesions provided by an embodiment of the present invention;

[0039] Figure 2 An overall hierarchical network model for a breast cancer lesion classification method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0042] The present invention provides a breast cancer lesion classification method, which is applied to the processing of three-dimensional ultrasound images and dynamic ultrasound angiography data when classifying breast cancer lesions. The breast cancer lesion classification method comprises the following steps:

[0043] S10. Continuously scan at multiple angles with an ultrasonic probe to obtain two-dimensional images and perform three-dimensional reconstruction; S20. Based on the three-dimensional ultrasonic images, use the 3D-Unet network architecture to perform three-dimensional segmentation; S30. Input the processed data into the U-point cloud model to obtain point cloud data; S40. Use the point cloud network to process the point cloud data and extract the point cloud features; S50. Use 3D-CNN to process dynamic ultrasound contrast data and extract dynamic contrast features; S60. Fuse the point cloud features and dynamic contrast features, and classify the lesions according to the fused features.

[0044] The method has simple process, convenient operation, high accuracy of ultrasonic image analysis, and improved accuracy of breast cancer lesion classification.

[0045] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0046] The present invention provides a breast cancer lesion classification method, which is applied to the processing of three-dimensional ultrasound images and dynamic ultrasound contrast data when classifying breast cancer lesions. Figure 1 and Figure 2 In this embodiment, the breast cancer lesion classification method comprises the following steps:

[0047] S10, continuously scanning at multiple angles with an ultrasonic probe to obtain two-dimensional images and perform three-dimensional reconstruction.

[0048] Specifically, in this embodiment, the ultrasound probe is used to continuously scan at multiple angles to acquire a series of two-dimensional images, which are then reconstructed in three dimensions using professional software to generate high-resolution three-dimensional ultrasound images. During the three-dimensional reconstruction process, technical steps such as image registration, stitching, and interpolation are used to ensure the accuracy and integrity of the image. In addition, by injecting microbubble contrast agents, image sequences of dynamic tissue changes are acquired in real time to provide blood flow and tissue information, thereby enriching the data content of the ultrasound image.

[0049] S20. Based on the three-dimensional ultrasound image, the 3D-Unet network architecture is used for three-dimensional segmentation.

[0050] Specifically, 3D-Unet is a deep learning architecture designed specifically for three-dimensional medical imaging. It uses an encoder-decoder structure to segment the input three-dimensional ultrasound images and extract clinically significant areas such as tumors and organs.

[0051] S30, inputting the processed data into a U-point cloud model (such as PointNet) to obtain point cloud data.

[0052] Specifically, in this embodiment, the processed data is input into the U-point cloud model, which consists of input, U-type network module, point cloud module and output. The U-type network architecture is mainly divided into an analysis path (i.e., encoder part) and a synthesis path (i.e., decoder part), each of which contains four resolution levels. The point cloud data is a data set composed of a large number of three-dimensional coordinate points, which can describe the geometric shape and boundary information of the target area in detail, and is used for subsequent point cloud feature extraction to provide the geometric features of the target area.

[0053] Furthermore, in this embodiment, in the analysis path, each layer consists of two 3×3×3 convolutional layers, each of which is followed by a ReLU activation function, followed by a 2×2×2 maximum pooling layer with a pooling step of 2 for extracting high-level features. This series of operations helps capture the global characteristics of the input data.

[0054] The convolutional layer is:

[0055]

[0056] in, Represents the convolution operation, x is the input, W is the convolution kernel, b is the bias term, y is the output, and ReLU is the activation function, which is defined as:

[0057] ReLU(z)=max(0,z).

[0058] The maximum pooling layer is:

[0059] Assume that the shape of the input data is H×W×C, where H is the height, W is the width, C is the number of channels, the pooling kernel size is k×k, and the step size is s. Divide the input feature map x into several non-overlapping local areas according to the size k×k of the pooling kernel and the step size s. The shape of each local area is k×k.

[0060] Select the maximum value in each local area to form a new feature map Y. The height H′ and width W′ of the new feature map can be calculated by the following formula:

[0061]

[0062]

[0063] The shape of the output feature map Y is H′×W′×C, and each element Y i，j，k is the maximum value of the corresponding local area in the input feature map x:

[0064]

[0065] Wherein, region(i, j) represents a k×k local region centered at position (i, j).

[0066] Furthermore, in this embodiment, in the synthesis path, each layer is first a 2×2×2 up-convolution layer (also called a deconvolution layer or a transposed convolution layer) with a step size of 2, which is used to restore the spatial resolution of the feature map. The up-convolution layer is followed by two 3×3×3 convolution layers, and each convolution layer is also followed by a ReLU activation function. After each upsampling, the obtained feature image is concatenated with the corresponding down-sampled feature image. The final Softmax layer of the network converts the output image into a segmentation result of the same size as the input image, and the segmentation result is a three-dimensional probability map in which each voxel is assigned a probability distribution.

[0067] Data z enters the upper convolution layer, and the convolution formula is:

[0068]

[0069] Among them, ConvTranspose is a transposed convolution operation, which is used to restore the spatial resolution of the feature map.

[0070] After the upper convolution is completed, the data enters the convolution layer for calculation. The convolution layer formula is:

[0071]

[0072] Then the upsampled feature map y u And the upsampled feature map y d The two feature maps are concatenated in the channel dimension through the Concatenate operation.

[0073] Finally, the final output P is obtained through the Softmax layer, which is the probability distribution result.

[0074] The Softmax function formula is:

[0075]

[0076] Furthermore, in this embodiment, after receiving the output result P, a threshold selection function is provided to segment the lesion location, and the data with a probability exceeding the threshold is recorded as 1 and vice versa. The three-dimensional position of the image of the data with a probability of 1 is then saved and passed to the next level point cloud network as input data of the point cloud.

[0077] S40, using the point cloud network to process the point cloud data and extract point cloud features.

[0078] Specifically, in this embodiment, the point cloud network is a deep learning model specifically used to process point cloud data. It can directly learn global and local features from point sets for 3D classification and segmentation tasks. In this model, only the classification function of the point cloud network is used. The point cloud receives an unordered point set as input, and each point is represented by its three-dimensional coordinates (x, y, z). The point cloud can be regarded as a set of discrete points in three-dimensional space, for example: {P i |i=1,...,n}, where each P i =(x i ,y i , z i ).

[0079] Furthermore, in this embodiment, the specific steps of step S40 are as follows:

[0080] S401, the point cloud network receives an unordered point set as input, each point is represented by its three-dimensional coordinates.

[0081] S401. The point cloud network aligns the input point cloud data through the input transformation network (Input Transform Net). The input transformation network consists of several convolutional layers and fully connected layers, and finally outputs a 3×3 transformation matrix. This transformation matrix is ​​used to perform affine transformation on the coordinates of the input point cloud, making the point cloud more regular and symmetrical in space. By aligning the input point cloud, the model can better capture the geometric structure of the point cloud. The following is the formula of the input transformation layer (InputTransformation):

[0082] Input: point set P i .

[0083] Output: The transformed point set P′.

[0084] formula:

[0085] T (input) =f STN (Pi)

[0086] P′=Pi·T (input) .

[0087] S403, the point cloud data output by the transformation matrix enters the shared multi-layer perceptron (Shared MLP), and local features are extracted for each point. The shared multi-layer perceptron consists of several convolutional layers. Each point undergoes the same convolution operation to extract local features. These local features retain the local geometric information of the point cloud and provide a basis for further feature extraction. The following is the formula of the shared multi-layer perceptron (Shared MLP):

[0088] Input: point set P′.

[0089] Output: feature vector f for each point i .

[0090] formula:

[0091] f i =MLP(p′).

[0092] S404. In order to further enhance the alignment of features, the point cloud also uses a feature transformation network (FeatureTransform Net) to align the extracted local features. The feature transformation network is also composed of several convolutional layers and fully connected layers, and finally outputs a 64×64 transformation matrix. This transformation matrix is ​​used to transform the feature vector to make the features more regular and symmetrical in the feature space. The following is the calculation formula of the feature transformation layer:

[0093] Input: feature vector fi for each point.

[0094] Output: The transformed point set P′.

[0095] formula:

[0096] T (feature) =f STN (F)

[0097] F′=F·T (feature) .

[0098] S405. After extracting local features and aligning them, the point cloud uses the maximum pooling layer to extract the global feature vector from the local features of all points. The maximum pooling layer aggregates the local features into a global feature vector by performing the maximum value operation on the features of all points. This global feature vector represents the global characteristics of the entire point cloud and can be used for subsequent classification and segmentation tasks. The global feature vector calculation formula is as follows:

[0099] Input: feature vector f′ of each point i .

[0100] Output: global eigenvector g.

[0101] formula:

[0102] g=max({f′1, f′2,…, f′ n}).

[0103] S406. For the classification task, the point cloud extracts the global feature vector through the fully connected layer of the classification branch and finally outputs the probability of each category. The classification branch consists of several fully connected layers, which process the global feature vector and generate the final classification result. In this way, the point cloud can classify the entire point cloud and identify the category to which the point cloud belongs. The following is the relevant calculation formula for classification:

[0104] Input: Global eigenvector g.

[0105] Output: probability distribution c of class labels.

[0106] formula:

[0107] c = MLP(g).

[0108] Activation function: ReLU activation function is usually used.

[0109] Output layer: Use the Softmax function to convert the output into a probability distribution.

[0110] c=softmax(MLP(g)).

[0111] S50. Use 3D-CNN to process dynamic ultrasound contrast imaging data and extract dynamic contrast imaging features.

[0112] Specifically, in this embodiment, the specific steps of step S50 are as follows:

[0113] S501. Input data to the input layer. Dynamic ultrasound contrast data is usually input to 3D-CNN in the form of a three-dimensional tensor. Assuming that we have a sequence of T frames of images, each frame has a size of H×W, then the shape of the input data is (T, H, W, C), where C represents the number of channels; the input layer directly passes the raw data to the next layer without any processing. Therefore, the output shape of the input layer is still (T, H, W, C).

[0114] S502: Send the input layer data to the convolution layer for operation.

[0115] Input: 3D tensor (T, H, W, C) received from the input layer.

[0116] Volume and operation: The 3D convolution layer uses a three-dimensional convolution kernel to perform convolution operations in three dimensions: time, height, and width. Assume that the size of the convolution kernel is (K t , K h , K w), the stride is (S t , S h , S w ), filled with (P t , P h , P w ), then the output shape of the convolutional layer is:

[0117]

[0118] Among them, F is the number of convolution kernels, that is, the number of output feature maps.

[0119] Convolution formula:

[0120]

[0121] in, is the value of the first output feature map at position (i, j, k), is the value of the qth channel of the input tensor at position (i+m, j+n, k+p), is the value of the qth channel of the first convolution kernel at position (m, n, p), b l It is the bias term of the first convolution kernel.

[0122] Output: The output of the convolutional layer is a new 3D tensor with the shape:

[0123]

[0124] S503. Process the data received by the convolution layer through an activation function to introduce nonlinearity and improve the expressiveness of the model.

[0125] Input: 3D tensor received from the convolutional layer.

[0126] ReLU formula:

[0127] ReLU(x)=max(0,x).

[0128] Output: The output of the activation function is still a three-dimensional tensor with the same shape, but the values ​​have undergone nonlinear transformation.

[0129] S504: Send the data to the pooling layer to reduce the spatial dimension of the feature map, reduce the amount of calculation, and retain important features.

[0130] Input: 3D tensor received from the activation function.

[0131] Max pooling formula:

[0132]

[0133] in, is the value of the first output feature map at position (i, j, k), is the value of the input feature map at position (i+m, j+n, k+p), P t , P h , P w are the pooling window sizes in time, height, and width, respectively.

[0134] Output: The output of the pooling layer is a new 3D tensor with shape Where T′, H′, W′ are the spatiotemporal dimensions after pooling.

[0135] S505, sending the acquired data in the pooling layer to the fully connected layer, flattening the output feature map of the previous layer into a one-dimensional vector, and then mapping these features to a vector of fixed length through a fully connected operation.

[0136] Input: 3D tensor received from the pooling layer

[0137] Full connection formula:

[0138]

[0139] Among them, O i is the value of the ith output unit, I j is the jth element of the input vector, W j，i is the weight between the jth input unit and the ith output unit, b i is the bias term for the i-th output unit.

[0140] Output: The output of the fully connected layer is a one-dimensional vector with shape (M).

[0141] S506, output data. The output result is usually used for classification tasks. The output of the fully connected layer is converted into a probability distribution through the Softmax function to represent the probability of each category.

[0142] Input: A one-dimensional vector (M) received from a fully connected layer.

[0143] Softmax formula:

[0144]

[0145] Where P(y=c|x) is the probability that input x belongs to category c, z c is the cth element in the output vector of the fully connected layer, and C is the total number of categories.

[0146] Finally, the 3D-CNN network outputs a one-dimensional vector with a shape of (C), which represents the probability of each category.

[0147] 3D-CNN processes multi-frame image sequences of dynamic ultrasound contrast imaging data through multi-layer convolution and pooling operations, extracting temporal and spatial features that reflect the dynamic changes of blood flow velocity, perfusion pattern, etc. of tissues, providing rich information for subsequent feature fusion.

[0148] S60, fusing the point cloud features and the dynamic angiography features, and classifying the lesions according to the fused features.

[0149] Specifically, in this embodiment, on the basis of feature extraction, the point cloud features and dynamic imaging features are fused to form a more comprehensive feature vector. First, the probability vector P output by the point cloud network is converted into prob and the probability vector C output by 3D-CNN to form a higher dimensional feature vector F, F = [P prob , V prob ], and then perform attention mechanism fusion, assign different weights to point cloud features and dynamic video features according to their importance, and generate a weighted fused feature vector F attention , the formula is:

[0150] F attention =α1·P prob +α2·V prob ;

[0151] Among them, α1 and α2 are attention weights, which are calculated by the attention mechanism. The calculation formula is:

[0152] α=softmmax( A ttention(P prob , V prob ));

[0153] Finally, the output of the deep neural network is converted into a probability distribution through the Softmax function, indicating the probability of each category.

[0154] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any one of the methods in the present embodiment is implemented.

[0155] An embodiment of the present invention further provides an electronic terminal, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0156] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0157] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0158] Compared with the prior art, the intelligent breast cancer lesion classification method, medium and terminal described in the above embodiments have some technical shortcomings in the process of breast cancer lesion classification when processing three-dimensional ultrasound images and dynamic ultrasound angiography data, resulting in low accuracy of ultrasound image analysis, and further resulting in low accuracy of breast cancer lesion classification. The present invention uses the 3D-Unet network architecture to perform high-precision three-dimensional segmentation of automatic volumetric ultrasound data, accurately identify the target area, effectively reduce the influence of noise and uneven background, and provide a high-quality foundation for subsequent point cloud data generation; extract point cloud data from the segmentation results, and describe the geometric shape and boundary information of the target area in detail; use PointNet or other point cloud analysis models to extract the geometric features of the point cloud data, such as the distance, angle, curvature, etc. between points, to effectively describe the shape and spatial distribution of the target area; use 3D-CNN to process dynamic ultrasound contrast data, extract time and space features, comprehensively describe the dynamic changes of tissues, and extract rich blood flow and tissue information through multi-frame image sequences, which makes up for the shortcomings of existing methods in dynamic feature extraction; through feature-level fusion, weighted fusion or neural network fusion and other methods, effectively combine point cloud features and dynamic contrast features to generate comprehensive feature vectors, ensure rich and accurate information, optimize the effect of feature fusion, and provide more reliable data support for subsequent classification. By integrating the above-mentioned key technologies, the present invention solves the deficiencies of the prior art in terms of image segmentation accuracy, dynamic feature extraction, feature fusion effect and classification model generalization ability, achieves accurate classification of lesions, improves the accuracy of ultrasound image analysis and the comprehensiveness of clinical diagnosis, and greatly improves the accuracy of breast cancer lesion classification.

[0159] Obviously, the embodiments described above are only preferred embodiments of the present invention, rather than all embodiments. The preferred embodiments of the present invention are shown in the accompanying drawings, but they do not limit the patent scope of the present invention. The present invention can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive. Although the present invention has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific embodiments, or to perform equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of the present invention, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present invention.

Claims

1. A method for classifying breast cancer lesions, characterized in that: The following steps are involved: S10, continuously scanning at multiple angles with an ultrasonic probe to obtain a two-dimensional image and perform three-dimensional reconstruction; S20, based on the three-dimensional ultrasound image, the 3D-Unet network architecture is used for three-dimensional segmentation; S30, inputting the processed data into the U-point cloud model to obtain point cloud data; S40, using a point cloud network to process point cloud data and extract point cloud features; S50, using 3D-CNN to process dynamic ultrasound contrast data and extract dynamic contrast features; S60, fusing the point cloud features and the dynamic angiography features, and classifying the lesions according to the fused features.

2. The method for intelligent classification of breast cancer lesions according to claim 1, characterized in that: In step S10, the three-dimensional reconstruction process includes image registration, stitching and interpolation steps, and by injecting microbubble contrast agents, an image sequence of dynamic tissue changes is acquired in real time.

3. The method for intelligent classification of breast cancer lesions according to claim 2, characterized in that: In step S30, the U-point cloud model includes an input layer, a U-type network module, a point cloud module and an output layer.

4. The method for intelligent classification of breast cancer lesions according to claim 3, characterized in that: The U-shaped network module includes an analysis path and a synthesis path, each of which contains four resolution levels. In the analysis path, each layer includes two 3×3×3 convolutional layers, each convolutional layer is followed by a ReLU activation function, followed by a 2×2×2 maximum pooling layer with a pooling step of 2, which is used to extract high-level features.

5. The method for intelligent classification of breast cancer lesions according to claim 4, characterized in that: In the synthesis path, each layer is first a 2×2×2 upconvolution layer with a stride of 2, which is used to restore the spatial resolution of the feature map. The upconvolution layer is followed by two 3×3×3 convolution layers, each of which is followed by a ReLU activation function. After each upsampling, the obtained feature image is concatenated with the corresponding downsampled feature image. The final softmax layer converts the output image into a segmentation result of the same size as the input image, and the segmentation result is a three-dimensional probability map in which each voxel is assigned a probability distribution.

6. The method for intelligent classification of breast cancer lesions according to claim 5, characterized in that: According to the output probability distribution result, a threshold selection function is provided for segmenting the lesion position. The data whose probability exceeds the threshold is recorded as 1, and the data whose probability does not exceed the threshold is recorded as 0. The three-dimensional position of the image of the data recorded as 1 is saved and used as the input data of the point cloud module.

7. The method for intelligent classification of breast cancer lesions according to claim 1, characterized in that: The specific steps of step S40 are as follows: S401, the point cloud network receives an unordered point set as input, each point is represented by its three-dimensional coordinates; S402, the point cloud network aligns the input point cloud data through an input transformation network, the input transformation network includes several convolutional layers and fully connected layers, and finally outputs a 3×3 transformation matrix for performing affine transformation on the coordinates of the input point cloud; S403, the point cloud data output by the transformation matrix enters a shared multi-layer perceptron, which includes several convolution layers, and each point undergoes the same convolution operation to extract local features; S404, the point cloud network uses a feature transformation network to align the extracted local features. The feature transformation network includes several convolutional layers and fully connected layers, and finally outputs a 64×64 transformation matrix for transforming the feature vector; S405, the point cloud network uses a maximum pooling layer to extract a global feature vector from the local features of all points; S406, the point cloud network passes the extracted global feature vector through the fully connected layer of the classification branch, and finally outputs the probability of each category.

8. The method for intelligent classification of breast cancer lesions according to claim 1, characterized in that: The specific steps of step S50 are as follows: S501, input data: dynamic ultrasound contrast imaging data is input into 3D-CNN in the form of three-dimensional tensor; S502, sending the data of the input layer to the convolution layer for calculation; S503, processing the data received by the convolution layer through an activation function to introduce nonlinearity and improve the expression ability of the model; S504: Send the data to the pooling layer to reduce the spatial dimension of the feature map, reduce the amount of calculation, and retain important features; S505, sending the data obtained in the pooling layer to the fully connected layer, flattening the output feature map of the previous layer into a one-dimensional vector, and mapping the features to a vector of fixed length through a fully connected operation; S506, output data: The output result is used for classification tasks. The output of the fully connected layer is converted into a probability distribution through the Softmax function to represent the probability of each category.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An electronic terminal, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the method according to any one of claims 1 to 8.

Images