Mammary nodule detection method and system based on ultrasonic medicine

Abstract

The invention discloses a breast nodule detection method and system based on ultrasonic medicine, and the method comprises the steps: firstly obtaining breast ultrasonic video data, and carrying out the preprocessing of interception, zooming, sampling, screening, enhancement and the like; 18 layers of ResNet18 are adopted for feature extraction, a linear classification network and an attention selection network are combined, and a detection model is constructed through optimization of a cross entropy and a center loss function. The system is correspondingly provided with a feature extraction module, a linear classification module and the like. And finally, to-be-tested video data are input into the model, so that whether breast nodules exist or not and the result of benign and malignant nodules can be quickly obtained, and efficient and accurate assistance is provided for breast disease diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the medical field, in particular to a breast nodule detection method and system based on ultrasonic medicine. BACKGROUND

[0002] Adenopathy is a common disease in women, and accurate detection and benign and malignant judgment of breast nodules are crucial for the treatment and prognosis of patients. Ultrasonic examination has become a common means of breast disease screening and diagnosis due to its non-invasive, convenient, real-time and other advantages. However, traditional ultrasonic examination relies on the experience and subjective judgment of ultrasonic physicians, and the following problems exist:

[0003] Low diagnostic efficiency: ultrasonic physicians need to observe ultrasonic videos frame by frame, which is time-consuming, and it is difficult to quickly and accurately complete the diagnosis when facing a large number of examination cases.

[0004] High misdiagnosis and missed diagnosis rate: the diagnosis result is greatly affected by the experience, visual fatigue and other factors of the physician, and for some small and atypical nodules, misdiagnosis or missed diagnosis may occur.

[0005] Lack of objectivity: different physicians may have different judgments on the same ultrasonic image, resulting in poor objectivity and consistency of the diagnosis result.

[0006] To solve the above problems, researchers have begun to explore computer technology-based breast nodule auxiliary diagnosis methods. At present, some breast ultrasonic computer-aided diagnosis systems are developed based on ultrasonic static images, but in clinical ultrasonic examination, physicians often make comprehensive judgments based on dynamic images, so it is of great significance to use artificial intelligence methods to deeply mine ultrasonic dynamic video information for breast nodule diagnosis. SUMMARY

[0007] The present application aims to at least solve one of the above technical problems.

[0008] To achieve the above purpose, the first aspect of the present application provides a breast nodule detection method and system based on ultrasonic medicine, comprising the following steps:

[0009] Obtaining breast ultrasonic video data and preprocessing

[0010] Image interception and scaling: intercept the maximum rectangular region of the original video data at the ultrasonic imaging site, and uniformly scale it to 256x256 pixel size. To reduce the data volume and retain key information, 20 frames are equally and sparsely sampled in the middle part of the video. In this way, the main information in the video can be obtained, and the computational load of subsequent processing can be reduced.

[0011] Image classification and screening: classify the intercepted ultrasound video images, manually or by preset algorithm to select the image slices containing complete nodule regions and clear image quality, and delete redundant images without the region of interest (i.e. not containing nodules). This operation can effectively remove invalid data, improve the efficiency and accuracy of subsequent model training and detection.

[0012] Data augmentation: the classified ultrasound video images are randomly cropped to 224x224 pixels in a random cropping manner. At the same time, the spatial geometric transformation method is used to perform affine transformation (including translation, rotation, scaling, etc.) on the ultrasound video images, and the transformed images are filled by interpolation method to keep the size consistent. Data augmentation can expand the data set, increase the diversity of data, and improve the generalization ability of the model, so that the model can accurately detect breast nodules in different scenarios.

[0013] The preprocessed data is trained by a transfer learning method to establish a breast nodule detection model

[0014] Feature extraction: the preprocessed video data is input into the feature extraction network ResNet to obtain the video feature vector. The present application uses ResNet18 with a network depth of 18 layers as the feature extraction network, which uses a convolutional layer with a convolution kernel size of 7x7 in the first layer to capture a larger receptive field and extract macro features in the image; then a maximum pooling layer with a step of 2 is connected to downsample the image and reduce the data dimension; in the last four layers, a convolutional layer with a convolution kernel size of 3x3 is used for feature extraction to gradually extract the detailed features in the image. After the video information is encoded by the ResNet18 network, a video feature vector sequence F={st|t∈(1,T),t∈N*} can be obtained, where st represents the feature vector corresponding to the t-th frame of the video.

[0015] Linear classification: the video feature vector is input into the linear classification network to detect the classification probability of each frame of the video. The linear classification network consists of a fully connected layer and a softmax function layer. The fully connected layer performs feature fusion on each dimension of the extracted feature vector, and through learning the weight, the classification process focuses on effective features and ignores useless features; the softmax function layer normalizes the output of the fully connected layer and converts the output into a probability distribution form to provide effective probabilities for the classification of each frame of image belonging to different categories. Given the extracted video feature vector F, the learning weight Wp∈R, and the classification probability P of the video is softmaxp(WpF+b), where b is a constant.

[0016] Attention selection: input the video feature vector into the attention selection network to obtain the weight of each frame of the video. The attention selection network also includes a fully connected layer and a softmax function layer. The fully connected layer performs feature fusion on the feature vector, and through the weight form, the model focuses on the effective features related to the breast nodule and ignores other irrelevant information; the softmax function layer normalizes the output to assign weights to each frame, highlighting the frames that are more important for nodule detection. Given the extracted video feature vector F, learn the weight Wa R, and the attention selection weight of the video A = softmaxa(WaF + b), where b is a constant.

[0017] Video detection: combine the classification probability given by the linear classification and the weight of each frame provided by the attention selection module to build a breast nodule detection model. This model considers the classification probability and weight of each frame to analyze the entire video, thereby more accurately determining whether a breast nodule exists in the video and the benignity or malignancy of the nodule.

[0018] Loss function optimization: the commonly used cross-entropy loss function and center loss function are used to optimize each module of the model. The cross-entropy loss function is used to measure the difference between the model's prediction results and the true labels, and the center loss function is used to make the feature vectors of the same class more clustered in the feature space and the feature vectors of different classes more dispersed. According to the detection probability and true value y of each video, the cross-entropy loss function and the center loss function are calculated respectively. Where N represents the size of the training data set, xi represents the feature before the fully connected layer, cyi represents the feature center of the ith class of the yith class, and lambda controls the proportion between the two. By continuously adjusting the model parameters, the loss function value is minimized, thereby improving the performance of the model.

[0019] Input the video data to be tested into the breast nodule detection model to obtain the result: input the preprocessed video data to be tested into the trained breast nodule detection model, and the model detects and judges the breast nodule in the video according to the learned features and classification rules, and outputs the result of whether there is a breast nodule and the benignity or malignancy of the nodule, providing auxiliary basis for clinical diagnosis.

[0020] (II) Breast nodule detection system based on ultrasound medicine

[0021] The breast nodule detection system of the present application comprises the following modules:

[0022] Feature extraction module: used for obtaining video feature vectors from preprocessed video data through feature extraction network ResNet, its working principle is consistent with the above feature extraction step, and the specific structure of ResNet18 network is used to encode the information of each frame of the video and extract key features.

[0023] Linear classification module: used for detecting the classification probability of each frame of the video by linear classification network of the video feature vector, containing a fully connected layer and a softmax function layer, performing feature fusion and normalization operation, and outputting the classification probability of each frame.

[0024] Attention selection module: used for obtaining the weight of each frame of the video by attention selection network of the video feature vector, composed of a fully connected layer and a softmax function layer, focusing on effective features and assigning weights to each frame.

[0025] Video detection module: used for combining the classification probability given by the linear classification and the weight of each frame provided by the attention selection module to obtain the breast nodule detection model, comprehensively evaluating the video information, and judging whether there is a breast nodule and the benignity or malignancy of the nodule.

[0026] Loss function module: used for optimizing each module of the model by adopting cross-entropy loss function and center loss function, calculating the loss function according to the detection probability and the true value, feeding back the model parameters, and improving the accuracy of the model.

[0027] The present application has the following advantages:

[0028] Data processing refinement: after obtaining the breast ultrasound video data, multidimensional preprocessing is performed. By intercepting and scaling, sparse sampling is used to reduce the data amount, retain key information, and reduce the calculation cost; image classification screening accurately removes redundancy and focuses on effective data; random cropping and affine transformation and other data enhancement methods are used to expand the diversity of the data set, significantly improve the generalization ability of the model, and enable it to also stably detect in complex actual scenes.

[0029] Model construction innovation: the breast nodule detection model is trained by adopting the transfer learning strategy, the feature extraction is performed based on the classic ResNet18 network, the convolutional layer and the pooling layer are designed in layers, the video features are efficiently extracted from macro to detail. The linear classification network and the attention selection network cooperate, the former accurately calculates the classification probability of each frame by using the fully connected layer and the softmax function, the latter dynamically allocates the weight of each frame, highlights the key frame, and finally constructs the model by comprehensively combining the two, realizes the accurate judgment of the breast nodule in the video. In addition, the cross-entropy loss function and the center loss function are combined to optimize the model, which optimizes from the prediction accuracy and the feature aggregation, and improves the performance of the model.

[0030] System architecture modularization: the detection system is composed of five modules of feature extraction, linear classification, attention selection, video detection and loss function, which have clear division and close cooperation. Each module corresponds to the key step of model training, and the modular design is convenient for system maintenance, upgrading and function expansion, and also conducive to subsequent targeted optimization according to clinical needs.

[0031] Diagnosis is efficient and accurate: through automated data processing and model analysis, compared with traditional ultrasound examination relying on the experience of doctors, the efficiency of breast nodule detection is greatly improved, and the workload of doctors is reduced. At the same time, with fine data processing and innovative model architecture, the misdiagnosis rate is reduced, and objective and reliable auxiliary basis is provided for clinical diagnosis, which has important significance for early detection and treatment of breast diseases. BRIEF DESCRIPTION OF DRAWINGS

[0032] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0033] Figure 1 is a schematic diagram of a system according to the present application; DETAILED DESCRIPTION

[0034] Embodiments of the present application are described in detail below with reference to examples shown in the accompanying drawings, in which the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0035] A breast nodule detection method and system based on ultrasound medicine are described below in conjunction with the accompanying drawings.

[0036] I. Data collection

[0037] Breast ultrasound video data was collected from three different regions of three first-class hospitals, a total of 2000 cases of ultrasound video were obtained, covering cases of different age stages (18-70 years old), different breast conditions (benign nodules, malignant nodules, normal breasts). Video acquisition devices include different models of ultrasound instruments such as Philips EPIQ 7, Siemens Acuson S2000, etc., the frame rate is 30 frames per second, the video duration is 10-30 seconds, and the data has wide diversity and representativeness.

[0038] II. Data preprocessing

[0039] Image cropping and scaling: for each ultrasound video, the largest rectangle containing the complete imaging area is automatically identified and cropped at the ultrasound imaging site, and the cropped image is uniformly scaled to 256x256 pixels. For example, for a video with a duration of 20 seconds and a frame rate of 30 frames / second, the total number of frames is 600, 20 frames are equally spaced and sparsely sampled in the middle part of the video (i.e. in the range of 150-450 frames), and the sampling interval is 15 frames, which reduces the data volume while retaining the key information of the video.

[0040] Image classification and screening: The 20-frame image is preliminarily classified by using a deep learning-based image classification algorithm (such as a pre-trained ResNet50), and image slices containing complete nodule regions and having a higher image clarity than a set threshold (such as a clarity score of 70 points or more, with a full score of 100 points) are identified. For images that do not meet the conditions, automatic marking and deletion are performed, and the proportion of valid images is finally retained at about 60%.

[0041] Data augmentation: The screened images are randomly cropped and resized to 224x224 pixels. At the same time, affine transformation operations are performed, including random translation (with a horizontal and vertical translation range of ±10 pixels), random rotation (with a rotation angle range of -15°-15°), and random scaling (with a scaling ratio range of 0.8-1.2 times), and bilinear interpolation is used to fill the transformed images to maintain a size of 224x224 pixels. After data augmentation, each original ultrasound video image can be expanded to 500 images, greatly enriching the data set.

[0042] III. Model training

[0043] Feature extraction: The preprocessed video data is input into the feature extraction network ResNet18. The first layer of the ResNet18 network uses a 7x7 convolution kernel with a step size of 2 to perform convolution operations on the input image and extract macro features of the image. Then, a maximum pooling layer with a step size of 2 is connected to reduce the image size from 128x128 to 64x64 and reduce the data dimension. In the subsequent four layers, each layer is composed of multiple 3x3 convolution layers, which gradually extract detailed features of the image through continuous convolution operations. For each video data containing 20 images, after processing by the ResNet18 network, 20 video feature vectors are obtained, forming a video feature vector sequence F.

[0044] Linear classification: The video feature vector sequence F is input into the linear classification network. The number of neurons in the fully connected layer of the network is set to 512 to perform feature fusion on the input feature vectors and learn the relationship between different feature dimensions. Then, the output of the fully connected layer is converted into a probability distribution form by a softmax function layer, and the classification probability P of each frame of the video belonging to different categories is calculated, including “benign nodule”, “malignant nodule”, and “normal tissue”.

[0045] Attention selection: The video feature vector sequence F is simultaneously input into the attention selection network. Similarly, the number of neurons in the fully connected layer is 512 to further fuse the feature vectors and highlight the effective features related to breast nodules. The softmax function layer assigns weights A to each frame of the video based on the fused features, so that the model can focus on the frames that are more important for nodule detection in subsequent processing.

[0046] Video detection and model construction: combine the classification probability P output by the linear classification network and the frame weight A provided by the attention selection network to construct a breast nodule detection model. The specific calculation method is to weight and sum the classification probability of each frame to obtain the comprehensive probability of the entire video for different categories, so as to judge whether there is a breast nodule in the video and the benignity and malignancy of the nodule.

[0047] Loss function optimization: cross-entropy loss function and center loss function are used to optimize the model. In the training process, the weight λ1 of the cross-entropy loss function is set to 0.8, and the weight λ2 of the center loss function is set to 0.2. The Adam optimizer is used, and the initial learning rate is set to 0.001. Every 10 epochs, the learning rate is decayed to 0.9 times of the original. Through continuous iterative training (a total of 50 epochs), according to the calculated loss value, the model parameters are adjusted using the back propagation algorithm to minimize the loss function value and improve the performance of the model.

[0048] Four, model testing and application

[0049] Model testing: from the collected 2000 cases of data, 500 cases of video data not participating in training are selected as the test set. After processing these test video data according to the above data preprocessing steps, they are input into the trained breast nodule detection model. By comparing with the actual clinical pathological diagnosis results, the performance indicators of the model are evaluated, and finally the accuracy of the model is 92%, the recall rate is 88%, and the F1 value is 90%.

[0050] Clinical application: practical application test in the ultrasonic department of a hospital. When doctors perform breast ultrasound examination for patients, they transmit the ultrasound video data to the detection system of the application in real time. The system can complete the processing and analysis of video data within 1 minute, output the detection results, and prompt whether there is a breast nodule and the benignity and malignancy of the nodule. Doctors combine the system detection results with their own clinical experience to comprehensively judge the patient's condition, significantly improve the diagnosis efficiency and accuracy, and provide a strong reference basis for the follow-up treatment of patients.

[0051] In the description of the present specification, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise specifically limited.

[0052] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are contained in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0053] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limiting the present application, and the person skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for detecting breast nodules based on ultrasound medicine, characterized in that, Includes the following steps: Acquire breast ultrasound video data and perform preprocessing: crop the largest rectangle of the original video data at the ultrasound imaging area, scale it to 256×256, and sparsely sample 20 frames at equal intervals in the middle of the video. The captured images are classified, and image slices containing complete nodular regions and clear images are selected. Redundant images are deleted. The classified images are randomly cropped into 224×224 pixels and then subjected to affine transformation and interpolation filling. A breast nodule detection model was established by training the preprocessed data using transfer learning: Preprocessed video data was processed through a ResNet feature extraction network to obtain video feature vectors; The video feature vectors are used to detect the classification probability of each frame of the video through a linear classification network; the video feature vectors are then used to obtain the weights of each frame of the video through an attention selection network; the classification probabilities given by the linear classification and the frame weights provided by the attention selection module are combined to obtain a breast nodule detection model; the cross-entropy loss function and the center loss function are used to optimize each module of the model. The video data to be tested is input into the breast nodule detection model to obtain the results of whether breast nodules exist and whether the nodules are benign or malignant.

2. The method for detecting breast nodules based on ultrasound medicine according to claim 1, characterized in that, The feature extraction network ResNet is a ResNet18 with a depth of 18 layers. Its first layer uses a convolutional layer with a kernel size of 7×7, followed by a max pooling layer with a stride of 2, and the last four layers use convolutional layers with a kernel size of 3×3 for feature extraction.

3. The method for detecting breast nodules based on ultrasound medicine according to claim 1, characterized in that, The linear classification network includes fully connected layers and softmax function layers. The fully connected layers perform feature fusion on the feature vectors, and the softmax function layers normalize the output to provide classification probabilities.

4. The method for detecting breast nodules based on ultrasound medicine according to claim 1, characterized in that, The attention selection network includes fully connected layers and softmax function layers. The fully connected layers perform feature fusion on the feature vectors, and the softmax function layers normalize the output to provide weights for each frame.

5. A breast nodule detection system based on ultrasound medicine, characterized in that, include: The feature extraction module is used to process the preprocessed video data through the ResNet feature extraction network to obtain video feature vectors; The linear classification module is used to detect the classification probability of each frame of the video by passing the video feature vector through a linear classification network; The attention selection module is used to obtain the weights of each frame of the video by passing the video feature vector through the attention selection network; The video detection module combines the classification probabilities given by the linear classification module with the frame weights provided by the attention selection module to obtain a breast nodule detection model. The loss function module is used to optimize the various modules of the model using the cross-entropy loss function and the center loss function.

6. The breast nodule detection system based on ultrasound medicine according to claim 5, characterized in that, The feature extraction network ResNet in the feature extraction module is a ResNet18 with a network depth of 18 layers. Its first layer uses a convolutional layer with a kernel size of 7×7, followed by a max pooling layer with a stride of 2, and the last four layers use convolutional layers with a kernel size of 3×3 for feature extraction.

7. The breast nodule detection system based on ultrasound medicine according to claim 5, characterized in that, The linear classification network in the linear classification module includes a fully connected layer and a softmax function layer. The fully connected layer performs feature fusion on the feature vector, and the softmax function layer normalizes the output to provide classification probability.

8. The breast nodule detection system based on ultrasound medicine according to claim 5, characterized in that, The attention selection network in the attention selection module includes a fully connected layer and a softmax function layer. The fully connected layer performs feature fusion on the feature vector, and the softmax function layer normalizes the output to provide weights for each frame.

Images