A training method for a breast tumor detection model and a breast tumor detection method

Abstract

The present application discloses a training method for a breast tumor detection model and a breast tumor detection method. The method includes controlling a feature module to determine a plurality of first feature maps of training breast images; controlling a pyramid module to determine a plurality of second feature maps based on the plurality of first feature maps; controlling a prediction module to determine a positive sample set and a negative sample set based on the plurality of second feature maps and determine a loss function based on the positive sample set and the negative sample set; and training a breast tumor detection model to be trained based on the loss function to obtain a breast tumor detection model. The present application uses a three-dimensional network model as the breast tumor detection model, enabling the breast tumor detection model to learn the global spatial information of full-field digital breast ultrasound images and improving the accuracy of lesion area detection. At the same time, the present application uses an adaptive threshold to select positive and negative samples, ensuring that all positive samples are around the labeled area and obtaining more representative positive samples to improve the model performance of the breast tumor detection model.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a training method for a breast tumor detection model and a breast tumor detection method. Background Art

[0002] Automated breast ultrasound (ABUS) can reconstruct the complete three-dimensional breast structure in one scan by using a high-frequency and wide-band probe. More importantly, it has a reproducible standardized acquisition process and image interpretation. Based on three-dimensional images, doctors can obtain the precise location, morphology, and edge information of the lesion. In addition, it can also provide coronal imaging to provide doctors with more and more accurate diagnostic information and improve diagnostic accuracy. Therefore, some scholars at home and abroad have carried out research on computer-aided diagnosis methods (CAD) based on fully automatic breast ultrasound images, which mainly follow the process of preprocessing, candidate region extraction, feature extraction and screening, image classification or segmentation, and evaluation.

[0003] However, the computer-aided diagnosis models currently used generally use traditional machine learning and image processing methods to screen breast tumor lesions on a two-dimensional plane, ignoring the global spatial information in fully-automatic breast ultrasound images. As a result, when existing computer-aided diagnosis models are applied to fully-automatic breast ultrasound images, there is often a problem of low detection accuracy.

[0004] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a breast tumor detection model training method and a breast tumor detection method in view of the deficiencies of the existing technology.

[0006] In order to solve the above technical problems, the first aspect of the embodiment of the present application provides a training method for a breast tumor detection model, wherein the breast tumor detection model is a three-dimensional network model, and includes a feature module, a pyramid module and a prediction module, and the training method includes:

[0007] Inputting a training breast image in a first training sample set into a feature module, and determining a plurality of first feature maps corresponding to the training breast image through the feature module;

[0008] Inputting a plurality of first feature maps into a pyramid module, and determining a plurality of second feature maps through the pyramid module;

[0009] Inputting a plurality of second feature maps into a prediction module, determining a positive sample set and a negative sample set corresponding to the training breast image through the prediction module, and determining a loss function based on the positive sample set and the negative sample set, wherein the positive sample set and the negative sample set are obtained by dividing the anchor point boxes in each second feature map based on an adaptive threshold, and the adaptive threshold is determined based on the plurality of second feature maps;

[0010] The breast tumor detection model to be trained is trained based on the loss function to obtain a breast tumor detection model.

[0011] The training method of the breast tumor detection model, wherein, before inputting the training breast image in the first training sample set into the feature module and determining the first feature graphs corresponding to the training breast image by the feature module, the method further comprises:

[0012] Acquire an initial sample set, wherein the initial sample set includes a plurality of initial training images, and each of the plurality of initial training images carries a plurality of lesion areas;

[0013] For each initial training image in the initial sample set, the shortest sides of several lesion regions are selected, and based on the shortest sides of the regions, it is determined whether the initial training image needs to be enhanced by Mosaic;

[0014] When the initial training image needs to be enhanced by Mosaic, a first image region is selected around the lesion region corresponding to the shortest side of the region, and a preset number of second image regions are randomly selected in the initial training image; data enhancement is performed on the first image region and the preset number of second image regions respectively, and the first image region after data enhancement and each second image region are spliced ​​to obtain a training breast image;

[0015] When the initial training image does not need to be enhanced by Mosaic, randomly selecting a third image region from the initial training image and performing data enhancement on the third image region, and using the data enhanced third image region as a training breast image;

[0016] The set consisting of all the obtained training breast images is used as the first training sample set.

[0017] The training method of the breast tumor detection model, wherein the initial training image is a fully automatic breast ultrasound image, and the area size of the third image area is equal to the image size of the spliced ​​image obtained by splicing the first image area after mosaic enhancement and each second image area.

[0018] The training method of the breast tumor detection model, wherein the determining of the positive sample set and the negative sample set corresponding to the training breast image by the prediction module specifically comprises:

[0019] Controlling the prediction module to determine candidate positive sample sets corresponding to respective second feature maps, wherein the candidate positive sample sets include a plurality of anchor point boxes;

[0020] An adaptive threshold is determined based on all determined candidate positive sample sets, and the anchor point boxes included in the second feature map are divided into positive samples and negative samples based on the adaptive threshold to obtain a positive sample set and a negative sample set.

[0021] The training method of the breast tumor detection model, wherein an adaptive threshold is determined based on all determined candidate positive sample sets, and the anchor point boxes included in the second feature map are divided into positive samples and negative samples based on the adaptive threshold to obtain positive sample sets and negative sample sets, specifically comprising:

[0022] Calculate the first IoU value between each candidate positive sample in each candidate positive sample set and each lesion area corresponding to the training breast image, and the second IoU value between the anchor point box in each second feature map and each lesion area corresponding to the training breast image;

[0023] Calculate the mean and variance of all the first IoU values ​​obtained, and determine the adaptive threshold according to the mean and variance;

[0024] The anchor box whose second IoU value is greater than the adaptive threshold is taken as a positive sample, and the anchor box whose second IoU value is less than or equal to the adaptive threshold is taken as a negative sample to obtain a positive sample set and a negative sample set.

[0025] The training method of the breast tumor detection model, wherein, after the breast tumor detection model to be trained is trained based on the loss function to obtain the breast tumor detection model, the method further comprises:

[0026] Acquire a second training sample set, wherein the second training sample set includes a plurality of training data pairs, and each of the plurality of training data pairs includes a false positive training image and a lesion training image;

[0027] Inputting the training data pairs of the two training sample sets into a feature extraction module in a preset network model, and determining a first feature vector corresponding to a false positive training image and a second feature vector corresponding to a lesion training image in the training data pairs through the feature extraction module;

[0028] Inputting the first feature vector and the second feature vector into an attention pairwise interaction module in a preset network model, and determining four attention feature vectors through the attention pairwise interaction module;

[0029] Inputting the four attention feature vectors into a classification module in a preset network model, and determining the prediction category corresponding to each attention feature vector through the classification module;

[0030] The preset network model is trained based on each prediction category, and after the preset network model training is completed, the attention pair interaction module in the preset network model is removed to obtain a classification model;

[0031] The breast tumor detection model is combined with the classification model, and the combined network model is used as the breast tumor detection model.

[0032] The training method of the breast tumor detection model, wherein the attention pair interaction module includes a mutual vector learning unit, a gate vector generation unit and a pair interaction unit, the first feature vector and the second feature vector are input into the attention pair interaction module in the preset network model, and the four attention feature vectors are determined by the attention pair interaction module specifically including:

[0033] Controlling the mutual vector learning unit to generate a fusion vector based on the first feature vector and the second feature vector;

[0034] Controlling the gate vector generating unit to generate a first gate vector based on the first feature vector and the fusion vector, and to generate a second gate vector based on the second feature vector and the fusion vector;

[0035] The paired interaction units are controlled to generate attention feature vectors based on the first feature vector and the first gate vector, the first feature vector and the second gate vector, the second feature vector and the first gate vector, and the second feature vector and the second gate vector, respectively, to obtain four attention feature vectors.

[0036] A second aspect of an embodiment of the present application provides a breast tumor detection method, the method using a breast tumor detection model trained using the training method of the breast tumor detection model as described above, the method comprising:

[0037] Inputting the breast ultrasound image to be detected into the breast tumor detection model;

[0038] The breast tumor detection model is used to output a plurality of diseased areas of the breast ultrasound image.

[0039] The breast tumor detection method, wherein the breast tumor detection model includes a classification model trained by the training method of the breast tumor detection model as described above; the method further includes:

[0040] Selecting a plurality of lesion images from the breast ultrasound image based on a plurality of lesion areas;

[0041] Controlling the classification model to determine the lesion category of each disease area based on each lesion image;

[0042] The lesion regions whose lesion categories are normal regions in the plurality of lesion regions are classified to obtain the target lesion region of the breast ultrasound image.

[0043] A third aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0044] The communication bus realizes the connection and communication between the processor and the memory;

[0045] When the processor executes the computer-readable program, the processor implements the steps of the training method of the breast tumor detection model described above, and / or implements the steps of the breast tumor detection method described above.

[0046] Beneficial effects: Compared with the prior art, the present application provides a training method for a breast tumor detection model and a breast tumor detection method, the method comprising inputting a training breast image in a first training sample set into a feature module, determining a plurality of first feature maps corresponding to the training breast image through the feature module; inputting a plurality of first feature maps into a pyramid module, determining a plurality of second feature maps through the pyramid module; inputting a plurality of second feature maps into a prediction module, determining a positive sample set and a negative sample set corresponding to the training breast image through the prediction module, and determining a loss function based on the positive sample set and the negative sample set; training a breast tumor detection model to be trained based on the loss function to obtain a breast tumor detection model. The present application adopts a three-dimensional network model as a breast tumor detection model, so that the breast tumor detection model can learn the global spatial information of the fully automatic breast ultrasound image, and improves the accuracy of the lesion area detection. At the same time, the present application also adopts an adaptive threshold to select positive samples and negative samples, ensuring that all positive samples are around the marked area, and obtaining more representative positive samples, thereby improving the model performance of the trained breast tumor detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0048] Figure 1 A flowchart of the training method for the breast tumor detection model provided in this application.

[0049] Figure 2This is a flow chart of the principles of the training method for the breast tumor detection model provided in this application.

[0050] Figure 3 This is the working principle diagram of FPN.

[0051] Figure 4 This is the working principle diagram of PANet.

[0052] Figure 5 This is the working principle diagram of NAS-FPN.

[0053] Figure 6 This is the working principle diagram of BiFPN.

[0054] Figure 7 Model structure principle diagram of the preset network model in the training method of the breast tumor detection model provided in this application

[0055] Figure 8 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0056] The present application provides a training method for a breast tumor detection model and a breast tumor detection method. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0058] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0059] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0060] The inventors have found through research that fully automated breast ultrasound (ABUS) can reconstruct a complete three-dimensional breast structure in one scan by using a high-frequency wide-band probe. More importantly, it has a reproducible standardized acquisition process and image interpretation. Based on three-dimensional images, doctors can obtain the precise location, morphology, and edge information of the lesion. In addition, coronal imaging can be provided to provide doctors with more and more accurate diagnostic information, thereby improving the accuracy of diagnosis. As a result, some scholars at home and abroad have conducted research on computer-aided diagnosis methods (CAD) based on fully automated breast ultrasound images, which mainly follow the process of preprocessing, candidate region extraction, feature extraction and screening, image classification or segmentation, and evaluation.

[0061] However, the computer-aided diagnosis models currently used generally use traditional machine learning and image processing methods to screen breast tumor lesions on a two-dimensional plane, ignoring the global spatial information in fully-automatic breast ultrasound images. As a result, when existing computer-aided diagnosis models are applied to fully-automatic breast ultrasound images, there is often a problem of low detection accuracy.

[0062] In order to solve the above problems, in an embodiment of the present application, the training breast images in the first training sample set are input into the feature module, and several first feature maps corresponding to the training breast images are determined by the feature module; several first feature maps are input into the pyramid module, and several second feature maps are determined by the pyramid module; several second feature maps are input into the prediction module, and the positive sample set and negative sample set corresponding to the training breast images are determined by the prediction module, and the loss function is determined based on the positive sample set and the negative sample set; the breast tumor detection model to be trained is trained based on the loss function to obtain the breast tumor detection model. The present application adopts a three-dimensional network model as a breast tumor detection model, so that the breast tumor detection model can learn the global spatial information of the fully automatic breast ultrasound image, and improve the accuracy of the lesion area detection. At the same time, the present application also adopts an adaptive threshold to select positive samples and negative samples, ensuring that all positive samples are around the marked area, and obtaining more representative positive samples, thereby improving the model performance of the trained breast tumor detection model.

[0063] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0064] The present embodiment provides a training method for a breast tumor detection model, wherein the breast tumor detection model is a three-dimensional network model, and includes a feature module, a pyramid module, and a prediction module, wherein the feature module is connected to the pyramid module, and the pyramid module is connected to the prediction module. The feature module is used to select image features of a training breast image to obtain a plurality of first feature maps, the pyramid module is used to fuse a plurality of first feature maps to obtain a plurality of second feature maps, and the prediction module is used to determine a positive sample set and a negative sample set based on a plurality of second feature maps, and determine a loss function based on the positive sample set and the negative sample set, so as to train the breast tumor detection model to be trained based on the loss function.

[0065] This embodiment provides a breast tumor detection model training method, such as Figure 1 and Figure 2 As shown, the method includes:

[0066] S10, inputting the training breast images in the first training sample set into a feature module, and determining a plurality of first feature maps corresponding to the training breast images through the feature module.

[0067] Specifically, the first training sample set includes a number of training breast images, each of which is determined based on a breast ultrasound image acquired by fully automatic breast ultrasound, and each of which includes a number of lesion areas. The first feature maps are determined by a feature module, and the image sizes of the first feature maps in the first feature maps are different. It can be understood that the first feature maps are output by different network layers of the feature module.

[0068] In an implementation of this embodiment, the feature module may include a plurality of downsampling units, the number of which is greater than the number of the first feature maps, and the first feature maps are output items of some of the downsampling units in the plurality of downsampling units, wherein the downsampling unit may adopt a ResBlock module, etc. For example, Figure 2 As shown, the feature module includes 5 cascaded downsampling units, the number of first feature maps is 4, and the 4 first feature maps are the output items of the last four feature extraction units. In addition, the inventors have found through research that the Z-axis direction in the lesion area in the breast ultrasound image is the shortest axis of the lesion area, so that in the feature module, the downsampling parameters of the first downsampling unit can be configured as (1, 2, 2), and the downsampling parameters of other downsampling units can be configured as (2, 2, 2), so that the resolution of the downsampled image obtained by downsampling can be guaranteed.

[0069] In an implementation of this embodiment, before inputting the training breast image in the first training sample set into the feature module and determining the first feature maps corresponding to the training breast image through the feature module, the method further includes:

[0070] Obtain an initial sample set;

[0071] For each initial training image in the initial sample set, the shortest sides of several lesion regions are selected, and based on the shortest sides of the regions, it is determined whether the initial training image needs to be enhanced by Mosaic;

[0072] When the initial training image needs to be enhanced by Mosaic, a first image region is selected around the lesion region corresponding to the shortest side of the region, and a preset number of second image regions are randomly selected in the initial training image; data enhancement is performed on the first image region and the preset number of second image regions respectively, and the first image region after data enhancement and each second image region are spliced ​​to obtain a training breast image;

[0073] When the initial training image does not need to be enhanced by Mosaic, randomly selecting a third image region from the initial training image and performing data enhancement on the third image region, and using the data enhanced third image region as a training breast image;

[0074] The set consisting of all the obtained training breast images is used as the first training sample set.

[0075] Specifically, the initial sample set includes a number of initial training images, each of which carries a number of lesion areas, wherein the initial training images are fully automatic breast ultrasound images. In other words, each initial training image is a breast ultrasound image acquired by fully automatic breast ultrasound.

[0076] Since the lesion areas in the breast ultrasound images obtained by fully automatic breast ultrasound acquisition are generally small lesion areas, and the small lesion areas are close to the surrounding tissues of the small lesion areas and have low resolution, after several downsamplings through the feature module, the image information of the small lesion areas will be seriously lost, so that the small lesion areas are easily missed. Therefore, this embodiment determines whether the initial training image needs to be enhanced by Mosaic based on the shortest side of the area in several lesion areas, which can improve the difference between the small lesion areas and the surrounding tissues of the small lesion areas, thereby improving the probability of small lesion areas being detected, enriching the distribution of small lesion areas in the training sample set, and thus improving the robustness of the breast tumor detection model to small lesion areas.

[0077] The shortest side of the region is the minimum value among the shortest sides of the first region of each lesion area in the plurality of lesion areas. That is to say, the process of determining the shortest side of the region can be: first select the shortest side of the first region of each lesion area in the plurality of lesion areas, then select the minimum value among all the selected first region shortest sides, and use the selected minimum value as the shortest side of the region in the plurality of lesion areas. In addition, after the shortest side of the region is determined, the probability of Mosaic enhancement can be determined based on the shortest side of the region, and then whether Mosaic enhancement is required can be determined based on the probability of Mosaic enhancement. The calculation formula of the probability of Mosaic enhancement can be:

[0078]

[0079] Among them, Prop(Mosaic) represents the probability of mosaic enhancement, th represents the adaptive threshold (for example, 16 pixels, etc.), and min(edge) represents the shortest edge of the region.

[0080] In addition, after obtaining the probability of Mosaic enhancement, determining whether Mosaic enhancement is required based on the probability of Mosaic enhancement can be to compare the probability of Mosaic enhancement with a preset probability threshold. When the probability of Mosaic enhancement is greater than the preset probability threshold, it indicates that the initial training image needs to be Mosaic enhanced. On the contrary, when the probability of Mosaic enhancement is less than or equal to the preset probability threshold, it indicates that the initial training image does not need to be Mosaic enhanced. The preset probability prediction is preset and can be determined according to actual needs, for example, 0.75, 0.5, etc. Of course, in actual applications, when judging whether the initial training image needs to be Mosaic enhanced based on the shortest side of the region, other methods can also be used to determine. For example, the shortest side of the region can be compared with a preset length. When the shortest side of the region is less than the preset length, it indicates that the initial training image needs to be Mosaic enhanced. On the contrary, when the shortest side of the region is greater than or equal to the preset length, it indicates that the initial training image does not need to be Mosaic enhanced.

[0081] The first image region includes the lesion region corresponding to the shortest side of the region, that is, when the initial training image needs to be enhanced by Mosaic, an image region including the lesion region corresponding to the shortest side of the region is randomly selected, and the selected image region is used as the first image region, wherein the region size of the first image region is smaller than the image size of the training breast image, for example, the region size of the first image region is 0.4-0.6 times the image size of the training breast image, etc. In addition, after selecting the first image region, a number of second image regions are randomly selected in the initial training image, the region size of each of the number of second image regions is smaller than the training breast image, and the image size of the spliced ​​image obtained by splicing the first image region and the number of second image regions is equal to the image size of the training breast image, wherein the number of regions of the number of second image regions can be set according to actual needs, for example, the number of regions of the number of second image regions is 4, etc.

[0082] Furthermore, after selecting the first image region and several second image regions, data enhancement is performed on the first image region and each second image region respectively, and the first image region and each second image region after data enhancement are spliced ​​to obtain a spliced ​​image, and the spliced ​​image is used as a training breast image, wherein the data enhancement may include random rotation and random brightness adjustment, etc. The present application increases the batch size by independently performing data enhancement on the first image region and several second image regions. This is because the training batch size can only be 1 due to the limitation of GPU video memory, and the data for mosaic enhancement is composed of several image regions, and each image region is independently data enhanced, so it is equivalent to implicitly increasing the batch size when the model batchnorm is used.

[0083] Furthermore, when the initial training image does not need to be enhanced by Mosaic, a third image region can be randomly selected from the initial training image, and data enhancement can be performed on the third image region, and the third image region after data enhancement can be used as a training breast image, wherein the size of the selected third image region is equal to the image size of the spliced ​​image obtained by splicing the first image region after data enhancement and each second image region. Thus, the spliced ​​image obtained by splicing the third image region and the first image region after data enhancement and each second image region can be used as a training breast image to obtain a first training sample set.

[0084] S20, inputting a plurality of first feature maps into a pyramid module, and determining a plurality of second feature maps through the pyramid module.

[0085] Specifically, the pyramid module can be used as follows Figure 3 The FPN (feature pyramid network) shown in Figure 4 The PANet shown in Figure 5 NAS-FPN as shown and Figure 6 BIFPN is shown in the figure. However, FPN uses a top-down approach to feature fusion, and this single-channel information flow also limits the performance of FPN; PANet adds an additional bottom-up feature fusion network based on FPN, and NAS-FPN is obtained using a neural architecture search. Both PANet and NAS-FPN lack interpretability, and the search process of NAS-FPN takes a lot of time. Therefore, in a typical implementation, the pyramid module uses BIFPN, which can fuse more features without increasing the computational cost too much, thereby improving the characteristic information carried by each second feature map.

[0086] In addition, when the pyramid module fuses the first feature maps, since different first feature maps have different resolutions and usually contribute differently to the output features, this embodiment uses a fast normalization method to add an additional weight coefficient to each first feature map to balance the importance of different features, wherein the calculation formula of the weight coefficient of each first feature map can be:

[0087]

[0088] Among them, ω i For learnable weights, we use ReLu to ensure ω i ≥0; ε is the preset data used to avoid numerical instability, for example, ε=0.0001; o is the weight coefficient of the first special graph; I i is the i-th first feature map.

[0089] S30, inputting a plurality of second feature maps into a prediction module, inputting a plurality of second feature maps into a prediction module, determining a positive sample set and a negative sample set corresponding to the training breast image through the prediction module, and determining a loss function based on the positive sample set and the negative sample set.

[0090] Specifically, the image size of each of the several second feature maps is different, the positive sample set includes several first anchor boxes, and the negative sample set also includes several second anchor boxes. Each of the first anchor boxes in the several first anchor boxes is contained in a second feature map among the several second feature maps, and each of the second anchor boxes in the several second anchor boxes is contained in a second feature map among the several second feature maps, and each first anchor box and each second anchor box are different from each other, wherein the union of the positive sample set and the negative sample set is equal to the set formed by the anchor boxes included in each second feature map.

[0091] In one implementation of this embodiment, the positive sample set and the negative sample set are obtained by dividing the anchor point boxes in each second feature map based on an adaptive threshold, and the adaptive threshold is determined based on several second feature maps, that is, the division basis of the positive sample and the negative sample is the adaptive threshold, and the adaptive threshold is determined based on several second feature maps. This implementation adopts an adaptive threshold to determine the positive sample set and the negative sample set, which can ensure that all positive samples are around the lesion area, and more representative positive samples can be obtained according to the adaptive threshold, thereby enhancing the learning ability of the network. In addition, the use of an adaptive threshold does not require manual setting of a fixed IoU threshold, but only needs to set the number of candidate samples included in the candidate positive sample set, which reduces the setting of hyperparameters of the network model and facilitates network optimization.

[0092] In an implementation of this embodiment, determining the positive sample set and the negative sample set corresponding to the training breast image by the prediction module specifically includes:

[0093] Controlling the prediction module to determine candidate positive sample sets corresponding to respective second feature maps, wherein the candidate positive sample sets include a plurality of anchor point boxes;

[0094] An adaptive threshold is determined based on all determined candidate positive sample sets, and the anchor point boxes included in the second feature map are divided into positive samples and negative samples based on the adaptive threshold to obtain a positive sample set and a negative sample set.

[0095] Specifically, each candidate positive sample in the candidate positive sample set is an anchor box, and the anchor box is contained in the second feature map corresponding to the candidate positive sample set, that is, the candidate positive sample set is determined by some anchor boxes in the second feature map. In one implementation, the candidate positive sample is determined based on the anchor box and the lesion area corresponding to the training breast image, wherein the process of determining the candidate positive sample can be: respectively calculating the candidate distances between each anchor box in the second feature map and each lesion area corresponding to the training breast image, and selecting a preset number of candidate distances from all the calculated candidate distances in order from small to large, and taking the anchor box corresponding to the selected candidate distance as the candidate positive sample to obtain the candidate positive sample set, wherein the candidate distance can be the L2 distance between the center of each anchor box and the regional center of the lesion area.

[0096] In an implementation of this embodiment, determining an adaptive threshold based on all determined candidate positive sample sets, and dividing the anchor point box included in the second feature map into positive samples and negative samples based on the adaptive threshold to obtain the positive sample set and the negative sample set specifically includes:

[0097] Calculate the first IoU value between each candidate positive sample in each candidate positive sample set and each lesion area corresponding to the training breast image, and the second IoU value between the anchor point box in each second feature map and each lesion area corresponding to the training breast image;

[0098] Calculate the mean and variance of all the first IoU values ​​obtained, and determine the adaptive threshold according to the mean and variance;

[0099] The anchor box whose second IoU value is greater than the adaptive threshold is taken as a positive sample, and the anchor box whose second IoU value is less than or equal to the adaptive threshold is taken as a negative sample to obtain a positive sample set and a negative sample set.

[0100] Specifically, the first IoU value is used to represent the IoU value of the selected positive sample and the lesion area, and the second IoU value is the IoU value of the anchor box in the second feature map in the lesion area. The mean is the mean of all first IoU values, the variance is the variance of all first IoU values, and the adaptive threshold can be equal to the sum of the mean and the variance, or the adaptive threshold can be obtained by weighting the mean and the variance. After obtaining the adaptive threshold, each second IoU value is compared with the adaptive threshold to obtain all second IoU values ​​greater than the adaptive threshold, and all second IoU values ​​less than or equal to the adaptive threshold, and the set consisting of the anchor boxes of the second IoU values ​​greater than the adaptive threshold is used as the positive sample set; the set consisting of the anchor boxes of the second IoU values ​​less than or equal to the adaptive threshold is used as the negative sample set.

[0101] In one implementation of this embodiment, after determining the positive sample set and the negative sample set, a loss function can be determined based on the positive sample set and the negative sample set, wherein the loss function includes a classification loss term and a regression loss term, the classification loss term uses focal loss, and the regression loss term uses smooth L1loss.

[0102] S40: training the breast tumor detection model to be trained based on the loss function to obtain a breast tumor detection model.

[0103] Specifically, after obtaining the loss function, the model parameters of the breast tumor detection model to be trained are optimized based on the loss function, and step S10 is continued until the breast tumor detection model to be trained meets the training end condition, wherein the training end condition is that the model parameters meet the preset requirements, or the number of training times reaches the threshold number.

[0104] In one implementation of this embodiment, the breast tumor detection model is also combined with a classification model, and the classification model is trained based on a preset network model, such as Figure 7 As shown, the prediction network model includes the prediction network model including a feature extraction module, an attention pair interaction module and a classification module. In addition, the training process of the classification model can be after the training process of the breast tumor detection model, or of course before. Here, it is after the training process of the breast tumor detection model. Correspondingly, after the breast tumor detection model to be trained is trained based on the loss function to obtain the breast tumor detection model, the method also includes:

[0105] Obtaining a second training sample set;

[0106] Inputting the training data pairs of the two training sample sets into a feature extraction module in a preset network model, and determining a first feature vector corresponding to a false positive training image and a second feature vector corresponding to a lesion training image in the training data pairs through the feature extraction module;

[0107] Inputting the first feature vector and the second feature vector into an attention pairwise interaction module in a preset network model, and determining four attention feature vectors through the attention pairwise interaction module;

[0108] Inputting the four attention feature vectors into a classification module in a preset network model, and determining the prediction category corresponding to each attention feature vector through the classification module;

[0109] The preset network model is trained based on each prediction category, and after the preset network model training is completed, the attention pair interaction module in the preset network model is removed to obtain a classification model;

[0110] The breast tumor detection model is combined with the classification model, and the combined network model is used as the breast tumor detection model.

[0111] Specifically, the second training sample set includes several training data pairs, each of which includes a false positive training image and a lesion training image, wherein the false positive training image is an image in which the lesion area in the training image is a normal tissue area but is determined as a tumor area by a breast tumor detection model, and the lesion training image is an image in which the lesion area in the training image is a tumor area but is determined as a tumor area by a detection model. In addition, the false positive training image carries a lesion area, the lesion training image also carries a lesion area, and the similarity between the false positive training image and the lesion training image is greater than a preset similarity threshold to improve the effectiveness of the candidate attention interaction behavior. In addition, the feature extraction module is used to extract feature maps of false positive training images and lesion training images. For example, the feature extraction module can use vgg13, etc.

[0112] In an implementation of this embodiment, when the preset network model is trained based on each prediction category, cross entropy is used as a classification loss function, wherein the calculation formula of the classification loss function may be:

[0113]

[0114] Among them, L ce represents the classification loss function, Represents the predicted classification corresponding to each attention feature vector, y 1 represents the annotated category of the false positive training image, y 2Indicates the annotated category of the lesion training image.

[0115] In one implementation of this embodiment, Figure 7 As shown, the attention pair interaction module includes a mutual vector learning unit, a gate vector generation unit and a pair interaction unit, and the first feature vector and the second feature vector are input into the attention pair interaction module in the preset network model, and the four attention feature vectors are determined by the attention pair interaction module. Specifically, it includes:

[0116] Controlling the mutual vector learning unit to generate a fusion vector based on the first feature vector and the second feature vector;

[0117] Controlling the gate vector generating unit to generate a first gate vector based on the first feature vector and the fusion vector, and to generate a second gate vector based on the second feature vector and the fusion vector;

[0118] The paired interaction units are controlled to generate attention feature vectors based on the first feature vector and the first gate vector, the first feature vector and the second gate vector, the second feature vector and the first gate vector, and the second feature vector and the second gate vector, respectively, to obtain four attention feature vectors.

[0119] Specifically, the fusion vector is determined by a mutual vector learning unit, wherein the mutual vector learning unit includes a connection block and a multi-layer perceptron MLP, the first feature vector and the second feature vector are connected by the connection block, and the connection vector obtained by the connection is input into the multi-layer perceptron MLP, and the fusion vector is obtained by the multi-layer perceptron MLP. The fusion vector can learn highly distinguishable features in the first feature vector and the second feature vector, for example, the edge of the lesion, etc.

[0120] The first gate vector is generated based on the first feature vector and the fusion vector, and the second gate vector is generated based on the second feature vector and the fusion vector. The first gate vector and the second gate vector are relative to the attention mechanism, and the difference features in the image can be strengthened through the first gate vector and the second gate vector. In one implementation, the generation formula of the first gate vector and the second gate vector can be:

[0121] g i =sigmoid(Xm⊙Xi),i∈{1,2}

[0122] Among them, g i represents the i-th gate vector, Xm represents the fusion vector, and Xi represents the i-th feature vector.

[0123] Each attention feature vector is used to reflect the difference between the first feature vector and the second feature vector, wherein the four attention feature vectors are based on the first gate vector, the second gate vector, the first feature vector and the second feature vector, wherein the calculation formulas of the four attention feature vectors can be respectively:

[0124]

[0125]

[0126]

[0127]

[0128] Among them, g 1 represents the first gate vector, g 2 represents the second gate vector, x 1 represents the first gate eigenvector, x 2 represents the second gate eigenvector, as well as Both represent attention feature vectors, With X i The dimensions are the same, consolidating and enhancing the features in the image. Activated by another input gate vector, it increases the model's ability to discriminate category features.

[0129] Furthermore, after the training of the preset network model is completed, the attention pairwise interaction module in the trained prediction network model is removed to obtain a classification model, and the classification model is combined with the breast detection model to obtain a joint module, and the joint model is used as the breast detection model. In this way, after detecting the lesion area in the breast image, the lesion area can be further classified to remove the false positive lesion area in the detected lesion area, thereby improving the accuracy of the lesion area.

[0130] In summary, this embodiment provides a training method for a breast tumor detection model, the method comprising inputting a training breast image in a first training sample set into a feature module, determining a number of first feature maps corresponding to the training breast image through the feature module; inputting a number of first feature maps into a pyramid module, determining a number of second feature maps through the pyramid module; inputting a number of second feature maps into a prediction module, determining a positive sample set and a negative sample set corresponding to the training breast image through the prediction module, and determining a loss function based on the positive sample set and the negative sample set; training a breast tumor detection model to be trained based on the loss function to obtain a breast tumor detection model. This application adopts a three-dimensional network model as a breast tumor detection model, so that the breast tumor detection model can learn the global spatial information of fully automatic breast ultrasound images, thereby improving the accuracy of lesion area detection. At the same time, this application also uses an adaptive threshold to select positive samples and negative samples, ensuring that all positive samples are around the labeled area, and obtaining more representative positive samples, thereby improving the model performance of the trained breast tumor detection model.

[0131] Based on the training method of the above-mentioned breast tumor detection model, this embodiment provides a breast tumor detection method, which applies the breast tumor detection model determined in the above-mentioned embodiment, and comprises:

[0132] Inputting the breast ultrasound image to be detected into the breast tumor detection model;

[0133] The breast tumor detection model is used to output a plurality of diseased areas of the breast ultrasound image.

[0134] The breast tumor detection method, wherein the breast tumor detection model includes a classification model; the method further includes:

[0135] Selecting a plurality of lesion images from the breast ultrasound image based on a plurality of lesion areas;

[0136] Controlling the classification model to determine the lesion category of each disease area based on each lesion image;

[0137] The lesion regions whose lesion categories are normal regions in the plurality of lesion regions are classified to obtain the target lesion region of the breast ultrasound image.

[0138] Specifically, the lesion image is included in the breast ultrasound image, and the lesion image includes a lesion area for determining the lesion image, that is, each lesion image includes a lesion area. After the lesion image is acquired, the lesion image is used as an input item of the classification model, and the lesion category of the lesion area in the lesion image is predicted by the classification model, wherein the lesion category is a benign category, a malignant category, and a normal area category. Since when detecting the lesion area, the lesion area with a benign category or a malignant category is detected, when the lesion category corresponding to the lesion image is determined to be a normal area category based on the classification model, it is indicated that the lesion area corresponding to the lesion image is a false positive lesion, so that the lesion area corresponding to the lesion image is removed, and all lesion areas after removing the false positive lesion areas are used as the target lesion areas of the breast ultrasound image.

[0139] Based on the above-mentioned breast tumor detection model training method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the breast tumor detection model training method as described in the above-mentioned embodiment.

[0140] Based on the training method of the above breast tumor detection model, the present application also provides a terminal device, such as Figure 8 As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0141] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0142] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0143] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0144] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A training method for a breast tumor detection model, characterized in that, the breast tumor detection model is a three-dimensional network model, and it includes a feature module, a pyramid module, and a prediction module. The training method includes: Input the training breast images in the first training sample set into the feature module, and determine a number of first feature maps corresponding to the training breast images through the feature module; Input the number of first feature maps into the pyramid module, and determine a number of second feature maps through the pyramid module; Input the number of second feature maps into the prediction module, and determine the positive sample set and negative sample set corresponding to the training breast images through the prediction module, and determine the loss function based on the positive sample set and the negative sample set. Among them, the positive sample set and negative sample set are obtained by dividing the anchor boxes in each second feature map based on an adaptive threshold, and the adaptive threshold is determined based on a number of second feature maps; Train the breast tumor detection model to be trained based on the loss function to obtain the breast tumor detection model; Among them, the specific process of determining the positive sample set and negative sample set corresponding to the training breast images through the prediction module includes: Control the prediction module to determine the candidate positive sample set corresponding to each second feature map, where the candidate positive sample set includes a number of anchor boxes; Calculate the first IoU value between each candidate positive sample in each candidate positive sample set and each lesion area corresponding to the training breast image, and the second IoU value between the anchor boxes in each second feature map and each lesion area corresponding to the training breast image; Calculate the mean and variance of all the obtained first IoU values, and determine the adaptive threshold according to the mean and variance; Take the anchor boxes with the second IoU value greater than the adaptive threshold as positive samples, and the anchor boxes with the second IoU value less than or equal to the adaptive threshold as negative samples to obtain the positive sample set and negative sample set.

2. The training method for the breast tumor detection model according to claim 1, characterized in that, before inputting the training breast images in the first training sample set into the feature module and determining a number of first feature maps corresponding to the training breast images through the feature module, the method further includes: Obtain an initial sample set, where the initial sample set includes a number of initial training images, and each initial training image in the number of initial training images carries a number of lesion areas; For each initial training image in the initial sample set, select the shortest side of the regions among the number of lesion areas, and determine whether the initial training image needs to be Mosaic enhanced based on the shortest side of the region; When the initial training image needs to be Mosaic enhanced, select a first image region around the lesion area corresponding to the shortest side of the region, and randomly select a preset number of second image regions in the initial training image; perform data enhancement on the first image region and the preset number of second image regions respectively, and splice the data-enhanced first image region and each second image region to obtain the training breast image; When the initial training image does not need to be enhanced by Mosaic, randomly select a third image region in the initial training image, perform data enhancement on the third image region, and use the third image region after data enhancement as the training breast image; Use the set composed of all the obtained training breast images as the first training sample set.

3. The training method of the breast tumor detection model according to claim 2, characterized in that, the initial training image is a full-field digital mammography ultrasound image, and the region size of the third image region is equal to the image size of the mosaic image obtained by splicing the first image region and each second image region after Mosaic enhancement.

4. The training method of the breast tumor detection model according to claim 1, characterized in that, after training the breast tumor detection model to be trained based on the loss function to obtain the breast tumor detection model, the method further includes: obtain a second training sample set, wherein the second training sample set includes a number of training data pairs, and each training data pair in the number of training data pairs includes a false positive training image and a lesion training image; input the training data pairs of the second training sample set into the feature extraction module in the preset network model, and determine the first feature vector corresponding to the false positive training image and the second feature vector corresponding to the lesion training image in the training data pair through the feature extraction module; input the first feature vector and the second feature vector into the attention pairwise interaction module in the preset network model, and determine four attention feature vectors through the attention pairwise interaction module; input the four attention feature vectors into the classification module in the preset network model, and determine the predicted category corresponding to each attention feature vector through the classification module; train the preset network model based on each predicted category, and after the preset network model is trained, remove the attention pairwise interaction module in the preset network model to obtain a classification model; combine the breast tumor detection model and the classification model, and use the combined network model as the breast tumor detection model.

5. The training method of the breast tumor detection model according to claim 4, characterized in that, the attention pairwise interaction module includes a mutual vector learning unit, a gate vector generation unit, and a pairwise interaction unit. The step of inputting the first feature vector and the second feature vector into the attention pairwise interaction module in the preset network model and determining four attention feature vectors through the attention pairwise interaction module specifically includes: control the mutual vector learning unit to generate a fusion vector based on the first feature vector and the second feature vector; control the gate vector generation unit to generate a first gate vector based on the first feature vector and the fusion vector, and generate a second gate vector based on the second feature vector and the fusion vector; control the pairwise interaction unit to generate attention feature vectors respectively based on the first feature vector and the first gate vector, the first feature vector and the second gate vector, the second feature vector and the first gate vector, and the second feature vector and the second gate vector to obtain four attention feature vectors.

6. A detection method for breast tumor images, characterized in that, the method applies a breast tumor detection model trained by using the training method of the breast tumor detection model described in any one of claims 1-5, and the method includes: inputting the breast ultrasound image to be detected into the breast tumor detection model; outputting several disease regions of the breast ultrasound image through the breast tumor detection model.

7. According to the detection method for breast tumor images described in claim 6, characterized in that, the breast tumor detection model includes a classification model trained by using the training method of the breast tumor detection model described in any one of claims 1; the method further includes: selecting several lesion images from the breast ultrasound image based on several lesion regions; controlling the classification model to determine the lesion categories of each disease region based on each lesion image; removing the lesion regions with the lesion category of normal region category among several lesion regions to obtain the target lesion regions of the breast ultrasound image.

8. A terminal device, characterized in that, it includes: a processor, a memory and a communication bus; a computer-readable program executable by the processor is stored on the memory; the communication bus realizes the connection communication between the processor and the memory; when the processor executes the computer-readable program, it realizes the steps of the training method of the breast tumor detection model described in any one of claims 1-5, and / or realizes the steps of the detection method for breast tumor images described in any one of claims 6-7.

Images