Method and system for structural distortion detection based on dual view digital breast tomosynthesis

Abstract

The present invention relates to the technical field of image analysis and proposes a method and system for detecting structural distortion based on dual-view digital breast tomosynthesis, comprising the following steps: acquiring a three-dimensional breast tomosynthesis image containing two projection position perspectives and processing the image into a two-dimensional image slice; marking the position of structural distortion in the two-dimensional image slice using a bounding box; pairing the marked two-dimensional image slices from the two projection position perspectives belonging to the same breast; predicting the two-dimensional detection frames corresponding to the position of structural distortion in the paired two-dimensional image slices using a computer-aided detection model and extracting a representation vector of the breast tissue structure within the two-dimensional detection frame; matching the two-dimensional detection frames in the image slices from the two projection position perspectives using the breast tissue structure representation vector of the paired two-dimensional image slice; fusing the two-dimensional detection frames into a three-dimensional detection frame; and eliminating part of the three-dimensional detection frame based on information from a single perspective and a dual perspective to obtain a final retained three-dimensional detection frame.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image analysis, and more particularly, to a structure distortion detection method and system based on double-view digital breast tomosynthesis. BACKGROUND

[0002] Traditional two-dimensional full-field digital mammography (FFDM) has a high false-positive probability in the screening of architectural distortion (AD).

[0003] Digital breast tomosynthesis (DBT) as a new breast imaging technology can provide more abundant internal information, and through tomographic imaging, it can effectively reduce the false-positive probability caused by overlapping of breast tissue.

[0004] The structure distortion detection model based on single-view digital breast tomosynthesis images performs poorly in detecting atypical structural distortion, because these structural distortions are often blocked by breast tissue without structural distortion, or are difficult to be correctly identified due to overlapping substantial tissue, resulting in low overall sensitivity and accuracy of the structure distortion detection model based on single-view digital breast tomosynthesis images. SUMMARY

[0005] To overcome the above-mentioned defects of the structure distortion detection model based on single-view digital breast tomosynthesis images, the present application provides a structure distortion detection method and system based on double-view digital breast tomosynthesis with higher overall sensitivity and accuracy.

[0006] To solve the above technical problems, the technical solutions of the present application are as follows:

[0007] Collect three-dimensional breast tomosynthesis images containing two projection position perspectives, and process them into two-dimensional image tomograms;

[0008] Use a bounding box to label the position of structural distortion in the two-dimensional image tomogram;

[0009] Pair the two-dimensional image tomograms with labeled structural distortion of the same breast at two projection position perspectives;

[0010] Use a computer-aided detection model to predict the two-dimensional detection box corresponding to the structural distortion position in the paired two-dimensional image tomograms and extract the feature vector of the breast tissue structure in the two-dimensional detection box;

[0011] The two-dimensional detection frame in the two-dimensional image slice is matched by using the breast tissue structure feature vector of the paired two-dimensional detection frame in the two-dimensional image slice.

[0012] The paired two-dimensional detection frame is fused into a three-dimensional detection frame.

[0013] The prediction probability deviation metric of the three-dimensional detection frame, the three-dimensional anatomical coordinate space position where the three-dimensional detection frame is located, and the breast parenchyma proportion of the three-dimensional detection frame are calculated respectively, the three-dimensional detection frame is removed according to the calculation results, and the remaining three-dimensional detection frame is obtained.

[0014] The proportion of the two-dimensional detection frame paired with the remaining three-dimensional detection frame that is removed is counted, the three-dimensional detection frame with a proportion greater than a preset threshold is removed, and the final remaining three-dimensional detection frame is obtained.

[0015] The application also provides a structure distortion prediction system based on double-view digital breast tomography, which is used to realize the structure distortion detection method based on double-view digital breast tomography.

[0016] An image processing module is configured to acquire three-dimensional breast tomography images containing two projection position angles and process the three-dimensional breast tomography images into two-dimensional image slices.

[0017] A labeling module is configured to label the positions of structure distortion in the two-dimensional image slices by using bounding boxes.

[0018] A pairing module is configured to pair the two-dimensional image slices of the same breast that have been labeled at two projection position angles.

[0019] A two-dimensional detection frame prediction module is configured to predict the two-dimensional detection frame corresponding to the position of structure distortion in the paired two-dimensional image slices by using a computer-aided detection model, extract the feature vector of the breast tissue structure in the two-dimensional detection frame, and match the two-dimensional detection frames in the two-dimensional image slices at the two projection position angles by using the feature vector of the breast tissue structure in the paired two-dimensional detection frame.

[0020] A three-dimensional detection frame prediction module is configured to fuse the paired two-dimensional detection frame output by the two-dimensional detection frame prediction module into a three-dimensional detection frame, calculate the prediction probability deviation metric of the three-dimensional detection frame, the three-dimensional anatomical coordinate space position where the three-dimensional detection frame is located, and the breast parenchyma proportion of the three-dimensional detection frame respectively, remove the three-dimensional detection frame according to the calculation results, obtain the remaining three-dimensional detection frame, count the proportion of the two-dimensional detection frame paired with the remaining three-dimensional detection frame that is removed, remove the three-dimensional detection frame with a proportion greater than a preset threshold, and output the final remaining three-dimensional detection frame as a structure distortion prediction result.

[0021] Compared with the prior art, the beneficial effects of the technical scheme of the present application are:

[0022] The three-dimensional breast tomography images containing two projection position perspectives are collected and processed into two-dimensional image sections, the two-dimensional image sections with the same breast and two projection position perspectives are paired, the paired two-dimensional detection boxes are predicted by the structure distortion positions in the paired two-dimensional image sections, the three-dimensional detection boxes are predicted by the paired two-dimensional detection boxes, and the three-dimensional detection boxes with labeling errors are removed by using multiple methods, so that the accurate detection probability of the structure distortion of the breast tissue which is not blocked by the breast tissue with structure distortion is improved by using the dual-perspective breast tomography images, and the accurate detection probability of the structure distortion at the overlapping substantial tissue is improved by determining whether to remove the three-dimensional detection box according to the breast substantial proportion, so that the present scheme has high sensitivity and precision. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flowchart of the structure distortion detection method based on dual-perspective digital breast tomography for embodiment 1 is shown.

[0024] Figure 2 A structure diagram of the computer-aided detection model for embodiment 1 is shown.

[0025] Figure 3 A detection flowchart of the computer-aided detection model for embodiment 1 is shown.

[0026] Figure 4 Performance display diagrams of the computer-aided detection model for embodiment 1 under different structure parameters are shown.

[0027] Figure 5 A prediction probability deviation threshold value diagram for embodiment 1 is shown.

[0028] Figure 6 A pectoral muscle voxel distance diagram for embodiment 1 is shown.

[0029] Figure 7 A breast substantial proportion diagram for embodiment 1 is shown.

[0030] Figure 8 A position quantile diagram for embodiment 1 is shown.

[0031] Figure 9 A performance comparison example diagram for embodiment 2 is shown.

[0032] Figure 10 An ablation experiment result diagram for embodiment 2 is shown.

[0033] Figure 11The overall framework diagram of the structure distortion prediction system based on dual-view digital breast tomosynthesis proposed in Embodiment 3. DETAILED DESCRIPTION

[0034] The accompanying drawings are only used for illustrative purposes and cannot be understood as a limitation on the embodiments;

[0035] In order to better illustrate the embodiments, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual size of the product;

[0036] It is understandable for those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0037] The technical solutions of the present application will be further described below in combination with the drawings and embodiments.

[0038] Embodiment 1

[0039] The present embodiment proposes a structure distortion detection method based on dual-view digital breast tomosynthesis, Figure 1 The flowchart of the structure distortion detection method based on dual-view digital breast tomosynthesis of the present embodiment;

[0040] The structure distortion detection method based on dual-view digital breast tomosynthesis proposed in the present embodiment includes the following steps:

[0041] S1: Collect three-dimensional breast tomosynthesis images containing two projection position perspectives, and process them into two-dimensional image slices;

[0042] S2: Use a bounding box to label the position of the structure distortion in the two-dimensional image slice;

[0043] S3: Pair the labeled two-dimensional image slices of the same breast at two projection position perspectives;

[0044] S4: Use a computer-aided detection model to predict the two-dimensional detection box corresponding to the structure distortion position in the paired two-dimensional image slices and extract the feature vector of the breast tissue structure in the two-dimensional detection box;

[0045] S5: Use the breast tissue structure feature vector of the two-dimensional detection box in the paired two-dimensional image slices to match the two-dimensional detection boxes in the two projection position perspective image slices;

[0046] S6: Fuse the paired two-dimensional detection boxes into a three-dimensional detection box;

[0047] S7: Calculate the prediction probability deviation metric of the three-dimensional detection box, the three-dimensional anatomical coordinate space position where the three-dimensional detection box is located, and the breast parenchymal proportion of the three-dimensional detection box, respectively, and perform rejection processing on the three-dimensional detection box according to the calculation results to obtain the retained three-dimensional detection box.

[0048] S8: Statistics the proportion of the paired two-dimensional detection boxes that are eliminated for the remaining three-dimensional detection box, and eliminates the three-dimensional detection box with a proportion greater than a preset threshold to obtain the final remaining three-dimensional detection box.

[0049] In the specific implementation process, three-dimensional breast tomography images containing two projection position perspectives are collected and processed into two-dimensional image tomography, two-dimensional image tomography with the same breast and two projection position perspectives are paired, the paired two-dimensional detection box is predicted by the structure distortion position in the paired two-dimensional image tomography, the three-dimensional detection box is predicted by the paired two-dimensional detection box, and the three-dimensional detection box with incorrect marking is eliminated by using multiple methods. The double-perspective breast tomography images improve the accurate detection probability of the structure distortion of the breast tissue that is not blocked by the structure distortion of the breast tissue, and the breast parenchyma proportion determines whether to eliminate the three-dimensional detection box to improve the accurate detection probability of the structure distortion at the overlapping parenchyma, so the scheme has high sensitivity and accuracy.

[0050] In an optional embodiment, the two projection position perspectives include: cranio-caudal position perspective and medial-lateral oblique position perspective;

[0051] The three-dimensional breast tomography images corresponding to the cranio-caudal position perspective include: three-dimensional breast tomography images corresponding to left cranio-caudal position and right cranio-caudal position;

[0052] The three-dimensional breast tomography images corresponding to the medial-lateral oblique position perspective include: three-dimensional breast tomography images corresponding to left medial-lateral oblique position and right medial-lateral oblique position;

[0053] When collecting three-dimensional breast tomography images, the three-dimensional breast tomography images are screened based on the inclusion criteria and the gold standard, and only the three-dimensional breast tomography images that meet the inclusion criteria and the gold standard at the same time are collected;

[0054] The inclusion criteria include: three-dimensional breast tomography images of patients who participate in breast cancer screening for the first time and have not undergone breast surgery and biopsy before participating in the screening;

[0055] The gold standard includes: three-dimensional breast tomography images of patients who are confirmed to have structure distortion in at least one breast tissue through biopsy, surgery or follow-up at a later stage;

[0056] After the collected three-dimensional breast tomography images are standardized and desensitized, the three-dimensional breast tomography images are split into two-dimensional tomography images;

[0057] When the bounding box is used to label the position of the structural distortion in the two-dimensional image tomography, in each structural distortion position, the most clear breast tomography and the two tomographies above and below the tomography are labeled by using the bounding box to label their positions;

[0058] When the two-dimensional image tomographies of the same breast from two projection position perspectives are paired, the two-dimensional image tomography corresponding to the left cranio-caudal position in the bounding box is paired with the two-dimensional image tomography corresponding to the left mediolateral oblique position, and the two-dimensional image tomography corresponding to the right cranio-caudal position in the bounding box is paired with the two-dimensional image tomography corresponding to the right mediolateral oblique position.

[0059] In the optional embodiment, the cross analysis of the same side view of the left cranio-caudal (LCC)-left mediolateral oblique (LMLO) or the right cranio-caudal (RCC)-right mediolateral oblique (RMLO) can reveal the structural distortion that may be ignored in a single view, thereby reducing the uncertainty of the structural distortion; research shows that the double-view screening can significantly reduce the missed cancer cases compared with the single-view screening;

[0060] As an exemplary illustration, the collected breast tomography images are stored in the original DICOM medical image format; the collecting device used is the Selenia Dimension breast X-ray imaging system of Hologic; the image data is stored in 16-bit gray scale, and the pixel size in the tomography is from 0.086 mm to 0.108 mm (mean ± standard deviation: 0.089 ± 0.005), and the physical interval of adjacent tomographies is 1 mm; as an exemplary illustration, a radiologist with more than three years of experience is used to report the image as a reference, and the position of each structural distortion lesion is labeled by using the bounding box on the most clear breast tomography layer and the two tomographies above and below the tomography, which is used as the three-dimensional gold standard of the detection task;

[0061] In the optional embodiment, without the labeling information of the diseased area, the nipple position, the areola position, and the axillary tail position of the breast, only the position of each structural distortion lesion is labeled by using the bounding box on the most clear breast tomography layer and the two tomographies above and below the tomography, and the two-dimensional image tomographies of the same breast from two projection position perspectives are paired.

[0062] In an optional embodiment, the computer-aided detection model comprises a feature map pyramid network, a task sharing subnetwork, a structural distortion score subnetwork, a detection frame subnetwork, and an image representation subnetwork.

[0063] The task sharing sub-network includes convolutional layers;

[0064] The detection box sub-network and the structural distortion score sub-network include 3×3 convolutional layers;

[0065] The image representation subnetwork includes 1×1 convolutional layers;

[0066] The steps of using a computer-aided detection model to predict a two-dimensional detection frame corresponding to a structural distortion position in a paired two-dimensional image slice and extracting a representation vector of the breast tissue structure within the two-dimensional detection frame include:

[0067] The paired two-dimensional image slices are simultaneously input into a feature map pyramid network. The feature map pyramid network is based on the feature maps of the paired two-dimensional image slices. The feature maps are input into a task sharing subnetwork. The task sharing subnetwork outputs features extracted from the feature maps, and the features are input into a structural distortion score subnetwork, a detection frame subnetwork, and an image representation subnetwork. The detection frame subnetwork is used to locate and output the location of the structural distortion, the structural distortion score subnetwork is used to calculate and output the structural distortion score corresponding to the location of the structural distortion, and the image representation subnetwork is used to extract and output a representation vector of the breast structural tissue within the location of the structural distortion. The output results of the structural distortion score subnetwork, the detection frame subnetwork, and the image representation subnetwork are used to form a two-dimensional detection candidate frame corresponding to the structural distortion location in the two-dimensional image slice, which includes the structural distortion location and the structural distortion score, and a representation vector of the breast tissue structure within the two-dimensional detection candidate frame.

[0068] In this optional embodiment, Figure 2 This is a schematic diagram of the structure of the computer-aided detection model proposed in this embodiment, as shown in FIG. Figure 2 As shown in the figure, a Feature Pyramid Network (FPN) is used to extract multi-scale features of images, and a single-view two-dimensional deep learning detection network is formed with the task sharing subnetwork, structural distortion score subnetwork, detection box subnetwork, and image representation subnetwork constructed by the fully convolutional network.

[0069] As an example, Figure 3 This is a schematic diagram of the detection process of the computer-aided detection model proposed in this embodiment, as shown in FIG. Figure 3 As shown in the figure, a Siamese network is constructed using a single-view two-dimensional detection network to simultaneously extract the image features of dual-view digital breast tomosynthesis images and obtain dual-view candidate detection frames. The representation of the tissue structure within the detection frame is achieved through comparative learning using triplet loss.

[0070] In an optional embodiment, before using the computer-aided detection model to predict the two-dimensional detection frame corresponding to the structural distortion position in the paired two-dimensional image slices and extracting the representation vector of the breast tissue structure in the two-dimensional detection frame, the paired two-dimensional image slices are divided into a training set according to a preset ratio. and validation set Using the training set The computer-aided detection model is trained for a preset number of rounds. In each round of training, based on the loss function The computer-aided detection model is trained until the loss function When the number of iterations is minimized or reaches a preset threshold, the training is stopped to obtain the trained computer-aided detection model corresponding to each round;

[0071] Using the validation set , verify the reliability test index of the trained computer-aided detection model corresponding to each round, and select the computer-aided detection model with the highest reliability test index as the final computer-aided detection model;

[0072] When using a computer-aided detection model to predict a two-dimensional detection frame corresponding to a structural distortion position in a paired two-dimensional image slice and extracting a representation vector of the breast tissue structure within the two-dimensional detection frame, the final computer-aided detection model is used for prediction and extraction;

[0073] Among them, the loss function The expression is:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] Where, represents the loss function used to train the structural distortion score sub-network, Indicates the number of predicted categories, Indicates the current predicted category, Indicates the category The predicted probability function, represents the adjustment factor for focus loss, represents the loss function used to train the detection box positioning, respectively represent the horizontal coordinate of the detection box center point, the vertical coordinate of the detection box center point, the width of the detection box, and the height of the detection box, represent the detection box parameters after the scale transformation, represent the label box parameters after the scale transformation, represent the smoothing function parameters, represent the loss scale adjustment factor, used to balance the scale size of the triplet loss and other losses, represent the triplet loss function, represent the number of selected positive samples, represent the positive sample set, represent a certain positive sample, represent the distance parameters between different categories, represent the distance between the current sample and the positive sample, represent the distance between the current sample and the negative sample, represent the negative sample set.

[0081] As an exemplary illustration, when the paired two-dimensional image slices are divided into a training set , a validation set and a test set at a preset ratio, the preset ratio is 5:2:3; the test set is used to evaluate the reliability test indicators of the final computer-aided detection model, and the evaluation indicators of the validation set are obtained, and the evaluation indicators of the validation set are used as the final evaluation indicators of the computer-aided detection model;

[0082] As an exemplary illustration, the data is randomly divided into a training set, a validation set and a test set at the patient level; the preset number of rounds is 25 rounds, and the reliability test indicator is MTPF (Mean Time Between Failure, Mean Time Between Failure); each round uses the training set to train in random order, and at the end of each training round, the MTPF indicator of the model is verified using the validation set; at the end of the entire training, the model with the highest MTPF indicator in the validation set is selected as the optimal model, and the test set is used for final verification;

[0083] As an exemplary illustration, is a Focal Loss (Focal Loss) function, is a Huber Loss (Smooth L1 Loss) function, is a Triplet Loss (Triplet Loss) function, and the computer-aided detection model learns the feature expression of the tissue structure in the detection box based on the Triplet Loss function through double-view image comparison learning.

[0084] As an example, Figure 4 This is a diagram showing the effectiveness of the computer-aided detection model proposed in this embodiment under different structural parameters; Figure 4 In the example, R@k indicates that the highest sensitivity R can be achieved while tolerating k false positives per image. “-” indicates that the sensitivity is less than 80%. The bold value is the highest value corresponding to the R@k item. As an example, the structural parameters of the final computer-aided detection model are 、 and They are: 3, 1 and 2;

[0085] In this optional embodiment, a twin network architecture is used to extract and compare information from two views, and a triplet loss module is introduced. The anatomical structure relationship between the ipsilateral views is used as supervisory information to guide the fusion of multi-view feature information. This method not only completes the detection and structural feature extraction tasks simultaneously during training, but also can effectively fuse multi-view information for detection and remove erroneous judgments, thereby improving the overall performance of the model.

[0086] In an optional embodiment, the step of matching the two-dimensional detection frames in the paired two-dimensional image slices using the breast tissue structure representation vector of the two-dimensional detection frames in the paired two-dimensional image slices includes:

[0087] Obtain the two-dimensional detection candidate frames corresponding to the head-foot position and the two-dimensional detection candidate frames corresponding to the medial and lateral positions in the paired two-dimensional image tomography, calculate the correlation between the two-dimensional detection candidate frames through the breast tissue structure representation vectors corresponding to the two-dimensional detection candidate frames, and based on the correlation, use the Hungarian matching algorithm to determine the one-to-one matching relationship between the two-dimensional detection candidate frames to obtain the paired two-dimensional detection frames, and use the formula Adjust the structural distortion score within the paired 2D detection candidate box;

[0088] Among them, the formula The expression is:

[0089] =

[0090] Where, represents the weight parameter, Represents the structural distortion score of the current two-dimensional candidate box, Represents Structural distortion scores for paired 2D proposals.

[0091] In an optional embodiment, the step of fusing the paired two-dimensional detection frames into a three-dimensional detection frame includes:

[0092] Based on the paired two-dimensional detection frames, the centroids of the two-dimensional detection frames on each slice of the paired two-dimensional image slices are calculated, and all the calculated centroids are projected onto the same plane. All the centroids projected onto the same plane are clustered using a density-based spatial clustering method to obtain a plurality of cluster centers.

[0093] All 2D detection frames belonging to the same cluster center are stacked in the upper and lower layers of the breast slice to form a series of 3D connected regions.

[0094] Use 3D morphological erosion and dilation operations to connect adjacent connected branches in the same cluster and remove connected branches that only appear in one or two faults;

[0095] Calculate the bounding box of each remaining three-dimensional connected branch, take the bounding box as the ROI in the three-dimensional space, take the maximum two-dimensional prediction probability in the ROI as the prediction probability of the entire ROI, and the three-dimensional detection box is the ROI.

[0096] In this optional embodiment, a three-dimensional structural distortion candidate region (ROI) is determined through the steps of two-dimensional centroid clustering, constructing three-dimensional connected regions, filling gaps, and eliminating isolated regions. In the two-dimensional centroid clustering step, a density-based spatial clustering method (Density Based Spatial Clustering of Applications with Noise, DBSCAN) is used for clustering.

[0097] In an optional embodiment, the steps of respectively calculating a prediction probability deviation measure of the three-dimensional detection frame, a three-dimensional anatomical coordinate spatial position of the three-dimensional detection frame, and a breast parenchyma ratio of the three-dimensional detection frame, and removing the three-dimensional detection frame based on the calculation results to obtain a retained three-dimensional detection frame include:

[0098] Calculating the prediction probability deviation metric of the three-dimensional detection frame and removing the three-dimensional detection frame whose prediction probability deviation metric is greater than a preset threshold, the steps include:

[0099] For any 3D detection frame, assuming that it contains K faults, the prediction probability of the computer-aided detection model on each fault is arranged from top to bottom according to the spatial position, forming a prediction probability sequence, which is recorded as ;in, Indicates the The predicted probability value of each fault;

[0100] With width Sliding window, calculate the sequence The sliding mean and sliding standard deviation of is the default value;

[0101] The maximum sliding mean value is selected as the prediction probability deviation metric of the three-dimensional detection frame;

[0102] The three-dimensional detection frame with a prediction probability deviation metric greater than a preset threshold is removed;

[0103] Obtain pectoral muscle position information and nipple position information, and construct a three-dimensional anatomical coordinate space based on the pectoral muscle position information and the nipple position information with a pectoral muscle axis, a nipple axis, and an up-down cross-sectional direction axis;

[0104] The pectoral muscle axis includes a junction line between the pectoral muscle region and the breast region;

[0105] The nipple axis includes a perpendicular line with the nipple position as the starting point and perpendicular to the pectoral muscle axis;

[0106] The up-down cross-sectional direction axis includes an axis line of the up-down layer between cross sections;

[0107] The pectoral muscle axis, the nipple axis, and the up-down cross-sectional direction axis are perpendicular to each other in pairs;

[0108] Calculate the three-dimensional anatomical coordinate space position of the three-dimensional detection frame, and remove the three-dimensional detection frame whose three-dimensional anatomical coordinate space position is outside the preset acceptance range, and the steps include:

[0109] Based on the three-dimensional anatomical coordinate space, count the frequency of the voxel distance of the geometric center of the structural distortion position of the paired two-dimensional image cross sections in the training set , and select the lower limit value of the voxel distance acceptance range and the upper limit value of the voxel distance acceptance range from the voxel distance frequency, the expression of the voxel distance acceptance range is: , wherein represents the lower limit value of the voxel distance acceptance range, represents the upper limit value of the voxel distance acceptance range;

[0110] Calculate the three-dimensional anatomical coordinate space position of the three-dimensional detection frame, and remove the three-dimensional detection frame whose three-dimensional anatomical coordinate space position is outside the preset acceptance range;

[0111] The three-dimensional anatomical coordinate space position of the three-dimensional detection frame includes the frequency of the voxel distance of the geometric center of the three-dimensional detection frame from the pectoral muscle line; and the preset acceptance range includes the voxel distance acceptance range .

[0112] As an exemplary illustration, the width of the sliding window is an odd number; as an exemplary illustration, ;

[0113] As an exemplary illustration, Figure 5 The prediction probability deviation metric threshold value diagram proposed for the present embodiment; Figure 5 The threshold value relationship between the MTPF and the prediction probability deviation metric calculated using the training set is shown as an exemplary illustration, according to Figure 5 The three-dimensional candidate box with a prediction probability deviation metric greater than 0.19 is determined as a false positive result and is removed by the present application; Figure 6 The pectoralis major voxel distance diagram proposed for the present embodiment, Figure 6 The cumulative distribution histogram of the pectoralis major voxel distance calculated using the training set at each threshold value is shown as an exemplary illustration, according to Figure 6 The present application selects , .

[0114] In an optional embodiment, the prediction probability deviation metric of the three-dimensional detection box, the three-dimensional anatomical coordinate space position where the three-dimensional detection box is located, and the breast parenchymal proportion of the three-dimensional detection box are calculated respectively, and the retained three-dimensional detection box is obtained by performing the removal processing on the three-dimensional detection box according to the calculation results. The step further includes:

[0115] The breast parenchymal proportion of the three-dimensional detection box is calculated, and the three-dimensional detection box with a breast parenchymal proportion less than a preset threshold value is removed, and the step includes:

[0116] The two-dimensional section where the three-dimensional geometric center of the three-dimensional prediction box is located is selected as the center section, the OTSU adaptive threshold segmentation algorithm is used on the center section to distinguish the parenchymal tissue and the adipose tissue in the breast region, and the breast parenchymal proportion of the three-dimensional detection box is calculated The expression of the is as follows:

[0117]

[0118] In the formula, represents the number of parenchymal tissue pixels of the three-dimensional prediction box in the breast region of the center section, represents the size of the three-dimensional prediction box of the center section;

[0119] The three-dimensional detection box with a breast parenchymal proportion less than a preset threshold value is removed, and the position quantile acceptance range is set , wherein, represents the lower limit value of the position quantile acceptance range, represents the upper limit value of the position quantile acceptance range, the three-dimensional prediction box in the center section range exceeding the position quantile acceptance range is removed, and the expression of the position quantile is as follows:

[0120]

[0121] wherein, denotes the position quantile, denotes the ordinal number of the layer where the three-dimensional prediction frame is located, denotes the total number of layers of the fault.

[0122] As an exemplary illustration, Figure 7 a schematic diagram of the breast parenchymal fraction proposed in the embodiment, Figure 7 shows the frequency distribution of the breast parenchymal fraction calculated using the training set in each interval; Figure 8 a schematic diagram of the position quantile proposed in the embodiment, Figure 8 shows the frequency distribution of the position quantile calculated using the training set in each interval; as an exemplary illustration, according to Figure 8 , the , .

[0123] In an optional embodiment, the step of obtaining the structural distortion detection result includes:

[0124] performing two-by-two inner product on the breast tissue structure representation vectors of the two-dimensional detection frame of the remaining three-dimensional detection frame and the two-dimensional detection frame paired therewith, to obtain the correlation degree between the paired two-dimensional detection frames, and determining one-to-one pairing relationship between the paired two-dimensional detection frames based on the correlation degree by using the Hungarian matching algorithm, and the mapping expression of the pairing relationship is:

[0125]

[0126] wherein, denotes the two-dimensional detection frame corresponding to one kind of projection position view in the paired two-dimensional detection frame, denotes the two-dimensional detection frame corresponding to another kind of projection position view in the paired two-dimensional detection frame;

[0127] combining the two-dimensional detection frames corresponding to all the remaining three-dimensional detection frames into a set , and based on the set statistically obtaining the proportion of the two-dimensional detection frames paired with the two-dimensional detection frames that are eliminated , and the proportion is measured by the pairing false alarm metric ;

[0128] eliminating the three-dimensional detection frames with the pairing false alarm metric greater than a preset threshold, to obtain the final remaining three-dimensional detection frames;

[0129] wherein, the calculation expression of the is:

[0130]

[0131]

[0132]

[0133] wherein, represents any remaining three-dimensional bounding box, represents the kth two-dimensional predicted bounding box that constitutes the three-dimensional bounding box.

[0134] As an example, three-dimensional bounding boxes with a pair-wise false positive metric greater than 1 are discarded.

[0135] Embodiment 2

[0136] This embodiment is based on the structure distortion detection method based on dual-view digital breast tomosynthesis proposed in Embodiment 1, and the following indicators are used for comparison with other common models:

[0137] (1) Mean True Positive Fraction (MTPF): The average correct detection rate in the range of 0.05 to 2.0 false positives (FPs) per volume. This indicator measures the ability of the model to correctly identify true structure distortions (AD) at different false positive thresholds.

[0138] (2) Sensitivity at x False Positives (FPs) per volume (R@x): The highest sensitivity that can be achieved under the condition that the number of false positives per breast tomosynthesis is not higher than x. This indicator reflects the sensitivity of the model under the condition of limited false positives.

[0139] (3) Number of FPs at 80% Sensitivity (FPs@0.8): The number of false positive results produced by the model while maintaining 80% sensitivity (i.e., correctly identifying 80% of true positives). This indicator is used to evaluate the number of false positive results while maintaining high sensitivity.

[0140] Figure 9 The performance comparison example graph for this embodiment, Figure 9 shows the performance comparison results of this method and other models, Figure 9 wherein R@k represents the highest sensitivity that can be achieved under the condition of tolerating k false positive results per image, “-” represents that the sensitivity does not reach 80%, and the bold value is the highest value corresponding to the R@k item;

[0141] ​The application removes false positive results of structural distortion positions based on dual-view, three-dimensional continuity of breast tissue and prior knowledge of breast anatomy. In order to compare the influence of the three false positive result removal methods on the present method, the present method performs ablation experiments on the three false positive result removal methods, Figure 10 The ablation experiment result schematic diagram for the present embodiment is shown in FIG. 6, Figure 10 R@k in FIG. 6 represents the highest sensitivity R achieved under the tolerance of k false positive results of each image, "-" represents that the sensitivity does not reach 80%, and the bold value is the highest value corresponding to R@k.

[0142] Embodiment 3

[0143] The present embodiment proposes a structural distortion prediction system based on dual-view digital breast tomography, which is used to implement the structural distortion detection method based on dual-view digital breast tomography proposed in Embodiment 1.

[0144] Figure 11 The overall framework diagram of the structural distortion prediction system based on dual-view digital breast tomography of the present embodiment is shown in FIG. 7.

[0145] The structural distortion prediction system based on dual-view digital breast tomography comprises:

[0146] An image processing module is configured to acquire three-dimensional breast tomography images containing two projection position perspectives and process them into two-dimensional image sections.

[0147] A labeling module is configured to label the positions of structural distortions in the two-dimensional image sections by using bounding boxes.

[0148] A pairing module is configured to pair the two-dimensional image sections with labeled structural distortions of the same breast.

[0149] A two-dimensional detection box prediction module is configured to predict two-dimensional detection boxes corresponding to the positions of structural distortions in the paired two-dimensional image sections by using a computer-aided detection model, extract feature vectors of breast tissue structures in the two-dimensional detection boxes, and match the two-dimensional detection boxes in the two-dimensional image sections of the two projection position perspectives by using the feature vectors of the breast tissue structures in the two-dimensional detection boxes.

[0150] The three-dimensional bounding box prediction module is configured to fuse the paired two-dimensional bounding boxes output by the two-dimensional bounding box prediction module into three-dimensional bounding boxes, and calculate a predicted probability deviation metric of the three-dimensional bounding boxes, a three-dimensional anatomical coordinate space position where the three-dimensional bounding boxes are located, and a breast parenchymal proportion of the three-dimensional bounding boxes, respectively, perform elimination processing on the three-dimensional bounding boxes according to the calculation results, obtain retained three-dimensional bounding boxes, and perform statistics on the retained three-dimensional bounding boxes on a proportion of the two-dimensional bounding boxes paired with the three-dimensional bounding boxes that are eliminated, eliminate the three-dimensional bounding boxes with the proportion greater than a preset threshold, and output a final retained three-dimensional bounding box as a structure distortion position prediction result.

[0151] In the embodiment, a three-dimensional breast tomography image containing two projection position perspectives is input into a structure distortion prediction system based on dual-view digital breast tomography, the system labels a position of structure distortion in a two-dimensional image tomography using a bounding box, and based on the labeling information, a final retained three-dimensional bounding box (a specific position of structure distortion in the two-dimensional image tomography) can be predicted, and a structure distortion position prediction result is obtained.

[0152] It can be understood that the structure distortion prediction system based on dual-view digital breast tomography of the embodiment improves the method of embodiment 1, and the optional items in the above embodiment 1 are also applicable to the embodiment, and thus are not described again here.

[0153] The same or similar reference numerals correspond to the same or similar components;

[0154] The terms describing the positional relationship in the drawings are only used for illustrative description, and cannot be understood as a limitation on the embodiment;

[0155] Obviously, the above-described embodiments of the present application are merely examples for clearly illustrating the present application, and are not intended to limit the implementation modes of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the implementation modes do not need to be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A method for detecting structural distortion based on dual-view digital breast tomosynthesis, characterized in that: The following steps are involved: Acquire three-dimensional breast tomosynthesis images containing two projection positions and view angles, and process them into two-dimensional image slices; Use bounding boxes to mark the locations of structural distortions in two-dimensional image slices; Pairing the annotated two-dimensional image slices of the same breast from two different projection positions and perspectives; A computer-aided detection model is used to predict the two-dimensional detection frame corresponding to the structural distortion position in the paired two-dimensional image slices and extract the representation vector of the breast tissue structure within the two-dimensional detection frame; Using the breast tissue structure representation vector of the two-dimensional detection frame in the paired two-dimensional image slices, the two-dimensional detection frames in the image slices of the two projection positions are matched; fusing the paired two-dimensional detection frames into a three-dimensional detection frame; Calculating the predicted probability deviation measure of the three-dimensional detection frame, the three-dimensional anatomical coordinate spatial position of the three-dimensional detection frame, and the breast parenchyma ratio of the three-dimensional detection frame respectively, and eliminating the three-dimensional detection frame according to the calculation results to obtain a retained three-dimensional detection frame; For the remaining 3D detection frames, the percentage of the 2D detection frames paired with them that have been eliminated is counted, and the 3D detection frames whose percentage is greater than a preset threshold are eliminated to obtain the final retained 3D detection frames; The computer-aided detection model includes: a feature map pyramid network, a task sharing subnetwork, a structural distortion score subnetwork, a detection box subnetwork and an image representation subnetwork; The task sharing sub-network includes convolutional layers; The detection box sub-network and the structural distortion score sub-network include 3×3 convolutional layers; The image representation subnetwork includes 1×1 convolutional layers; The steps of using a computer-aided detection model to predict a two-dimensional detection frame corresponding to a structural distortion position in a paired two-dimensional image slice and extracting a representation vector of the breast tissue structure within the two-dimensional detection frame include: The paired two-dimensional image slices are simultaneously input into a feature map pyramid network. The feature map pyramid network is based on the feature maps of the paired two-dimensional image slices. The feature maps are input into a task sharing subnetwork. The task sharing subnetwork outputs features extracted from the feature maps, and the features are input into a structural distortion score subnetwork, a detection frame subnetwork, and an image representation subnetwork. The detection frame subnetwork is used to locate and output the location of the structural distortion, the structural distortion score subnetwork is used to calculate and output the structural distortion score corresponding to the location of the structural distortion, and the image representation subnetwork is used to extract and output a representation vector of the breast structural tissue within the location of the structural distortion. The output results of the structural distortion score subnetwork, the detection frame subnetwork, and the image representation subnetwork are used to form a two-dimensional detection candidate frame corresponding to the structural distortion location in the two-dimensional image slice, which includes the structural distortion location and the structural distortion score, and a representation vector of the breast tissue structure within the two-dimensional detection candidate frame.

2. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to claim 1, wherein: The two projection position angles include: head-foot angle and medial-lateral angle; The three-dimensional breast tomosynthesis images corresponding to the head-foot view angle include: three-dimensional breast tomosynthesis images corresponding to the left head-foot view angle and the right head-foot view angle; The three-dimensional breast tomosynthesis images corresponding to the internal and external oblique lateral views include: three-dimensional breast tomosynthesis images corresponding to the left internal and external oblique lateral view and the right internal and external oblique lateral view; When acquiring 3D breast tomosynthesis images, the images were screened based on the inclusion criteria and the gold standard, and only those that met both the inclusion criteria and the gold standard were acquired; The inclusion criteria include: 3D breast tomosynthesis images of patients who participated in breast cancer screening for the first time and had not undergone breast surgery or biopsy before participating in the screening; The gold standard includes: 3D breast tomosynthesis imaging of patients with architectural distortion in at least one breast tissue confirmed by biopsy, surgery, or follow-up; After standardization and desensitization processing of the acquired 3D breast tomosynthesis images, the 3D breast tomosynthesis images are split into 2D tomographic images; When using a bounding box to mark the location of structural distortion in a two-dimensional image slice, at each location of structural distortion, the location of the clearest breast slice, the two slices above it, and the two slices below it are marked with a bounding box; When pairing two annotated two-dimensional image slices belonging to the same breast from two projection positions and perspectives, the two-dimensional image slice corresponding to the left head-foot position within the bounding box is paired with the two-dimensional image slice corresponding to the left medial-lateral oblique position, and the two-dimensional image slice corresponding to the right head-foot position within the bounding box is paired with the two-dimensional image slice corresponding to the right medial-lateral oblique position.

3. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to claim 1, wherein: Before using the computer-aided detection model to predict the 2D detection frame corresponding to the structural distortion position in the paired 2D image slices and extracting the representation vector of the breast tissue structure within the 2D detection frame, the paired 2D image slices are divided into a training set according to a preset ratio. and validation set Using the training set The computer-aided detection model is trained for a preset number of rounds. In each round of training, based on the loss function The computer-aided detection model is trained until the loss function When the number of iterations is minimized or reaches a preset threshold, the training is stopped to obtain the trained computer-aided detection model corresponding to each round; Using the validation set , verify the reliability test index of the trained computer-aided detection model corresponding to each round, and select the computer-aided detection model with the highest reliability test index as the final computer-aided detection model; When using a computer-aided detection model to predict a two-dimensional detection frame corresponding to a structural distortion position in a paired two-dimensional image slice and extracting a representation vector of the breast tissue structure within the two-dimensional detection frame, the final computer-aided detection model is used for prediction and extraction; Among them, the loss function The expression is: Where, represents the loss function used to train the structural distortion score sub-network, Indicates the number of predicted categories, Indicates the current predicted category, Indicates the category The predicted probability function, represents the adjustment factor for focus loss, represents the loss function used to train the detection box positioning, They represent the horizontal coordinate of the center point of the detection frame, the vertical coordinate of the center point of the detection frame, the width of the detection frame, and the height of the detection frame respectively. Represents the detection box parameters after scale transformation, Represents the parameters of the annotation box after scale transformation, represents the smoothing function parameters, Represents the loss scale adjustment factor, which is used to balance the scale of triplet loss and other losses. represents the triplet loss function, represents the number of positive samples selected, represents the positive sample set, Represents the current sample, Represents the distance parameter between different categories, Represents the distance between the current sample and the positive sample, Represents the distance between the current sample and the negative sample, represents the negative sample set.

4. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to claim 3, wherein: The steps of matching the two-dimensional detection frames in the paired two-dimensional image slices using the breast tissue structure representation vector of the two-dimensional detection frames in the paired two-dimensional image slices include: Obtain the two-dimensional detection candidate frames corresponding to the head-foot position and the two-dimensional detection candidate frames corresponding to the medial and lateral positions in the paired two-dimensional image tomography, calculate the correlation between the two-dimensional detection candidate frames through the breast tissue structure representation vectors corresponding to the two-dimensional detection candidate frames, and based on the correlation, use the Hungarian matching algorithm to determine the one-to-one matching relationship between the two-dimensional detection candidate frames to obtain the paired two-dimensional detection frames, and use the formula Adjust the structural distortion score within the paired 2D detection candidate box; Among them, the formula The expression is: Where, represents the weight parameter, Represents the structural distortion score of the current two-dimensional candidate box, Represents Structural distortion scores for paired 2D proposals.

5. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to any one of claims 1 to 4, characterized in that: The step of fusing the paired two-dimensional detection frames into a three-dimensional detection frame includes: Based on the paired two-dimensional detection frames, the centroids of the two-dimensional detection frames on each slice of the paired two-dimensional image slices are calculated, and all the calculated centroids are projected onto the same plane. All the centroids projected onto the same plane are clustered using a density-based spatial clustering method to obtain a plurality of cluster centers. All 2D detection frames belonging to the same cluster center are stacked in the upper and lower layers of the breast slice to form a series of 3D connected regions. Use 3D morphological erosion and dilation operations to connect adjacent connected branches in the same cluster and remove connected branches that only appear in one or two faults; Calculate the bounding box of each remaining three-dimensional connected branch, take the bounding box as the ROI in the three-dimensional space, take the maximum two-dimensional prediction probability in the ROI as the prediction probability of the entire ROI, and the three-dimensional detection box is the ROI.

6. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to claim 5, characterized in that: The steps of respectively calculating a prediction probability deviation measure of the three-dimensional detection frame, a three-dimensional anatomical coordinate spatial position of the three-dimensional detection frame, and a breast parenchyma ratio of the three-dimensional detection frame, and eliminating the three-dimensional detection frame according to the calculation results to obtain a retained three-dimensional detection frame include: Calculating the prediction probability deviation metric of the three-dimensional detection frame and removing the three-dimensional detection frame whose prediction probability deviation metric is greater than a preset threshold, the steps include: For any 3D detection frame, assuming that it contains K faults, the prediction probability of the computer-aided detection model on each fault is arranged from top to bottom according to the spatial position, forming a prediction probability sequence, which is recorded as ;in, Indicates the The predicted probability value of each fault; With width Sliding window, calculate the sequence The sliding mean and sliding standard deviation of is the default value; The sliding standard deviation corresponding to the maximum sliding mean is selected as the prediction probability deviation metric of the 3D detection frame; Eliminate the 3D detection frames whose predicted probability deviation measure is greater than the preset threshold; Obtaining pectoralis major muscle position information and nipple position information, and constructing a three-dimensional anatomical coordinate space based on the pectoralis major muscle position information and nipple position information using the pectoralis major muscle axis, the nipple axis, and the upper and lower fault direction axes; The pectoralis major axis includes: a boundary line between the pectoralis major region and the breast region; The nipple axis includes: a vertical line starting from the nipple position and perpendicular to the pectoralis major axis; The upper and lower fault direction axes include: the direction axes of the upper and lower layers between the faults; The pectoralis major axis, the nipple axis and the upper and lower fault direction axes are perpendicular to each other; Calculating the 3D anatomical coordinate space position of the 3D detection frame and removing the 3D detection frame whose 3D anatomical coordinate space position is outside a preset acceptance range, the steps include: Based on the three-dimensional anatomical coordinate space, statistical training set The voxel distance frequency between the geometric center of the structural distortion position of the paired two-dimensional image tomography and the pectoralis major line is selected from the voxel distance frequency, and the lower limit value and the upper limit value of the voxel distance acceptance range are selected. The expression of the voxel distance acceptance range is: ,in, Indicates the lower limit of the voxel distance acceptance range, Indicates the upper limit of the voxel distance acceptance range; Calculating the 3D anatomical coordinate space position of the 3D detection frame, and removing the 3D detection frame whose 3D anatomical coordinate space position is outside a preset acceptance range; The three-dimensional anatomical coordinate space position of the three-dimensional detection frame includes: the voxel distance frequency from the geometric center of the three-dimensional detection frame to the pectoralis major line; the preset acceptance range includes: voxel distance acceptance range .

7. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to claim 6, wherein: The steps of respectively calculating a prediction probability deviation measure of the three-dimensional detection frame, a three-dimensional anatomical coordinate spatial position of the three-dimensional detection frame, and a breast parenchyma ratio of the three-dimensional detection frame, and removing the three-dimensional detection frame according to the calculation results to obtain a retained three-dimensional detection frame further include: Calculating the breast parenchyma ratio of the 3D detection frame and removing the 3D detection frame whose breast parenchyma ratio is less than a preset threshold, the steps include: The 2D fault where the 3D geometric center of the 3D prediction frame is located is selected as the central fault. The OTSU adaptive threshold segmentation algorithm is used on the central fault to distinguish the solid tissue and fat tissue in the breast area, and the breast solid proportion of the 3D detection frame is calculated. , The expression is: Where, represents the number of solid tissue pixels in the breast area of ​​the central slice of the 3D prediction box, Indicates the size of the 3D prediction box of the central fault; Breast parenchyma ratio Eliminate 3D detection frames that are smaller than the preset threshold and set the position quantile acceptance range ,in, Indicates the lower limit of the acceptance range of the position quantile, It represents the upper limit of the position quantile acceptance range. The three-dimensional prediction frame within the central fault range that exceeds the position quantile acceptance range is eliminated. The expression of the position quantile is: Where, represents the position quantile, Indicates the ordinal number of the layer where the 3D prediction box is located, Indicates the total number of fault layers.

8. The method for detecting structural distortion based on dual-view digital breast tomosynthesis according to claim 7, wherein: The steps of calculating the ratio of the 2D detection frames that are paired with the remaining 3D detection frames and discarding them, and discarding the 3D detection frames whose ratio is greater than a preset threshold, to obtain the structural distortion detection results include: The inner product of the breast tissue structure representation vectors of the 2D detection frame of the retained 3D detection frame and its paired 2D detection frame is performed to obtain the correlation between the paired 2D detection frames. Based on the correlation, the Hungarian matching algorithm is used to determine the one-to-one pairing relationship between the paired 2D detection frames. The mapping expression of the pairing relationship is: Where, Indicates the two-dimensional detection frame corresponding to a projection position angle in the paired two-dimensional detection frame, Indicates the two-dimensional detection frame corresponding to another projection position perspective in the paired two-dimensional detection frame; Combine the two-dimensional detection frames corresponding to all the remaining three-dimensional detection frames into a set , based on the set Count the percentage of 2D detection frames that are paired with the 2D detection frames of the retained 3D detection frames, and measure the percentage as pairing false positives. to measure; Pairwise false positive metric The 3D detection frames larger than the preset threshold are removed to obtain the final remaining 3D detection frames.

9. A structural distortion prediction system based on dual-view digital breast tomosynthesis, used to implement the structural distortion detection method based on dual-view digital breast tomosynthesis according to any one of claims 1 to 8, characterized in that: include: An image processing module is used to acquire three-dimensional breast tomography images including two projection position viewing angles and process them into two-dimensional image slices; A labeling module for labeling the location of structural distortion in a two-dimensional image slice using a bounding box; A pairing module is used to pair two annotated two-dimensional image slices belonging to the same breast from two projection positions and perspectives; A two-dimensional detection frame prediction module is used to use a computer-aided detection model to predict the two-dimensional detection frame corresponding to the structural distortion position in the paired two-dimensional image slices and extract the representation vector of the breast tissue structure within the two-dimensional detection frame; and use the breast tissue structure representation vector of the two-dimensional detection frame in the paired two-dimensional image slices to match the two-dimensional detection frames in the image slices of the two projection positions and perspectives; The three-dimensional detection frame prediction module is used to fuse the paired two-dimensional detection frames output by the two-dimensional detection frame prediction module into a three-dimensional detection frame, and respectively calculate the prediction probability deviation measurement of the three-dimensional detection frame, the three-dimensional anatomical coordinate space position of the three-dimensional detection frame, and the breast substance proportion of the three-dimensional detection frame. According to the calculation results, the three-dimensional detection frame is eliminated to obtain a retained three-dimensional detection frame; for the retained three-dimensional detection frame, the proportion of the two-dimensional detection frame paired with the two-dimensional detection frame that has been eliminated is counted, and the three-dimensional detection frame with a proportion greater than a preset threshold is eliminated, and the final retained three-dimensional detection frame is output as the structural distortion position prediction result.

Images