Physician-interpretable ultrasound-based breast cancer pathological subtype detection system

Abstract

Disclosed in the present invention is a physician-interpretable ultrasound-based breast cancer pathological subtype detection system, comprising: a segmentation module, which uses a U-Net network and incorporates a foreground optimization network to segment a breast and a nodule; a texture channel, which extracts a texture channel feature; an edge channel, which extracts boundary information; a position channel, which extracts position information, so as to quantize the overall relationship between the nodule and the boundary information of the breast to extract a breast nodule growth position feature; a shape channel, which calculates an edge aspect ratio on the basis of a segmented nodule region, so as to obtain a predicted interval value as a shape channel feature; an echo channel, which calculates the echo difference between the interior and exterior of the nodule to serve as an echo channel feature value; and a joint prediction module, which uses a multinomial logistic regression model to classify and predict breast cancer pathological subtypes on the basis of output features of the texture channel, the edge channel, the echo channel, the shape channel and the position channel. An artificial intelligence diagnosis tool that is interpretable to physicians is built, thereby providing an automated theoretical basis for clinical diagnosis and decision-making.

Description

Ultrasound-based breast cancer pathology subtype detection system understandable by doctors TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical artificial intelligence, and particularly relates to an ultrasound-based breast cancer pathology subtype detection system understandable by doctors. BACKGROUND

[0002] Breast cancer is one of the most common malignant tumors in women and a major cause of cancer-related deaths worldwide. Early detection and treatment of breast cancer are critical to improve survival and reduce mortality. As a non-invasive, low-cost and easy-to-operate imaging technology, ultrasound imaging has become an important tool for breast cancer screening and diagnosis. However, due to the heterogeneity of breast cancer, it is still challenging to accurately classify only relying on ultrasound imaging features. TECHNICAL PROBLEM

[0003] In recent years, artificial intelligence (AI) technology has made significant progress in medical image analysis, especially deep learning algorithms have shown strong performance in image recognition and classification tasks. Based on this, researchers have begun to explore the use of AI technology to automatically classify breast cancer ultrasound images to improve the accuracy and efficiency of diagnosis. However, the lack of explainability of AI has been criticized, hindering the understanding and application of medical staff, and the accuracy achieved by current technology is also difficult to satisfy. TECHNICAL SOLUTION

[0004] To solve the above problems, the present application provides an ultrasound-based breast cancer pathology subtype detection system understandable by doctors, which extracts features through separate channels, performs multi-logistic regression classification prediction of four subtypes of benign, non-invasive, early invasive and invasive based on numerical features of texture, edge, echo, shape and position channels, builds an artificial intelligence diagnosis tool that doctors can understand, provides an automated theoretical basis for clinical diagnosis and decision-making, and analyzes the contribution of each factor to the overall prediction based on analysis of variance (ANOVA).

[0005] To achieve the above purpose, the technical scheme of the present application is as follows: an ultrasound-based breast cancer pathology subtype detection system understandable by doctors, comprising:

[0006] A segmentation module adopts a U-Net network and embeds a foreground optimization network to segment the breast and the nodule, wherein the foreground-background relationship module improves the discrimination of the nodule itself by associating the surrounding semantic information related to the nodule shape pixels, adjusts the gradient descent direction to improve the recognition rate, inputs the segmentation output and the original ultrasound image into multiple channels for feature extraction, and the multiple channels include a texture channel, an edge channel, an echo channel, a shape channel and a position channel;

[0007] Texture channel, based on Resnet-18, texture-based malignancy risk prediction is performed to obtain texture channel features;

[0008] Edge channel, based on the U-Net network and foreground optimization network segmentation, obtains the boundary information of white lines without background texture;

[0009] A position channel is used to register the segmented breast and nodule ultrasound images in a pre-configured breast template based on the boundary information, thereby obtaining the position information in the current registered image and quantifying the overall relationship between the nodule and the breast boundary information to extract the breast nodule growth position feature;

[0010] Shape channel, according to the segmented nodule area, calculates the edge aspect ratio and obtains the prediction interval value as the shape channel feature;

[0011] Echo channel: the difference between the echo inside and outside the nodule is calculated as the echo channel characteristic value;

[0012] The joint prediction module classifies and predicts breast cancer pathological subtypes based on the output features of the texture channel, edge channel, echo channel, shape channel and position channel through a multinomial logistic regression model. The breast cancer pathological subtypes include benign, non-invasive, early invasive and invasive subtypes.

[0013] Preferably, the prospect optimization network is: , where f i is the i-th layer in the pyramid feature map, represents a set of FPN feature maps with the same number of channels d, M represents the lateral connection of the learnable convolutional layer, and N represents the nearest neighbor upsampling;

[0014] A branch is attached to C5 to generate scene features C6 through global context aggregation. In order to transform the feature map, a scale-aware projection function is introduced in the following equation: ,in yes The learnable parameters of

[0015] Transformed feature map : , in order to obtain the relationship between the mappings, use To calculate the scene embedding vector v, , where I is the projection function implemented by the learnable convolutional layer and the vector v is shared for each pyramid;

[0016] By point-by-point inner product The similarity estimation achieved is used to obtain the relationship mapping r i , the encoder input pyramid feature map f ire-encoding to obtain a re-encoded feature map , the encoder has learnable parameters : ;

[0017] , thereby obtaining the relationship-enhanced foreground feature map g i : .

[0018] Preferably, the baseline of the foreground optimization network of the segmentation module is a ResNet-50 model, pre-trained on ImageNet, and the sizes of the convolution kernels include 4x3x4, 8x3x8, 16x3x16 and 32x3x32; the learning rate is 0.007, and the maximum number of training iterations is set to 60,000; when training the U-Net network, the batch-size is set to 8, the Adam optimizer is used, the learning rate is set to 0.001, and the maximum number of iterations is set to 300.

[0019] Preferably, the texture channel adopts Resnet-18 as the feature extraction framework to obtain the boundary information of the image output by the segmentation module, and then extract the texture features.

[0020] Preferably, the edge channel adopts Resnet-18 as the feature extraction framework, adopts the random mosaic method to extract a large number of random blocks from the edge region of the breast nodule, and splices them into a complete image, which ensures the consistency of the extracted features in the case of different sizes of different nodules, maximizes the preservation of the details of the original image to preserve the edge information, and uses the parameters of the ImageNet pre-trained model for training.

[0021] Preferably, the position channel registers the segmented breast and nodule ultrasound images based on the boundary information to obtain a deformation field, maps each nodule to a breast template, and automatically obtains the position information in the current registration image. Specifically, the software package ANTs based on the symmetric differential homeomorphism method is used for registration. The software package ANTs assumes that the differential homeomorphism is defined on the image domain , and an affine transformation is retained at the boundary, so that , where is an affine mapping applied to the identity transformation, mapping a family of differential homeomorphisms parameterized by time is generated by integrating a time-varying, smooth velocity field; is generated by the ordinary differential equation , The deformation field generated is , when decomposed into two components and , and define , , , where ;

[0022] The standard large deformation diffeomorphic metric matching (LDDMM) equation is generated as follows:

[0023]

[0024] The method of registration is'syn', the method of interpolation adopts linear interpolation, the optimization metric in the registration process is mutual information, and the final evaluation index is mean square error:

[0025] ,

[0026] ,

[0027] Where J is the breast image before registration, J ’ is the breast image after registration, M and N represent the width and height of the image respectively, after segmentation and registration operation, the breast is mapped to a fixed position, and the actual physical position information of the nodule and the breast is converted into relative position information in the current registration picture.

[0028] Preferably, the shape channel calculates the edge aspect ratio according to the segmented nodule region, obtains a prediction interval value as a shape channel feature; the echo channel calculates the difference between the average value of the echo in the outer breast region of the nodule and the average value of the echo in the nodule, and then fits the distribution of the echo difference through a Gaussian function to obtain a prediction interval as an echo channel feature value.

[0029] Preferably, for the texture feature, the position feature and the edge feature, the corresponding prediction value is obtained as the value range of the channel through a deep learning method, for the shape feature, the aspect ratio of the shape is taken as the value range of the shape channel, and for the echo feature, the echo difference between the inside and outside of the nodule is taken as the value range of the echo channel.

[0030] Preferably, the formula of the multiple logistic regression model of the joint prediction module is: , wherein x1, x2, x3, x4 and x5 are the prediction probabilities of malignancy of the texture channel, the echo channel, the edge channel, the shape channel and the position channel respectively, and y is the prediction result of the model.

[0031] Preferably, the joint prediction module further performs effect evaluation based on a verification set and an external test set, and finally analyzes the contribution of each factor to the overall prediction classification based on analysis of variance. Beneficial effects

[0032] The present application has the following advantages and positive effects compared with the prior art by adopting the above technical scheme:

[0033] 1、The technical scheme of the present application realizes the artificial intelligence detection system that doctors can understand by extracting and jointly predicting the features with specific biological significance through multiple channels, scoring the texture, edge, echo, shape and position of the breast cancer ultrasound image, and jointly giving the detection results of benign, non-infiltrative, early infiltrative and infiltrative in the pathological subtype of breast cancer, thereby providing an automatic theoretical basis for clinical diagnosis and decision-making.

[0034] 2、The technical scheme of the present application adopts a foreground optimization segmentation network with a feature pyramid network (FPN) as the backbone, enhances the context correlation of the foreground by calculating a global similarity matrix, thereby solving the problem of the fuzzy and complex background of breast cancer ultrasound, and accurately segmenting the position of the nodule for subsequent processing. The design of the texture, edge, echo, shape and position channels facilitates medical staff to read the breast cancer ultrasound report and focus on and understand the content.

[0035] 3、In the scheme of the present application, the position channel needs to predict the risk of malignancy according to the position relationship, adopts a graph convolution network structure, can construct the position relationship in a higher dimension, and more effectively excavates the corresponding relationship implied in the information. Therefore, the position relationship between the breast and the nodule is converted into a graph structure in this embodiment, the distance between the nodule and the breast is represented by the edge weight in the graph. A deep graph convolution network is used to extract the mapped structure features. Finally, the position channel can obtain a specific score of the position features.

[0036] 4、The scheme of the present application adopts a graph convolution network structure, can construct the position relationship in a higher dimension, and more effectively excavates the corresponding relationship implied in the information. Therefore, the position relationship between the breast and the nodule is converted into a graph structure in this embodiment, the distance between the nodule and the breast is represented by the edge weight in the graph. A registration method based on differential homeomorphism is used to homogenously locate the nodule in various patient ultrasound images, and a deep graph convolution network is used to extract the mapped structure features, and the position information of the breast cancer nodule is predicted. BRIEF DESCRIPTION OF DRAWINGS

[0037] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings, in which:

[0038] FIG. 1 is a flowchart of the construction process of the detection system of the present application;

[0039] FIG. 2 is an example of a benign breast cancer ultrasound;

[0040] FIG. 3 is an example of a non-infiltrative breast cancer ultrasound;

[0041] Fig. 4 is an example of an ultrasound image of early invasive breast cancer;

[0042] Fig. 5 is an example of an ultrasound image of invasive breast cancer. Embodiments of the present application

[0043] The present application will be further described with reference to the drawings and specific examples. The advantages and features of the present application will become apparent from the following description of the preferred embodiments and from the claims. It is to be understood that the drawings are designed solely for the purpose of illustration and are not drawn to precise scale, in which:

[0044] It should be noted that all directional references (e.g., upper, lower, left, right, front, rear, etc.) are in relation to the exemplary embodiment of the present application as illustrated in the drawings and are made only by way of descriptions and illustration and are not to be construed as limiting the present application. First embodiment

[0045] As shown in Fig. 1, the present embodiment provides a doctor-understandable ultrasound-based breast cancer pathological subtype detection system, comprising:

[0046] A segmentation module, which uses a U-Net network and embeds a foreground optimization network to segment the breast and the nodule, wherein the foreground-background relationship module improves the discrimination of the nodule itself by associating the surrounding semantic information related to the nodule shape pixels, adjusts the gradient descent direction to improve the recognition rate, and inputs the segmentation output and the original ultrasound image into multiple channels for feature extraction, the multiple channels including a texture channel, an edge channel, an echo channel, a shape channel and a position channel;

[0047] The texture channel is based on Resnet-18 for texture-based malignant risk prediction to obtain texture channel features;

[0048] The edge channel obtains the boundary information of the white line without background texture based on the breast and the nodule segmented by the U-Net network and the foreground optimization network;

[0049] The position channel registers the segmented breast and nodule ultrasound image in a pre-configured breast template according to the boundary information, thereby obtaining the position information in the current registration image to quantify the overall relationship between the nodule and the breast boundary information and extract the breast nodule growth position feature;

[0050] The shape channel calculates the edge aspect ratio according to the segmented nodule region to obtain the prediction interval value as the shape channel feature;

[0051] The echo channel calculates the echo difference inside and outside the nodule as the echo channel feature value;

[0052] The joint prediction module classifies and predicts the pathological subtypes of breast cancer, including benign, non-invasive, early invasive and invasive, through a multinomial logistic regression model based on the output features of the texture channel, the edge channel, the echo channel, the shape channel and the position channel.

[0053] The embodiment extracts features with specific biological significance through multiple channels for joint prediction, and gives the detection results of benign, non-invasive, early invasive and invasive in the pathological subtypes of breast cancer based on the texture, edge, echo, shape and position scores of the breast cancer ultrasound image, so as to realize an artificial intelligence detection system that can be understood by doctors and provide an automatic theoretical basis for clinical diagnosis and decision-making.

[0054] Referring to FIG. 1, it is a flowchart of the construction process of the detection system of the present application. In order to analyze the characteristics of the nodules, the nodules must be effectively segmented as the input for classification. However, unlike natural images or other medical images, breast cancer ultrasound images have the following characteristics: large scale variation, low inter-organ contrast, blurred background, relatively small detection target, and small number. Due to the large intra-class variance of the background, more serious misjudgment may occur, bringing great challenges to the segmentation task. Therefore, the present application uses a foreground optimization method to accurately segment the breast nodules. The foreground optimization FPN in the method can solve the problem of multi-scale input; the foreground-background relationship module can improve the discrimination of the nodules themselves by associating the surrounding semantic information related to the nodule shape pixels. The purpose is to enhance the difference between the nodules and the surrounding background. In order to pay attention to the classification error samples that are misjudged as nodules in the background, the gradient descent direction is adjusted to improve the recognition rate, and a foreground optimization module is embedded in the model. In addition, many models produce jagged edges when segmenting breasts. Considering the segmentation performance and reducing the jagged edges, after many model comparison experiments, we use the U-Net network as the breast segmentation module.

[0055] Specifically, the foreground optimization network is: , wherein f i is the i-th layer in the pyramid feature map, indicates a set of FPN feature maps with the same channel number d, M represents a horizontal connection of a learnable convolutional layer, and N represents nearest neighbor up-sampling;

[0056] A branch is added to C5 to generate scene features C6 by global context aggregation. In order to transform the feature map, a scale-aware projection function is introduced in the following equation: , wherein is a learnable parameter of ;

[0057] The transformed feature map To obtain the relationship between the mappings, we use to compute the scene embedding vector v, where I is a projection function implemented by a learnable convolutional layer, and the vector v is shared for each pyramid;

[0058] The relationship mapping r is obtained by a similarity estimation implemented by a point-wise inner product i The encoder re-encodes the input pyramid feature maps f i to obtain the re-encoded feature maps The encoder has learnable parameters

[0059] Thus, we obtain the relationship-enhanced foreground feature maps g i

[0060] Based on the above principles, we implement a foreground-optimized segmentation model to extract nodule information. For the nodule segmentation module, the baseline of the foreground-optimized network is a ResNet-50 model, which has been pre-trained on ImageNet.

[0061] The foreground-optimized segmentation network with a feature pyramid network (FPN) as the backbone is adopted. By calculating the global similarity matrix, the context correlation of the foreground is enhanced to solve the problem of fuzzy and complex background of breast cancer ultrasound. The location of the nodule is accurately segmented for subsequent processing. The design of texture, edge, echo, shape, and location channels facilitates medical staff to read the breast cancer ultrasound report and focus on the content that is easy to understand.

[0062] Preferably, the baseline of the foreground-optimized network of the segmentation module is a ResNet-50 model, which is pre-trained on ImageNet. The size of the convolution kernel includes 4x3x4, 8x3x8, 16x3x16, and 32x3x32. The learning rate is 0.007, and the maximum training iteration number is set to 60,000. When training the U-Net network, the batch-size is set to 8, the Adam optimizer is used, the learning rate is set to 0.001, and the maximum iteration number is set to 300.

[0063] Thereafter, five channels of texture, edge, echo, shape, and location are constructed. Specifically, the segmentation output and the original image are jointly used as input, and five channels of texture, edge, echo, shape, and location that are mentioned in the ultrasound report and can be understood by doctors are defined.

[0064] ​​​​​​Preferably, the texture channel adopts Resnet-18 as the framework of feature extraction to obtain the boundary information of the image output by the segmentation module, and then extracts the texture features.

[0065] The segmented data is modeled, Resnet-18 is used for texture-based malignant risk prediction, and texture channel features are obtained.

[0066] Preferably, the edge channel adopts Resnet-18 as the framework of feature extraction, adopts a random mosaic method to extract a large number of random blocks from the edge area of the breast nodule, and splices them into a complete image, which ensures the consistency of the extracted features in the case of inconsistent sizes of different nodules, maximizes the preservation of the details of the original image to preserve the edge information, and uses the parameters of the ImageNet pre-trained model for training.

[0067] Even experienced doctors cannot provide accurate criteria for how to filter edge features from the entire ultrasound image. This embodiment adopts a new "random mosaic" method to maximize the preservation of the details of the original image, thereby preserving the edge information. In combination with the segmentation task, the edge area is determined, which exhibits good performance. In the experiment, we set the size of each image block to 16x16 pixels, and each spliced image has 14x14 image blocks. Then, ResNet-18 is used to predict the risk of malignant tumors, and the input is the mosaic image.

[0068] Analyzing the impact of the growth position of the breast nodule on the malignant prediction, the overall relationship between the nodule and the breast edge information is quantified. First, the U-Net network and the foreground optimization network are used to segment the breast and the nodule respectively, and the boundary information (white lines without background texture) is obtained. Then, the position channel predicts the malignant risk according to the boundary information. This embodiment pre-configures a unified breast template, and registers the segmented breast and nodule ultrasound images to the template. According to the deformation field obtained by registering the breast, each nodule is mapped to the breast template to automatically obtain the position information in the current registered image.

[0069] Specifically, the position channel registers the segmented breast and nodule ultrasound images based on the boundary information to obtain a deformation field, maps each nodule to the breast template, and automatically obtains the position information in the current registered image. Specifically, the software package ANTs based on symmetric differential homeomorphism method is used for registration. The software package ANTs assumes that the differential homeomorphism is defined in the image domain and preserves an affine transformation at the boundary, so that , where is an affine mapping applied to the identity transformation, mapping A family of parametrized diffeomorphisms is generated over time by integrating a time-dependent, smooth velocity field; by a system of ordinary differential equations , The deformation field generated is When is decomposed into two components and , and define , , where ;

[0070] The standard large deformation diffeomorphic metric matching (LDDMM) equation is generated as follows:

[0071]

[0072] The method of registration is'syn', the method of interpolation is linear interpolation, the optimization metric in the registration process is mutual information, and the final evaluation index is mean square error:

[0073] ,

[0074] ,

[0075] Where J is the breast image before registration, J ’ is the breast image after registration, M and N represent the width and height of the image respectively, after segmentation and registration operation, the breast is mapped to a fixed position, and the actual physical position information of the nodule and the breast is converted into the relative position information in the current registration picture.

[0076] In the scheme of the embodiment, the position channel needs to predict the risk of malignancy according to the position relationship, adopts a graph convolution network structure, can construct the position relationship in a higher dimension, and more effectively excavate the corresponding relationship hidden in the information. Therefore, the embodiment converts the position relationship between the breast and the nodule into a graph structure, and uses an edge weight in the graph to represent the distance between the nodule and the breast. A registration method based on a diffeomorphism is used to homogeneously locate the nodule in various patient ultrasound images, and a deep graph convolution network is used to extract the mapped structural features, so as to predict the position information of the breast cancer nodule. Finally, the position channel can obtain a specific score of the position feature.

[0077] Preferably, the shape channel calculates an edge length-width ratio according to a segmented nodule region, obtains a prediction interval value as a shape channel feature; and the echo channel calculates a difference between an average echo value of a breast region outside the nodule and an average echo value inside the nodule, and then fits a distribution of the echo difference by a Gaussian function to obtain a prediction interval as an echo channel feature value.

[0078] The technical solution of the embodiment adopts multiple feature extraction channels with specific biological significance for joint prediction, which on one hand improves the explainability of artificial intelligence medical treatment, and on the other hand is beneficial to providing an automatic theoretical basis for clinical diagnosis and decision-making.

[0079] Preferably, for the texture feature, the position feature and the edge feature, corresponding prediction values are obtained by a deep learning method as the value range of the channel, for the shape feature, the aspect ratio of the shape is taken as the value range of the shape channel, and for the echo feature, the echo difference between inside and outside the nodule is taken as the value range of the echo channel.

[0080] Preferably, a formula of the multiple logistic regression models of the joint prediction module is as follows: , wherein x1, x2, x3, x4 and x5 are respectively prediction malignant probabilities of the texture channel, the echo channel, the edge channel, the shape channel and the position channel, and y is a prediction result of the model.

[0081] The technical solution combines the outputs of the texture channel, the edge channel, the echo channel, the shape channel and the position channel, constructs a multiple logistic regression model from the perspective of these features, and performs classification prediction on four subtypes of benign, non-invasive, early invasive and invasive.

[0082] Preferably, the joint prediction module further performs effect evaluation based on a verification set and an external test set, and finally analyzes the contribution of each factor to the overall prediction classification based on variance analysis.

[0083] The embodiment also uses training, verification and external test sets in the five channels. The logistic regression model is trained using the training set, and effect evaluation is performed on the verification set and the external test set. Finally, the contribution of each factor to the overall prediction classification is analyzed based on variance analysis (ANOVA). In actual application, a single input image will obtain five channel prediction results, respective interval values, overall prediction results and the contribution of each factor to the overall result.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific execution of the identification of the system and the device described above can refer to the corresponding process in the foregoing method embodiment.

[0085] The embodiments of the present application are explained in detail above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments. Even if various changes are made to the present application, if the changes fall within the scope of the claims of the present application and equivalents thereof, they are still within the protective scope of the present application.

Claims

1. A physician interpretable ultrasound based breast cancer pathology subtype detection system, characterized in that, Comprise: The segmentation module adopts a U-Net network and embeds a foreground optimization network to segment the breast and the nodule, wherein the foreground-background relationship module improves the discrimination of the nodule itself by associating the surrounding semantic information related to the nodule shape pixels, adjusts the gradient descent direction to improve the recognition rate, and inputs the segmentation output and the original ultrasound image into multiple channels for feature extraction, the multiple channels including a texture channel, an edge channel, an echo channel, a shape channel and a position channel; The texture channel is based on Resnet-18 for texture-based malignant risk prediction to obtain texture channel features; The edge channel obtains the boundary information of the white line without background texture based on the breast and the nodule segmented by the U-Net network and the foreground optimization network; The position channel registers the segmented breast and nodule ultrasound image in the pre-configured breast template according to the boundary information, thereby obtaining the position information in the current registration image to quantify the overall relationship between the nodule and the breast boundary information and extract the breast nodule growth position feature; The shape channel calculates the edge aspect ratio according to the segmented nodule region to obtain a prediction interval value as the shape channel feature; The echo channel calculates the echo difference value inside and outside the nodule as the echo channel feature value; The joint prediction module classifies and predicts the breast cancer pathological subtypes based on the output features of the texture channel, the edge channel, the echo channel, the shape channel and the position channel through a multinomial logistic regression model, the breast cancer pathological subtypes including benign, non-invasive, early invasive and invasive four subtypes.

2. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 1, wherein, The foreground optimization network is: where f i is the i-th layer in the pyramid feature map, represents a set of FPN feature maps with the same number of channels d, M represents a horizontal connection of a learnable convolutional layer, and N represents nearest neighbor up-sampling. A branch is added on C5, and the scene feature C6 is generated by global context aggregation. In order to transform the feature map, a scale-aware projection function is introduced: wherein is The learnable parameters of the joint prediction module include: transformed feature map ： , to obtain the relationship between the mappings, use to compute a scene embedding vector v, Where I is a projection function implemented by a learnable convolutional layer, and the vector v is shared for each pyramid; by pointwise inner product A similar estimation is implemented to obtain the relation mapping r i The encoder re-encodes the input pyramid feature map f i to obtain a re-encoded feature map , the encoder has learnable parameters ： ； Thus, the relation-enhanced foreground feature map g is obtained i : 。 3. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 2, wherein, The baseline of the foreground optimization network of the segmentation module is a ResNet-50 model, which is pre-trained on ImageNet, and the size of the convolution kernel includes 4x3x4, 8x3x8, 16x3x16 and 32x3x32; the learning rate is 0.007, and the maximum number of training iterations is set to 60,000; when training the U-Net network, the batch-size is set to 8, the Adam optimizer is used, the learning rate is set to 0.001, and the maximum number of iterations is set to 300.

4. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 1, wherein, The texture channel uses Resnet-18 as the feature extraction framework to obtain the boundary information of the image output by the segmentation module, and then extracts the texture features.

5. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 1, wherein, The edge channel uses Resnet-18 as the feature extraction framework, uses the random mosaic method to extract a large number of random blocks from the edge region of the breast nodule, and splices them into a complete image, which ensures the consistency of the extracted features in the case of different nodule sizes, maximizes the preservation of the details of the original image to preserve the edge information, and uses the parameters of the ImageNet pre-trained model for training.

6. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 5, wherein, The position channel registers the segmented breast and nodule ultrasound images based on the boundary information to obtain a deformation field, maps each nodule to a breast template, and automatically obtains position information in a current registration image; specifically, registration is performed by using a software package ANTs based on a symmetric differential homeomorphism method, the software package ANTs assumes a differential homeomorphism defined in an image domain and retains an affine transformation at a boundary, so that where is an affine mapping applied to an identity transformation, and the mapping parameterizes a family of differential homeomorphisms over time, which is generated by integrating a smooth velocity field that changes over time; is generated by an ordinary differential equation , the generated deformation field is when is decomposed into two components and and , , , where ; The standard large deformation differential homomorphism metric matching equation is generated as follows: ， The registration method is'syn', the interpolation method uses linear interpolation, the optimization metric in the registration process is mutual information, and the final evaluation index is mean square error: ， ， where J is the pre-registration breast image, J ’ is the post-registration breast image, M and N represent the width and height of the image, respectively. After segmentation and registration, the breast is mapped to a fixed position, and the actual physical location information of the nodule and breast is converted to relative location information in the current registration picture.

7. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 1, wherein, The shape channel calculates an edge length-width ratio according to the segmented nodule region, obtains a prediction interval value as a shape channel feature; and the echo channel calculates a difference between an average value of an echo of a breast region outside the nodule and an average value of an echo inside the nodule, and then fits a distribution of the echo difference by using a Gaussian function to obtain a prediction interval as an echo channel feature value.

8. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 7, wherein, For the texture feature, the position feature and the edge feature, corresponding prediction values are obtained by using a deep learning method as a value range of the channel; for the shape feature, an aspect ratio of the shape is taken as a value range of the shape channel; and for the echo feature, an echo difference between inside and outside the nodule is taken as a value range of the echo channel.

9. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 1, wherein, The multiple logistic regression model formula of the joint prediction module is: , wherein x1, x2, x3, x4 and x5 are respectively the predicted malignant probability of the texture channel, the echo channel, the edge channel, the shape channel and the position channel, and y is the prediction result of the model.

10. The physician interpretable ultrasound-based breast cancer pathology subtype detection system of claim 1, wherein, The joint prediction module further performs effect evaluation based on the verification set and the external test set, and finally analyzes contributions of various factors to overall prediction classification based on variance analysis.

Images