Ultrasonic-based breast cancer pathological subtype detection system capable of being understood by doctors

Abstract

The invention discloses an ultrasound-based breast cancer pathological subtype detection system understandable by doctors, which comprises a segmentation module for segmenting a breast and a nodule by adopting a U-Net network and embedding a foreground optimization network; texture channel extraction: extracting texture channel features; the edge channel is used for extracting boundary information; the position channel is used for extracting position information so as to quantify the overall relation between nodules and breast boundary information and extract breast nodule growth position features; a shape channel, calculating an edge length-width ratio according to the segmented nodule region, and obtaining a prediction interval value as a shape channel feature; an echo channel: calculating an echo difference value inside and outside a nodule as an echo channel characteristic value; and the combined prediction module is used for carrying out classification prediction on the breast cancer pathological subtypes through a multi-term logistic regression model based on the output characteristics of the texture channel, the edge channel, the echo channel, the shape channel and the position channel. An artificial intelligence diagnosis tool which can be understood by a doctor is built, and an automatic theoretical basis is provided for clinical diagnosis and decision making.

Description

Technical Field

[0001] The present invention belongs to the field of medical artificial intelligence technology, and in particular relates to an ultrasound-based breast cancer pathological subtype detection system that is understandable to doctors. Background Art

[0002] Ultrasound imaging, as a non-invasive, low-cost, and easy-to-use imaging technique, has become an important tool for breast cancer screening and diagnosis. However, due to the heterogeneity of breast cancer, accurate classification based solely on ultrasound imaging features remains challenging.

[0003] In recent years, artificial intelligence (AI) technology has made significant progress in medical image analysis, with deep learning algorithms, in particular, demonstrating strong performance in image recognition and classification tasks. Consequently, researchers have begun exploring the use of AI to automatically classify breast cancer ultrasound images to improve diagnostic accuracy and efficiency. However, AI's lack of interpretability has been criticized, hindering its understanding and application by medical professionals. Furthermore, the accuracy achieved by current technologies is far from satisfactory. Summary of the Invention

[0004] To address the above problems, the present invention provides a doctor-understandable ultrasound-based breast cancer pathology subtype detection system. By extracting features by channel, multiple logistic regression classification predictions of four subtypes, namely benign, non-invasive, early invasive, and invasive, are performed based on the numerical features of five channels: texture, edge, echo, shape, and position. This builds an artificial intelligence diagnostic tool that doctors can understand, provides an automated theoretical basis for clinical diagnosis and decision-making, and analyzes the contribution of each factor to the overall prediction based on analysis of variance (ANOVA).

[0005] To achieve the above objectives, the technical solution of the present invention is: a doctor-understandable ultrasound-based breast cancer pathological subtype detection system, comprising:

[0006] The segmentation module uses a U-Net network embedded in a foreground optimization network to segment breasts and nodules. The foreground-background relationship module improves the discrimination of nodules by associating surrounding semantic information related to nodule morphology pixels. The gradient descent direction is adjusted to improve the recognition rate. The segmentation output and the original ultrasound image are input into multiple channels for feature extraction, including texture channel, edge channel, echo channel, shape channel, and position channel.

[0007] Texture channel, based on Resnet-18, texture-based malignancy risk prediction is performed to obtain texture channel features;

[0008] Edge channel, based on the U-Net network and foreground optimization network segmentation, obtains the boundary information of white lines without background texture;

[0009] A position channel is used to register the segmented breast and nodule ultrasound images in a pre-configured breast template based on the boundary information, thereby obtaining the position information in the current registered image and quantifying the overall relationship between the nodule and the breast boundary information to extract the breast nodule growth position feature;

[0010] Shape channel, according to the segmented nodule area, calculates the edge aspect ratio and obtains the prediction interval value as the shape channel feature;

[0011] Echo channel: the difference between the echo inside and outside the nodule is calculated as the echo channel characteristic value;

[0012] The joint prediction module classifies and predicts breast cancer pathological subtypes based on the output features of the texture channel, edge channel, echo channel, shape channel and position channel through a multinomial logistic regression model. The breast cancer pathological subtypes include benign, non-invasive, early invasive and invasive subtypes.

[0013] Preferably, the prospect optimization network is: where f i is the i-th layer in the pyramid feature map, C i (i=2, 3, 4, 5) represents a set of FPN feature maps with the same number of channels d, M represents the lateral connection of the learnable convolutional layer, and N represents the nearest neighbor upsampling;

[0014] A branch is attached to C5 to generate scene features C6 through global context aggregation. In order to transform the feature map, a scale-aware projection function is introduced in the following equation: where θ i yes The learnable parameters of

[0015] Transformed feature map To obtain the relationship between the mappings, we use I(·) on C6 to compute the scene embedding vector v, v = I(C6), where I is the projection function implemented by the learnable convolutional layer and the vector v is shared for each pyramid.

[0016] The relationship mapping r is obtained by similarity estimation achieved by point-by-point inner product sim(·) i , the encoder’s input pyramid feature map f i Re-encode and obtain the re-encoded feature map The encoder has learnable parameters ω i :

[0017] Thus, the relationship-enhanced foreground feature map g is obtained i :

[0018] Preferably, the baseline of the foreground optimization network of the segmentation module is the ResNet-50 model, which is pre-trained on ImageNet, and the convolution kernel sizes include 4×3×4, 8×3×8, 16×3×16 and 32×3×32; the learning rate is 0.007, and the maximum number of training iterations is set to 60,000; when training the U-Net network, the batch-size is set to 8, the Adam optimizer is used, the learning rate is set to 0.001, and the maximum number of iterations is set to 300.

[0019] Preferably, the texture channel uses Resnet-18 as a feature extraction framework to obtain the boundary information of the image output by the segmentation module, and then extract texture features.

[0020] Preferably, the edge channel uses Resnet-18 as the framework for feature extraction, and adopts a random mosaic method to extract a large number of random blocks from the edge area of the breast nodule and splice them into a complete image. When the sizes of different nodules are inconsistent, the size of the extracted features is guaranteed to be consistent, so as to maximize the retention of the details of the original image to retain the edge information, and the parameters of the ImageNet pre-trained model are used to train it.

[0021] Preferably, the position channel registers the segmented breast and nodule ultrasound images based on the boundary information to obtain a deformation field, maps each nodule to the breast template, and automatically obtains the position information in the current registration image. Specifically, the registration is performed using the software package ANTs based on the symmetric differential homeomorphism method. The software package ANTs assumes differential homeomorphism. is defined on the image domain Ω and preserves an affine transformation at the boundary such that Where A(Id) is an affine mapping applied to the identity transformation, mapping Over time, parameterize a family of diffeomorphisms Produced by integrating a time-varying, smooth velocity field; v: Ω×t→R d , through the ordinary differential equation The resulting deformation field is When Decompose into two components and And define int∈[0,0.5],v(x,t)=v1(x,t),v(x,t)=v2(x,1-t), where t∈[0.5,1];

[0022] The standard Large Deformation Diffeomorphism Metric Matching (LDDMM) equation is generated as follows:

[0023]

[0024] The registration method is 'syn', the interpolation method is linear interpolation, the optimization metric in the registration process is mutual information, and the final evaluation indicator is mean square error:

[0025]

[0026]

[0027] Where J is the breast image before registration, J' is the breast image after registration, M and N represent the width and height of the image, respectively. After segmentation and registration, the breast is mapped to a fixed position, and the actual physical position information of the nodule and breast is converted into relative position information in the current registered image.

[0028] Preferably, the shape channel calculates the edge aspect ratio based on the segmented nodule area to obtain a predicted interval value as the shape channel feature; the echo channel calculates the difference between the average echo value of the breast area outside the nodule and the average echo value inside the nodule, and then fits the distribution of the echo difference through a Gaussian function to obtain a predicted interval as the echo channel feature value.

[0029] Preferably, for texture features, position features and edge features, the corresponding predicted values are obtained through deep learning methods as the value range of the channel. For shape features, the aspect ratio of the shape is used as the value range of the shape channel. For echo features, the echo difference between the inside and outside of the nodule is used as the value range of the echo channel.

[0030] Preferably, the multinomial logistic regression model formula of the joint prediction module is: logit(y)=β0+β1x1+β2x2+β3x3+β4x4+β5x5, where x1, x2, x3, x4 and x5 are the predicted malignancy probabilities of the texture channel, echo channel, edge channel, shape channel and position channel respectively, and y is the prediction result of the model.

[0031] Preferably, the joint prediction module also performs effect evaluation based on the validation set and the external test set, and finally analyzes the contribution of each factor to the overall prediction classification based on variance analysis.

[0032] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art:

[0033] 1. The technical solution of the present invention extracts features with specific biological significance through multiple channels for joint prediction. Based on the texture, edge, echo, shape, and position scores of breast cancer ultrasound images, the detection results of benign, non-invasive, early invasive, and invasive breast cancer pathological subtypes are jointly given, thereby realizing an artificial intelligence detection system that can be understood by doctors and providing an automated theoretical basis for clinical diagnosis and decision-making.

[0034] 2. The technical solution of this invention utilizes a foreground optimization segmentation network based on a feature pyramid network (FPN) as its backbone. By calculating a global similarity matrix, it enhances the contextual relevance of the foreground. This solves the problem of blurred and complex backgrounds in breast cancer ultrasound images and accurately segments nodules for subsequent processing. The design of texture, edge, echo, shape, and location channels facilitates the interpretation of key aspects of breast cancer ultrasound reports by medical personnel.

[0035] 3. In the solution of the present invention, the location channel needs to predict the risk of malignancy based on the positional relationship. By using a graph convolutional network structure, the positional relationship can be constructed at a higher dimension, and the implicit correspondence in the information can be more effectively mined. Therefore, this embodiment converts the positional relationship between the breast and the nodule into a graph structure, and uses edge weights in the graph to represent the distance between the nodule and the breast. A deep graph convolutional network is used to extract the structural features of the mapping. Finally, the location channel can obtain a specific score for the positional feature.

[0036] 4. The solution of the present invention adopts a graph convolutional network structure, which can construct positional relationships at a higher dimension and more effectively mine the implicit correspondences in the information. Therefore, this embodiment converts the positional relationship between the breast and the nodule into a graph structure, and uses edge weights in the graph to represent the distance between the nodule and the breast. A registration method based on differential homeomorphism is used to homogeneously locate nodules in ultrasound images of various patients, and a deep graph convolutional network is used to extract the mapped structural features to predict the location information of breast cancer nodules. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings, wherein:

[0038] Figure 1 Schematic diagram of the process of constructing the detection system of the present invention;

[0039] Figure 2 This is an example ultrasound image of benign breast cancer;

[0040] Figure 3 This is an example ultrasound image of non-invasive breast cancer;

[0041] Figure 4 This is an example ultrasound image of early invasive breast cancer;

[0042] Figure 5 This is an example of an ultrasound image of invasive breast cancer. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims. It should be noted that the drawings are greatly simplified and not to exact ratios, and are intended solely to facilitate and clearly illustrate the embodiments of the present invention.

[0044] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0045] First embodiment

[0046] like Figure 1 As shown, this embodiment provides a doctor-understandable ultrasound-based breast cancer pathological subtype detection system, including:

[0047] The segmentation module uses a U-Net network embedded in a foreground optimization network to segment breasts and nodules. The foreground-background relationship module improves the discrimination of nodules by associating surrounding semantic information related to nodule morphology pixels. The gradient descent direction is adjusted to improve the recognition rate. The segmentation output and the original ultrasound image are input into multiple channels for feature extraction, including texture channel, edge channel, echo channel, shape channel, and position channel.

[0048] Texture channel, based on Resnet-18, texture-based malignancy risk prediction is performed to obtain texture channel features;

[0049] Edge channel, based on the U-Net network and foreground optimization network segmentation, obtains the boundary information of white lines without background texture;

[0050] A position channel is used to register the segmented breast and nodule ultrasound images in a pre-configured breast template based on the boundary information, thereby obtaining the position information in the current registered image and quantifying the overall relationship between the nodule and the breast boundary information to extract the breast nodule growth position feature;

[0051] Shape channel, according to the segmented nodule area, calculates the edge aspect ratio and obtains the prediction interval value as the shape channel feature;

[0052] Echo channel: the difference between the echo inside and outside the nodule is calculated as the echo channel characteristic value;

[0053] The joint prediction module classifies and predicts breast cancer pathological subtypes based on the output features of the texture channel, edge channel, echo channel, shape channel and position channel through a multinomial logistic regression model. The breast cancer pathological subtypes include benign, non-invasive, early invasive and invasive subtypes.

[0054] This embodiment extracts features with specific biological significance through multiple channels for joint prediction. Based on the texture, edge, echo, shape, and position scores of breast cancer ultrasound images, the system jointly provides detection results for benign, non-invasive, early invasive, and invasive breast cancer pathological subtypes. This creates an artificial intelligence detection system that doctors can understand and provides an automated theoretical basis for clinical diagnosis and decision-making.

[0055] See also Figure 1 This is a flow chart illustrating the process of constructing the detection system of the present invention. Analyzing the impact of nodule characteristics requires effective nodule segmentation as input for classification. However, unlike natural or other medical images, breast cancer ultrasound images exhibit the following characteristics: large-scale variations, low inter-organ contrast, blurred backgrounds, and relatively small and few detected targets. Due to the large intra-class variance of the background, misclassifications can occur, posing a significant challenge to the segmentation task. Therefore, the present invention utilizes a foreground optimization method to accurately segment breast nodules. The foreground-optimized FPN in this method addresses the multi-scale input problem. The foreground-background relationship module improves the differentiation of the nodule itself by associating surrounding semantic information related to nodule morphology pixels. The goal is to enhance the distinction between nodules and their surrounding background. To address misclassified samples in the background that are mistakenly identified as nodules, the foreground optimization module is embedded in the model, adjusting the gradient descent direction to improve recognition rate. Furthermore, many models produce jagged edges when segmenting the breast. Considering segmentation performance and reducing jagged edges, after multiple model comparison experiments, we adopted the U-Net network as the breast segmentation module.

[0056] Specifically, the prospect optimization network is: where f i is the i-th layer in the pyramid feature map, C i (i=2, 3, 4, 5) represents a set of FPN feature maps with the same number of channels d, M represents the lateral connection of the learnable convolutional layer, and N represents the nearest neighbor upsampling;

[0057] A branch is attached to C5 to generate scene features C6 through global context aggregation. In order to transform the feature map, a scale-aware projection function is introduced in the following equation: where θ i yes The learnable parameters of

[0058] Transformed feature map To obtain the relationship between the mappings, we use I(·) on C6 to compute the scene embedding vector v, v = I(C6), where I is the projection function implemented by the learnable convolutional layer and the vector v is shared for each pyramid.

[0059] The relationship mapping r is obtained by similarity estimation achieved by point-by-point inner product sim(·) i , the encoder’s input pyramid feature map f i Re-encode and obtain the re-encoded feature map The encoder has learnable parameters ω i :

[0060] Thus, the relationship-enhanced foreground feature map g is obtained i :

[0061] Based on the above principles, we implemented a foreground-optimized segmentation model to extract nodule information. For the nodule segmentation module, the baseline based on the optimized foreground network is the ResNet-50 model, which has been pre-trained on ImageNet.

[0062] A foreground optimization segmentation network based on a Feature Pyramid Network (FPN) is employed to enhance the contextual relevance of the foreground by calculating a global similarity matrix. This approach addresses the fuzzy and complex backgrounds of breast cancer ultrasound images and accurately segments nodules for subsequent processing. The design of texture, edge, echo, shape, and location channels facilitates the identification of key and easily understood content within breast cancer ultrasound reports.

[0063] Preferably, the baseline of the foreground optimization network of the segmentation module is the ResNet-50 model, which is pre-trained on ImageNet, and the convolution kernel sizes include 4×3×4, 8×3×8, 16×3×16 and 32×3×32; the learning rate is 0.007, and the maximum number of training iterations is set to 60,000; when training the U-Net network, the batch-size is set to 8, the Adam optimizer is used, the learning rate is set to 0.001, and the maximum number of iterations is set to 300.

[0064] After that, the five channels of texture, edge, echo, shape, and position are constructed. Specifically, the segmentation output and the original image are used as input to define the five channels mentioned in the ultrasound report that doctors can understand: texture, edge, echo, shape, and position.

[0065] Preferably, the texture channel uses Resnet-18 as a feature extraction framework to obtain the boundary information of the image output by the segmentation module, and then extract texture features.

[0066] The segmented data is modeled and Resnet-18 is used to perform texture-based malignancy risk prediction to obtain texture channel features.

[0067] Preferably, the edge channel uses Resnet-18 as the framework for feature extraction, and adopts a random mosaic method to extract a large number of random blocks from the edge area of the breast nodule and splice them into a complete image. When the sizes of different nodules are inconsistent, the size of the extracted features is guaranteed to be consistent, so as to maximize the retention of the details of the original image to retain the edge information, and the parameters of the ImageNet pre-trained model are used to train it.

[0068] Even experienced doctors cannot come up with accurate criteria for filtering edge features from entire ultrasound images. This example uses a novel "random mosaicking" method to maximize the preservation of details in the original image, thereby preserving edge information. This method, combined with the segmentation task to determine edge regions, demonstrates good performance. In experiments, we set the size of each image block to 16×16 pixels, and each mosaicked image has 14×14 image blocks. We then used ResNet-18 to predict the risk of malignant tumors, using the mosaicked image as input.

[0069] Analyze the impact of the growth position of breast nodules on malignancy prediction and quantify the overall relationship between nodules and breast edge information. First, use the U-Net network and foreground optimization network to segment the breast and nodules respectively, and obtain these boundary information (white lines without background texture). Then the position channel predicts the malignancy risk based on these boundary information. This embodiment pre-configures a unified breast template and aligns the segmented breast and nodule ultrasound images to the template. According to the deformation field obtained by breast registration, each nodule is mapped to the breast template, and the position information in the current registration image is automatically obtained.

[0070] Specifically, the position channel registers the segmented breast and nodule ultrasound images based on the boundary information to obtain a deformation field, maps each nodule to the breast template, and automatically obtains the position information in the current registration image. Specifically, the registration is performed using the software package ANTs based on the symmetric differential homeomorphism method. The software package ANTs assumes differential homeomorphism. is defined on the image domain Ω and preserves an affine transformation at the boundary such that Where A(Id) is an affine mapping applied to the identity transformation, mapping Over time, parameterize a family of diffeomorphisms Produced by integrating a time-varying, smooth velocity field; v: Ω×t→R d , through the ordinary differential equation The resulting deformation field is When Decompose into two components and And define int∈[0,0.5],v(x,t)=v1(x,t),v(x,t)=v2(x,1-t), where t∈[0.5,1];

[0071] The standard Large Deformation Diffeomorphism Metric Matching (LDDMM) equation is generated as follows:

[0072]

[0073] The registration method is 'syn', the interpolation method is linear interpolation, the optimization metric in the registration process is mutual information, and the final evaluation indicator is mean square error:

[0074]

[0075]

[0076] Where J is the breast image before registration, J' is the breast image after registration, M and N represent the width and height of the image, respectively. After segmentation and registration, the breast is mapped to a fixed position, and the actual physical position information of the nodule and breast is converted into relative position information in the current registered image.

[0077] In the solution of this embodiment, the position channel needs to predict the risk of malignancy based on the positional relationship. By using a graph convolutional network structure, the positional relationship can be constructed at a higher dimension, and the implicit correspondence in the information can be more effectively mined. Therefore, this embodiment converts the positional relationship between the breast and the nodule into a graph structure, and uses edge weights in the graph to represent the distance between the nodule and the breast. A registration method based on differential homeomorphism is used to homogeneously locate the nodules in ultrasound images of various patients, and a deep graph convolutional network is used to extract the mapped structural features to predict the location information of breast cancer nodules. Finally, the position channel can obtain a specific score for the position feature.

[0078] Preferably, the shape channel calculates the edge aspect ratio based on the segmented nodule area to obtain a predicted interval value as the shape channel feature; the echo channel calculates the difference between the average echo value of the breast area outside the nodule and the average echo value inside the nodule, and then fits the distribution of the echo difference through a Gaussian function to obtain a predicted interval as the echo channel feature value.

[0079] The technical solution of this embodiment uses multiple feature extraction channels with specific biological significance for joint prediction, which on the one hand improves the interpretability of artificial intelligence medical care, and on the other hand is beneficial to provide an automated theoretical basis for clinical diagnosis and decision-making.

[0080] Preferably, for texture features, position features and edge features, the corresponding predicted values are obtained through deep learning methods as the value range of the channel. For shape features, the aspect ratio of the shape is used as the value range of the shape channel. For echo features, the echo difference between the inside and outside of the nodule is used as the value range of the echo channel.

[0081] Preferably, the multinomial logistic regression model formula of the joint prediction module is: logit(y)=β0+β1x1+β2x2+β3x3+β4x4+β5x5, where x1, x2, x3, x4 and x5 are the predicted malignancy probabilities of the texture channel, echo channel, edge channel, shape channel and position channel respectively, and y is the prediction result of the model.

[0082] This technical solution combines the outputs of texture, edge, echo, shape and position channels, and constructs a multinomial logistic regression model from the perspective of these features to classify and predict the four subtypes: benign, non-invasive, early invasive and invasive.

[0083] Preferably, the joint prediction module also performs effect evaluation based on the validation set and the external test set, and finally analyzes the contribution of each factor to the overall prediction classification based on variance analysis.

[0084] This embodiment also uses training, validation, and external test sets from the five channels. The logistic regression model is trained using the training set, and the performance is evaluated on the validation set and external test set. Finally, the contribution of each factor to the overall prediction classification is analyzed based on analysis of variance (ANOVA). In practical applications, a single input image will obtain the prediction results of the five channels, their respective interval values, the overall prediction result, and the contribution of each factor to the overall result.

[0085] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the identification content specifically executed by the above-described system and device can refer to the corresponding process in the aforementioned method embodiment.

[0086] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the scope of protection of the present invention.

Claims

1. A doctor-understandable ultrasound-based breast cancer pathological subtype detection system, characterized by: include: The segmentation module uses a U-Net network embedded in a foreground optimization network to segment breasts and nodules. The foreground-background relationship module improves the discrimination of nodules by associating surrounding semantic information related to nodule morphology pixels. The gradient descent direction is adjusted to improve the recognition rate. The segmentation output and the original ultrasound image are input into multiple channels for feature extraction, including texture channel, edge channel, echo channel, shape channel, and position channel. Texture channel, based on Resnet-18, texture-based malignancy risk prediction is performed to obtain texture channel features; Edge channel, based on the U-Net network and foreground optimization network segmentation, obtains the boundary information of white lines without background texture; A position channel is used to register the segmented breast and nodule ultrasound images in a pre-configured breast template based on the boundary information, thereby obtaining the position information in the current registered image and quantifying the overall relationship between the nodule and the breast boundary information to extract the breast nodule growth position feature; Shape channel, according to the segmented nodule area, calculates the edge aspect ratio and obtains the prediction interval value as the shape channel feature; Echo channel: the difference between the echo inside and outside the nodule is calculated as the echo channel characteristic value; The joint prediction module classifies and predicts breast cancer pathological subtypes based on the output features of the texture channel, edge channel, echo channel, shape channel and position channel through a multinomial logistic regression model. The breast cancer pathological subtypes include benign, non-invasive, early invasive and invasive subtypes.

2. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 1, characterized in that: The prospect optimization network is: where f i is the i-th layer in the pyramid feature map, C i (i=2, 3, 4, 5) represents a set of FPN feature maps with the same number of channels d, M represents the lateral connection of the learnable convolutional layer, and N represents the nearest neighbor upsampling; A branch is attached to C5 to generate scene features C6 through global context aggregation. In order to transform the feature map, a scale-aware projection function is introduced: where θ i yes The learnable parameters of Transformed feature map To obtain the relationship between the mappings, we use I(·) on C6 to compute the scene embedding vector v, v = I(C6), where I is the projection function implemented by the learnable convolutional layer and the vector v is shared for each pyramid. The relationship mapping r is obtained by similarity estimation achieved by point-by-point inner product sim(·) i , the encoder’s input pyramid feature map f i Re-encode and obtain the re-encoded feature map The encoder has learnable parameters ω i : Thus, the relationship-enhanced foreground feature map g is obtained i :

3. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 2, characterized in that: The baseline of the foreground optimization network of the segmentation module is the ResNet-50 model, which is pre-trained on ImageNet. The convolution kernel sizes include 4×3×4, 8×3×8, 16×3×16, and 32×3×32. The learning rate is 0.007, and the maximum number of training iterations is set to 60,000. When training the U-Net network, the batch size is set to 8, the Adam optimizer is used, the learning rate is set to 0.001, and the maximum number of iterations is set to 300.

4. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 1, characterized in that: The texture channel uses Resnet-18 as the feature extraction framework to obtain the boundary information of the image output by the segmentation module, and then extracts texture features.

5. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 1, characterized in that: The edge channel uses Resnet-18 as the feature extraction framework and a random mosaic method to extract a large number of random blocks from the edge area of the breast nodule and splice them into a complete image. When the sizes of different nodules are inconsistent, the size of the extracted features is guaranteed to be consistent, so as to maximize the retention of the details of the original image to retain the edge information, and the parameters of the ImageNet pre-trained model are used for training.

6. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 5, characterized in that: The position channel registers the segmented breast and nodule ultrasound images based on the boundary information to obtain a deformation field, maps each nodule to the breast template, and automatically obtains the position information in the current registration image. Specifically, the registration is performed using the software package ANTs based on the symmetric differential homeomorphism method. The software package ANTs assumes differential homeomorphism. is defined on the image domain Ω and preserves an affine transformation at the boundary such that Where A(Id) is an affine mapping applied to the identity transformation, mapping Over time, parameterize a family of diffeomorphisms Produced by integrating a time-varying, smooth velocity field; v: Ω×t→R d , through the ordinary differential equation The resulting deformation field is When Decompose into two components and And define int∈[0,0.5],v(x,t)=v1(x,t),v(x,t)=v2(x,1-t), where t∈[0.5,1]; The standard large deformation diffeomorphism metric matching equation is generated as follows: The registration method is 'syn', the interpolation method is linear interpolation, the optimization metric in the registration process is mutual information, and the final evaluation indicator is mean square error: Where J is the breast image before registration, J' is the breast image after registration, M and N represent the width and height of the image, respectively. After segmentation and registration, the breast is mapped to a fixed position, and the actual physical position information of the nodule and breast is converted into relative position information in the current registered image.

7. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 1, characterized in that: The shape channel calculates the edge aspect ratio based on the segmented nodule area to obtain a prediction interval value as the shape channel feature; the echo channel calculates the difference between the average echo value of the breast area outside the nodule and the average echo value inside the nodule, and then fits the distribution of the echo difference through a Gaussian function to obtain a prediction interval as the echo channel feature value.

8. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 7, characterized in that: For texture features, position features, and edge features, the corresponding predicted values are obtained through deep learning methods as the value range of the channel. For shape features, the aspect ratio of the shape is used as the value range of the shape channel. For echo features, the echo difference between the inside and outside of the nodule is used as the value range of the echo channel.

9. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 1, characterized in that: The multinomial logistic regression model formula of the joint prediction module is: logit(y)=β0+β1x1+β2x2+β3x3+β4x4+β5x5, where x1, x2, x3, x4 and x5 are the predicted malignancy probabilities of the texture channel, echo channel, edge channel, shape channel and position channel respectively, and y is the prediction result of the model.

10. The doctor-understandable ultrasound-based breast cancer pathological subtype detection system according to claim 1, characterized in that: The joint prediction module also performs effect evaluation based on the validation set and the external test set, and finally analyzes the contribution of each factor to the overall prediction classification based on variance analysis.

Images