Breast mass ultrasound image classification method and system based on deep learning

Abstract

The present invention provides a method and system for breast mass ultrasound image classification based on deep learning, which relates to the technical field of ultrasound image classification. The present invention realizes integrated processing of multiple ultrasound image types through a unified feature extraction model, thereby improving diagnostic efficiency. At the same time, the image quality score generated based on physical basic features quantifies the impact of probe pressure unevenness, equipment noise interference and motion artifacts on imaging quality, and dynamically adjusts the knowledge enhancement strength accordingly so that pathological sensitive features can be accurately compensated with the support of medical knowledge graphs. A two-level cascade classification network establishes a hierarchical decision-making mechanism to initiate subtype deep analysis of malignant lesions under the premise of ensuring rapid diagnosis, thereby improving the recognition rate. Further, through the image reliability factor reflecting the credibility of equipment acquisition, a model cognitive factor based on the self-consistency of decision logic is combined to form a dual-factor dynamic calibration system to provide support for efficient and accurate classification of breast mass ultrasound images.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasound image classification, and in particular to a method and system for breast mass ultrasound image classification based on deep learning. Background Art

[0002] Breast cancer is one of the most common malignant tumors among women worldwide. Ultrasound scanning of breast lumps to obtain ultrasound images is a common method for early diagnosis of the breast. Early diagnosis is of great significance for improving cure and survival rates. Ultrasound imaging is widely used in the clinical screening and diagnosis of breast lumps due to its advantages such as being non-invasive, radiation-free, and highly real-time. Currently, breast ultrasound imaging mainly includes three modalities: B-mode ultrasound, elastography, and color Doppler, which respectively reflect the morphological structure, tissue hardness, and blood flow distribution characteristics of the tumor. This information is comprehensively used in clinical judgment to determine whether the tumor is benign or malignant. However, due to the limited resolution of ultrasound imaging, severe noise interference, and the fact that image quality is greatly affected by equipment parameters and operator techniques, manual interpretation is highly subjective and diagnostic consistency is insufficient. In recent years, medical image classification technology based on deep learning has made significant progress. However, most existing methods focus on a single modality or shallow fusion strategies, which makes it difficult to fully explore the potential correlations between multiple modalities. At the same time, there is a lack of effective modeling of image quality and equipment interference factors, which affects the generalization ability and reliability of the model. There is an urgent need for an intelligent classification method that can adaptively identify image types, effectively decouple multidimensional features, combine knowledge enhancement mechanisms, and dynamically calibrate confidence levels to improve the accuracy and credibility of breast mass ultrasound image classification analysis.

[0003] Therefore, it is necessary to provide a deep learning-based breast mass ultrasound image classification method and system to solve the above technical problems. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a deep learning-based breast mass ultrasound image classification method and system, which achieves the beneficial effect of accurately and efficiently classifying breast mass ultrasound images.

[0005] The present invention provides a deep learning-based breast mass ultrasound image classification method, comprising:

[0006] S1: Use a pre-trained unified feature extraction model to identify the image type of the currently input breast mass ultrasound image and extract physical basic features, pathological sensitivity features, and device interference features. Image types include B-mode ultrasound images, elastography images, and color Doppler images.

[0007] S2: Generate an image quality score based on physical basic features, and determine the enhancement processing strength based on the image quality score. Based on the enhancement processing strength, the pathology sensitive features are input into the pre-trained cross-modal knowledge enhancer to generate cross-modal reinforcement features;

[0008] S3: Use a pre-trained two-stage cascade classification network to process pathology-sensitive features and cross-modal reinforcement features to obtain image classification results and initial confidence;

[0009] S4: Calculate the image reliability factor based on the device interference characteristics, calculate the model recognition factor based on the image classification results, and calibrate the initial confidence level using the image reliability factor and the model recognition factor to obtain the final confidence level.

[0010] S5: Output the image classification results and final confidence as the final classification results.

[0011] Preferably, step S1 includes the following steps:

[0012] S101: extracting a deep feature map of the currently input breast mass ultrasound image through a shared encoder in a pre-trained unified feature extraction model;

[0013] S102: Input the deep feature map into the type recognition branch of the pre-trained unified feature extraction model to generate an image type probability vector, and take the type with the largest probability value in the image type probability vector as the image type;

[0014] S103: Calculating feature modulation weights based on the image type probability vector, and performing weighted processing on the depth feature map using the feature modulation weights to obtain a weighted feature map;

[0015] S104: The weighted feature maps are processed separately by three parallel feature decoupling branches in the pre-trained unified feature extraction model to obtain physical basic features, pathological sensitive features, and device interference features.

[0016] Preferably, in step S2, the method for calculating the image quality score includes:

[0017] The mean acoustic impedance gradient of the mass edge region of the acoustic impedance distribution matrix extracted from the physical basic features was calculated. A similarity score was calculated based on the mean acoustic impedance gradient and a preset ideal gradient threshold. The similarity score was truncated using a preset score truncation interval to obtain the image quality score of the B-mode ultrasound image.

[0018] Identify continuous regions whose strain variance is less than a preset strain variance threshold from the strain field feature tensor of the physical basic features, and calculate the area ratio of the continuous region as the image quality score of the elastic imaging image;

[0019] Connected regions with velocities greater than a preset blood flow velocity threshold are extracted from the blood flow velocity field distribution of the physical basic characteristics, and the area ratio of the connected regions is calculated as the image quality score of the color Doppler image.

[0020] Preferably, in step S2, the enhancement processing strength is determined by a preset score and strength mapping table.

[0021] Preferably, in step S2, the steps of processing the input pathology-sensitive features by the pre-trained cross-modal knowledge enhancer include:

[0022] Reconstruct the input pathology-sensitive features into a lesion semantic topology map;

[0023] Based on the determined enhanced processing strength, a pre-constructed medical knowledge graph is hierarchically retrieved to obtain a knowledge association vector set;

[0024] The lesion semantic topology map and the knowledge association vector set are integrated to generate cross-modal reinforcement features.

[0025] Preferably, in step S3, the pre-trained two-level cascade classification network includes a first-level network and a second-level network, wherein the first-level network processes pathological sensitive features to generate pathological property classification results and first-level confidence, and the second-level network processes cross-modal reinforcement features to generate malignant subtype classification results and second-level confidence.

[0026] Preferably, if and only if the pathological property classification result generated by the first-level network is malignant, the second-level network is activated to process the cross-modal reinforcement features, and the malignant subtype classification result is used as the image classification result, and the second-level confidence is used as the initial confidence;

[0027] When the pathological property classification result generated by the first-level network is not malignant, the second-level network is skipped, and the pathological property classification result is used as the image classification result, and the first-level confidence is used as the initial confidence.

[0028] Preferably, in step S4, the step of calculating the image reliability factor includes:

[0029] Perform three-dimensional frequency domain decomposition on the device interference characteristics to obtain the high-frequency components of reverberation artifacts, the temporal components of motion blur, and the spatial components of device noise;

[0030] The energy per unit bandwidth of the high-frequency component of the reverberation artifact within the preset frequency band is calculated as the reverberation energy density. The maximum tissue displacement amplitude of the variable component of motion blur between consecutive image frames is detected as the motion offset peak. The Shannon entropy value of the spatial pixel distribution of the spatial component of the device noise is calculated as the noise spatial entropy.

[0031] The reverberation energy density, motion offset peak and noise spatial entropy are weighted and summed based on the preset weighted weights to obtain a composite interference intensity coefficient;

[0032] The attenuation coefficient is selected based on the image type and the preset type coefficient mapping table. After the composite interference intensity coefficient is multiplied by the attenuation coefficient, it is converted into an image reliability factor through an exponential attenuation function.

[0033] Preferably, in step S4, the initial confidence level is calibrated using an exponential fusion function to obtain a final confidence level.

[0034] The present invention also provides a deep learning-based breast mass ultrasound image classification system, which is applied to a deep learning-based breast mass ultrasound image classification method, comprising:

[0035] A multimodal feature decoupling module is used to identify the image type of the currently input breast mass ultrasound image and extract physical basic features, pathological sensitivity features, and device interference features using a pre-trained unified feature extraction model. Image types include B-mode ultrasound images, elastography images, and color Doppler images.

[0036] The knowledge enhancement feature reinforcement module is used to generate image quality scores based on physical basic features, determine the enhancement processing strength based on the image quality scores, and input pathology sensitive features into the pre-trained cross-modal knowledge enhancer based on the enhancement processing strength to generate cross-modal reinforcement features;

[0037] Image classification decision module, which uses a pre-trained two-stage cascade classification network to process pathology-sensitive features and cross-modal reinforcement features to obtain image classification results and initial confidence;

[0038] The confidence calibration module is used to calculate the image reliability factor based on the device interference characteristics, calculate the model recognition factor based on the image classification results, and calibrate the initial confidence level through the image reliability factor and the model recognition factor to obtain the final confidence level;

[0039] The classification result output module is used to output the image classification results and final confidence as the final classification results.

[0040] Compared with related technologies, the deep learning-based breast mass ultrasound image classification method and system provided by the present invention has the following beneficial effects:

[0041] This paper achieves integrated processing of multiple ultrasound image types through a unified feature extraction model, breaking the barriers of traditional segmented analysis of B-type ultrasound elastography and color Doppler images and significantly improving diagnostic efficiency. Furthermore, the image quality score generated based on physical fundamental features scientifically quantifies the impact of uneven probe pressure, equipment noise interference, and motion artifacts on imaging quality. Based on this, the knowledge enhancement strength is dynamically adjusted to accurately compensate for pathologically sensitive features with the support of the medical knowledge graph, effectively overcoming the feature distortion problem caused by low-quality images. Furthermore, a hierarchical decision-making mechanism is established based on a two-level cascade classification network to initiate in-depth subtype analysis of malignant lesions while ensuring rapid diagnosis of benign cases. This significantly improves the recognition rate of easily missed lesions such as intraductal carcinoma and mucinous carcinoma. More importantly, the image reliability factor constructed through equipment interference feature analysis objectively reflects the credibility of equipment acquisition. Combined with the model cognitive factor based on the self-consistency of the decision logic, a dual-factor dynamic calibration system is formed. This system differs from the traditional confidence assessment model that relies solely on classification probability. This provides high-precision and high-reliability intelligent decision support for early breast cancer screening, achieving efficient and accurate classification of breast mass ultrasound images based on deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a deep learning-based breast mass ultrasound image classification method of the present invention;

[0043] Figure 2 This is a module structure diagram of a deep learning-based breast mass ultrasound image classification system of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all of the structures. Furthermore, the embodiments of the present invention and the features of the embodiments may be combined with one another unless there is a conflict.

[0045] It should also be noted that, for ease of description, only portions relevant to the present invention are shown in the accompanying drawings, rather than all of the contents. Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the various operations (or steps) as being processed sequentially, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operations are completed, but may also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0046] Example 1, a breast mass ultrasound image classification method based on deep learning, in the specific implementation process, such as Figure 1 As shown, it shows a flowchart of a breast mass ultrasound image classification method based on deep learning, including:

[0047] Step S1: Use a pre-trained unified feature extraction model to identify the image type of the currently input breast mass ultrasound image and extract physical basic features, pathological sensitive features, and equipment interference features, where the image types include B-type ultrasound images, elastic imaging images, and color Doppler images.

[0048] Specifically, step S1 includes the following steps:

[0049] S101: extracting a deep feature map of the currently input breast mass ultrasound image through a shared encoder in a pre-trained unified feature extraction model;

[0050] S102: Input the deep feature map into the type recognition branch of the pre-trained unified feature extraction model to generate an image type probability vector, and take the type with the largest probability value in the image type probability vector as the image type;

[0051] S103: Calculating feature modulation weights based on the image type probability vector, and performing weighted processing on the depth feature map using the feature modulation weights to obtain a weighted feature map;

[0052] S104: The weighted feature maps are processed separately by three parallel feature decoupling branches in the pre-trained unified feature extraction model to obtain physical basic features, pathological sensitive features, and device interference features.

[0053] During the specific implementation process, a pre-trained unified feature extraction model is first constructed. The model is trained using a multimodal breast mass dataset including B-type ultrasound images, elastic imaging images and color Doppler images. During the training process, a three-task joint optimization strategy is adopted, namely, the type discrimination ability is constrained by the image type classification loss, the feature semantic representation is strengthened by the lesion area segmentation loss, and the feature decoupling performance is improved by the modality reconstruction loss. Finally, a complete architecture including a shared encoder, a type recognition branch and three parallel feature decoupling branches is formed; when the step S1 is specifically executed, for example, the currently input breast mass ultrasound image is first input into the shared encoder to generate a 128-dimensional deep feature map, and the deep feature map is simultaneously input into the type recognition branch for processing: the type recognition branch consists of three fully connected layers, and the last layer uses a softmax activation function to output an image type probability vector ,in, , , Probability estimation corresponding to three types of images: B-type ultrasound images, elastic imaging images, and color Doppler images; feature modulation weights are calculated based on the probability vector of this type ,in is a learnable parameter matrix that contains the knowledge learned by the model during training, that is, how to generate preliminary feature modulation signals based on the information of the probability distribution of image types. The bias matrix is ​​added after the nonlinear transformation to add a bias term to each dimension, which makes the model more flexible in constructing the modulated signal. represents the sigmoid function, To modify the linear unit activation function, this formula maps the image type probability vector to a 128-dimensional modulation weight vector with the same number of channels as the depth feature map; then, channel-level weighted processing is performed on the original depth feature map: the Hadamard product is multiplied channel by channel by the modulation weight vector to generate a weighted feature map; this weighted feature map is input in parallel into three feature decoupling branches. The physical basis feature branch uses a two-dimensional convolutional network combined with a spectral analysis layer to output an acoustic impedance distribution matrix (for B-type ultrasound images) or a strain field feature tensor (for elastic imaging images) or a blood flow velocity field distribution (for color Doppler images); the pathology-sensitive feature branch extracts a 512-dimensional high-dimensional semantic vector related to the biological characteristics of the lesion through an attention mechanism; the device interference feature branch uses residual separation technology to extract a 64-dimensional interference feature vector of quantization artifact noise; and finally, three decoupled features that are adaptive to the input image type are output, namely, physical basis features, pathology-sensitive features, and device interference features, which are used in subsequent processing flows.

[0054] Step S2: Generate an image quality score based on the physical basic features, and determine the enhancement processing strength based on the image quality score. Based on the enhancement processing strength, the pathology sensitive features are input into the pre-trained cross-modal knowledge enhancer to generate cross-modal reinforcement features.

[0055] Specifically, in step S2, the method for calculating the image quality score includes:

[0056] The mean acoustic impedance gradient of the mass edge region of the acoustic impedance distribution matrix extracted from the physical basic features was calculated. A similarity score was calculated based on the mean acoustic impedance gradient and a preset ideal gradient threshold. The similarity score was truncated using a preset score truncation interval to obtain the image quality score of the B-mode ultrasound image.

[0057] Identify continuous regions whose strain variance is less than a preset strain variance threshold from the strain field feature tensor of the physical basic features, and calculate the area ratio of the continuous region as the image quality score of the elastic imaging image;

[0058] Connected regions with velocities greater than a preset blood flow velocity threshold are extracted from the blood flow velocity field distribution of the physical basic characteristics, and the area ratio of the connected regions is calculated as the image quality score of the color Doppler image.

[0059] Specifically, in step S2, the enhancement processing strength is determined by a preset score and strength mapping table.

[0060] Specifically, in step S2, the pre-trained cross-modal knowledge enhancer processes the input pathology-sensitive features in the following steps:

[0061] Reconstruct the input pathology-sensitive features into a lesion semantic topology map;

[0062] Based on the determined enhanced processing strength, a pre-constructed medical knowledge graph is hierarchically retrieved to obtain a knowledge association vector set;

[0063] The lesion semantic topology map and the knowledge association vector set are integrated to generate cross-modal reinforcement features.

[0064] During the specific implementation process, the image quality score is first calculated based on the physical basic features. For B-type ultrasound images, the acoustic impedance distribution matrix in the physical basic features is extracted, the edge area of ​​the mass is located, and the average value of the acoustic impedance gradient in the area is calculated. This average value is compared with the preset ideal gradient threshold, where the ideal gradient threshold is set according to the typical gradient of the clear mass edge in the high-quality B-ultrasound image. Then, the inverse of the absolute difference between the two is calculated as the basis for the similarity score, and the preset score truncation interval is used to constrain the score to be within a reasonable range. Finally, the truncated value is output as the B-type ultrasound image quality score; for elastic imaging images, the strain variance is identified from the strain field feature tensor of the physical basic features. All continuous spatial regions with a velocity value less than a preset strain variance threshold are extracted, wherein the preset strain variance threshold reflects the stability of uniform deformation of the tissue, and then the ratio of the pixel area of ​​these stable regions to the pixel area of ​​the entire lesion region is calculated as its image quality score; for color Doppler images, all connected regions with velocity values ​​greater than the preset blood flow velocity threshold are extracted from the blood flow velocity field distribution map of the physical basic characteristics, wherein the preset blood flow velocity threshold is used to exclude noise and low-speed background flow, and then the ratio of the area of ​​these effective blood flow signal regions to the area of ​​the entire field of view is calculated as its quality score; then, according to the quality score of the corresponding image type calculated above, the preset score and intensity mapping table is queried, and the score is compared with the image quality score. The intensity mapping table has the score as the horizontal axis and the enhancement processing intensity as the vertical axis, showing a piecewise linear relationship. The cross-modal knowledge enhancement processing intensity required for the current image is determined based on the score and the intensity mapping table; then the pathological sensitive features are input into the pre-trained cross-modal knowledge enhancer for processing: when constructing the enhancer, a medical knowledge graph embedding module must first be pre-trained. The medical knowledge graph embedding module uses graph representation learning methods such as TransR to train on a breast medical knowledge graph that includes anatomical structures, pathological features, and biomarker associations to generate low-dimensional vectors of entities and relationships. At the same time, a feature-to-graph mapping network is pre-trained, which is exemplarily designed as a multi-layer perceptron that receives pathological sensitive feature input and outputs A query vector; in the specific processing steps, illustratively, first, the input 512-dimensional pathology-sensitive features are spatially semantically reorganized through a reconstruction layer to generate a lesion semantic topology map containing key semantic nodes, such as mass core, calcification points, burr areas, and their relative spatial topological relationships; based on the obtained enhancement processing intensity, a hierarchical search is performed on the medical knowledge graph. When the enhancement processing intensity is less than a preset first enhancement intensity threshold, a sparse search is performed: only the knowledge entities that are semantically closest to the top three nodes with the highest confidence in the topology map are searched; when the enhancement processing intensity is between the preset first enhancement intensity threshold and the preset second enhancement intensity threshold, a standard search is performed: knowledge entities related to the top five nodes in the topology map are searched;When the enhancement intensity exceeds a preset second enhancement intensity threshold, a deep search is performed: additional knowledge entities with second-order relationships with the node are augmented. The retrieval mechanism calculates the cosine similarity between each node vector in the lesion semantic topology graph and all entity vectors in the knowledge graph. The top k most relevant entities and their relationship paths are selected according to the enhancement intensity value to form a knowledge association vector set. The fusion module then uses a graph attention mechanism to use the nodes in the lesion semantic topology graph as query vectors and the knowledge association vector set as key-value pairs. The attention weights of the knowledge vectors on the lesion node features are calculated, and then weighted aggregation is performed to generate a 512-dimensional cross-modal enhancement feature. This feature incorporates the prior pathology knowledge provided by the medical knowledge graph. The enhancer is trained using a three-stage strategy: in the first stage, the knowledge graph embedding and mapping networks are independently trained to minimize the matching loss between features and graph entities. In the second stage, the reconstruction layer and the fusion layer are jointly trained to minimize the KL divergence between the enhancement features and the gold standard pathology features. Auxiliary supervision from graph entity matching is also added. In the third stage, end-to-end fine-tuning is performed using classification task feedback to optimize the final enhancement effect.

[0065] Step S3: Use the pre-trained two-stage cascade classification network to process pathology-sensitive features and cross-modal reinforcement features to obtain image classification results and initial confidence.

[0066] Specifically, in step S3, the pre-trained two-level cascade classification network includes a first-level network and a second-level network, wherein the first-level network processes pathology-sensitive features to generate pathology property classification results and first-level confidence, and the second-level network processes cross-modal reinforcement features to generate malignant subtype classification results and second-level confidence.

[0067] Specifically, if and only if the pathological classification result generated by the first-level network is malignant, the second-level network is activated to process the cross-modal reinforcement features, and the malignant subtype classification result is used as the image classification result, and the second-level confidence is used as the initial confidence;

[0068] When the pathological property classification result generated by the first-level network is not malignant, the second-level network is skipped, and the pathological property classification result is used as the image classification result, and the first-level confidence is used as the initial confidence.

[0069] During the specific implementation process, a pre-trained two-level cascade classification network is first constructed. The network is trained using a breast tumor ultrasound image dataset containing benign and malignant labels and malignant subtype labels. A joint optimization strategy is adopted during the training process, that is, the first-level network learns to distinguish the basic pathological properties of the tumor, which includes but is not limited to benign, malignant or undetermined. At the same time, the second-level network specifically learns to distinguish malignant subtypes, which includes but is not limited to invasive ductal carcinoma, ductal carcinoma in situ, and phyllodes tumor. The loss function of the two-level network includes standard cross entropy loss and KL divergence loss to coordinate feature transfer and confidence calibration; the pre-trained two-level cascade classification network includes a first-level network and a second-level network; wherein the first-level network is a classifier composed of a fully connected layer and a softmax layer, with an input of 512-dimensional pathological sensitive features and an output of pathological property classification results including three categories of "benign", "malignant" and "uncertain" and the corresponding first-level confidence; the second-level network is a deeper fully connected The classifier consists of a network and a softmax layer. Its input is cross-modal reinforcement features, and its output is the classification results of multiple preset malignant subtypes and the corresponding secondary confidence levels. When performing classification, the first-level network is first run to process the input pathological sensitive features. When and only when the pathological property classification result generated by the first-level network is "malignant", the second-level network will be activated to process the input cross-modal reinforcement features. At this time, the malignant subtype classification result output by the second-level network is used as the final image classification result, and the secondary confidence level output by the second-level network is used as the initial confidence level. When the pathological property classification result generated by the first-level network is not "malignant", the execution of the second-level network is skipped, and the pathological property classification result output by the first-level network is directly used as the final image classification result, and the first-level confidence level output by the first-level network is used as the initial confidence level. This design realizes cascade decision-making of classification tasks, significantly improves the resource utilization efficiency of the malignant subtype classification task, and ensures a unified output interface for the entire system.

[0070] Step S4: Calculate the image reliability factor based on the device interference feature, calculate the model recognition factor based on the image classification result, and calibrate the initial confidence level by the image reliability factor and the model recognition factor to obtain the final confidence level.

[0071] Specifically, in step S4, the steps of calculating the image reliability factor include:

[0072] Perform three-dimensional frequency domain decomposition on the device interference characteristics to obtain the high-frequency components of reverberation artifacts, the temporal components of motion blur, and the spatial components of device noise;

[0073] The energy per unit bandwidth of the high-frequency component of the reverberation artifact within the preset frequency band is calculated as the reverberation energy density. The maximum tissue displacement amplitude of the variable component of motion blur between consecutive image frames is detected as the motion offset peak. The Shannon entropy value of the spatial pixel distribution of the spatial component of the device noise is calculated as the noise spatial entropy.

[0074] The reverberation energy density, motion offset peak and noise spatial entropy are weighted and summed based on the preset weighted weights to obtain a composite interference intensity coefficient;

[0075] The attenuation coefficient is selected based on the image type and the preset type coefficient mapping table. After the composite interference intensity coefficient is multiplied by the attenuation coefficient, it is converted into an image reliability factor through an exponential attenuation function.

[0076] Specifically, it is characterized in that, in step S4, the initial confidence is calibrated using an exponential fusion function to obtain the final confidence.

[0077] During the specific implementation process, the interference characteristics of the equipment are first processed by three-dimensional frequency domain decomposition, and the reverberation artifact frequency band is divided by bandpass filtering. For example, the 6-10 MHz frequency band is a preset frequency band range. The unit bandwidth energy value is calculated as the reverberation energy density within the preset frequency band range. The optical flow method is used to track five consecutive frames of images to extract the maximum tissue displacement amplitude as the motion offset peak. The noise spatial distribution discreteness is quantified by wavelet transform as the noise spatial entropy. The three indicators are weighted according to the preset weight ratio. For example, they are weighted according to 0.5:0.3:0.2 to synthesize the composite interference intensity coefficient. Then, based on the determined image type, the attenuation coefficient is selected. For example, 0.8 is used for B-type ultrasound imaging, 1.2 is used for elastic imaging imaging, and 0.9 is used for color Doppler imaging. The attenuation coefficient is multiplied by the composite intensity and input into the exponential function to generate the image reliability factor. The specific calculation formula is: ,in, is the image reliability factor, is the attenuation coefficient, is the composite strength coefficient, the image reliability factor represents the reliability of the equipment acquisition, and the model recognition factor is dynamically calculated according to the image classification results. For example, when it is classified as malignant, the first-level network malignancy probability is extracted. The highest subtype probability with the second-level network , calculate the model cognitive factor, the specific formula is: ,in It is the model cognitive factor, quantifying the self-consistency of decision logic. If the diagnosis is benign, the edge regularity is extracted from the physical basic features. , internal uniformity , rear features , the model cognitive factor is calculated by the Sigmoid activation function, the formula is ,in, Add a weight to the preset edge regularity. The weight of the preset internal uniformity is Weights for the preset posterior features, Represents the Sigmoid activation function, and finally the exponential fusion function is used to calibrate the initial confidence. The specific calculation formula is: ,in, is the final confidence level, is the initial confidence, exponent and According to the preset image classification results, the confidence of the image classification results was dynamically adjusted by two factors, which improved the accuracy of breast mass ultrasound image classification.

[0078] Step S5: Output the image classification result and the final confidence as the final classification result.

[0079] During the specific implementation process, the image classification results and final confidence levels are converted into a structured diagnostic report and output to achieve the final breast mass ultrasound image classification.

[0080] The working principle of the deep learning-based breast mass ultrasound image classification method provided by the present invention is as follows:

[0081] First, the shared encoder in the shared feature extraction network is used to extract the deep feature map. The deep feature map is then processed through the type recognition branch to obtain the image type probability vector, thereby determining the image type. The image types include B-type ultrasound images, elastic imaging images, and color Doppler images. Subsequently, based on the identified image type, a dedicated processing channel is started to focus on analyzing the changes in the acoustic impedance gradient at the edge of the mass for B-type ultrasound images, accurately measure the strain field stability for elastic imaging images, and focus on the blood flow connectivity area for color Doppler images. After completing the quality assessment of image type adaptation, the knowledge enhancement strength is dynamically adjusted according to the image quality score, and the pathological features of the current image are reconstructed into a lesion structure map for on-demand retrieval. The matching medical knowledge base generates enhanced diagnostic features, and then a two-level cascade decision-making process is executed. The first-level network quickly determines the benign or malignant nature of the mass. When the test result is malignant, the second-level network is automatically activated for in-depth analysis, and the specific subtype is analyzed in combination with medical knowledge. A two-factor confidence system is simultaneously constructed throughout the diagnostic process. The reliability of image acquisition is quantified by separating the interference features of the equipment, and the self-consistency of the diagnostic logic is verified. Finally, the confidence is scientifically calibrated by integrating the dual factors of equipment reliability and decision rationality to output a customized diagnostic conclusion for the image and its precise credibility assessment, realizing a complete closed-loop processing from single image input to precise diagnosis, thereby improving the accuracy and reliability of ultrasound imaging diagnosis of breast masses.

[0082] Example 2, a breast mass ultrasound image classification system based on deep learning, is applied to a breast mass ultrasound image classification method based on deep learning. In the specific implementation process, Figure 2 As shown, it shows a module structure diagram of a breast mass ultrasound image classification system based on deep learning, including:

[0083] The multimodal feature decoupling module 100 is used to identify the image type of the currently input breast mass ultrasound image and extract physical basic features, pathological sensitivity features, and device interference features using a pre-trained unified feature extraction model, where the image types include B-mode ultrasound images, elastic imaging images, and color Doppler images;

[0084] The knowledge enhancement feature reinforcement module 200 is used to generate an image quality score based on the physical basic features, determine the enhancement processing strength according to the image quality score, and input the pathology sensitive features into the pre-trained cross-modal knowledge enhancer based on the enhancement processing strength to generate cross-modal reinforcement features;

[0085] An image classification decision module 300 is used to process pathology-sensitive features and cross-modal reinforcement features using a pre-trained two-stage cascade classification network to obtain image classification results and initial confidence levels;

[0086] Confidence calibration module 400, used to calculate the image reliability factor based on the device interference characteristics, calculate the model recognition factor based on the image classification results, and calibrate the initial confidence level using the image reliability factor and the model recognition factor to obtain the final confidence level;

[0087] The classification result output module 500 is used to output the image classification result and the final confidence as the final classification result.

[0088] The working principle of the deep learning-based breast mass ultrasound image classification system provided by the present invention is as follows:

[0089] First, the multimodal feature decoupling module 100 receives a single breast mass ultrasound image input and identifies its dominant type through a pre-trained unified model, automatically distinguishing one of the B-type ultrasound images, elastic imaging effects or color Doppler images, and decoupling the physical basic features, pathological sensitive features and equipment interference features. Then, the knowledge enhancement feature reinforcement module 200 performs type-specific quality assessment based on the physical basic features, adaptively selects the enhanced processing intensity according to the image quality score results, reconstructs the pathological sensitive features into a lesion semantic topology map, and then fuses the matching hierarchical medical knowledge to generate cross-modal reinforcement features. Then, the image classification decision module 300 is started The first-level classification network processes pathological sensitive features to judge benign or malignant. When malignant is detected, the second-level network is automatically activated to combine cross-modal reinforcement features to analyze subtypes such as intraductal carcinoma. Then the confidence calibration module 400 operates in parallel to separate reverberation artifacts, motion blur and noise components from device interference features to calculate the image reliability factor. At the same time, the model cognitive factor is calculated in combination with the diagnosis result type. Finally, the dual-factor driven exponential fusion calibrates the initial confidence to obtain the final confidence. The classification result output module 500 integrates the results of the previous modules to output customized image classification conclusions and their scientific credibility assessments to form a complete closed-loop processing chain from single image input to diagnosis output.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, magnetic disk storage, or magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0092] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

Claims

1. A breast mass ultrasound image classification method based on deep learning, characterized in that: The classification method comprises the following steps: S1: Use a pre-trained unified feature extraction model to identify the image type of the currently input breast mass ultrasound image and extract physical basic features, pathological sensitivity features, and device interference features. Image types include B-mode ultrasound images, elastography images, and color Doppler images. S2: Generate an image quality score based on physical basic features, and determine the enhancement processing strength based on the image quality score. Based on the enhancement processing strength, the pathology sensitive features are input into the pre-trained cross-modal knowledge enhancer to generate cross-modal reinforcement features; S3: Use a pre-trained two-stage cascade classification network to process pathology-sensitive features and cross-modal reinforcement features to obtain image classification results and initial confidence; S4: Calculate the image reliability factor based on the device interference characteristics, calculate the model recognition factor based on the image classification results, and calibrate the initial confidence level using the image reliability factor and the model recognition factor to obtain the final confidence level. S5: Output the image classification result and the final confidence as the final classification result; The physical basic characteristics include the acoustic impedance distribution matrix for B-mode ultrasound images, the strain field characteristic tensor for elastic imaging images, and the blood flow velocity field distribution for color Doppler images; The enhancement processing intensity is determined by a preset score and intensity mapping table; The steps of calculating the image reliability factor include: Perform three-dimensional frequency domain decomposition on the device interference characteristics to obtain the high-frequency components of reverberation artifacts, the temporal components of motion blur, and the spatial components of device noise; The energy per unit bandwidth of the high-frequency component of the reverberation artifact within a preset frequency band is calculated and used as the reverberation energy density. The maximum tissue displacement amplitude of the time-varying component of motion blur between consecutive image frames is detected and used as the motion offset peak. The Shannon entropy value of the spatial pixel distribution of the spatial component of device noise is calculated and used as the noise spatial entropy. The reverberation energy density, motion offset peak and noise spatial entropy are weighted and summed based on the preset weighted weights to obtain a composite interference intensity coefficient; The attenuation coefficient is selected based on the image type and the preset type coefficient mapping table. After the composite interference intensity coefficient is multiplied by the attenuation coefficient, it is converted into an image reliability factor through an exponential attenuation function.

2. The method for breast mass ultrasound image classification based on deep learning according to claim 1, characterized in that: Step S1 includes the following steps: S101: extracting a deep feature map of the currently input breast mass ultrasound image through a shared encoder in a pre-trained unified feature extraction model; S102: Input the deep feature map into the type recognition branch of the pre-trained unified feature extraction model to generate an image type probability vector, and take the type with the largest probability value in the image type probability vector as the image type; S103: Calculating feature modulation weights based on the image type probability vector, and performing weighted processing on the depth feature map using the feature modulation weights to obtain a weighted feature map; S104: The weighted feature maps are processed separately by three parallel feature decoupling branches in the pre-trained unified feature extraction model to obtain physical basic features, pathological sensitive features, and device interference features.

3. The method for breast mass ultrasound image classification based on deep learning according to claim 2, characterized in that: In step S2, the method for calculating the image quality score includes: The mean acoustic impedance gradient of the mass edge region of the acoustic impedance distribution matrix extracted from the physical basic features was calculated. A similarity score was calculated based on the mean acoustic impedance gradient and a preset ideal gradient threshold. The similarity score was truncated using a preset score truncation interval to obtain the image quality score of the B-mode ultrasound image. Identifying continuous regions whose strain variance is less than a preset strain variance threshold from the strain field feature tensor of the physical basic feature, and using the calculated area ratio of the continuous region as the image quality score of the elastic imaging image; Connected regions with velocities greater than a preset blood flow velocity threshold are extracted from the blood flow velocity field distribution of the physical basic characteristics, and the calculated area ratio of the connected regions is used as the image quality score of the color Doppler image.

4. The method for breast mass ultrasound image classification based on deep learning according to claim 3, characterized in that: In step S2, the processing steps of inputting the pathology-sensitive features into the pre-trained cross-modal knowledge enhancer include: Reconstruct the input pathology-sensitive features into a lesion semantic topology map; Based on the determined enhanced processing strength, a pre-constructed medical knowledge graph is hierarchically retrieved and a knowledge association vector set is obtained; The lesion semantic topology map and the knowledge association vector set are integrated to generate cross-modal reinforcement features.

5. The method for breast mass ultrasound image classification based on deep learning according to claim 4, characterized in that: In step S3, the pre-trained two-level cascade classification network includes a first-level network and a second-level network, wherein the first-level network processes pathology-sensitive features to generate pathology property classification results and first-level confidence, and the second-level network processes cross-modal reinforcement features to generate malignant subtype classification results and second-level confidence.

6. The method for breast mass ultrasound image classification based on deep learning according to claim 5, characterized in that: If and only if the pathological classification result generated by the first-level network is malignant, the second-level network is activated to process the cross-modal reinforcement features, and the malignant subtype classification result is used as the image classification result, and the second-level confidence is used as the initial confidence; When the pathological property classification result generated by the first-level network is not malignant, the second-level network is skipped, and the pathological property classification result is used as the image classification result, and the first-level confidence is used as the initial confidence.

7. The method for breast mass ultrasound image classification based on deep learning according to claim 6, characterized in that: In step S4, the initial confidence level is calibrated using an exponential fusion function to obtain a final confidence level.

8. A deep learning-based breast mass ultrasound image classification system, characterized by: Applied to a deep learning-based breast mass ultrasound image classification method as described in any one of claims 1 to 7, the classification system comprises: A multimodal feature decoupling module is used to identify the image type of the currently input breast mass ultrasound image and extract physical basic features, pathological sensitivity features, and device interference features using a pre-trained unified feature extraction model. Image types include B-mode ultrasound images, elastography images, and color Doppler images. The knowledge enhancement feature reinforcement module is used to generate image quality scores based on physical basic features, determine the enhancement processing strength based on the image quality scores, and input pathology sensitive features into the pre-trained cross-modal knowledge enhancer based on the enhancement processing strength to generate cross-modal reinforcement features; Image classification decision module, which uses a pre-trained two-stage cascade classification network to process pathology-sensitive features and cross-modal reinforcement features to obtain image classification results and initial confidence; The confidence calibration module is used to calculate the image reliability factor based on the device interference characteristics, calculate the model recognition factor based on the image classification results, and calibrate the initial confidence level through the image reliability factor and the model recognition factor to obtain the final confidence level; The classification result output module is used to output the image classification results and final confidence as the final classification results.

Images