Large-scale pre-training distillation alignment method and device for ultrasonic diagnosis

Abstract

This invention discloses a large-scale pre-trained distillation alignment method and apparatus for ultrasound diagnosis. The method includes: acquiring an ultrasound image; inputting the ultrasound image into a pre-trained semantic distillation model to obtain feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer, which comprises two functionally complementary linear transformation layers with shared weight transpose. It can adapt to large-scale pre-training scenarios for ultrasound of organs such as the breast, axillary lymph nodes, abdomen, and neck, achieving lossless, accurate, and efficient heterogeneous model semantic distillation and feature alignment, thus enabling precise localization of lesions in ultrasound images.

Description

Technical Field

[0001] This invention relates to the field of ultrasound image processing technology, and in particular to a large-scale pre-training distillation alignment method and apparatus, storage medium, and electronic device for ultrasound diagnosis. Background Technology

[0002] In the field of large-scale pre-training and lightweight model deployment for intelligent breast ultrasound detection, knowledge distillation is a core technology for achieving efficient knowledge transfer from a large-parameter teacher model to a lightweight student model. It can significantly reduce the number of model inference parameters and latency while effectively preserving the model's ability to represent the texture and anatomical layering features of breast lesions and its detection performance, thus meeting the terminal deployment requirements for real-time breast ultrasound screening.

[0003] However, in large-scale self-supervised pre-training and heterogeneous model distillation scenarios for breast ultrasound images, there is an inherent problem of feature dimension mismatch. Mainstream high-parameter teacher models, such as DINOv3-large, can output 1024-dimensional high-dimensional global semantic features, which can fully capture the complex anatomical structure and subtle lesion features of the breast. However, lightweight student models adapted for clinical terminal deployment, such as ViT-Base, can only output 768-dimensional low-dimensional features. The physical difference between high- and low-dimensional feature spaces makes it impossible for heterogeneous teacher and student model features to be directly and accurately aligned and efficiently transferred, which seriously restricts the utilization efficiency of large-scale unlabeled breast ultrasound data. At present, the mainstream heterogeneous model feature alignment methods have many inherent defects. The dimension matching method of simple truncation or brute-force interpolation will directly lose the core semantic information such as breast microcalcifications and spiculated edges learned by the teacher model, or introduce invalid noise features, which will significantly reduce the quality of knowledge transfer. The method of using two sets of independent linear layers to transform the dimensions of teacher and student model features respectively lacks the constraint of the topological structure of the feature space, which can easily cause the feature space to be distorted and destroy the relative feature associations of glands, fat and lesions between breast samples. Traditional point-to-point feature alignment methods only constrain the approximation of single feature values, which cannot capture high-order contextual information such as the layered structure of breast ultrasound and the subtle texture of lesions, and make it difficult to transfer the global anatomical cognition ability of the teacher model. Summary of the Invention

[0004] To address the aforementioned issues, this application proposes a large-scale pre-training distillation and alignment method, apparatus, storage medium, and electronic device for breast ultrasound diagnosis. This method is adaptable to large-scale pre-training scenarios for breast ultrasound and can achieve non-destructive, accurate, and efficient semantic distillation and feature alignment of heterogeneous models.

[0005] This application proposes a large-scale pre-trained distillation alignment method for ultrasound diagnosis, comprising:

[0006] Acquire ultrasound images;

[0007] The ultrasound image is input into a pre-trained semantic distillation model to obtain the feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer. The transposed shared linear layer contains two functionally complementary linear transformation layers with weight transposed sharing.

[0008] In some embodiments, the pre-trained semantic distillation model includes a teacher model branch, a student model branch, and a transposed shared linear layer alignment module, wherein the transposed shared linear layer alignment module is disposed between the teacher model branch and the student model branch.

[0009] In some embodiments, the training process of the semantic distillation model includes:

[0010] Obtain ultrasound image samples after masking;

[0011] The masked ultrasound image samples are input into the teacher model and the student model respectively, and the heterogeneous feature alignment between the teacher model and the student model is realized based on the transposed shared linear layer alignment module.

[0012] The transposed shared linear layer alignment module is trained synchronously with the student model through backpropagation to obtain the semantic distillation model.

[0013] In some embodiments, heterogeneous feature alignment between the teacher model and the student model is achieved based on the transposed shared linear layer alignment module, including:

[0014] The ultrasound image sample is input into the teacher model to obtain the first feature, and the first feature is then dimensionality-reduced and aligned to obtain the dimensionality-reduced feature;

[0015] The reduced-dimensional features are then reconstructed by increasing their dimensionality to obtain reconstructed features, thereby achieving heterogeneous feature alignment between the teacher model and the student model.

[0016] In some embodiments, the dimensional reconstruction is achieved through the following formula:

[0017] in, This represents the features after dimensional reconstruction. This represents the dimensionality-reduced features after dimensionality reduction and alignment. These represent the weight matrices of the two transposed linear layers, The first feature.

[0018] In some embodiments, the transposed shared linear layer alignment module is trained synchronously with the student model via backpropagation to obtain the semantic distillation model, including:

[0019] The masked ultrasound image samples were input into the teacher model and the student model respectively, and the feature reconstruction loss, feature alignment loss and mask reconstruction loss were obtained respectively.

[0020] The transposed shared linear layer alignment module is backpropagated based on the feature reconstruction loss, and the student model branch is backpropagated based on the feature alignment loss and the mask reconstruction loss to obtain the semantic distillation model.

[0021] In some embodiments, the loss function of the semantic distillation model is composed of a weighted fusion of the mask reconstruction loss, the feature alignment loss, and the feature reconstruction loss.

[0022] This application also proposes a large-scale pre-trained distillation alignment device for ultrasound diagnosis, comprising:

[0023] The image acquisition module is used to acquire ultrasound images;

[0024] The image processing module is used to input the ultrasound image into a pre-trained semantic distillation model to obtain the feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer. The transposed shared linear layer contains two functionally complementary linear transformation layers with weight transposed sharing.

[0025] This application also proposes a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method described above.

[0026] This application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the method described above.

[0027] The method described in the above embodiments acquires ultrasound images and inputs them into a pre-trained semantic distillation model to obtain feature information of the target object in the ultrasound image. The semantic distillation model, through a lightweight and innovative design using a transposed shared linear layer, fundamentally solves core problems in large-scale pre-training of breast ultrasound models, such as feature dimension mismatch, loss of subtle semantic information, and feature space distortion. It accurately adapts to self-supervised pre-training of breast ultrasound and subsequent lesion detection tasks, demonstrating significant technical advantages and clinical screening application value. The method in the above embodiments only adds one set of transposed shared linear layers, requiring minimal modification, facilitating engineering deployment, and resulting in fewer parameters and stronger stability during large-scale training, effectively avoiding the shortcomings of traditional alignment methods.

[0028] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0029] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0030] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0031] Figure 1 This is a flowchart of a large-scale pre-trained distillation alignment method for ultrasound diagnosis provided in the embodiments of this application;

[0032] Figure 2 This is a schematic diagram of the semantic distillation model structure framework provided in the embodiments of this application;

[0033] Figure 3 This is a schematic diagram comparing the feature space PCA dimensionality reduction provided in the embodiments of this application;

[0034] Figure 4 These are schematic diagrams illustrating the pre-training effects of the two models provided in the embodiments of this application;

[0035] Figure 5 This is a structural block diagram of the large-scale pre-trained distillation alignment device for ultrasound diagnosis provided in the embodiments of this application. Detailed Implementation

[0036] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0037] In the semantic distillation MAE self-supervised pre-training framework built for breast ultrasound, the above-mentioned technical defects are further highlighted. It is impossible to achieve efficient feature alignment between the DINOv3-large teacher model and the ViT-Base student model. As a result, the self-supervised learning value of massive unlabeled breast ultrasound video and image data cannot be fully explored. The model's ability to extract the low-level features of occult and small breast lesions is limited, which ultimately restricts the improvement of accuracy and robustness of downstream breast lesion detection and benign / malignant classification tasks.

[0038] To address the aforementioned problems, this application proposes a large-scale pre-trained distillation alignment method for ultrasound diagnosis, using breast ultrasound images as an example in the following embodiments. Figure 1 As shown, it includes:

[0039] Step 101: Acquire ultrasound images.

[0040] In this embodiment, ultrasound images of the breast are acquired for model training.

[0041] Step 102: Input the ultrasound image into a pre-trained semantic distillation model to obtain the feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer. The transposed shared linear layer contains two functionally complementary linear transformation layers with weight transposed sharing.

[0042] In this embodiment, the target object refers to organs and tissues such as the breast, armpit, and abdomen. The feature information of the target object refers to, for example, the speckle noise distribution and tissue texture unique to breast ultrasound. The pre-trained semantic distillation model effectively solves problems such as feature dimension mismatch, feature space distortion, and subtle semantic loss in heterogeneous teacher-student models, providing a highly robust low-level image feature extraction capability for subsequent tasks such as organ stratification, lesion detection, and benign / malignant differentiation.

[0043] The method described in the above embodiments acquires ultrasound images and inputs them into a pre-trained semantic distillation model to obtain feature information of the target object in the ultrasound image. The semantic distillation model, through a lightweight and innovative design using a transposed shared linear layer, fundamentally solves core problems in large-scale pre-training of breast ultrasound models, such as feature dimension mismatch, loss of subtle semantic information, and feature space distortion. It accurately adapts to self-supervised pre-training of breast ultrasound and subsequent lesion detection tasks, demonstrating significant technical advantages and clinical screening application value. The method in the above embodiments only adds one set of transposed shared linear layers, requiring minimal modification, facilitating engineering deployment, and resulting in fewer parameters and stronger stability during large-scale training, effectively avoiding the shortcomings of traditional alignment methods.

[0044] In some embodiments, the pre-trained semantic distillation model includes a teacher model branch, a student model branch, and a transposed shared linear layer alignment module.

[0045] refer to Figure 2Taking breast ultrasound images as an example, the overall architecture of the large-scale pre-trained semantic distillation model (MAE) consists of three core modules: a teacher model with frozen weights, a transposed shared linear layer alignment module, and a trainable student model. The transposed shared linear layer feature alignment module, deployed between the teacher model branch and the student model branch, plays a crucial role in losslessly aligning and adapting the 1024-dimensional high-dimensional global semantic features of the breast tissue output by the DINOv3-large teacher model to the 768-dimensional low-dimensional feature space of the ViT-Base student model, thus adapting to the self-supervised learning scenario of large-scale breast ultrasound data.

[0046] In some embodiments, the training process of the semantic distillation model includes:

[0047] 1. Obtain breast ultrasound image samples after masking.

[0048] In this embodiment, masking randomly covers most of the image patch regions in a breast ultrasound image, leaving only a small number of visible patches. This forces the model to infer global semantics based on incomplete information, thereby learning a more robust feature representation. Specific implementation steps:

[0049] (1) Image patching: The input breast ultrasound image (e.g., 512×512 pixels) is divided into non-overlapping image patch grids of a fixed size (e.g., 16×16 pixels). Taking a 512×512 input and a 16×16 patch as an example, a total of 32×32=1024 patches are obtained.

[0050] (2) Generate masking ratio: Set the masking ratio. In this embodiment, the teacher model uses a 75% masking ratio, that is, randomly cover 75% of the patches and keep only 25% of the visible patches. For 1024 patches, 256 visible patches are kept and 768 patches are covered.

[0051] Random sampling of visible patches: A uniform random sampling strategy is adopted, and 256 patches are randomly selected without replacement from 1024 patches as the visible patch set, while the remaining 768 patches are masked patches.

[0052] (3) Generate a binary mask matrix: Construct a 32×32 binary mask matrix M, where the positions corresponding to visible patches are marked as 1, and the positions corresponding to masked patches are marked as 0. This mask matrix M is subsequently used for MAE mask reconstruction loss calculation.

[0053] (4) Input visible patches: Only the visible patches (256) are input into the encoders of the teacher model and the student model. The masked patches will not participate in the forward propagation calculation, which greatly reduces the amount of computation.

[0054] Through the masking process described above, both the teacher model (DINOv3-large) and the student model (ViT-Base) receive breast ultrasound images with the same mask, and encoding is performed only based on 25% of the visible patches. The teacher model outputs 1024-dimensional high-dimensional global semantic features containing breast glandular tissue, fat, and lesion texture; the student model outputs 768-dimensional features, and knowledge distillation is performed between the student and teacher features through a transposed shared linear layer alignment module.

[0055] 2. Input the masked breast ultrasound image samples into the teacher model and student model respectively, and realize the heterogeneous feature alignment between the teacher model and student model based on the transposed shared linear layer alignment module.

[0056] In this embodiment, a large-scale breast ultrasound image sample, after being masked, is input into the teacher model, and a high-dimensional global semantic feature containing breast glandular tissue, fat, and lesion texture is output. Combined with... Figure 2 The teacher model (DINOv3-Large) freezes the weights during training without modifying them, and extracts 1024-dimensional global semantic features from the masked ultrasound image.

[0057] Freezing weights is a technique that fixes model parameters during training, preventing them from being updated. After freezing, the teacher model calculates output features normally during forward propagation, but does not calculate gradients or update parameters during backpropagation. Since the teacher model (DINOv3-large) has been pre-trained on large-scale natural images and possesses powerful general visual semantic representation capabilities, freezing its weights serves as a fixed "semantic anchor," preventing semantic drift in the teacher model's representation during large-scale iterative training and ensuring that the student model always learns towards a stable and high-quality target.

[0058] During training, the teacher model's weights are frozen throughout, serving only as anchor points for fixed breast semantic features to avoid semantic drift during large-scale iterative training. Both the teacher and student models are inputted with identical masked breast ultrasound images, forcing the teacher model to infer global breast anatomical semantics and lesion features based on incomplete image information, thus providing high-quality supervision signals to the student model.

[0059] Furthermore, the transpose shared linear layer alignment module completes the high-dimensional feature dimensionality reduction alignment and feature reconstruction constraints, for example... Figure 2As shown, in this embodiment, the transposed shared linear layer alignment module completes the dimensional transformation and semantic constraints of the heterogeneous ultrasound features through a set of shared weights and transposed linear layers L1 and L2, which are equivalent to dynamic principal component screening. This accurately preserves the core semantic features learned by the teacher model, such as breast microcalcifications, spiculated boundaries, and layered anatomy, avoiding the feature distortion and loss of subtle lesion information problems of traditional alignment methods, and adapting to the feature learning needs of large-scale pre-training of breast ultrasound.

[0060] (4) The transposed shared linear layer alignment module is trained synchronously with the student model through backpropagation to obtain the semantic distillation model.

[0061] In this embodiment, the transposed shared linear layer alignment module is trained synchronously with the student model via backpropagation; that is, the transposed shared linear layer and the encoder of the student model are trained synchronously and iteratively. In one training iteration, the weight matrix W of the transposed shared linear layer and the parameters of the student ViT-Base encoder simultaneously receive gradients and are updated simultaneously. The synchronous iterative training of the transposed shared linear layer and the encoder of the student model allows for lossless transfer of the high-dimensional global breast semantics of the teacher model to the student model. Simultaneously, the student model, relying on the MAE mask reconstruction task, learns local detailed features such as speckle texture of breast ultrasound, the boundary between glandular and fat layers, and the fine structure of lesions on a large scale, achieving bidirectional fusion of global breast semantics and local detailed features, and comprehensively improving the feature representation capability of the lightweight student model for breast ultrasound scenarios.

[0062] In some embodiments, heterogeneous feature alignment between the teacher model and the student model is achieved based on the transposed shared linear layer alignment module, including:

[0063] (1) Input the ultrasound image sample into the teacher model to obtain the first feature, and then perform dimension reduction and alignment on the first feature to obtain the dimension reduction feature.

[0064] refer to Figure 2 The transpose-shared linear layer alignment module contains two complementary linear transformation layers with shared transpose weights, strictly satisfying the weight constraint relationship: the weight matrix of linear layer L1 is... It is responsible for reducing the 1024-dimensional high-dimensional breast features of the teacher model to a 768-dimensional feature space suitable for the student model; the weight matrix of the linear layer L2 is... , which is the transpose of the L1 weight matrix, is responsible for upscaling and reconstructing the aligned low-dimensional features into the original high-dimensional features, enabling retrospective verification of breast semantic information. Taking breast ultrasound images as an example, the dimensionality reduction and alignment process is as follows:

[0065] The first feature output by the teacher model, namely the primitive breast feature. (B is the training batch size) Input a linear L1 layer to perform dimensionality reduction and alignment of high-dimensional features, and output dimensionality-reduced features that match the student model's dimensions. ,Right now The specific implementation process is as follows:

[0066] Input tensor The shape is [B, 1024], for example, when the batch size B=64. The shape is [64, 1024], and the weight matrix W is [1024, 768], which is a trainable parameter matrix initialized to a Xavier uniform distribution.

[0067] Perform matrix multiplication [64, 1024] × [1024, 768] → [64, 768], yielding the dimensionality-reduced alignment features. This process projects 1024-dimensional high-dimensional features onto a 768-dimensional subspace. Through subsequent feature alignment loss constraints, it forces the column vectors of W to select the principal component directions with the largest variance and the most core semantic information in the teacher features.

[0068] For example, assuming batch size B=2 (processing 2 ultrasound images at a time), the teacher model branch outputs... The matrix is ​​2×1024. The weights W of the linear layer L1 are a 1024×768 matrix. Matrix multiplication. The result It is a 2×768 matrix, meaning that each ultrasound image corresponds to a 768-dimensional feature vector, which perfectly matches the output dimension of the student model branch ViT-Base.

[0069] (2) The reduced features are reconstructed by increasing their dimensions to obtain the reconstructed features, thereby achieving the alignment of heterogeneous features between the teacher model and the student model.

[0070] Dimensionally reduced and aligned breast features Inputting the transposed linear layer L2, we achieve feature dimensionality upscaling and reconstruction, restoring the original feature dimensions. The calculation formula for dimensionality upscaling and reconstruction is as follows:

[0071]

[0072] That is, the mammary gland features after dimensionality reduction and alignment. Input transpose linear layer L2, through Calculate, where, , This achieves the dimensionality reconstruction of reduced features, restoring the original feature dimensions. Specific implementation process:

[0073] Input tensor The shape is [B, 768], and the weight matrix is... for The transpose of , with shape [768, 1024], does not introduce additional trainable parameters (it shares the same set of weights W as L1).

[0074] Perform matrix multiplication [B, 768] × [768, 1024] → [B, 1024], yielding the reconstructed features. . Characteristics of the original teacher With consistent dimensionality (both 1024 dimensions), feature reconstruction loss constraints ensure that core semantic information is not lost during the closed-loop process of dimensionality reduction-up. For example, continuing with the example of B=2 above, It is a 2×768 matrix. It is a 768×1024 matrix. get It is a 2×1024 matrix. Dimensions and original Completely identical. (Through comparison) and The difference (L2 distance) can be used to determine the 768-dimensional difference. Whether the core information in the original 1024-dimensional features has been fully preserved.

[0075] In some embodiments, the transposed shared linear layer alignment module is trained synchronously with the student model via backpropagation, including:

[0076] (1) Input the masked breast ultrasound image samples into the teacher model and the student model respectively, and obtain the feature reconstruction loss, feature alignment loss and mask reconstruction loss respectively.

[0077] The student model, relying on the aligned semantic supervision of the teacher model, learns breast ultrasound-specific hierarchical structures and lesion texture features on a large scale using the MAE mask reconstruction task. The student model's encoder (MAE ViT-Base), under aligned semantic supervision of the teacher model, learns the ultrasound low-level texture features using the MAE mask reconstruction task, resulting in the mask reconstruction loss. .

[0078] Feature Reconstruction Loss To constrain the consistency between the original high-dimensional breast features and the reconstructed features, and to supervise the weight updates of the transposed shared linear layer, ensuring that core semantic information such as subtle breast lesions and hierarchical structures is not lost during the large-scale dimensionality reduction and alignment process, the L2 loss method is used, and the formula is as follows:

[0079] ,

[0080] In the formula, The original group of breast features output by the teacher model. Let B represent the i-th group of breast features after dimensionality upgrading and reconstruction, and let B be the total number of samples in the batch. This represents the squared L2 norm, which is the average squared Euclidean distance between the teacher model features and the reconstructed aligned features of a single sample. This represents the average of the squared Euclidean distances among all samples in the batch.

[0081] Feature alignment loss This is used to constrain the consistency between the breast features output by the student model and the breast features after alignment with the teacher model, driving the student model to accurately learn the core breast semantic representation of the teacher model. It is adapted to large-scale breast data pre-training and uses the L2 loss calculation method, as shown in the following formula:

[0082] ,

[0083] In the formula, For the i-th group of breast features output by the student model, Here, B represents the dimensionality-reduced and aligned breast features of the i-th group in the teacher model, and B is the total number of samples in the batch. Let L2 be the squared value, which is the average squared Euclidean distance between the student model features and the teacher model features after alignment. By minimizing this loss, the student model is driven to learn the core semantic representations (mammary glands, fat, lesion texture, etc.) of the teacher model after dimensionality reduction, achieving lossless transfer of knowledge from the large model to the small model.

[0084] (2) Backpropagation is performed on the transposed shared linear layer alignment module based on feature reconstruction loss, and backpropagation is performed on the student model branch based on feature alignment loss and mask reconstruction loss to obtain the semantic distillation model.

[0085] The encoder parameters of the student model (ViT-Base) and the weight matrix W of the transposed shared linear layer alignment module L1 / L2 are all trainable parameters. During training, the transposed shared linear layer alignment module and the student encoder are trained synchronously and iteratively, and the high-dimensional global breast semantics of the teacher model branch are transferred to the student model branch without loss. At the same time, the student model branch relies on the MAE mask reconstruction task to learn local detailed features such as breast ultrasound speckle texture, gland and fat layer boundary, and lesion fine structure on a large scale, so as to realize the bidirectional fusion of breast global semantics and local detailed features, and comprehensively improve the feature representation ability of the lightweight student model for breast ultrasound scene.

[0086] The total loss of the model training is composed of a weighted fusion of MAE mask reconstruction loss, feature alignment loss, and feature reconstruction loss, achieving joint optimization of multiple feature dimensions of breast tissue. The total loss formula is as follows:

[0087] ,

[0088] In the formula, The native mask reconstruction loss of the breast semantic distillation MAE framework is used to constrain the reconstruction capability of the decoder in the student model (only updating the student model parameters); feature alignment loss. Simultaneously constrain the student model encoder (through...) ) and transposed shared linear layer (via ).if and The difference is large, and the gradient will be fed back to both the student encoder and the linear layer W, so that the outputs of the two tend to be consistent in the alignment space. This is the weight for the feature alignment loss, with a default value of 1.0. This sets the weights for the feature reconstruction loss; the default value is 0.5. Feature Reconstruction Loss Only constrain the transpose shared linear layer (through) Ensure that the dimensionality reduction-up closure loop does not lose core semantics.

[0089] For example, considering the inherent characteristics of breast ultrasound images, such as severe speckle noise, complex layering of glandular and fatty tissues, blurred lesion boundaries, highly concealed microcalcifications, and high similarity between benign and malignant lesions, the method in this application has been specifically adapted and optimized to meet the needs of large-scale pre-training of breast ultrasound. On one hand, a noise robustness optimization strategy is added. Before the breast feature input transpose shared linear layer alignment module of the teacher model, the high-dimensional semantic features are Gaussian smoothed to filter out random speckle noise and scanning artifacts in breast ultrasound, preventing noise features from affecting the dimensional alignment accuracy and the semantic transfer effect of subtle lesions. On the other hand, considering the complex structure and difficult-to-distinguish concealed lesions in breast ultrasound, a breast-specific adaptive weight adjustment mechanism is designed to uniformly optimize the loss weight ratio and adjust the feature reconstruction loss weight. Fine-tuned to 0.6, it enhances the preservation of weak core semantic features such as microcalcifications and spiculated edges in the breast, while taking into account the convergence speed and feature learning accuracy of large-scale pre-training, and is suitable for self-supervised training scenarios with massive amounts of unlabeled breast ultrasound data.

[0090] Furthermore, principal component analysis (PCA) was performed on the encoder features of the student models trained using the method of this application embodiment and pure MAE pre-training, and the dimensionality was reduced to 2D for visualization, as shown below. Figure 3 The results are shown. The specific steps of PCA are as follows:

[0091] (1) The original data, namely the N×768-dimensional global semantic feature matrix extracted from N breast ultrasound images by the student model, is standardized.

[0092] In the specific implementation process, the mean of each column (each feature dimension) of the N×768-dimensional feature matrix is ​​subtracted and the standard deviation of that column is divided to make the mean of each dimension of the feature matrix 0 and the variance 1.

[0093] (2) Calculate the covariance matrix and solve for the eigenvectors to obtain the principal components sorted by variance contribution.

[0094] Specifically, calculate the 768×768 covariance matrix (the pairwise covariance of N samples in the 768-dimensional feature space), solve for the eigenvalues ​​and eigenvectors of the covariance matrix, sort them in descending order of eigenvalues ​​(variance contribution), and obtain 768 principal component directions. The larger the eigenvalue, the larger the variance of the feature in the corresponding principal component direction, and the richer the breast semantic information it carries. (3) Select the first two principal components (PC1 and PC2) as the new coordinate axes and project the data onto this two-dimensional space.

[0095] (4) The output is a two-dimensional scatter plot, where the X-axis is PC1, representing the direction of the largest variance in the data; the Y-axis is PC2, representing the direction orthogonal to PC1 and representing the second largest variance. The color / shape of the points often represents different labels. For example... Figure 3 The features of the pure MAE pre-training model shown are scattered in 2D space, and the cluster boundaries of features of glands, fat and lesion regions are blurred; while in the feature space of the semantic distillation model MAE in this embodiment, the three types of regions show obvious cluster separation, which verifies that the transposed shared linear layer alignment module effectively preserves the high-order semantic information of the teacher model (DINOv3).

[0096] Furthermore, a comparison of the final lesion detection performance of the semantic distillation model pre-trained using the embodiments of this application and the pure MAE-trained model is shown below. Figure 4As shown, the model in this embodiment of the application achieves accurate lesion detection. In the subsequent task of breast ultrasound lesion detection, the F1 score of the model in this embodiment of the application is improved from 0.564 in pure MAE pre-training to 0.614 (an improvement of 8.9%), mAP@50 from 0.515 to 0.546, and mAP@50:95 from 0.245 to 0.288. This significantly improves the lightweight model's ability to identify breast microcalcifications, occult lesions, and malignant lesions with blurred boundaries, as well as the accuracy of benign and malignant classification diagnosis. It effectively reduces the false positive and false negative rates of breast ultrasound detection, enhances the robustness of the semantic distillation model in complex breast backgrounds, and can be widely adapted to the large-scale pre-training and lightweight terminal deployment scenarios of breast ultrasound intelligent screening systems. It should be noted that the F1 score is a comprehensive measure of the precision and recall of the semantic distillation model. The F1 score ranges from [0, 1]. The closer it is to 1, the better the semantic distillation model has achieved in terms of "no false positives" and "no false negatives".

[0097] In some embodiments, such as Figure 5 As shown, a large-scale pre-trained distillation alignment device for ultrasound diagnosis is also proposed, comprising:

[0098] Image acquisition module 501 is used to acquire ultrasound images;

[0099] Image processing module 502 is used to input the ultrasound image into a pre-trained semantic distillation model to obtain feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer. The transposed shared linear layer contains two functionally complementary linear transformation layers with transposed weights.

[0100] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the methods described in the above embodiments.

[0101] In some embodiments, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the methods described in the various embodiments above.

[0102] The principles by which the above-mentioned devices, computer storage media, and electronic devices in the embodiments of this application solve the problem are similar to those of the aforementioned methods. Therefore, their implementation can refer to the implementation of the aforementioned methods, and repeated details will not be repeated.

[0103] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0104] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of equivalents of this invention, this invention is also intended to include these modifications and variations.

Claims

1. A large-scale pre-trained distillation alignment method for ultrasound diagnosis, characterized in that, include: Acquire ultrasound images; The ultrasound image is input into a pre-trained semantic distillation model to obtain the feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer. The transposed shared linear layer contains two functionally complementary linear transformation layers with weight transposed sharing.

2. The method as described in claim 1, characterized in that, The pre-trained semantic distillation model includes a teacher model branch, a student model branch, and a transposed shared linear layer alignment module, which is located between the teacher model branch and the student model branch.

3. The method as described in claim 2, characterized in that, The training process of the semantic distillation model includes: Obtain ultrasound image samples after masking; The masked ultrasound image samples are input into the teacher model and the student model respectively, and the heterogeneous feature alignment between the teacher model and the student model is realized based on the transposed shared linear layer alignment module. The transposed shared linear layer alignment module is trained synchronously with the student model through backpropagation to obtain the semantic distillation model.

4. The method as described in claim 3, characterized in that, The alignment module based on the transposed shared linear layer realizes the alignment of heterogeneous features between the teacher model and the student model, including: The ultrasound image sample is input into the teacher model to obtain the first feature, and the first feature is then dimensionality-reduced and aligned to obtain the dimensionality-reduced feature; The reduced-dimensional features are then reconstructed by increasing their dimensionality to obtain reconstructed features, thereby achieving heterogeneous feature alignment between the teacher model and the student model.

5. The method as described in claim 4, characterized in that, The dimensional reconstruction is achieved through the following formula: ; in, This represents the features after dimensional reconstruction. This represents the dimensionality-reduced features after dimensionality reduction and alignment. These represent the weight matrices of the two transposed linear layers, The first feature.

6. The method as described in claim 3, characterized in that, The transposed shared linear layer alignment module is trained synchronously with the student model through backpropagation to obtain the semantic distillation model, including: The masked ultrasound image samples were input into the teacher model and the student model respectively, and the feature reconstruction loss, feature alignment loss and mask reconstruction loss were obtained respectively. The transposed shared linear layer alignment module is backpropagated based on the feature reconstruction loss, and the student model branch is backpropagated based on the feature alignment loss and the mask reconstruction loss to obtain the semantic distillation model.

7. The method as described in claim 6, characterized in that, The loss function of the semantic distillation model is composed of a weighted fusion of the mask reconstruction loss, the feature alignment loss, and the feature reconstruction loss.

8. A large-scale pre-trained distillation alignment device for ultrasound diagnosis, characterized in that, include: The image acquisition module is used to acquire ultrasound images; The image processing module is used to input the ultrasound image into a pre-trained semantic distillation model to obtain the feature information of the target object in the ultrasound image. The semantic distillation model achieves heterogeneous feature alignment based on a transposed shared linear layer. The transposed shared linear layer contains two functionally complementary linear transformation layers with weight transposed sharing.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

10. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

Images