Domain confrontation federated learning method for cross-style multi-center ultrasonic image classification

Abstract

The invention relates to the technical field of image classification, in particular to a cross-style multi-center ultrasound image classification-oriented domain confrontation federated learning method, which comprises the following steps of: acquiring fetal ultrasound image data of multiple centers, establishing a multi-domain image data set, and constructing local and global models based on a federated learning framework of a preset standard, designing a basic feature extraction network, and initializing a local model; in a local training process, a domain confrontation module is introduced, a semantic consistency alignment module based on singular value decomposition is designed, and after each round of local training and feature alignment are completed, global model aggregation is performed, and next round of training is performed until a federal learning medical image classification model with privacy protection and cross-domain generalization capabilities is generated. Therefore, the problems that data security cannot be ensured and cross-style multi-center non-independent synchronous image data cannot be effectively adapted due to the fact that a traditional ultrasonic image classification method adopts a centralized training architecture and ultrasonic images have remarkable style differences among different collection centers are solved.

Description

Technical Field

[0001] The present application relates to the field of image classification technology, and in particular to a domain adversarial federated learning method for cross-style multi-center ultrasound image classification. Background Art

[0002] Prenatal screening ultrasound image classification is a key step in the early diagnosis of fetal developmental abnormalities, and its accuracy directly impacts the timeliness of clinical intervention and the health prognosis of both mother and baby. Traditional fetal ultrasound image screening methods rely primarily on the experience of clinicians for manual analysis and judgment. In practice, different hospitals and physicians typically use two-dimensional grayscale ultrasound images to determine and classify fetal development and structural abnormalities. This method places extremely high demands on the physician's professional expertise and operational experience, and diagnostic results are easily influenced by subjective factors, resulting in a certain degree of inconsistency in judgment criteria. In addition, due to the uneven distribution of medical resources, some primary healthcare institutions lack highly skilled ultrasound physicians, making it difficult to ensure the accuracy and consistency of image screening. In a multi-center collaborative scenario, significant differences between hospitals due to factors such as equipment models, imaging protocols, and operator techniques further exacerbate the diversity of image style and quality, making traditional methods face serious generalization issues in cross-center applications.

[0003] Among related technologies, deep learning demonstrates powerful feature extraction and automatic recognition capabilities in medical image classification tasks, particularly in processing fetal ultrasound images with complex structures and unclear textures. In recent years, numerous studies have attempted to automatically classify ultrasound images by building end-to-end convolutional neural network models, reducing manual intervention and improving diagnostic efficiency.

[0004] However, existing deep learning methods typically employ a centralized training architecture, requiring images from different hospitals to be uploaded to a central server for training. This approach not only carries a significant risk of privacy breaches but also struggles to meet the stringent data security requirements of medical scenarios. Furthermore, ultrasound images can exhibit significant style differences (such as brightness, texture, and artifacts) between different acquisition centers. Traditional centralized deep learning methods struggle to effectively adapt to this cross-style, multi-center, non-independent and identically distributed (non-IID) data, resulting in insufficient model generalization and difficulty maintaining stable performance in practical deployments.

[0005] In summary, among the related technologies, the ultrasound image classification method based on deep learning cannot guarantee data security; and cannot effectively adapt to non-independent synchronous image data across styles and multiple centers, making it difficult for the model to maintain stable performance in actual deployment; and it is in urgent need of improvement. Summary of the Invention

[0006] This application provides a domain-adversarial federated learning method for cross-style multi-center ultrasound image classification to solve the problems in related technologies such as the inability of deep learning-based ultrasound image classification methods to guarantee data security; and the inability to effectively adapt to cross-style multi-center non-independent synchronized image data, which makes it difficult for the model to maintain stable performance in actual deployment.

[0007] The first aspect of the present application provides a domain adversarial federated learning method for cross-style multi-center ultrasound image classification, including the following steps: collecting fetal ultrasound image data from multiple centers to establish a multi-domain image dataset, and based on a preset standard federated learning framework and the multi-domain image dataset, constructing a local model and a global model, designing a basic feature extraction network, and initializing the local model; in the local training process, introducing a domain adversarial module to use adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements; designing a semantic consistency alignment module based on singular value decomposition to perform low-rank decomposition and feature alignment on the local model features and global model features corresponding to the local model and the global model, and after completing each round of local training and feature alignment, using a preset federal averaging strategy to perform global model aggregation, and enter the next round of training until a federated learning medical image classification model with both privacy protection and cross-domain generalization capabilities is generated.

[0008] Through the above technical solution, the embodiment of the present application can deploy a local model based on the federated learning framework, combine the domain adversarial module (DoA) to suppress image style differences, and enable the feature extraction network to learn domain invariant features through adversarial training; and further design a semantic consistency alignment module (SVD-Semantic Alignment, SVD-SA) based on singular value decomposition (SVD) to perform low-rank decomposition of local and global features to eliminate inter-domain semantic offsets; by introducing a dual optimization mechanism of domain adversarial training and semantic consistency alignment, the problem of model performance degradation caused by image style differences in traditional methods in multi-center scenarios is solved. Compared with centralized deep learning, the embodiment of the present application can avoid sharing of original data through a federated architecture and effectively protect patient privacy; compared with existing federated learning methods, the DoA module can suppress interference from non-independent and identically distributed data, and the SVD-SA module can be combined to enhance cross-domain semantic consistency, achieving SOTA performance on a four-center fetal ultrasound dataset.

[0009] Optionally, in one embodiment of the present application, after establishing the multi-domain image dataset, it also includes: performing image enhancement processing and standardization processing on the data of the multi-domain image dataset to obtain processed data; and eliminating image samples with artifacts or quality problems in the processed data to obtain the final multi-domain image dataset.

[0010] Through the above technical solutions, the embodiments of the present application can reduce the inter-domain differences, enhance the intra-domain diversity, and ultimately improve the performance of the model in unknown domains by enhancing and standardizing the constructed multi-domain image dataset.

[0011] Optionally, in one embodiment of the present application, the introduction of a domain adversarial module to utilize adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements includes: utilizing the discriminator of the domain adversarial module to minimize classification loss and the feature extractor of the domain adversarial module to maximize the discriminator classification loss as the goal, so as to adopt an alternating optimization strategy to suppress style information.

[0012] Through the above technical solution, the embodiment of the present application can suppress image style information through adversarial training based on the domain adversarial module, making it difficult for the feature extraction network to distinguish different styles by the style discriminator, thereby improving the style invariance of the features; this alternating optimization adversarial training strategy can effectively improve the generalization ability of the model on cross-domain images, and can perform well on ultrasound images between different medical centers.

[0013] Optionally, in one embodiment of the present application, the design is a semantic consistency alignment module based on singular value decomposition to perform low-rank decomposition and feature alignment processing on the local model features and global model features corresponding to the local model and the global model, including: expanding the feature tensor to express it in matrix form; performing singular value decomposition on the local feature matrix and the global feature matrix respectively, extracting the corresponding main singular vectors and performing spatial alignment operations; using the similarity measure of the aligned features as a loss function, guiding the local features to align to the global semantic space while maintaining the semantic information of the local domain.

[0014] Through the above technical solution, the embodiment of the present application can perform singular value decomposition on the local feature matrix and the global feature matrix, extract their main singular vectors and perform spatial alignment operations; use the similarity measurement of the aligned features as the loss function to guide the local features to align to the global semantic space while maintaining the semantic information of the domain, thereby alleviating the semantic offset problem caused by differences in labeling methods and anatomical structures between domains, reducing the semantic differences between different center data, and further improving the classification effect.

[0015] Optionally, in one embodiment of the present application, after adopting the preset federated averaging strategy to perform global model aggregation, the method further includes: sending the generated global model to the target end.

[0016] Through the above technical solution, the embodiment of the present application can integrate local training, domain adversarial training, semantic consistency alignment and global aggregation process into an end-to-end training process based on the training process formed by the above steps. Finally, a federated learning medical image classification method with both privacy protection and cross-domain generalization capabilities is obtained, thereby realizing efficient and accurate cross-domain fetal ultrasound image classification.

[0017] The second aspect of the present application provides a domain adversarial federated learning device for cross-style multi-center ultrasound image classification, including: a modeling module for collecting fetal ultrasound image data from multiple centers to establish a multi-domain image dataset, and based on a preset standard federated learning framework and the multi-domain image dataset, constructing a local model and a global model, designing a basic feature extraction network, and initializing the local model; a training module for introducing a domain adversarial module during the local training process, so as to use adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements; a federated learning module for designing a semantic consistency alignment module based on singular value decomposition, so as to perform low-rank decomposition and feature alignment on the local model features and global model features corresponding to the local model and the global model, and after completing each round of local training and feature alignment, adopting a preset federal averaging strategy to perform global model aggregation, and enter the next round of training until a federated learning medical image classification model with both privacy protection and cross-domain generalization capabilities is generated.

[0018] Through the above technical solution, the embodiment of the present application can deploy a local model based on the federated learning framework, combine the domain adversarial module (DoA) to suppress image style differences, and enable the feature extraction network to learn domain invariant features through adversarial training; and further design a semantic consistency alignment module (SVD-Semantic Alignment, SVD-SA) based on singular value decomposition (SVD) to perform low-rank decomposition of local and global features to eliminate inter-domain semantic offsets; by introducing a dual optimization mechanism of domain adversarial training and semantic consistency alignment, the problem of model performance degradation caused by image style differences in traditional methods in multi-center scenarios is solved. Compared with centralized deep learning, the embodiment of the present application can avoid sharing of original data through a federated architecture and effectively protect patient privacy; compared with existing federated learning methods, the DoA module can suppress interference from non-independent and identically distributed data, and the SVD-SA module can be combined to enhance cross-domain semantic consistency, achieving SOTA performance on a four-center fetal ultrasound dataset.

[0019] Optionally, in one embodiment of the present application, it further includes: a preprocessing module, which is used to perform image enhancement and standardization processing on the data of the multi-domain image dataset after establishing the multi-domain image dataset to obtain processed data; and a elimination module, which is used to eliminate image samples with artifacts or quality problems in the processed data to obtain the final multi-domain image dataset.

[0020] Through the above technical solutions, the embodiments of the present application can reduce the inter-domain differences, enhance the intra-domain diversity, and ultimately improve the performance of the model in unknown domains by enhancing and standardizing the constructed multi-domain image dataset.

[0021] Optionally, in one embodiment of the present application, the training module includes: a suppression unit, configured to utilize the discriminator of the domain adversarial module to minimize classification loss and the feature extractor of the domain adversarial module to maximize the discriminator classification loss, so as to adopt an alternating optimization strategy to suppress style information.

[0022] Through the above technical solution, the embodiment of the present application can suppress image style information through adversarial training based on the domain adversarial module, making it difficult for the feature extraction network to distinguish different styles by the style discriminator, thereby improving the style invariance of the features; this alternating optimization adversarial training strategy can effectively improve the generalization ability of the model on cross-domain images, and can perform well on ultrasound images between different medical centers.

[0023] Optionally, in one embodiment of the present application, the federated learning module includes: an expansion unit for expanding the feature tensor to express it in matrix form; an alignment unit for performing singular value decomposition on the local feature matrix and the global feature matrix respectively, extracting the corresponding main singular vectors and performing spatial alignment operations; a guiding unit for guiding the local features to align to the global semantic space while maintaining the semantic information of the local domain based on the similarity measure of the aligned features as a loss function.

[0024] Through the above technical solution, the embodiment of the present application can perform singular value decomposition on the local feature matrix and the global feature matrix, extract their main singular vectors and perform spatial alignment operations; use the similarity measurement of the aligned features as the loss function to guide the local features to align to the global semantic space while maintaining the semantic information of the domain, thereby alleviating the semantic offset problem caused by differences in labeling methods and anatomical structures between domains, reducing the semantic differences between different center data, and further improving the classification effect.

[0025] Optionally, in one embodiment of the present application, it further includes: a sending module, which is used to send the generated global model to the target end after the global model aggregation is performed using the preset federal averaging strategy.

[0026] Through the above technical solution, the embodiment of the present application can integrate local training, domain adversarial training, semantic consistency alignment and global aggregation process into an end-to-end training process based on the training process formed by the above steps. Finally, a federated learning medical image classification method with both privacy protection and cross-domain generalization capabilities is obtained, thereby realizing efficient and accurate cross-domain fetal ultrasound image classification.

[0027] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the domain adversarial federated learning method for cross-style multi-center ultrasound image classification as described in the above embodiment.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned domain adversarial federated learning method for cross-style multi-center ultrasound image classification.

[0029] The fifth aspect of the present application provides a computer program product, which stores a computer program that, when executed by a processor, implements the above-mentioned domain adversarial federated learning method for cross-style multi-center ultrasound image classification.

[0030] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0032] Figure 1 A flowchart of a domain adversarial federated learning method for cross-style multi-center ultrasound image classification provided according to an embodiment of the present application;

[0033] Figure 2 This is a flowchart of cross-domain fetal ultrasound image classification based on federated learning according to a specific embodiment of the present application;

[0034] Figure 3 A schematic diagram of a process for performing feature optimization by combining domain confrontation and semantic consistency alignment according to a specific embodiment of the present application;

[0035] Figure 4 Schematic diagram of a domain adversarial federated learning device for cross-style multi-center ultrasound image classification according to an embodiment of the present application;

[0036] Figure 5 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0038] The following describes the domain adversarial federated learning method for cross-style multi-center ultrasound image classification of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the ultrasound image classification method based on deep learning in the related technology mentioned in the background technology center cannot guarantee data security; and cannot effectively adapt to the non-independent synchronous image data of cross-style multi-center, which makes it difficult for the model to maintain stable performance in actual deployment, the present application provides a domain adversarial federated learning method for cross-style multi-center ultrasound image classification, in which fetal ultrasound image data can be collected from multiple medical centers, a cross-domain dataset can be constructed and standardized preprocessing can be performed; based on the federated learning framework, a local model is deployed and initialized, and the image style difference is suppressed by combining the domain adversarial module, and the feature extraction network learns domain invariant features through adversarial training; further design a semantic consistency alignment module based on singular value decomposition, and perform low-rank decomposition on local and global features to eliminate semantic offset between domains; adopt a federated averaging strategy to aggregate global model weights, determine the training process based on the cross-domain dataset, network and objective function, complete the training, and form a classification model with both privacy protection and cross-domain generalization capabilities after iterative optimization. This solves the problems in related technologies such as the inability of deep learning-based ultrasound image classification methods to guarantee data security and the inability to effectively adapt to non-independent synchronized image data across multiple styles and centers, making it difficult for the model to maintain stable performance in actual deployment.

[0039] Specifically, Figure 1 A flowchart of a domain adversarial federated learning method for cross-style multi-center ultrasound image classification provided in an embodiment of the present application.

[0040] like Figure 1 As shown in FIG, the domain adversarial federated learning method for cross-style multi-center ultrasound image classification includes the following steps:

[0041] In step S101, fetal ultrasound image data from multiple centers are collected to establish a multi-domain image dataset. Based on a preset standard federated learning framework and the multi-domain image dataset, a local model and a global model are constructed, a basic feature extraction network is designed, and the local model is initialized.

[0042] Federated learning is a distributed machine learning framework that enables multiple participants (such as devices and institutions) to collaboratively train models on local data without directly sharing the original data. Data remains locally, and collaboration is achieved only through encrypted model parameters (or gradients), enabling privacy-preserving learning where "the data remains static, but the model moves."

[0043] In the embodiments of this application, the local model can be understood as a model that each client participating in federated learning independently trains on its own private data. The model does not share the original data and is trained only based on the local dataset to generate model parameters (such as the weights of the neural network). The global model can be understood as an aggregate model maintained by the server. It integrates the knowledge of all clients and can integrate the model updates of each client through the federated aggregation algorithm to form a more general model.

[0044] In actual implementation, the embodiments of the present application can use a standard federated learning framework based on the multi-domain dataset constructed in the above steps to deploy a local model on each data holder. The model uses a unified lightweight backbone network and is initialized through pre-training. Each local model is trained using its own dataset to improve the model's adaptability to local data. To enhance cross-domain modeling capabilities, a federated averaging (FedAvg) strategy is used to aggregate the weights of the global model.

[0045] The embodiment of the present application can use a federal averaging strategy to ensure that the weight update after each local training does not expose the specific data content, thereby realizing local model training under privacy protection and realizing cross-center modeling capabilities under privacy protection.

[0046] In step S102, during the local training process, a domain adversarial module is introduced to utilize adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements.

[0047] The domain adversarial module can be understood as a neural network component used to reduce differences between different data distributions (domains), while adversarial training can be understood as a training method that enhances model robustness by introducing adversarial examples. Its core idea is to enable the model to actively learn to resist carefully constructed input perturbations during training, thereby improving its resistance to noise, attacks, or domain shift.

[0048] Specifically, the embodiment of the present application can introduce a domain adversarial module in the local training process. The module is composed of an image style discriminator and a feature extraction network to form an adversarial structure. Among them, the style discriminator is intended to identify the domain (i.e., different centers) from which the image comes, and the feature extraction network continuously learns to make its output features difficult for the discriminator to accurately identify the image style, thereby achieving the suppression of style information. The specific training method adopts an alternating optimization strategy. The discriminator aims to minimize the classification loss and enhance the style recognition ability; the feature extractor aims to maximize the discriminator classification loss and weaken the style differences between images. This strategy effectively improves the style invariance of the extracted features and enhances the generalization performance of the model on images with different centers.

[0049] Through the above technical solution, the embodiment of the present application can introduce a domain adversarial module to perform adversarial training between the style discriminator and the feature extraction network, so that the feature extraction network can learn more "domain-invariant" features, thereby suppressing the differences between multi-center multi-style images, successfully solving the performance degradation problem caused by inter-domain distribution differences in multi-center fetal ultrasound images, and significantly improving the generalization ability and robustness of the model between different data centers.

[0050] In step S103, a semantic consistency alignment module based on singular value decomposition is designed to perform low-rank decomposition and feature alignment on the local model features and global model features corresponding to the local model and the global model. After completing each round of local training and feature alignment, a preset federal averaging strategy is used to aggregate the global model and enter the next round of training until a federated learning medical image classification model with both privacy protection and cross-domain generalization capabilities is generated.

[0051] Singular value decomposition can be understood as an important matrix decomposition method in linear algebra, which can be applied to signal processing, statistics, machine learning and other fields.

[0052] Specifically, the embodiments of the present application can perform singular value decomposition based on the feature maps extracted from the local and global models, extract the main semantic components of the features, and construct a semantic space representation with singular values ​​and their corresponding singular vectors. By measuring the cosine similarity between the local and global semantic components, a semantic consistency loss is formed, and used as one of the objective functions of model training to achieve feature alignment. After completing each round of local training and feature alignment, a federal averaging strategy is used to aggregate the global model and enter the next round of training.

[0053] The embodiment of the present application can perform singular value decomposition on local and global feature maps through the above-mentioned semantic consistency alignment module, extract their semantic principal components, and align them, thereby alleviating the problem of deep learning performance degradation caused by semantic offset of different center images. It can also improve the generalization performance of deep learning models in multi-center data environments; based on the federated learning framework, by combining local model training with global model aggregation, it can effectively protect data privacy while improving the accuracy of cross-center image classification and avoiding the privacy leakage risks brought about by data centralization in traditional methods.

[0054] Optionally, in one embodiment of the present application, after establishing the multi-domain image dataset, the process further includes: performing image enhancement processing and standardization processing on the data of the multi-domain image dataset to obtain processed data; and eliminating image samples with artifacts or quality problems in the processed data to obtain the final multi-domain image dataset.

[0055] During the actual implementation process, the embodiment of the present application may collect fetal ultrasound image data from different medical centers, construct a multi-domain image dataset, and then perform data preprocessing on the constructed multi-domain image dataset; the preprocessing process may include: unifying the image resolution (224×224), random cropping, random horizontal flipping, center cropping, scaling and other data enhancement operations, and normalizing the image (for example, using the ImageNet mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] for normalization), while eliminating images containing obvious artifacts or quality problems.

[0056] The embodiments of the present application can enhance and standardize the constructed multi-domain image dataset, thereby reducing the differences between domains, enhancing the diversity within the domain, and further improving the performance of the model in unknown domains.

[0057] Optionally, in one embodiment of the present application, a domain adversarial module is introduced to utilize adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements, including: utilizing the discriminator of the domain adversarial module to minimize classification loss and the feature extractor of the domain adversarial module to maximize the discriminator classification loss, so as to adopt an alternating optimization strategy to suppress style information.

[0058] The relevant explanations about the domain adversarial module and adversarial training have been explained in detail above and will not be repeated here.

[0059] During the actual implementation process, the embodiment of the present application may introduce a domain adversarial module in the local training stage. This module suppresses the image style information through adversarial training, so that the feature extraction network learns more domain-invariant features. Specifically, the module includes two main parts: a style discriminator and a feature extraction network. The task of the style discriminator is to determine the specific style of the input image (such as single view / dual view, color image / grayscale image, etc.), while the feature extraction network uses adversarial training to make the image style difficult for the style discriminator to accurately identify, thereby suppressing the style differences between different central images. This training process adopts an alternating optimization strategy, in which the goal of the style discriminator is to minimize the classification loss, while the feature extraction network enhances domain invariance by maximizing the classification loss; the loss function of this part includes label classification loss (CELoss) and domain classification loss (CE Loss).

[0060] During the local training process, the embodiment of the present application can suppress image style information through adversarial training based on the domain adversarial module, making it difficult for the feature extraction network to distinguish different styles by the style discriminator, thereby improving the style invariance of the features; this alternating optimization adversarial training strategy can effectively improve the generalization ability of the model on cross-domain images, and can perform well on ultrasound images between different medical centers.

[0061] Optionally, in one embodiment of the present application, a semantic consistency alignment module based on singular value decomposition is designed to perform low-rank decomposition and feature alignment processing on the local model features and global model features corresponding to the local model and the global model, including: expanding the feature tensor to represent it in matrix form; performing singular value decomposition on the local feature matrix and the global feature matrix respectively, extracting the corresponding main singular vectors and performing spatial alignment operations; using the similarity measure of the aligned features as the loss function, guiding the local features to align to the global semantic space while maintaining the semantic information of the local domain.

[0062] Spatial alignment, as it's understood, refers to the process of making different data (such as images, point clouds, and feature maps) consistent across spatial dimensions through geometric transformations or feature adjustments. Its core goal is to eliminate spatial inconsistencies caused by factors like perspective, displacement, scale, or deformation, thereby improving the accuracy of subsequent tasks (such as matching, fusion, and classification).

[0063] To further alleviate the problem of cross-domain semantic drift, a semantic consistency alignment module based on singular value decomposition is introduced. This module extracts the semantic principal components of local and global feature maps by performing singular value decomposition and aligning them. Specifically, the local feature tensor and the global feature tensor are expanded and low-rank decomposed, and the first k singular values ​​and their left and right singular vectors are extracted to construct a semantic space representation. The distance between the semantic features and the residual features is then measured using cosine similarity, forming a semantic consistency loss as part of the training objective.

[0064] In the embodiment of the present application, the local feature matrix and the global feature matrix can be subjected to singular value decomposition to extract their main singular vectors and perform spatial alignment operations; by using the similarity measure of the aligned features as a loss function, the local features are guided to align to the global semantic space while maintaining the semantic information of the domain, thereby alleviating the semantic offset problem caused by differences in annotation methods and anatomical structures between domains, reducing the semantic differences between different center data, and further improving the classification effect.

[0065] like Figure 2 As shown, the embodiment of the present application can suppress image style differences by introducing a domain adversarial module (DoA), and enable the feature extraction network to learn domain invariant features through adversarial training; further design a semantic consistency alignment module (SVD-SA) based on singular value decomposition, and perform low-rank decomposition on local and global features to eliminate inter-domain semantic offsets; so that the model after the final optimized iterative training has high privacy protection capabilities and cross-domain adaptability, can be deployed in multiple medical centers, adapt to ultrasound image data from different centers, and improve the performance of cross-domain medical image classification tasks while ensuring data security, and has strong practical application value and promotion potential; can be widely used in cross-institutional medical image analysis, and provide efficient, safe, and highly generalized intelligent diagnostic support for prenatal screening.

[0066] Optionally, in one embodiment of the present application, after adopting a preset federated averaging strategy to perform global model aggregation, the method further includes: sending the generated global model to a target end.

[0067] In some embodiments, a complete training objective function can be defined based on the introduction of a domain adversarial module to train optimized local feature representations and network parameters based on a semantic consistency alignment module based on singular value decomposition. The total loss function consists of three parts: label classification loss, domain classification loss, and semantic consistency loss (based on the cosine similarity between semantic and style components after singular value decomposition). After each round of training is completed, the FedAvg strategy in federated learning is used to aggregate the weights of each local model. Specifically, the global model weight is generated by calculating the weighted average of each local model parameter. Figure 3As shown in the figure, the global model weights will be distributed to each data holder for use in the next round of local training.

[0068] Based on the training process formed by the above steps, the embodiment of the present application can integrate local training, domain adversarial, semantic consistency alignment and global aggregation process into an end-to-end training process. Ultimately, a federated learning medical image classification method with both privacy protection and cross-domain generalization capabilities is obtained, thereby achieving efficient and accurate cross-domain fetal ultrasound image classification.

[0069] According to the domain adversarial federated learning method for cross-style multi-center ultrasound image classification proposed in the embodiment of the present application, fetal ultrasound image data can be collected from multiple medical centers, a cross-domain dataset can be constructed and standardized preprocessing can be performed; local models can be deployed and initialized based on the federated learning framework, and the image style differences can be suppressed by combining the domain adversarial module, and the feature extraction network can learn domain invariant features through adversarial training; a semantic consistency alignment module based on singular value decomposition is further designed to perform low-rank decomposition on local and global features to eliminate inter-domain semantic offsets; a federated averaging strategy is used to aggregate global model weights, and based on the cross-domain dataset, network and objective function, the training process is determined, the training is completed, and after iterative optimization, a classification model with both privacy protection and cross-domain generalization capabilities is formed. In this way, the problems of ultrasound image classification methods based on deep learning in related technologies cannot guarantee data security; and cannot effectively adapt to non-independent synchronized image data across multiple styles, making it difficult for the model to maintain stable performance in actual deployment are solved.

[0070] Next, refer to the attached Figure 4 The present invention describes a domain adversarial federated learning device for cross-style multi-center ultrasound image classification proposed in an embodiment of the present application.

[0071] Figure 4 It is a block diagram of a domain adversarial federated learning device for cross-style multi-center ultrasound image classification according to an embodiment of the present application.

[0072] like Figure 4 As shown, the domain adversarial federated learning device 10 for cross-style multi-center ultrasound image classification includes: a modeling module 100, a training module 200, and a federated learning module 300.

[0073] The modeling module 100 is used to collect fetal ultrasound image data from multiple centers to establish a multi-domain image dataset, and based on a preset standard federated learning framework and a multi-domain image dataset, build a local model and a global model, design a basic feature extraction network, and initialize the local model.

[0074] The training module 200 is used to introduce a domain adversarial module during the local training process, so as to utilize adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements.

[0075] The federated learning module 300 is used to design a semantic consistency alignment module based on singular value decomposition to perform low-rank decomposition and feature alignment on the local model features and global model features corresponding to the local model and the global model. After completing each round of local training and feature alignment, a preset federal averaging strategy is used to aggregate the global model and enter the next round of training until a federated learning medical image classification model with both privacy protection and cross-domain generalization capabilities is generated.

[0076] Optionally, in one embodiment of the present application, the domain adversarial federated learning device 10 for cross-style multi-center ultrasound image classification also includes: a preprocessing module and a elimination module; the preprocessing module is used to perform image enhancement and standardization processing on the data of the multi-domain image dataset after establishing the multi-domain image dataset to obtain processed data; the elimination module is used to eliminate image samples with artifacts or quality problems in the processed data to obtain the final multi-domain image dataset.

[0077] Optionally, in one embodiment of the present application, the training module 200 includes: a suppression unit, which is used to utilize the discriminator of the domain adversarial module to minimize the classification loss and the feature extractor of the domain adversarial module to maximize the discriminator classification loss, so as to adopt an alternating optimization strategy to suppress style information.

[0078] Optionally, in one embodiment of the present application, the federated learning module 300 includes: an expansion unit, an alignment unit, and a guidance unit; wherein the expansion unit is used to expand the feature tensor to express it in matrix form; the alignment unit is used to perform singular value decomposition on the local feature matrix and the global feature matrix respectively, extract the corresponding main singular vectors and perform spatial alignment operations; the guidance unit is used to guide the local features to align to the global semantic space while maintaining the semantic information of the local domain based on the similarity measure of the aligned features as a loss function.

[0079] Optionally, in one embodiment of the present application, the domain adversarial federated learning device 10 for cross-style multi-center ultrasound image classification also includes: a sending module, which is used to send the generated global model to the target end after the global model is aggregated using a preset federal averaging strategy.

[0080] It should be noted that the above explanation of the embodiment of the domain adversarial federated learning method for cross-style multi-center ultrasound image classification is also applicable to the domain adversarial federated learning device for cross-style multi-center ultrasound image classification in this embodiment, and will not be repeated here.

[0081] According to the domain adversarial federated learning device for cross-style multi-center ultrasound image classification proposed in the embodiment of the present application, fetal ultrasound image data can be collected from multiple medical centers, a cross-domain dataset can be constructed and standardized preprocessing can be performed; local models can be deployed and initialized based on the federated learning framework, and the image style differences can be suppressed by combining the domain adversarial module, and the feature extraction network can learn domain invariant features through adversarial training; a semantic consistency alignment module based on singular value decomposition is further designed to perform low-rank decomposition on local and global features to eliminate semantic shifts between domains; a federated averaging strategy is used to aggregate global model weights, and based on the cross-domain dataset, network and objective function, the training process is determined, the training is completed, and after iterative optimization, a classification model with both privacy protection and cross-domain generalization capabilities is formed. In this way, the problems of ultrasound image classification methods based on deep learning in related technologies cannot guarantee data security; and cannot effectively adapt to non-independent synchronized image data across multiple styles, making it difficult for the model to maintain stable performance in actual deployment are solved.

[0082] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0083] Memory 501 , processor 502 , and computer programs stored in the memory 501 and executable on the processor 502 .

[0084] When the processor 502 executes the program, the domain adversarial federated learning method for cross-style multi-center ultrasound image classification provided in the above embodiment is implemented.

[0085] Furthermore, the electronic device further includes:

[0086] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0087] The memory 501 is used to store computer programs that can be run on the processor 502 .

[0088] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0089] If the memory 501, the processor 502, and the communication interface 503 are implemented independently, the communication interface 503, the memory 501, and the processor 502 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0091] The processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0092] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned domain adversarial federated learning method for cross-style multi-center ultrasound image classification.

[0093] An embodiment of the present application also provides a computer program product on which a computer program is stored. When the program is executed by a processor, the above-mentioned domain adversarial federated learning method for cross-style multi-center ultrasound image classification is implemented.

[0094] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0095] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0096] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0097] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0098] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0099] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0100] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0101] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A domain adversarial federated learning method for cross-style multi-center ultrasound image classification, characterized by: The following steps are involved: Collecting fetal ultrasound image data from multiple centers to establish a multi-domain image dataset, and building a local model and a global model based on a preset standard federated learning framework and the multi-domain image dataset, designing a basic feature extraction network, and initializing the local model; During the local training process, a domain adversarial module is introduced to utilize adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements; A semantic consistency alignment module based on singular value decomposition is designed to perform low-rank decomposition and feature alignment on the local model features and global model features corresponding to the local model and the global model. After completing each round of local training and feature alignment, a preset federal averaging strategy is used to aggregate the global model and enter the next round of training until a federated learning medical image classification model with both privacy protection and cross-domain generalization capabilities is generated.

2. The method according to claim 1, characterized in that After establishing the multi-domain image dataset, the method further includes: performing image enhancement processing and standardization processing on the data of the multi-domain image dataset to obtain processed data; Image samples with artifacts or quality problems are removed from the processed data to obtain a final multi-domain image dataset.

3. The method according to claim 1, characterized in that The domain adversarial module is introduced to utilize adversarial training to suppress image style information so that the basic feature extraction network meets preset requirements, including: The discriminator of the domain adversarial module is used to minimize the classification loss, and the feature extractor of the domain adversarial module is used to maximize the discriminator classification loss, so as to adopt an alternating optimization strategy to suppress style information.

4. The method according to claim 1, wherein The semantic consistency alignment module based on singular value decomposition is designed to perform low-rank decomposition and feature alignment processing on the local model features and the global model features corresponding to the local model and the global model, including: Expand the feature tensor to express it in matrix form; Performing singular value decomposition on the local feature matrix and the global feature matrix respectively, extracting corresponding main singular vectors and performing spatial alignment operations; The similarity measure of the aligned features is used as the loss function to guide local features to align to the global semantic space while maintaining the semantic information of the local domain.

5. The method according to claim 1, wherein After adopting the preset federated averaging strategy to perform global model aggregation, the method further includes: Send the generated global model to the target end.

6. A domain adversarial federated learning device for cross-style multi-center ultrasound image classification, characterized by: include: a modeling module for collecting fetal ultrasound image data from multiple centers to establish a multi-domain image dataset, constructing a local model and a global model based on a pre-set standard federated learning framework and the multi-domain image dataset, designing a basic feature extraction network, and initializing the local model; A training module is used to introduce a domain adversarial module during the local training process, so as to suppress image style information by adversarial training so that the basic feature extraction network meets preset requirements; The federated learning module is used to design a semantic consistency alignment module based on singular value decomposition to perform low-rank decomposition and feature alignment on the local model features and global model features corresponding to the local model and the global model. After completing each round of local training and feature alignment, a preset federated averaging strategy is used to aggregate the global model and enter the next round of training until a federated learning medical image classification model with both privacy protection and cross-domain generalization capabilities is generated.

7. The device according to claim 6, characterized in that Also includes: a preprocessing module, configured to perform image enhancement and standardization processing on the data of the multi-domain image dataset after establishing the multi-domain image dataset to obtain processed data; The elimination module is used to eliminate image samples with artifacts or quality problems in the processed data to obtain a final multi-domain image dataset.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the domain adversarial federated learning method for cross-style multi-center ultrasound image classification as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the domain adversarial federated learning method for cross-style multi-center ultrasound image classification as described in any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that The computer program is executed to implement the domain adversarial federated learning method for cross-style multi-center ultrasound image classification according to any one of claims 1 to 5.

Images