Frequency domain adaptive CT pulmonary nodule image segmentation method based on federated learning

Abstract

The present invention relates to a frequency domain adaptive CT lung nodule image segmentation method based on federated learning, which belongs to the field of image processing technology. The method comprises the following steps: S1: performing data preparation, data labeling, data processing, data enhancement and data set division on CT lung nodule image data; S2: constructing a frequency domain adaptive CT lung nodule image segmentation model FASNet based on federated learning; S3: training, verifying and testing the frequency domain adaptive CT lung nodule image segmentation model FASNet based on federated learning; and S4: performing CT lung nodule image segmentation using the trained frequency domain adaptive CT lung nodule image segmentation model FASNet based on federated learning.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and relates to a frequency domain adaptive CT lung nodule image segmentation method based on federated learning. Background Art

[0002] Computed tomography (CT) is the primary tool for detecting pulmonary nodules. However, due to their diverse morphology and fuzzy boundaries, manual segmentation of pulmonary nodules in CT images is time-consuming and error-prone. In recent years, deep learning, particularly convolutional neural networks (CNNs), has rapidly developed, enabling automatic image feature extraction and high-precision segmentation. Automatic pulmonary nodule segmentation methods based on deep learning have become a hot topic of research. Due to the closed nature of medical data, using only medical data from a single clinical site is insufficient to build accurate and robust deep learning models for critical medical applications. Federated learning (FL) offers a solution to this problem by enabling distributed data entities to collaboratively train a shared global model under privacy-preserving constraints. In the FL paradigm, each client stores its own private data to train a local model, and only the model parameters, not the raw data, are transmitted between the local client and the central server. However, the data features across clients are not independent and identically distributed, making global models difficult to train.

[0003] Several methods currently utilize deep learning for medical image segmentation. Zhu proposed DeepLung, a deep learning-based automatic pulmonary nodule detection and classification system. This system utilizes a deep 3D dual-pathway network architecture, combining the deep feature extraction capabilities of ResNet with an additional path specifically designed for pulmonary nodules, improving detection sensitivity and accuracy. However, DeepLung was only used on a single public dataset, the LIDC-IDRI, and lacks good transferability and generalization to real-world hospital CT image data. Ananya proposed a federated learning-based pulmonary nodule segmentation method that combines an attention mechanism, a residual network, and a U-Net architecture. This method is trained in a federated learning setting and optimized using the FedProx optimizer. Security is enhanced through a Transport Layer Security (TLS) link, alleviating data privacy concerns. However, this method operates only in the spatial domain and does not consider the details and overall structural information of the image represented in the frequency space. Wang designed a novel federated semi-supervised learning (FSSL) framework with dynamic model aggregation to improve pulmonary nodule segmentation performance. This method proposes a dynamically updated algorithm to handle model parameter aggregation in FSSL, which fully utilizes the quality and quantity of client data. Its results are better than traditional deep learning methods and recent federated semi-supervised learning methods. However, due to the complexity of dynamically updating the model aggregation, it introduces a huge computational overhead. Zhang designed a dual-subnetwork structure consisting of a multi-information flow subnetwork and a classification-assisted correction subnetwork, which respectively provide lung nodule image feature information and simulated lung nodule malignancy and visual features. Nan used the objective function to establish the pixel membership to the cluster center as the optimization goal, and proposed an improved superpixel clustering algorithm SLIC+ based on the original SLIC algorithm. This method improves the feasibility of the superpixel clustering algorithm for practical use in detecting lung nodule images, but the performance needs to be further improved in practical applications. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a frequency domain adaptive CT lung nodule image segmentation method based on federated learning, and a frequency domain adaptive segmentation network FASNet (Frequency Domain Adaptive Segmentation Network) is designed.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A frequency-domain adaptive CT lung nodule image segmentation method based on federated learning includes the following steps:

[0007] S1: Data preparation, data annotation, data processing, data enhancement and data set division for CT lung nodule image data;

[0008] S2: Construct a frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning;

[0009] S3: training, validating, and testing the frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning;

[0010] S4: Use the trained frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning to perform CT lung nodule image segmentation.

[0011] Furthermore, the data preparation, data labeling, data processing, data enhancement and data set division of the CT lung nodule image data in step S1 include the following steps:

[0012] S11: Data preparation: Ensure that all CT image data are in a unified format, check data quality, remove incomplete, damaged, or non-compliant images, and ensure data integrity and validity;

[0013] S12: Data annotation: Use medical image annotation tools to annotate CT lung nodule images;

[0014] S13: Data processing: All CT images were resized to the same size or resolution, interpolated or cropped to a fixed size as network input, and the data were individually normalized to zero mean and unit variance in intensity values;

[0015] S14: Data Augmentation: Apply data augmentation techniques to increase data diversity, including rotation, translation, scaling, and flipping;

[0016] S15: Dataset partitioning: Divide the data into training set, validation set and test set.

[0017] Furthermore, the frequency-domain adaptive CT pulmonary nodule image segmentation model FASNet based on federated learning includes an image data reconstruction process and a segmentation network; the image data reconstruction process includes an adaptive mask generator and a spatial feature extraction module; the computational process of the frequency-domain adaptive CT pulmonary nodule image segmentation model FASNet based on federated learning is as follows:

[0018] S21: Input the CT lung nodule image into the FASNet model, roughly extract features through a convolution structure, and use Fourier transform to convert the input image from the spatial domain to the frequency domain. Then, a shift operation is performed on the Fourier transform result to move the zero-frequency component to the center.

[0019] S22: generating a corresponding adaptive mask by an adaptive mask generator to separate high-frequency information and low-frequency information in the frequency domain;

[0020] S23: performing inverse shift and inverse Fourier transform on the high-frequency information and the low-frequency information in the frequency domain, and converting them back to the spatial domain;

[0021] S24: using a spatial feature extraction module to extract features of the high-frequency information and the low-frequency information in the spatial domain respectively, adding the high-frequency and low-frequency information bit by bit, and generating reconstructed image data with independent and identical distribution;

[0022] S25: Input the reconstructed image data into the segmentation network and output the segmentation result.

[0023] Furthermore, the adaptive mask generator is used to generate an image mask adapted to the high-frequency information and low-frequency information of the image, thereby separating the high-frequency and low-frequency components. The process of generating the adaptive mask is as follows:

[0024] For input I, it first passes through a convolution layer with a convolution kernel size of 3 to map the input three-channel image to a higher-dimensional space. For an input with a shape size of h×w×3, it is mapped to h×w×c, and the obtained features are recorded as P; then, through Fourier transform and shift, the spatial domain features are converted to the frequency domain. The obtained features are recorded as F, which can be expressed as:

[0025] P = conv 3×3 (I)

[0026] F=s(f(P))

[0027] Among them, conv() represents the conventional convolution operation, the default convolution kernel size is 1, conv 3×3 () represents the convolution operation with a convolution kernel size of 3; f() represents the Fourier transform, and s() represents the shift operation;

[0028] Calculate the mask, the height h of the mask area - and width w - The image height h and width w are divided by a constant n, and then multiplied by the corresponding threshold value to obtain h - and w - Then convert to an integer;

[0029] The threshold is calculated by first using adaptive average pooling on the feature P, performing global average pooling on each channel of the feature map, pooling each channel into a value, and outputting a size of 1×1×c; passing through a convolution layer with a convolution kernel size of 1, setting a compression rate r to compress the number of channels, and obtaining a feature map size of 1×1×c / r; then passing through an activation function GELU, mapping the features to a nonlinear space; passing through a convolution layer with a convolution kernel size of 1, further compressing the number of channels to 2, obtaining a feature map size of 1×1×2, and finally passing through a Sigmoid function to convert the values ​​of the two channels into values ​​between (0,1) and add them up to 1. The values ​​of the two channels are the learnable adaptive thresholds; the above process is expressed as follows:

[0030] T=gap(P)

[0031] T = conv(δ(conv(T)))

[0032] T=σ(T)

[0033] Among them, gap() represents the adaptive average pooling operation, δ represents the GELU activation function, σ represents the Sigmoid function, and T represents the threshold, whose size is 1×1×2;

[0034] After generating the threshold, calculate the height h of the mask area - and width w - , the formula is described as follows:

[0035]

[0036]

[0037] Where T[0] and T[1] represent the values ​​of the two channels of the threshold respectively, and int() represents the rounding operation;

[0038] Fill the mask matrix with values ​​for a matrix of size h - ×w - The mask matrix in the height direction is from (h / / 2-h) to (h / / 2+h - ), in the width direction from (w / / 2-w) to (w / / 2+w - ) is set to 1, and the remaining areas are filled with 0, to obtain the adaptive mask matrix M, as shown below:

[0039]

[0040] At this point, the adaptive mask generator generates a mask matrix with the center area as 1 and the edge as 0. The size of the mask matrix and the center area is generated by the adaptive threshold;

[0041] Generate a mask matrix to separate the high-frequency information and low-frequency information of the frequency domain information, specifically including: the frequency domain information F is first multiplied bit by bit with the mask matrix M, the low-frequency components in the central area are retained, and the high-frequency components outside the central area are removed, thereby separating the low-frequency information F l For the separation of high-frequency information, F is multiplied by the inverse of the mask matrix M, that is, multiplied by (1-M). The low-frequency components in the central area are removed, and the high-frequency components outside the central area are extracted, which is expressed as F h The separated high-frequency information and low-frequency information are then inverse shifted and inverse Fourier transformed to convert the separated frequency domain information back to the spatial domain, which is represented by S l , S h , the formula is as follows:

[0042] F l =F⊙M

[0043] F h =F⊙(1-M)

[0044] S i =f -1 (S -1 (F i ))

[0045] Among them, ⊙ represents the bit-by-bit multiplication operation, i∈{l,h}.

[0046] Furthermore, the spatial feature extraction module first passes through a simple convolution structure, the output is recorded as R, and R is used as the residual connection later; then passes through a depth convolution; then passes through a linear layer, an activation layer and a linear layer in sequence; the first linear layer increases the input dimension to four times the original as the input after passing through the GELU activation function, and the second linear layer restores the increased dimension to the original dimension; finally, a residual connection is added, and the result of the addition with R is input into a convolution layer, and then the corresponding low-frequency information X is output. l and high-frequency information X h ; The process is expressed mathematically as:

[0047] R i =conv(S i )

[0048]

[0049] X i =X i +R i

[0050] X i =conc(X i )

[0051] Among them, R represents the structure obtained after the first convolution, which is used for residual connection later. represents a depthwise convolution with a kernel size of 7, and linear represents a linear layer;

[0052] The low-frequency information S is extracted by the spatial feature extraction module l and high frequency information S h After further feature extraction, the low-frequency and high-frequency features X l and X h Add and output the final reconstructed image The reconstructed image is expressed as follows:

[0053]

[0054] Furthermore, through federated learning, the frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning is trained on the local client, and then the trained network weights are uploaded to the central server. The central server aggregates the weights of each client to update the model; the trained model of the central server is then deployed in an actual clinical environment to verify the model performance and continuously optimize it based on feedback.

[0055] Furthermore, in the initial training stage, the central server initializes a global model; after the local client training is completed, an updated set of model parameters is generated based on the training results of the local data set; the local client transmits the updated model parameters to the central server, and the central server uses the aggregated model parameters to update the global model. The updated global model will become the starting point for the next round of training, and the updated global model parameters will be distributed to the local clients; the local client trains, updates the model, and transmits parameters based on its own data set, and the central server aggregates and updates the global model, and this cycle iterates until the model training is completed.

[0056] The beneficial effects of the present invention are as follows: the present invention mines the frequency domain features of CT lung nodule images, designs an adaptive mask generator to decompose the image features into high-frequency and low-frequency information, and then further extracts the features of the high-frequency and low-frequency information in the spatial domain to reconstruct the input image and eliminate the differences in different data distributions.

[0057] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0059] Figure 1 This is a flowchart of model training based on local clients and central servers in federated learning;

[0060] Figure 2 This is the overall structure diagram of the network, as well as the structure diagram of the adaptive mask generator and spatial feature extraction module;

[0061] Figure 3 This is an example diagram of network test results. DETAILED DESCRIPTION

[0062] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0063] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0064] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0065] like Figure 1 As shown, the present invention provides a frequency domain adaptive CT lung nodule image segmentation method based on federated learning, which is specifically implemented in the following steps:

[0066] The first step is to prepare the dataset.

[0067] (1) Prepare data.

[0068] For CT lung nodule image data on the local client, ensure that all CT image data is in a unified format (such as DICOM). At the same time, check the data quality and remove incomplete, damaged, or DICOM-compliant images to ensure data integrity and validity.

[0069] (2) Data labeling.

[0070] Professional medical image annotation tools, such as ITK-SNAP and 3D Slicer, are used for annotation, and experienced radiologists annotate CT lung nodule images to ensure accuracy and consistency.

[0071] (3)Data processing.

[0072] For preprocessing, all CT images are adjusted to the same size or resolution, usually using interpolation techniques or cropping to a fixed size as network input, and the data are individually normalized to zero mean and unit variance in intensity values.

[0073] (4) Data enhancement.

[0074] Various data augmentation techniques are applied to increase the diversity of the data, such as rotation, translation, scaling, flipping, etc. In this way, the features learned by the model are more robust, which improves the generalization performance of the model in practical applications.

[0075] (5) Dataset division.

[0076] Split the dataset into training, validation, and test sets to evaluate model performance and avoid overfitting.

[0077] The second step is to train the frequency domain adaptive segmentation network FASNet.

[0078] (1) Overall network training process

[0079] like Figure 2The main network structure above is shown. In this figure, FFT stands for Fourier transform, IFFT stands for inverse Fourier transform, shift stands for shift operation, unshift stands for inverse shift operation, and Conv stands for convolution. A local client CT lung nodule image I is input into the frequency-domain adaptive segmentation network FASNet. First, a convolutional structure is used to roughly extract features. Then, a Fourier transform is used to convert the input image from the spatial domain to the frequency domain. The Fourier transform result is then shifted using a shift operation, moving the zero-frequency component to the center. Next, to obtain the high- and low-frequency components of the frequency-domain features, the shifted result is multiplied by the corresponding mask for frequency division. This mask is generated by the adaptive mask generator. The resulting high- and low-frequency components are then inversely shifted and inverse Fourier transformed, converting them back to the spatial domain. The spatial feature extraction module then extracts features from the high- and low-frequency information in the spatial domain. Finally, the input image is reconstructed through bitwise addition. The reconstructed image then serves as the input to the segmentation network, which outputs the segmentation result. Next, through federated learning, the network weights trained by the local client are uploaded to the central server, which aggregates the weights from each client to update the model. Ultimately, the trained model on the central server is deployed in a real-world clinical setting to verify its performance and continuously optimize based on feedback.

[0080] (2) Adaptive mask generator: This module is used to generate an image mask that is adapted to the high-frequency and low-frequency information of the image, and is used to separate the high-frequency and low-frequency components. The structure of the adaptive mask generator is as follows: Figure 2 As shown in the lower left corner, GAP represents the maximum pooling layer, Conv represents convolution, GELU represents the GELU activation function, and Sigmoid represents the Sigmoid activation function. The process of generating an adaptive mask is as follows:

[0081] For input I, it first passes through a convolution layer with a convolution kernel size of 3, mapping the input three-channel image to a higher-dimensional space. That is, for an input of shape size h×w×3, it is mapped to h×w×c, and the resulting features are recorded as P. Then, through Fourier transform and shift, the spatial domain features are converted to the frequency domain, and the resulting features are recorded as F. It can be expressed as:

[0082] P = conv 3×3 (I)

[0083] F=s(f(P))

[0084] Among them, conv() represents the conventional convolution operation, the default convolution kernel size is 1, conv 3×3 () represents a convolution operation with a kernel size of 3. F() represents Fourier transform, and s() represents a shift operation.

[0085] Next, calculate the mask, the height h of the mask area - and width w - The image height h and width w are divided by a constant n, and then multiplied by the corresponding threshold value to obtain h - and w - Then convert to an integer.

[0086] In the present invention, the constant n is 128. The threshold is calculated by first using adaptive average pooling on the feature P, and performing global average pooling on each channel of the feature map. Each channel is pooled into a value, and the output size is 1×1×c. Next, a compression rate r is set to compress the number of channels through a convolution layer with a convolution kernel size of 1. The present invention sets r to 8, and the resulting feature map size is 1×1×c / r. Then, after passing through an activation function GELU, the feature is mapped to a nonlinear space. After passing through a convolution layer with a convolution kernel size of 1, the number of channels is further compressed to 2, and the feature map size is 1×1×2. Finally, after passing through the Sigmoid function, the values ​​of the two channels are converted into values ​​between (0,1) and added to 1. The values ​​of the two channels are the learnable adaptive thresholds. The above process can be expressed by the formula:

[0087] T=gap(P)

[0088] T = conv(δ(conv(T)))

[0089] T=σ(T)

[0090] Among them, gap() represents the adaptive average pooling operation, δ represents the GELU activation function, σ represents the Sigmoid function, and T represents the threshold, whose size is 1×1×2.

[0091] After generating the threshold, the height h of the mask area can be calculated - and width w - , the formula is described as follows:

[0092]

[0093] Where T[0] and T[1] represent the values ​​of the two channels of the threshold respectively, and int() represents the rounding operation.

[0094] Next fill the values ​​of the mask matrix, for a size of h - ×w - The mask matrix in the height direction is from (h / / 2-h) to (h / / 2+h - ), in the width direction from (w / / 2-w) to (w / / 2+w -) is set to 1, and the remaining areas are filled with 0, and the adaptive mask matrix M is obtained, as shown below:

[0095]

[0096] At this point, the adaptive mask generator generates a mask matrix with the center area as 1 and the edge as 0. The size of the mask matrix and the center area is generated by the adaptive threshold.

[0097] Generate a mask matrix to separate the high-frequency information and low-frequency information of the frequency domain information. The method is to multiply the frequency domain information F by the mask matrix M bit by bit. The low-frequency components in the central area are retained, and the high-frequency components outside the central area are removed, thereby separating the low-frequency information F. l ; For the separation of high-frequency information, it is the inverse multiplication of F and the mask matrix M, that is, multiplication by (1-M). The low-frequency components in the central area are removed, and the high-frequency components outside the central area are extracted, which is expressed as F h The separated high-frequency information and low-frequency information are then inverse-shifted and inverse Fourier transformed to convert the separated frequency domain information back to the spatial domain, which is represented by S l , S h , the formula is as follows:

[0098] F l =F⊙M

[0099] F h =F⊙(1-M)

[0100] S i =f -1 (s -1 (F i ))

[0101] Among them, ⊙ represents the bit-by-bit multiplication operation, i∈{l,h}.

[0102] (3) Spatial feature extraction module: separated high-frequency and low-frequency features S i , further extract features through a spatial feature extraction module structure. The spatial feature extraction module structure is as follows Figure 2As shown in the lower right corner, in this figure, Conv represents convolution, DWConv represents depth convolution layer, Linear represents linear layer, and GELU represents GELU activation function. First, through a simple convolution structure, the output is recorded as R, and R is used as the residual connection later. Then pass through a depth convolution. Compared with the standard convolution, the computational complexity of the depth convolution is significantly reduced while maintaining the effect of feature extraction. Then pass through a linear layer, an activation layer, and a linear layer in sequence. The first linear layer increases the dimension of the input to four times the original as the output, so that a higher-dimensional space representation of the feature can be obtained. After the GELU activation function, the second linear layer restores the increased maintenance to the original dimension. Finally, add a residual connection, and the result of adding R is input into a convolution layer, and then the corresponding low-frequency information X is output. l and high-frequency information X h The process can be expressed mathematically as:

[0103] R i =conv(S i )

[0104]

[0105] X i =X i +R i

[0106] X i =conv(X i )

[0107] Among them, R represents the structure obtained after the first convolution, which is used for residual connection later. represents a depthwise convolution with a kernel size of 7, and linear represents a linear layer.

[0108] The low-frequency information S is extracted by the spatial feature extraction module l and high frequency information S h After further feature extraction, the low-frequency and high-frequency features X l and X h Add and output the final reconstructed image The reconstructed image eliminates the influence of specific domains and has the property of independent and identical distribution. It can be used to directly input the segmentation network to predict the segmentation results. The segmentation network can use a network structure such as UNet, Transformer, Mamba and other common network structures. The reconstructed image can be expressed by the following formula:

[0109]

[0110] Finally, the weights of the entire local network training are uploaded to the central server. The central server aggregates the parameters of the local model trained from the local client data to iteratively optimize the global model, and finally completes the training of the central server model.

[0111] like Figure 3 As shown in the figure, the original CT image, the mask image of the real nodule and the predicted nodule mask image are displayed.

[0112] In the above embodiments, references to "this embodiment" in the specification indicate that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily refer to the same embodiment.

[0113] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the present invention are intended to encompass all such alternatives, modifications, and variations that fall within the broad scope of the appended claims.

[0114] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0115] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0116] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0117] Regarding the computer-readable storage medium in this embodiment, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0118] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used for communication, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes the various steps of the above method.

[0119] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0120] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0121] The present invention can be used in a wide variety of general-purpose or special-purpose computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above.

[0122] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A frequency-domain adaptive CT lung nodule image segmentation method based on federated learning, characterized by: The following steps are involved: S1: Data preparation, data annotation, data processing, data enhancement and data set division for CT lung nodule image data; S2: Construct a frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning; S3: training, validating, and testing the frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning; S4: CT lung nodule image segmentation is performed using the trained frequency-domain adaptive CT lung nodule image segmentation model FASNet based on federated learning; The frequency-domain adaptive CT pulmonary nodule image segmentation model FASNet based on federated learning includes an image data reconstruction process and a segmentation network; the image data reconstruction process includes an adaptive mask generator and a spatial feature extraction module; the computational process of the frequency-domain adaptive CT pulmonary nodule image segmentation model FASNet based on federated learning is as follows: S21: Input the CT lung nodule image into the FASNet model, roughly extract features through a convolution structure, and use Fourier transform to convert the input image from the spatial domain to the frequency domain. Then, a shift operation is performed on the Fourier transform result to move the zero-frequency component to the center. S22: generating a corresponding adaptive mask by an adaptive mask generator to separate high-frequency information and low-frequency information in the frequency domain; S23: performing inverse shift and inverse Fourier transform on the high-frequency information and the low-frequency information in the frequency domain, and converting them back to the spatial domain; S24: using a spatial feature extraction module to extract features of the high-frequency information and the low-frequency information in the spatial domain respectively, adding the high-frequency and low-frequency information bit by bit, and generating reconstructed image data with independent and identical distribution; S25: input the reconstructed image data into the segmentation network and output the segmentation result; The adaptive mask generator is used to generate an image mask that is adapted to the high-frequency information and low-frequency information of the image, thereby separating the high-frequency and low-frequency components. The process of generating the adaptive mask is as follows: For input , first through a convolution layer with a convolution kernel size of 3, the input three-channel image is mapped to a higher-dimensional space. The input is mapped to The obtained features are recorded as ; Then, through Fourier transform and shift, the features of the spatial domain are converted to the frequency domain, and the obtained features are recorded as , expressed as: in, Indicates a conventional convolution operation, with a default convolution kernel size of 1. Represents a convolution operation with a convolution kernel size of 3; represents the Fourier transform, It is the shift operation; Calculate the mask, the height of the mask area h _ and width w _By the height of the image and width Divide by a constant , then multiplied by the corresponding threshold, the obtained h _and w _ and then convert to integer; The threshold is calculated by first Using adaptive average pooling, global average pooling is performed on each channel of the feature map, each channel is pooled into one value, and the output size is ; Set a compression rate through a convolution layer with a convolution kernel size of 1 Compress the number of channels to obtain the feature map size ; Then pass through an activation function GELU to map the features into a nonlinear space; then pass through a convolution layer with a convolution kernel size of 1 to further compress the number of channels to 2, and obtain a feature map size of Finally, after the Sigmoid function, the values ​​of the two channels are converted into values ​​between (0, 1) and the sum is 1. The values ​​of the two channels are the learnable adaptive thresholds. The above process is expressed as follows: in, represents the adaptive average pooling operation, represents the GELU activation function, represents the Sigmoid function, represents the threshold, and its size is ; After generating the threshold, calculate the height of the mask area and width , the formula is described as follows: in and Represent the values ​​of the two channels of the threshold respectively, Indicates rounding operation; Fill the mask matrix with values ​​of size The mask matrix of arrive , width direction from arrive The area size is set to 1, and the remaining area is filled with 0 to obtain an adaptive mask matrix , as shown below: At this point, the adaptive mask generator generates a mask matrix with the center area as 1 and the edge as 0. The size of the mask matrix and the center area is generated by the adaptive threshold; Generate a mask matrix to separate the high-frequency information and low-frequency information of the frequency domain information, specifically including: frequency domain information First with the mask matrix M By multiplying bit by bit, the low-frequency components in the central area are retained, and the high-frequency components outside the central area are removed, thereby separating the low-frequency information ; For the separation of high-frequency information, With the mask matrix The reverse multiplication of (1- M ), the low-frequency components in the central area are removed, and the high-frequency components outside the central area are extracted, which is expressed as ; The separated high-frequency information and low-frequency information are then inverse shifted and inverse Fourier transformed to convert the separated frequency domain information back to the spatial domain, which is expressed as , , the formula is as follows: in, Represents a bitwise multiplication operation, .

2. The frequency-domain adaptive CT pulmonary nodule image segmentation method based on federated learning according to claim 1, characterized in that: Step S1 includes the following steps: data preparation, data labeling, data processing, data enhancement, and data set division for CT lung nodule image data: S11: Data preparation: Ensure that all CT image data are in a unified format, check data quality, remove incomplete, damaged, or non-compliant images, and ensure data integrity and validity; S12: Data annotation: Use medical image annotation tools to annotate CT lung nodule images; S13: Data processing: All CT images were resized to the same size or resolution, interpolated or cropped to a fixed size as network input, and the data were individually normalized to zero mean and unit variance in intensity values; S14: Data Augmentation: Apply data augmentation techniques to increase data diversity, including rotation, translation, scaling, and flipping; S15: Dataset partitioning: Divide the data into training set, validation set and test set.

3. The frequency-domain adaptive CT pulmonary nodule image segmentation method based on federated learning according to claim 1, characterized in that: The spatial feature extraction module first passes through a simple convolution structure, and the output is recorded as R , R It is used as a residual connection afterwards; it passes through a depth convolution; it passes through a linear layer, an activation layer, and a linear layer in sequence; the first linear layer increases the input dimension to four times the original as the output, and after the GELU activation function, the second linear layer restores the increased dimension to the original dimension; finally, a residual connection is added, and R The result of the addition is input into a convolutional layer, which then outputs the corresponding low-frequency information. and high-frequency information ; The process is expressed mathematically as: in, R It means that it is obtained after the first convolution structure and is used for residual connection later. represents a depthwise convolution with a kernel size of 7. represents a linear layer; The low-frequency information is extracted through the spatial feature extraction module and high-frequency information After further feature extraction, the low-frequency and high-frequency features obtained and Add and output the final reconstructed image ; The reconstructed image is expressed as follows: 。 4. The frequency-domain adaptive CT pulmonary nodule image segmentation method based on federated learning according to claim 1, characterized in that: Through federated learning, the federated learning-based frequency-domain adaptive CT lung nodule image segmentation model FASNet is trained on the local client, and then the trained network weights are uploaded to the central server. The central server aggregates the weights of each client to update the model; the trained model of the central server is then deployed in an actual clinical environment to verify the model performance and continuously optimize it based on feedback.

5. The frequency-domain adaptive CT pulmonary nodule image segmentation method based on federated learning according to claim 4, characterized in that: In the initial stage of training, the central server initializes a global model; after the local client completes training, it generates an updated set of model parameters based on the training results of the local dataset; the local client transmits the updated model parameters to the central server, and the central server uses the aggregated model parameters to update the global model. The updated global model becomes the starting point for the next round of training, and the updated global model parameters are distributed to the local clients; the local client trains, updates the model, and transmits parameters based on its own dataset, and the central server aggregates and updates the global model, and this cycle iterates until the model training is completed.

Images