Medical pathology prediction auxiliary system and method

Through the federated learning architecture and data distribution strategy, combined with the 3D-UNet model and the pathological prediction auxiliary model, the problem of medical data silos and non-independent homogeneous distribution is solved, and the accuracy of the pathological prediction auxiliary system and model training effect are improved.

CN118969305AInactive Publication Date: 2024-11-15TAIZHOU UNIV (ZHEJIANG)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410932812.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The problem of medical data silos and non-independent homogeneous distribution results in low accuracy of machine learning training, making it difficult to effectively use medical image data for pathological prediction.

Method used

Using a medical pathological prediction assistance system based on federated learning, through a horizontal federated learning architecture and data distribution strategy, image segmentation and prediction assistance are performed using 3D-UNet model and pathological prediction assistance model, realizing data iterative training between local databases and servers and model updates.

Benefits of technology

Overcoming the data island problem, improving the accuracy of the pathological model, achieving more accurate pathological prediction assisted image generation, and enhancing the effect of machine learning training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969305A_ABST
    Figure CN118969305A_ABST
Patent Text Reader

Abstract

The invention discloses a medical pathology prediction auxiliary system and method, and relates to the field of intelligent medical treatment, and the medical pathology prediction auxiliary system comprises a pathology prediction auxiliary module and a model training module. For the Nth training process, the pathology prediction auxiliary module is used for processing a local database corresponding to the Nth training process by adopting a pathology model corresponding to the (N-1) th training process transmitted by the server to obtain an image database corresponding to the Nth training process; the model training module is used for adopting the training set corresponding to the Nth training process to carry out iterative training on the pathological model corresponding to the (N-1) th training process transmitted by the server to obtain the pathological model trained in the Nth training process, and sending the pathological model trained in the Nth training process to the server, according to the invention, the accuracy of the pathological model obtained by machine learning training can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent medical care, and in particular to a medical pathology prediction auxiliary system and method. Background Art

[0002] The application of artificial intelligence is also expanding with the development of machine learning, creating conditions for intelligent medical care. The intelligent prediction auxiliary model can obtain pathology prediction auxiliary images, which can highlight the characteristics of pathology.

[0003] In the field of image segmentation, training deep neural networks to obtain intelligent prediction auxiliary models can efficiently process medical image information. However, due to the fact that medical data is not allowed to be directly exchanged and data owners are unwilling to share, data islands are formed, making it impossible to bring data together for machine learning training, resulting in low accuracy of machine learning training. At the same time, since medical data is not independent and identically distributed, the accuracy of the pathology model obtained by machine learning training is also low. Summary of the invention

[0004] The purpose of this application is to provide a medical pathology prediction auxiliary system and method, which can improve the accuracy of the pathology model obtained by machine learning training.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a medical pathology prediction auxiliary system based on federated learning, comprising: a pathology prediction auxiliary module and a model training module connected in sequence; the model training module and the pathology prediction auxiliary module are both connected to a server;

[0007] For the Nth training process, the pathology prediction auxiliary module is used to use the pathology model corresponding to the N-1th training process transmitted by the server, process the local database corresponding to the Nth training process, and obtain the image database corresponding to the Nth training process; the pathology model includes a 3D-UNet model and a pathology prediction auxiliary model connected in sequence; the local database includes multiple pathological images to be processed; the image database corresponding to the Nth training process includes the 3D-UNet model in the pathology model corresponding to the N-1th training process sent by the server, process the local database corresponding to the Nth training process, obtain the pathology image segmentation map database corresponding to the Nth training process, and the pathology prediction auxiliary model in the pathology model corresponding to the N-1th training process sent by the server, process the pathology image segmentation map database corresponding to the Nth training process, and obtain the pathology prediction auxiliary image database corresponding to the Nth training process;

[0008] The model training module is used to use the training set corresponding to the Nth training process to iteratively train the pathological model corresponding to the N-1th training process transmitted by the server, obtain the pathological model trained by the Nth training process, and send the pathological model trained by the Nth training process to the server; when N=1, the training set corresponding to the Nth training process includes the local database corresponding to the first training process and the image database corresponding to the first training process; when N>1, the training set corresponding to the Nth training process includes the shared database corresponding to the Nth training process and the image database corresponding to the Nth training process; the shared database corresponding to the Nth training process includes the local database corresponding to the Nth training process of each medical pathology prediction auxiliary system and the shared database corresponding to the N-1th training process.

[0009] In a second aspect, the present application provides a medical pathology prediction assistance method, which is applied to the above-mentioned medical pathology prediction assistance system, and the medical pathology prediction assistance method includes:

[0010] During the Nth training process, the local database corresponding to the Nth training process is processed using the case model corresponding to the N-1th training process to obtain an image database corresponding to the Nth training process; the image database corresponding to the Nth training process includes a pathological image segmentation map database corresponding to the Nth training process obtained by processing the local database corresponding to the Nth training process using the 3D-UNet model in the pathological model corresponding to the N-1th training process, and a pathological prediction auxiliary image database corresponding to the Nth training process obtained by processing the pathological image segmentation map database corresponding to the Nth training process using the pathological prediction auxiliary model in the pathological model corresponding to the N-1th training process;

[0011] The training set corresponding to the Nth training process is used to iteratively train the pathological model corresponding to the N-1th training process transmitted by the server to obtain the pathological model trained by the Nth training process; when N=1, the training set corresponding to the Nth training process includes the local database corresponding to the first training process and the image database corresponding to the first training process; when N>1, the training set corresponding to the Nth training process includes the shared database corresponding to the Nth training process and the image database corresponding to the Nth training process; the shared database corresponding to the Nth training process includes the local database corresponding to the Nth training process of each medical pathology prediction auxiliary system and the shared database corresponding to the N-1th training process.

[0012] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0013] The present application provides a medical pathology prediction assistance system, which solves the problem of data islands by using the local database provided by each medical pathology prediction assistance system to train the pathology model during the training process, and improves the accuracy of the pathology model obtained by machine learning training, so that the pathology prediction auxiliary image finally obtained by using the pathology model is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 is a workflow diagram of a medical pathology prediction auxiliary system provided by an embodiment of the present application;

[0016] Figure 2 This is a data distribution strategy diagram of a horizontal federated learning framework in a medical pathology prediction auxiliary system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0019] The present application provides a medical pathology prediction auxiliary system, specifically a medical pathology prediction auxiliary system based on federated learning. The medical pathology prediction auxiliary system under the horizontal federated learning architecture proposes a data distribution strategy for the problem of non-independent and identically distributed medical data, which not only overcomes the data island problem, but also provides an intelligent solution for medical pathology prediction. The medical pathology prediction auxiliary system includes: a pathology prediction auxiliary module and a model training module connected in sequence; the model training module and the pathology prediction auxiliary module are both connected to a server.

[0020] For the Nth training process, the pathology prediction auxiliary module is used to use the pathology model corresponding to the N-1th training process transmitted by the server, process the local database corresponding to the Nth training process, and obtain the image database corresponding to the Nth training process; the pathology model includes a 3D-UNet model and a pathology prediction auxiliary model connected in sequence; the local database includes multiple pathological images to be processed; the image database corresponding to the Nth training process includes the 3D-UNet model in the pathology model corresponding to the N-1th training process sent by the server, and processes the local database corresponding to the Nth training process to obtain the pathology image segmentation map database corresponding to the Nth training process (including the local database corresponding to the Nth training process The pathological image segmentation map database corresponding to the Nth training process is processed to obtain the pathological prediction auxiliary image database corresponding to the Nth training process (including the pathological prediction auxiliary images corresponding to each pathological image to be processed in the local database corresponding to the Nth training process); when N=1, the pathological model corresponding to the N-1th training process transmitted by the server is a pathological model constructed according to the shared database corresponding to the first training process; the shared database corresponding to the first training process includes the local database and image database corresponding to the first training process uploaded by each medical pathology prediction auxiliary system.

[0021] The model training module is used to use the training set corresponding to the Nth training process to iteratively train the pathological model corresponding to the N-1th training process transmitted by the server, obtain the pathological model trained by the Nth training process, and send the pathological model trained by the Nth training process to the server; when N=1, the training set corresponding to the Nth training process includes the local database corresponding to the first training process and the image database corresponding to the first training process; when N>1, the training set corresponding to the Nth training process includes the shared database corresponding to the Nth training process and the image database corresponding to the Nth training process; the shared database corresponding to the Nth training process includes the local database corresponding to the Nth training process of each medical pathology prediction auxiliary system and the shared database corresponding to the N-1th training process.

[0022] In practical applications, the data processing steps of this system are as follows:

[0023] Step 101: Submit the image.

[0024] Step 102: Convert the image to numpy format.

[0025] Step 103: Upload the processed image to the 3D-UNet model.

[0026] Step 104: Use the 3D-UNet model to perform pathological image segmentation and output a pathological image segmentation map.

[0027] Step 105: Process the pathology image segmentation map using the pathology prediction auxiliary model to obtain a pathology prediction auxiliary image.

[0028] Step 106: If the amount of output data reaches the threshold set by the system, it participates in model training.

[0029] Step 107: If the number of outputs does not reach the threshold set by the system, the pathological image segmentation map and the pathological prediction auxiliary image are output, and then the data upload module is executed.

[0030] Specifically, when the submitted image is a DICOM image set in DICOM format, the data processing steps of the system are:

[0031] Step 1.1: Preprocess the DICOM image set, that is, convert the DICOM image set into NumPy format. The image size in NumPy format is 512*512.

[0032] Step 1.2: Use the 3D-UNet model to segment the numpy format image to obtain the image of the lesion site, that is, the pathological image segmentation map.

[0033] Step 1.3: Change the image size of the pathological image segmentation map to 224*336.

[0034] Step 1.4: Assist pathology prediction through the pathology prediction auxiliary model to obtain a pathology prediction auxiliary image.

[0035] In practical applications, the steps for storing the pathology image segmentation map in the pathology prediction auxiliary module data are as follows:

[0036] Step 2.1: The dicom image set is stored in the . / normal file.

[0037] Step 2.2: Convert the dicom image set into numpy format with image size of 512*512 and store it in . / dataset / npy.

[0038] Step 2.3: After the 3D-UNet model, the image is converted into a segmented numpy dataset, which is the pathological image segmentation map. The pathological image segmentation map is stored in . / unet_reslut / npy.

[0039] Step 2.4: Adjust the size of the pathological image segmentation map to 224*336 and store it in . / reslut / npy.

[0040] In actual applications, the medical pathology prediction assistance system also includes: a data upload module connected to the pathology prediction assistance module, the model training module and the server respectively; for the Nth training process, the data upload module is used to upload the local database corresponding to the Nth training process to the pathology prediction assistance module, the model training module and the server.

[0041] In practical applications, the medical pathology prediction auxiliary system also includes: a data processing module, through which the data upload module and the pathology prediction auxiliary module are connected; the data processing module is used to convert the format of each pathology image to be processed included in the local database, and the specific local database includes dicom images. The data processing module converts the dicom image into numpy format, and the image size in numpy format is 512×512.

[0042] In practical applications, data in medical pathology prediction auxiliary systems may have problems such as redundancy and missing data. Therefore, different data processing solutions need to be selected according to the actual tasks. The following introduces the DICOM image data preprocessing steps in the data processing module.

[0043] Step 3.1: Convert the measurement units of the DICOM image data to Hounsfield Unit (HU).

[0044] Step 3.2: Determine the threshold of the HU window (e.g. -1200 to 600HU) and then convert the dicom image to numpy format by linearly normalizing the values ​​from 16 bits to 8 bits, i.e. 0-255.

[0045] Step 3.3: Resample the above numpy data to the same spatial resolution (such as 512*512).

[0046] In practical applications, the medical pathology prediction auxiliary system further processes the image by resizing the image segmentation (3D-Uent model) and the subsequent pathology judgment (pathology prediction auxiliary model), specifically including: first, for each CT volume (pathology image to be processed), the image output by the trained 3D-UNet model forms a mask volume, and then the CT volume is connected with the mask volume to obtain a CT-mask volume. Without changing the number of slices for training and testing the judgment model, the CT-mask volume is resampled to a fixed spatial resolution (such as 224×336).

[0047] In practical applications, the 3D-UNet model includes: a contraction path (encoder), an expansion path and a (decoder) output layer (a 1×1 convolution layer); the contraction path includes a plurality of (4) downsampling modules connected in sequence; the expansion path includes a plurality of (4) upsampling modules connected in sequence; the downsampling module includes a first convolution module (two 3x3 convolution layers) and a first pooling layer (2x2 pooling layer) connected in sequence; the upsampling module includes a feature splicing layer and a deconvolution layer and a second convolution module (two 3x3 convolution layers) connected in sequence; the output end of the first pooling layer of the first downsampling module and the input end of the second convolution module of the fourth upsampling module are connected through the fourth upsampling module The output end of the first pooling layer of the second down-sampling module is connected to the input end of the second convolution module of the third up-sampling module through the feature splicing layer of the third up-sampling module; the output end of the first pooling layer of the third down-sampling module is connected to the input end of the second convolution module of the second up-sampling module through the feature splicing layer of the second up-sampling module; the output end of the first pooling layer of the fourth down-sampling module is connected to the input end of the second convolution module of the first up-sampling module through the feature splicing layer of the first up-sampling module; the output layer is connected to the output end of the second convolution module of the fourth up-sampling module; the output end of the first pooling layer of the fourth down-sampling module is connected to the input end of the deconvolution layer of the first up-sampling module.

[0048] The data processing steps of the 3D-UNet model are as follows:

[0049] The pathological image (512×512 numpy format data) first enters the contraction path for downsampling: through two 3×3 convolutional layers (RELU), the depth of the feature map is increased and the spatial dimension of the feature map is reduced, and then through the pooling layer, the size of the feature map is further reduced to extract the abstract features. Repeat 4 times to form a downsampling path until the deepest layer of the network is reached (the size of the feature map is 32×32 at this time); after reaching the deepest layer, the image features begin to be upsampled through the expansion path: through the deconvolution layer, the spatial dimension of the feature map is gradually restored, and the number of feature channels is reduced. At the same time, the feature concatenation layer concatenates the feature map of the corresponding layer in the contraction path with the feature map of the current layer in each upsampling step, and concatenates the feature map in the contraction path with the feature map in the expansion path through the jump connection. Repeat 4 times to form an upsampling path until the output layer of the network is reached (the size of the feature map is 512×512 at this time); Output layer: Finally, the feature map after feature fusion passes through a 1×1 convolution layer, and uses the Softmax activation function in the output layer to assign a category label to each pixel. The output layer then outputs the final segmentation result, i.e., the pathological image segmentation map.

[0050] In practical applications, the pathology prediction auxiliary model includes: a backbone network, a residual network and a progressive classifier connected in sequence; the backbone network includes sequentially connected convolutional layers (ordinary convolutions with a kernel size of 3×3), a first batch of normalization layers and a second pooling layer; the residual network includes sequentially connected second batches of normalization layers and shortcut connections (including 3×3 convolutions); the progressive classifier includes sequentially connected convolutional modules and fully connected layers; the convolutional module includes multiple sequentially connected convolutional layers, specifically three 3×3 convolutional layers.

[0051] The data processing steps of the above pathology prediction auxiliary model are as follows:

[0052] After the pathological image segmentation map output by the 3D-UNet model is converted into a pixel matrix, a convolution operation with non-zero padding is performed on the pixel matrix, and then a convolution operation is performed on this feature map using a 3×3 ordinary convolution kernel, followed by a pooling operation to reduce the data flow while retaining the features. After the batch normalization layer unifies the size of the feature map, the size is corrected by the shortcut connection of the 3×3 convolution and then aligned. Finally, the progressive classifier fuses the feature maps through three 3×3 convolution layers and one fully connected layer, and uses the Softmax activation function in the fully connected layer to assign a category label to each pixel. After gradually extracting the information in the pathological image segmentation map, the pathology prediction auxiliary image is directly output.

[0053] In practical applications, the model training module is used to:

[0054] Taking the pathological image to be processed in the training set corresponding to the Nth training process as input, and the pathological image segmentation map database in the image database corresponding to the Nth training process as output, the 3D-UNet model in the pathological model transmitted by the server is iteratively trained to obtain the 3D-UNet model trained in the Nth training process.

[0055] Taking the pathological image segmentation map database in the image database corresponding to the Nth training process as input, and the pathological prediction auxiliary image database in the image database corresponding to the Nth training process as output, the pathological prediction auxiliary model in the pathological model transmitted by the server is iteratively trained to obtain the pathological prediction auxiliary model trained in the Nth training process.

[0056] In practical applications, the pathology prediction auxiliary module includes: a pathology prediction auxiliary unit, a storage unit, a judgment unit, a first determination unit and a second determination unit.

[0057] For the Nth training process, at the kth prediction number, the pathology prediction auxiliary unit is used to use the pathology model corresponding to the N-1th training process transmitted by the server to process the pathology image to be processed at the kth prediction number, and obtain the pathology image segmentation map and pathology prediction auxiliary image corresponding to the pathology image to be processed at the kth prediction number. Specifically, the pathology image to be processed is input into the pre-trained 3D-UNet model to obtain the pathology image segmentation map output by the 3D-UNet model; then the pathology image segmentation map is input into the pathology prediction auxiliary model to obtain the pathology prediction auxiliary image output by the pathology prediction auxiliary model. The pathology prediction auxiliary image can highlight the characteristics of the pathology and assist doctors in making judgments.

[0058] The storage unit is used to store the pathological image segmentation map and the pathological prediction auxiliary image corresponding to the pathological image to be processed under the first k prediction times.

[0059] The judging unit is used to judge whether the number of pathological image segmentation maps and pathological prediction auxiliary images corresponding to the pathological image to be processed under the first k prediction times stored in the storage unit reaches a set threshold.

[0060] The first determination unit is used to determine that the pathological image to be processed under the first k prediction times is the local database corresponding to the Nth training process if the set threshold is reached, and determine the pathological image segmentation map and pathological prediction auxiliary image corresponding to the pathological image to be processed under the first k prediction times as the corresponding image database corresponding to the Nth training process, and then execute the model training module, which uses the local database and the shared database to train the 3D-UNet model and the pathological prediction auxiliary model. During the training stage, the parameters of the 3D-UNet model and the pathological prediction auxiliary model are optimized by the cross entropy loss function to improve the auxiliary prediction accuracy. Finally, the 3D-UNet model and the pathological prediction auxiliary model are synthesized into a local pathological model and the local pathological model is uploaded to the server.

[0061] The second determination unit is used to update the prediction times k if the set threshold is not reached, and then return to the pathology prediction auxiliary unit.

[0062] In practical applications, the training process of the 3D-UNet model and the pathology prediction auxiliary model is the same. The 3D-UNet model is trained using pathology images, and the pathology prediction auxiliary model is trained using the pathology image segmentation map output by the 3D-UNet model. In this way, this application only introduces the 3D-UNet model training process, and the pathology prediction auxiliary model training only needs to change the input data.

[0063] The specific process of 3D-UNet model training includes:

[0064] Step 4.1: Initialize the logger.

[0065] Create a `Logger` object `Unet_train_txt` to record the training loss of each epoch.

[0066] Step 4.2: Define the training function (`Train_Unet`), whose parameters include:

[0067] `net`: 3D-UNet model.

[0068] `device`: Used to specify whether the model and data should be run on the CPU or CUDA device.

[0069] `data_path`: data training path.

[0070] `batch_size`: The number of samples in each training batch.

[0071] `epochs`: The total number of training rounds.

[0072] `lr`: learning rate of the optimizer.

[0073] Step 4.3: Load the dataset.

[0074] Use the `SelfDataSet` class to load the dataset, and create an iterator `train_loader` through `DataLoader` to batch load the data.

[0075] Step 4.4: Define the optimizer and loss function.

[0076] Create an Adam optimizer using PyTorch's `optim.Adam` to update the model's weights. And use the `nn.BCEWithLogitsLoss` loss function.

[0077] Step 4.5: Training loop.

[0078] For each epoch, the following steps are performed:

[0079] Step 4.5.1: Set the 3D-UNet model to training mode (`net.train()`).

[0080] Step 4.5.2: Initialize the running loss `running_loss` and batch counter `i`.

[0081] Step 4.5.3: Record the epoch start time.

[0082] Step 4.5.4: Iterate over each batch in `train_loader`, which includes:

[0083] (1) Use `opt.zero_grad()` to clear the previous gradient.

[0084] (2) Move the pathological image and label data (pathological image segmentation map) to the specified device (CPU or CUDA) and convert them to float32 type.

[0085] (3) Perform forward propagation through the D-UNet model to obtain prediction auxiliary results.

[0086] (4) Calculate the loss between the predicted auxiliary results and the true labels.

[0087] (5) Perform backpropagation (ˋloss.backward()ˋ).

[0088] (6) Update the weights of the D-UNet model (ˋopt.step()ˋ).

[0089] (7) Update running loss and batch counters.

[0090] Step 4.5.5: Calculate the average loss of the current epoch.

[0091] Step 4.5.6: Record the average loss into a log file.

[0092] Step 4.5.7: Print the information of the current epoch, including the number, average loss, and training time.

[0093] Step 4.5.8: If the average loss of the current epoch is lower than the previous best loss, update the 3D-UNet model.

[0094] The workflow of the medical pathology prediction auxiliary system provided in the above embodiment is as follows: Figure 1 As shown, the data upload module uploads the dicom database and guides the data processing module. The data processing module performs image preprocessing to obtain the numpy_data image; the pathology prediction auxiliary model inputs the numpy_data image into the 3D-UNet model for image segmentation, and inputs the segmentation result into the pathology prediction auxiliary model for image prediction, and stores the prediction result. When the data amount is greater than the threshold, the model is trained through the model training module and then sent to the server. If the data amount is not greater than the threshold, the pathology prediction auxiliary result, i.e., the pathology prediction auxiliary image, is directly output.

[0095] The above-mentioned medical pathology prediction auxiliary system includes a horizontal federated learning framework and a data distribution strategy. For the specific working process, please refer to Figure 2 Among them, the horizontal federated learning framework includes:

[0096] Step 5.1: The server sends the latest pathology model to each medical pathology prediction auxiliary system.

[0097] Step 5.2: After the model training module of the medical pathology prediction auxiliary system trains the pathology model using the DICOM database of the shared database, the local pathology model is uploaded to the server by gradient.

[0098] Step 5.3: The server aggregates the gradient models of various medical pathology prediction auxiliary systems and updates the pathology model in a timely manner.

[0099] Step 5.4: The server returns the updated pathology model to the medical pathology prediction auxiliary system.

[0100] Step 5.5: The medical pathology prediction auxiliary system updates its respective local pathology model.

[0101] Among them, the data distribution strategy, the detailed steps are as follows:

[0102] Step 6.1: Each medical pathology prediction auxiliary system uploads the local database to the server.

[0103] Step 6.2: The server aggregates various local databases to form a DICOM shared database.

[0104] Step 6.3: The server uses the DICOM shared database to build the pathology model.

[0105] Step 6.4: The server sends the pathology model to the medical pathology prediction auxiliary system for training.

[0106] Step 6.5: If the epoch is 1, the server sends the pathology model to the medical pathology prediction auxiliary system.

[0107] Step 6.6: The medical pathology prediction auxiliary system downloads the model and uses the local database to train until the pathology model converges, then uploads it to the server and updates the pathology model.

[0108] Step 6.7: Before running the nth epoch, the server sends the amount of data in the shared database according to the scale of the medical pathology prediction auxiliary system.

[0109] Step 6.8: The medical pathology prediction auxiliary system receives the shared database.

[0110] Step 6.9: Train the model based on the shared database until the model converges, upload it to the server and update the pathology model.

[0111] Step 6.10: The server determines whether epoch is greater than n (n is the number of server training times set in advance). If it is greater than n, the training ends; if it is less than n, go to step 6.9.

[0112] In this application, the data distribution strategy provides optimization functions for the model during model training to achieve model updates. Model training is based on model optimization under the horizontal federated learning framework, breaking through the limitations of data silos. The data distribution function can also overcome the phenomenon of non-independent and identically distributed data in the federated learning environment.

[0113] The present application also provides a medical pathology prediction auxiliary method, which is applied to the above-mentioned medical pathology prediction auxiliary system. The medical pathology prediction auxiliary method includes:

[0114] During the Nth training process, the case model corresponding to the N-1th training process is used to process the local database corresponding to the Nth training process to obtain an image database corresponding to the Nth training process; the image database corresponding to the Nth training process includes a pathological image segmentation map database corresponding to the Nth training process obtained by processing the local database corresponding to the Nth training process using the 3D-UNet model in the pathological model corresponding to the N-1th training process, and a pathological prediction auxiliary image database corresponding to the Nth training process obtained by processing the pathological image segmentation map database corresponding to the Nth training process using the pathological prediction auxiliary model in the pathological model corresponding to the N-1th training process.

[0115] The training set corresponding to the Nth training process is used to iteratively train the pathological model corresponding to the N-1th training process transmitted by the server to obtain the pathological model trained by the Nth training process; when N=1, the training set corresponding to the Nth training process includes the local database corresponding to the first training process and the image database corresponding to the first training process; when N>1, the training set corresponding to the Nth training process includes the shared database corresponding to the Nth training process and the image database corresponding to the Nth training process; the shared database corresponding to the Nth training process includes the local database corresponding to the Nth training process of each medical pathology prediction auxiliary system and the shared database corresponding to the N-1th training process.

[0116] In practical applications, during the Nth training process, the case model corresponding to the N-1th training process is used to process the local database corresponding to the Nth training process to obtain the image database corresponding to the Nth training process, which specifically includes:

[0117] For the Nth training process, at the kth prediction number, the pathological model corresponding to the N-1th training process is used to process the pathological image to be processed at the kth prediction number, and the pathological image segmentation map and pathological prediction auxiliary image corresponding to the pathological image to be processed at the kth prediction number are obtained.

[0118] It is determined whether the number of pathological image segmentation maps and pathological prediction auxiliary images corresponding to the pathological image to be processed under the first k prediction times reaches a set threshold.

[0119] If the set threshold is reached, the pathological image to be processed under the first k prediction times is determined to be the local database corresponding to the Nth training process, and the pathological image segmentation map and pathological prediction auxiliary image corresponding to the pathological image to be processed under the first k prediction times are determined to be the corresponding image database corresponding to the Nth training process.

[0120] If the set threshold is not reached, the prediction number k is updated and the next prediction is performed.

[0121] In practical applications, when N=1, the pathological model corresponding to the N-1th training process is constructed based on the shared database corresponding to the first training process.

[0122] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A medical pathology prediction auxiliary system, characterized in that: The medical pathology prediction auxiliary system comprises: a pathology prediction auxiliary module and a model training module connected in sequence; the model training module and the pathology prediction auxiliary module are both connected to a server; For the Nth training process, the pathology prediction auxiliary module is used to use the pathology model corresponding to the N-1th training process transmitted by the server, process the local database corresponding to the Nth training process, and obtain the image database corresponding to the Nth training process; the pathology model includes a 3D-UNet model and a pathology prediction auxiliary model connected in sequence; the local database includes multiple pathological images to be processed; the image database corresponding to the Nth training process includes the 3D-UNet model in the pathology model corresponding to the N-1th training process sent by the server, process the local database corresponding to the Nth training process, obtain the pathology image segmentation map database corresponding to the Nth training process, and the pathology prediction auxiliary model in the pathology model corresponding to the N-1th training process sent by the server, process the pathology image segmentation map database corresponding to the Nth training process, and obtain the pathology prediction auxiliary image database corresponding to the Nth training process; The model training module is used to use the training set corresponding to the Nth training process to iteratively train the pathological model corresponding to the N-1th training process transmitted by the server, obtain the pathological model trained by the Nth training process, and send the pathological model trained by the Nth training process to the server; when N=1, the training set corresponding to the Nth training process includes the local database corresponding to the first training process and the image database corresponding to the first training process; when N>1, the training set corresponding to the Nth training process includes the shared database corresponding to the Nth training process and the image database corresponding to the Nth training process; the shared database corresponding to the Nth training process includes the local database corresponding to the Nth training process of each medical pathology prediction auxiliary system and the shared database corresponding to the N-1th training process.

2. The medical pathology prediction auxiliary system according to claim 1, characterized in that: The medical pathology prediction auxiliary system further includes: a data uploading module connected to the pathology prediction auxiliary module, the model training module and the server respectively; For the Nth training process, the data uploading module is used to upload the local database corresponding to the Nth training process to the pathology prediction auxiliary module, the model training module and the server.

3. The medical pathology prediction auxiliary system according to claim 2, characterized in that: The medical pathology prediction auxiliary system further includes: a data processing module, through which the data upload module and the pathology prediction auxiliary module are connected; the data processing module is used to perform format conversion on each to-be-processed pathology image included in the local database.

4. The medical pathology prediction assistance system according to claim 1, characterized in that: The 3D-UNet model includes: a contraction path, an expansion path and an output layer; The contraction path includes a plurality of sequentially connected downsampling modules; the expansion path includes a plurality of sequentially connected upsampling modules; the downsampling module includes a first convolution module and a first pooling layer connected in sequence; the upsampling module includes a feature concatenation layer and a deconvolution layer and a second convolution module connected in sequence; The output end of the first pooling layer of the first down-sampling module is connected to the input end of the second convolution module of the fourth up-sampling module through the feature splicing layer of the fourth up-sampling module; the output end of the first pooling layer of the second down-sampling module is connected to the input end of the second convolution module of the third up-sampling module through the feature splicing layer of the third up-sampling module; the output end of the first pooling layer of the third down-sampling module is connected to the input end of the second convolution module of the second up-sampling module through the feature splicing layer of the second up-sampling module; the output end of the first pooling layer of the fourth down-sampling module is connected to the input end of the second convolution module of the first up-sampling module through the feature splicing layer of the first up-sampling module; the output layer is connected to the output end of the second convolution module of the fourth up-sampling module; the output end of the first pooling layer of the fourth down-sampling module is connected to the input end of the deconvolution layer of the first up-sampling module.

5. The medical pathology prediction auxiliary system according to claim 1, characterized in that: The pathology prediction auxiliary model includes: a backbone network, a residual network and a progressive classifier connected in sequence; the backbone network includes a convolutional layer, a first batch of normalization layers and a second pooling layer connected in sequence; the residual network includes a second batch of normalization layers and shortcut connections connected in sequence; the progressive classifier includes a convolutional module and a fully connected layer connected in sequence; the convolutional module includes multiple convolutional layers connected in sequence.

6. The medical pathology prediction assistance system according to claim 1, characterized in that: The model training module is used to: Taking the pathological image to be processed in the training set corresponding to the Nth training process as input and the pathological image segmentation map database in the image database corresponding to the Nth training process as output, the 3D-UNet model in the pathological model transmitted by the server is iteratively trained to obtain the 3D-UNet model trained in the Nth training process; Taking the pathological image segmentation map database in the image database corresponding to the Nth training process as input, and the pathological prediction auxiliary image database in the image database corresponding to the Nth training process as output, the pathological prediction auxiliary model in the pathological model transmitted by the server is iteratively trained to obtain the pathological prediction auxiliary model trained in the Nth training process.

7. The medical pathology prediction assistance system according to claim 1, characterized in that: The pathology prediction auxiliary module includes: a pathology prediction auxiliary unit, a storage unit, a judgment unit, a first determination unit and a second determination unit; For the Nth training process, at the kth prediction number, the pathology prediction auxiliary unit is used to use the pathology model corresponding to the N-1th training process transmitted by the server to process the pathology image to be processed at the kth prediction number, and obtain the pathology image segmentation map and pathology prediction auxiliary image corresponding to the pathology image to be processed at the kth prediction number; The storage unit is used to store the pathological image segmentation map and the pathological prediction auxiliary image corresponding to the pathological image to be processed under the first k prediction times; The judging unit is used to judge whether the number of pathological image segmentation graphs and pathological prediction auxiliary images corresponding to the pathological image to be processed under the first k prediction times stored in the storage unit reaches a set threshold; A first determination unit is used to determine, if a set threshold is reached, that the pathological image to be processed under the previous k prediction times is a local database corresponding to the Nth training process, determine that the pathological image segmentation map and the pathological prediction auxiliary image corresponding to the pathological image to be processed under the previous k prediction times are the corresponding image database corresponding to the Nth training process, and then execute the model training module; The second determination unit is used to update the prediction times k if the set threshold is not reached, and then return to the pathology prediction auxiliary unit.

8. A medical pathology prediction auxiliary method, characterized in that: The medical pathology prediction assistance method is applied to the medical pathology prediction assistance system described in any one of claims 1 to 7, and the medical pathology prediction assistance method includes: During the Nth training process, the local database corresponding to the Nth training process is processed using the case model corresponding to the N-1th training process to obtain an image database corresponding to the Nth training process; the image database corresponding to the Nth training process includes a pathological image segmentation map database corresponding to the Nth training process obtained by processing the local database corresponding to the Nth training process using the 3D-UNet model in the pathological model corresponding to the N-1th training process, and a pathological prediction auxiliary image database corresponding to the Nth training process obtained by processing the pathological image segmentation map database corresponding to the Nth training process using the pathological prediction auxiliary model in the pathological model corresponding to the N-1th training process; The training set corresponding to the Nth training process is used to iteratively train the pathological model corresponding to the N-1th training process transmitted by the server to obtain the pathological model trained by the Nth training process; when N=1, the training set corresponding to the Nth training process includes the local database corresponding to the first training process and the image database corresponding to the first training process; when N>1, the training set corresponding to the Nth training process includes the shared database corresponding to the Nth training process and the image database corresponding to the Nth training process; the shared database corresponding to the Nth training process includes the local database corresponding to the Nth training process of each medical pathology prediction auxiliary system and the shared database corresponding to the N-1th training process.

9. The medical pathology prediction auxiliary method according to claim 7, characterized in that: In the Nth training process, the case model corresponding to the N-1th training process is used to process the local database corresponding to the Nth training process to obtain the image database corresponding to the Nth training process, which specifically includes: For the Nth training process, at the kth prediction number, the pathological model corresponding to the N-1th training process is used to process the pathological image to be processed at the kth prediction number, and the pathological image segmentation map and pathological prediction auxiliary image corresponding to the pathological image to be processed at the kth prediction number are obtained; Determine whether the number of pathological image segmentation images and pathological prediction auxiliary images corresponding to the pathological image to be processed under the first k prediction times reaches a set threshold; If the set threshold is reached, the pathological image to be processed under the first k prediction times is determined to be the local database corresponding to the Nth training process, and the pathological image segmentation map and pathological prediction auxiliary image corresponding to the pathological image to be processed under the first k prediction times are determined to be the corresponding image database corresponding to the Nth training process; If the set threshold is not reached, the prediction number k is updated and the next prediction is performed.

10. The medical pathology prediction auxiliary method according to claim 7, characterized in that: When N=1, the pathological model corresponding to the N-1th training process is constructed based on the shared database corresponding to the first training process.