A Tooth Disease Classification Method Based on Federated Learning

Abstract

The present invention discloses a method for classifying dental diseases based on federated learning. A method for classifying dental diseases based on federated learning includes the following steps: S1, obtaining the oral information of users; S2, transmitting the oral tooth images to a local server for preprocessing; S3, a single local server performing model training according to the preprocessed oral tooth image set and the model of the cloud total server; S4, transmitting the trained model parameters to the cloud total server by means of digital encryption; S5, each local server performing fusion and classification according to the model parameters decrypted and distributed by the cloud total server. In the method of the present invention, the classification is accurate, there is no need to centrally process data, the oral information is in a strictly confidential state, and there is no privacy problem; and the model parameters are encrypted during the process, further ensuring security.

Description

Technical Field

[0001] The present invention relates to the technical field of stomatology, and in particular to a tooth disease classification method based on federated learning. Background Art

[0002] Currently, the diagnosis of oral medicine is carried out by taking periapical films, panoramic films and dental CT examinations, and then the type of tooth disease is judged through the diagnosis of doctors. However, for some small cities with relatively low medical levels, due to the small number of patients and the relatively insufficient professional skills of clinical doctors, it is impossible to diagnose patients well. On the premise of protecting the privacy of patients, a tooth disease classification method based on federated learning is proposed to help doctors make medical diagnoses using big data.

[0003] Disclosed in the Chinese patent literature "A method for segmenting three-dimensional CBCT tooth images based on feature transformation", with the publication number CN113744275A, specifically relates to a method for segmenting three-dimensional CBCT tooth images based on feature transformation. The method includes: acquiring CBCT image data in real time and preprocessing the data; inputting the preprocessed CBCT image data into a trained CBCT image tooth segmentation model for segmentation processing; evaluating and analyzing the segmentation results; the CBCT image tooth segmentation model is an improved 3D convolutional neural network, and the improved 3D convolutional neural network includes an encoder, a spatial transformation module STM, a class transformation module CTM, a feature fusion module, a decoder and an output layer; the invention adopts a 3D convolutional neural network model combining spatial feature transformation and class feature transformation modules, combines spatial global information and class global information, effectively improves the segmentation effect and improves the classification result. However, the invention only relates to the processing process of tooth images. Summary of the Invention

[0004] The present invention solves the problem that there is no reliable auxiliary classification method for existing tooth diseases, and proposes a tooth disease classification method based on federated learning. After collecting the oral information of each user, the oral tooth images are transmitted to the local server, and trained according to the oral tooth images and the model issued by the cloud general server. The trained model parameters are encrypted and sent to the cloud general server, and the cloud general server distributes the model parameters to each local server for fusion comparison; the method in the present invention has accurate classification, does not require centralized processing of data, and the oral information is in a strictly confidential state, not involving privacy issues; and the model parameters are encrypted during the process to further ensure security.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A tooth disease classification method based on federated learning, comprising the following steps:

[0006] S1, Obtain the oral information of the user;

[0007] S2, Transmit the oral tooth images to the local server for preprocessing;

[0008] S3, A single local server performs model training according to the preprocessed oral tooth image set and the model of the cloud total server; S4, Transmit the trained model parameters to the cloud total server through digital encryption means;

[0009] S5, Each local server performs fusion and classification according to the model parameters decrypted and distributed by the cloud total server. In the present invention, the oral information of the patient user is obtained through the acquisition end. The acquisition end can not only obtain the oral information, but also perform preliminary segmentation processing on the acquired oral tooth images; perform preprocessing on the segmented oral tooth images to remove some images that do not meet the requirements; the preprocessed images are used as training samples and trained in the model issued by the cloud total server; after training, the model parameters are uploaded to the cloud total server through digital encryption means; finally, the cloud total server decrypts and distributes the model parameters to each local server for fusion comparison, and classifies dental diseases according to the fusion comparison results; the method of the present invention protects the privacy of patient users while utilizing big data.

[0010] Preferably, the oral information in step S1 includes oral tooth images and the number of teeth. The oral tooth images are subjected to image segmentation processing according to the number of teeth. The oral tooth images after image segmentation processing form an oral tooth image data set numbered in the order of the number of teeth and are temporarily stored in the acquisition end. In the present invention, the number of teeth of each patient user is different. The oral tooth images are segmented into individual tooth images and numbered, and after numbering, they are arranged in order to form an oral tooth image data set, which is convenient for subsequent image processing.

[0011] Preferably, step S2 includes the following steps:

[0012] S21, Upload the oral tooth image data set to the local server;

[0013] S22, Adjust each oral tooth image to the same pixel and the same size, segment the target from the background, and filter out obvious error information to obtain the preprocessed oral tooth image data set;

[0014] S23, Extract the useful features of the oral tooth images in the oral tooth image data set, including the number of teeth, the tooth alignment, the tooth spacing, and the gum information. In the present invention, for the oral tooth image data set, it is adjusted and screened. The segmented images need to be adjusted in size and pixels to facilitate subsequent data processing.

[0015] Preferably, the step S3 includes the following steps:

[0016] S31, the original federated learning model in the cloud total server is sent to the local server;

[0017] S32, based on the original federated learning model in the cloud total server, combined with the oral tooth images of the local server for training, and a single local federated learning model is established. In the present invention, the learning model sent by the cloud total server is the initial model. Subsequently, after being sent to the local server, the local server trains the initial model with the continuously input preprocessed oral tooth image set to obtain new model parameters.

[0018] Preferably, in the step S32, it specifically includes obtaining the oral tooth image set of the local server and using the oral tooth image set of the local server as a sample for training. After the federated learning model recognizes the images in the oral tooth image set to reach the specified value, finally, the model parameters are generated and stored in the local federated learning model. In the present invention, the preprocessed oral tooth image set is transmitted to the graph convolutional layer, then the redundant features are processed by the graph pooling layer, and finally, the local server federated learning model is output through the fully connected layer.

[0019] Preferably, the step S4 includes the following steps:

[0020] S41, the model parameters are also output in the form of an image data set, and the output model parameters are mapped to binary data for representation. Each tooth forms an independent data matrix An, and the output oral tooth images are arranged in order to form a 6*6 matrix about An, and the insufficient positions are temporarily vacant;

[0021] S42, if the number of teeth is odd and the number of zeros filled is also odd, the entire 6*6 matrix is rearranged to make the 6*6 matrix complete. For each column containing the insufficient position with a temporary vacancy, zeros are filled. The zero-filling positions are set in order after the An corresponding to the maximum rank of An in each column of the 6*6 matrix, and the An below the zero-filling position is moved down in order; if there are two insufficient positions in a column, the zero-filling position is set after the An corresponding to the second largest value of the rank of An in that column; if there are multiple maximum ranks of An in a column, the An in the last row of that column is taken;

[0022] S43, if the number of teeth is even, the zero-filling positions are also rearranged. The zero-filling positions are set in order before the An corresponding to the minimum rank of An in each column of the 6*6 matrix, and the An below is moved down; if there are two insufficient positions in a column, the zero-filling position is set before the An corresponding to the second smallest value of the rank of An in that column; if there are multiple minimum ranks of An in a column, the An in the first row of that column is taken;

[0023] S44, record the zero-padding positions for each item and form a zero-padding matrix. Subsequently, invert the zero-padding matrix to form an encryption matrix, and send it to the cloud server in the form of the encryption matrix. In the present invention, the model parameters are converted and mapped into the form of binary data. Taking a single tooth image as a unit matrix An, the entire oral tooth image of the patient user, that is, all tooth images form a 6*6 matrix with respect to An. Insert the zero-padding positions according to the parity of the number of teeth of the patient user and then invert it; the model parameters of the patient user are protected, and to a certain extent, the order of the tooth images is disrupted, but the data inside the matrix An of a single tooth image is not damaged, and the traceability is high.

[0024] Preferably, the step S5 includes the following steps:

[0025] S51, the cloud server decrypts the digital model parameters and distributes them.

[0026] S52, the local server combines the output result of the local federated learning model and the model parameters distributed by the cloud server for fusion comparison, and classifies the dental diseases according to the fusion comparison result. In the present invention, after receiving the model parameters sent by the local server, the cloud server decrypts and distributes the model parameters to each local server. The decryption step is opposite to the process of step S4. The model parameters of the cloud server received by each local server include the model parameters of each other local server, and the data range is larger.

[0027] Preferably, the step S52 includes the following steps:

[0028] S521, sort out the model parameters distributed by the cloud server. First, traverse each model parameter, divide them into large categories according to the number of teeth, randomly insert interference color blocks into the corresponding model parameters. Different colors of interference color blocks represent different types of dental diseases. The insertion positions of the same interference color block are determined by the occurrence positions of the dental diseases. The randomly inserted positions of the interference color blocks are recorded in the color block memory in the local server.

[0029] S522, compare the similarity between the output result of the local federated learning model and the model parameters distributed by the cloud server. Traverse to find the model parameters distributed by the cloud server that are close, and determine the type of dental disease according to the color and insertion position of the interference color block. In the present invention, during the fusion comparison process, first sort out the model parameters of the cloud server, then divide them, and then insert interference color blocks, and judge the type of dental disease according to the color and position of the interference color block.

[0030] The beneficial effects of the present invention are as follows: In the method of the present invention, by collecting the oral information of each user, the oral tooth images are transmitted to the local server, trained according to the oral tooth images and the model issued by the cloud general server, and the trained model parameters are encrypted and sent to the cloud general server. The cloud general server distributes the model parameters to each local server for fusion and comparison; in the method of the present invention, the classification is accurate, there is no need to centrally process the data, the oral information is in a strictly confidential state, and there is no privacy problem; and the model parameters are encrypted during the process to further ensure security. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flowchart of the present invention;

[0032] Figure 2 is a schematic diagram of the local federated learning model of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] Embodiment:

[0034] This embodiment proposes a tooth disease classification method based on federated learning, referring to Figure 1 , which specifically includes the following multiple steps. Step S1, obtain the oral information of the user; specifically, the oral information mainly includes oral tooth images and the number of teeth. Perform image segmentation on the oral tooth images, specifically segment according to the number of teeth. The segmented oral tooth images are sequentially arranged to form an oral tooth image data set, specifically arranged in the order of the tooth number. The oral tooth image data set can be temporarily stored at the acquisition end.

[0035] Step S2, transmit the oral tooth images to the local server and perform preprocessing; specifically, the oral tooth images are sent from the acquisition end to the local server; specifically, it includes the following multiple steps. Step S21, upload the oral tooth image data set to the local server; the upload method is not limited here.

[0036] Step S22, adjust each oral tooth image to the same pixel and the same size, segment the target and the background, and filter out obvious error information to obtain the preprocessed oral tooth image data set; during the preprocessing process, standardize the oral tooth images and filter out some obvious error information.

[0037] Step S23, extract the useful features of the oral tooth images in the oral tooth image data set, including the number of teeth, tooth alignment, tooth spacing, and gum information.

[0038] Step S3, a single local server performs model training according to the preprocessed oral tooth image set and the model of the cloud total server; specifically, the model of the cloud total server is an initial model, which is sent by the cloud total server, and specifically includes the following two steps. Step S31, first, the cloud total server sends the original learning model therein to the local server. Specifically, the model can also be updated according to the actual needs of a single local server.

[0039] Step S32, the original learning model in the cloud total server is combined with the preprocessed oral tooth images of the local server for model training to establish a separate local federated learning model. Specifically, in Step S32, the oral tooth image set of the local server is obtained and used as a sample for training. After the federated learning model recognizes the images in the oral tooth image set to reach the specified value, the model parameters are finally generated and stored in the local federated learning model. In the present invention, the preprocessed oral tooth image set is transmitted to the graph convolutional layer, and then the redundant features are processed by the graph pooling layer, and finally output to the local server federated learning model through the fully connected layer.

[0040] Reference Figure 2 , for the local federated learning model, it includes a graph convolutional layer, a graph pooling layer and a fully connected layer. The graph convolutional layer is mainly used for feature extraction of oral tooth images in the oral tooth image dataset, and can replace the CNN and have a good effect. Specifically as follows:

[0041]

[0042] Among them, the superscript l represents the number of layers, is the identity matrix added on the basis of A; is the degree matrix, w l is the trainable weight parameter, h l represents the input image features;

[0043] After obtaining a large number of image features after the graph convolutional layer, a pooling operation is required, that is, in the graph pooling layer; in the fully connected layer, the features after the graph convolutional layer and the graph pooling layer are cross-layer fused and the dimensions are changed, and finally output to the local server federated learning model. Specifically as follows:

[0044]

[0045] Among them, N l represents the number of nodes, MAX(·) represents the max pooling operation, V fcIt is the feature finally input into the fully connected layer. Different from traditional CNNs, which perform a single average pooling or max pooling operation before inputting the features extracted by convolution into the fully connected layer, in the present invention, the results of the two poolings are concatenated (the || symbol represents the concatenation operation). First, the features obtained after each layer of graph convolution and graph pooling operations are respectively subjected to average pooling and max pooling, and then the two results are concatenated. After these steps, the results obtained from each layer are accumulated to achieve the effect of cross-layer fusion.

[0046] Step S4: Transmit the trained model parameters to the cloud central server through digital encryption means; specifically, ensure the security of the entire data transmission through certain encryption means. Specifically, it includes the following multiple steps. Step S41: The trained model parameters are transmitted in the form of an image data set. The model parameters to be transmitted are mapped into binary data for representation. Each tooth constitutes an independent data matrix, denoted as An, and the oral tooth images are arranged in a matrix in sequence. The matrix is specifically a 6*6 matrix regarding An, and the insufficient positions are temporarily vacant.

[0047] Step S42: When the number of teeth is odd, the number of filled 0s is also odd. Rearrange the entire 6*6 matrix to fill the 6*6 matrix. For each column with temporarily vacant insufficient positions, 0s are filled. The filling positions of 0s are set in sequence behind the An corresponding to the maximum rank of An in each column of the 6*6 matrix, and the An below the filling position of 0s is shifted downwards in sequence; if there are two insufficient positions in a column, then the filling position of 0s is set behind the An corresponding to the second largest value of the rank of An in this column; if there are multiple maximum ranks of An in a column, then take the An in the last row of this column.

[0048] Step S43: When the number of teeth is even, also perform rearrangement of the filling positions of 0s. The filling positions of 0s are set in sequence in front of the An corresponding to the minimum rank of An in each column of the 6*6 matrix, and the An below is shifted downwards; if there are two insufficient positions in a column, then the filling position of 0s is set in front of the An corresponding to the second smallest value of the rank of An in this column; if there are multiple minimum ranks of An in a column, then take the An in the frontmost row of this column.

[0049] Step S44: Record each filling position of 0s, and at the same time form a matrix of filled 0s. Then invert the matrix of filled 0s to form an encryption matrix, and send it to the cloud central server in the form of the encryption matrix.

[0050] Step S5: Each local server performs fusion and classification according to the model parameters decrypted and distributed by the cloud central server numerically; specifically, the cloud central server distributes them to the corresponding local server according to the actual requirements of each local server; it includes the following two steps. Step S51: First, the cloud central server distributes the model parameters after numerical decryption; the specific decryption process is the opposite of encryption.

[0051] Step S52: Subsequently, the local server comprehensively fuses and compares the output results of the local federated learning model and the model parameters distributed by the cloud central server, and classifies dental diseases according to the fusion and comparison results. Specifically, this step also includes the following two steps. Step S521: Sort out the data of the model parameters distributed by the cloud central server. First, traverse each model parameter, divide it into large categories according to the number of teeth, and then randomly insert interference color blocks into the corresponding model parameters. Different colors of interference color blocks represent different types of dental diseases, and the insertion positions of the same interference color block are determined by the occurrence positions of dental diseases. The randomly inserted positions of the interference color blocks are recorded in the color block memory in the local server.

[0052] Step S522: Compare the similarity between the output results of the local federated learning model and the model parameters distributed by the cloud central server, find the model parameters distributed by the cloud central server that are the closest, and determine the type of dental disease from the color and insertion position of the interference color block.

[0053] For the final classification of dental diseases, it relies on similarity comparison. The similarity comparison specifically uses the method of double comparison of images and data. After traversing and finding the closest cloud central server, the histogram similarity is used for image comparison. If the similarity is less than 0.5%, the comparison is successful, and then data comparison is carried out. If the data comparison is less than 1%, it is regarded as a successful comparison. After the comparison is successful, obtain the color and position of the interference color block of the model parameter. The color of the interference color block represents one of the dental diseases, such as dental caries, non-carious diseases of dental hard tissues, and infectious diseases, etc. The smaller the color difference, the more similar the type of dental disease; the insertion position of the interference color block represents the occurrence position of the dental disease, which can be specific to a single tooth.

[0054] In the present invention, the oral information of the patient user is obtained through the acquisition terminal. The acquisition terminal can not only obtain the oral information, but also perform preliminary segmentation processing on the acquired oral tooth images; the segmented oral tooth images are preprocessed to remove some images that do not meet the requirements; the preprocessed images are used as training samples and trained in the model sent by the cloud general server; after training, the model parameters are uploaded to the cloud general server by means of digital encryption; finally, the cloud general server decrypts the digital model parameters and distributes them to each local server for fusion comparison, and classifies dental diseases according to the results of the fusion comparison; the method of the present invention protects the privacy of patient users while utilizing big data.

[0055] In the present invention, the number of teeth of each patient user is different. The oral tooth images are segmented into individual tooth images and numbered. After numbering, they are arranged in order to form an oral tooth image data set, which is convenient for subsequent image processing.

[0056] In the present invention, for the oral tooth image data set, it is adjusted and screened. The segmented images need to be adjusted in size and pixels to facilitate subsequent data processing. The extracted features are used as reference information in subsequent fusion comparison.

[0057] In the present invention, the learning model sent by the cloud general server is an initial model. Subsequently, after being sent to the local server, the local server trains the initial model with the continuously input preprocessed oral tooth image sets to obtain new model parameters.

[0058] In the present invention, the model parameters are converted into the form of binary data, and a single tooth image is used as a unit matrix An. The entire oral tooth image of the patient user, that is, all tooth images form a 6*6 matrix about An. Zeros are inserted according to the parity of the number of teeth of the patient user, and it is inverted; the model parameters of the patient user are protected, and to a certain extent, the order of the tooth images is disrupted, but the data inside the matrix An of a single tooth image is not damaged, and the traceability is high.

[0059] In the present invention, after receiving the model parameters sent by the local server, the cloud general server decrypts and distributes the model parameters to each local server. The decryption step is opposite to the process of step S4. The model parameters of the cloud general server received by each local server include the model parameters of each other local server, and the data range is larger.

[0060] In the present invention, during the fusion comparison process, first, the model parameters of the cloud general server are sorted out, then divided, and then interference color blocks are inserted. The type of dental disease is judged according to the color and position of the interference color blocks.

[0061] The above embodiments are further elaborations and explanations of the present invention for easy understanding, and are not any limitations on the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A tooth disease classification method based on federated learning, characterized in that, Including the following steps: S1. Obtain the oral information of the user; S2. Transmit the oral tooth images to the local server for preprocessing; S3. A single local server performs model training according to the preprocessed oral tooth image set and the model of the cloud total server; S4. Transmit the trained model parameters to the cloud total server by means of digital encryption; S5. Each local server performs fusion and classification according to the model parameters decrypted and distributed by the cloud total server; S5 includes: S51. The cloud total server decrypts and distributes the model parameters digitally; S52. The local server combines the output results of the local federated learning model and the model parameters distributed by the cloud total server for fusion comparison, and classifies dental diseases according to the fusion comparison results; S52 includes: organizing the data of the model parameters distributed by the cloud total server, randomly inserting interference color blocks into the corresponding model parameters, and the insertion positions of the same interference color blocks are determined by the occurrence positions of dental diseases; comparing the similarity between the output results of the local federated learning model and the model parameters distributed by the cloud total server, traversing to find the model parameters distributed by the cloud total server that are close, and determining the type of dental disease according to the color and insertion position of the interference color block.

2. The method for classifying dental diseases based on federated learning according to claim 1, wherein, The oral information in step S1 includes oral tooth images and the number of teeth. The oral tooth images are subjected to image segmentation processing according to the number of teeth. The oral tooth images after image segmentation processing form an oral tooth image data set in the order of tooth numbering and are temporarily stored in the acquisition end.

3. A method for classifying dental diseases based on federated learning according to claim 1 or 2, characterized in that, Step S2 includes the following steps: S21. Upload the oral tooth image data set to the local server; S22. Adjust each oral tooth image to the same pixel and the same size, segment the target from the background, and filter out obvious error information to obtain the preprocessed oral tooth image data set; S23. Extract the useful features of the oral tooth images in the oral tooth image data set, including the number of teeth, tooth alignment, tooth spacing, and gum information.

4. A method for classifying dental diseases based on federated learning according to claim 1 or 2, characterized in that, Step S3 includes the following steps: S31. The original federated learning model in the cloud total server is sent to the local server; S32. Based on the original federated learning model in the cloud total server, combine with the oral tooth images of the local server for training to establish a single local federated learning model.

5. A method for classifying dental diseases based on federated learning according to claim 4, characterized in that, Specifically in step S32, it includes obtaining the oral tooth image set of the local server, using the oral tooth image set of the local server as a sample for training. After the federated learning model recognizes the images in the oral tooth image set to reach the specified value, finally generate model parameters and store them in the local federated learning model.

6. A method for classifying dental diseases based on federated learning according to claim 4, characterized in that, Step S4 includes the following steps: S41. Output the model parameters in the form of an image data set, map the output model parameters to binary data for representation, each tooth forms an independent data matrix An, and the output oral tooth images are arranged in order to form a 6*6 matrix about An, and the insufficient bits are temporarily vacant; S42. If the number of teeth is odd and the number of zeros to be filled is also odd, rearrange the entire 6×6 matrix to make it complete. For each column with insufficient positions (temporarily vacant), fill in zeros. The positions to fill in zeros are sequentially set after the An with the maximum rank in each column of the 6×6 matrix. The An below the zero-filling positions is shifted down sequentially. If there are two insufficient positions in a column, the zero-filling position is set after the An corresponding to the second-largest rank value of An in that column. If there are multiple An with the maximum rank in a column, take the An in the last row of that column. S43. If the number of teeth is even, also rearrange the zero-filling positions. The zero-filling positions are sequentially set before the An corresponding to the minimum rank of An in each column of the 6×6 matrix, and the An below is shifted down. If there are two insufficient positions in a column, the zero-filling position is set before the An corresponding to the second-smallest rank value of An in that column. If there are multiple An with the minimum rank in a column, take the An in the frontmost row of that column. S44. Record the positions of each zero filling and form a zero-filling matrix. Then invert the zero-filling matrix to form an encryption matrix, and send it to the cloud server in the form of the encryption matrix.

7. A method for classifying dental diseases based on federated learning according to claim 1, characterized in that, The step S52 includes the following steps: S521. Organize the data of the model parameters sent by the cloud server. First, traverse each model parameter, classify them into major categories according to the number of teeth, randomly insert interference color blocks into the corresponding model parameters. Different colors of interference color blocks represent different types of dental diseases. The insertion positions of the same interference color block are determined by the occurrence positions of dental diseases, and the randomly inserted positions of the interference color blocks are recorded in the color block memory in the local server. S522. Compare the similarity between the output result of the local federated learning model and the model parameters sent by the cloud server. Traverse to find the model parameters sent by the cloud server that are close, and determine the type of dental disease according to the color and insertion position of the interference color block.

Images