Method, system, computer equipment and medium for automatic recognition of oral pathology images

Abstract

The present invention provides an oral pathology image automatic recognition method, system, computer equipment and medium. By acquiring oral pathology tissue slide microscopic images, an oral pathology image data set is established, and after dividing the data set into a training set and a test set according to a preset ratio, a first preset number of deep neural network models are trained according to the training set to obtain candidate feature extraction models, and then each candidate feature extraction model is tested according to the test set to screen and obtain a second preset number of feature extraction models for feature extraction of the oral pathology image to be recognized. After obtaining the image features to be optimized, a nearest component analysis method is used to perform dimensionality reduction optimization and input the features into a preset integrated classifier for predictive recognition to obtain the recognition result. The method not only effectively solves the problems of high recognition training cost and low learning efficiency in the prior art, but also improves the recognition accuracy, and also provides a reliable basis for oral tumor biological behavior evaluation, individualized treatment plan formulation and efficacy monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of biomedical image processing, and in particular to a method, system, computer equipment and storage medium for automatic recognition of oral pathology images based on hybrid machine learning. Background Art

[0002] Oral cancer refers to malignant tumors that occur in the oral cavity. Most of them are squamous cell carcinomas, which are so-called mucosal mutations. It is one of the more common malignant tumors in the head and neck. In clinical practice, it includes gingival cancer, tongue cancer, soft and hard palate cancer, jaw cancer, floor of mouth cancer, oropharyngeal cancer, salivary gland cancer, lip cancer, maxillary sinus cancer, and cancers that occur in the skin and mucous membranes of the face. The existing oral cancer treatment plan generally uses a single treatment method of surgery or radiotherapy for patients with early oral malignancies, while for advanced patients, combined treatment is generally required. It can be seen that the typing, grading and staging of oral tumors are not only important indicators for evaluating the biological behavior of tumors, but also have an important impact on guiding the formulation of individualized treatment plans and monitoring the efficacy.

[0003] The medical images obtained through various imaging methods not only play an important role in the precise design of surgical procedures and preoperative simulation, but also are the main basis for qualitative and quantitative evaluation of tumors. They have gradually become the basis for many scholars to improve the classification, grading and staging of oral tumors. However, most of the existing automatic recognition methods of medical images used in tumor research consider that different imaging methods have different advantages and disadvantages and indications, and will obtain different levels of diagnostic information. Based on multimodal oral tumor data, deep learning fusion, training and prediction are performed to mine oral tumor disease characteristics to achieve automatic classification, grading and staging based on medical images. However, the application of this method not only requires the acquisition of a large amount of multimodal data, which increases the learning cost, but also requires the processing of a large amount of data during the training process, resulting in low learning efficiency.

[0004] Therefore, there is an urgent need to provide an automatic recognition method for oral pathology images that can effectively reduce data collection and learning costs and effectively improve learning efficiency and prediction recognition accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide a method, system, computer device and storage medium for automatic recognition of oral pathology images. By using multiple pre-trained deep convolutional neural network models screened based on prediction accuracy and computational efficiency as the basic architecture, different features of the oral pathology images to be recognized are extracted respectively, and then the nearest component analysis optimization technology is used to select important features from multiple groups of different image features and input them into an integrated classifier to obtain an automatic recognition result. The method effectively solves the defects of high recognition training cost and low learning efficiency in the prior art, while further improving the accuracy of the recognition results, and providing a reliable basis for the evaluation of the biological behavior of oral tumors, the formulation of individualized treatment plans and the monitoring of efficacy.

[0006] In order to achieve the above objectives, it is necessary to provide a method, system, computer device and storage medium for automatic recognition of oral pathology images in response to the above technical problems.

[0007] In a first aspect, an embodiment of the present invention provides a method for automatically identifying oral pathology images, the method comprising the following steps:

[0008] Obtain microscopic images of oral pathological tissue slides;

[0009] Establishing an oral pathology image dataset based on the oral pathology tissue slide microscopic image, and dividing the oral pathology image dataset into a training set and a test set according to a preset ratio;

[0010] A first preset number of deep neural network models are trained according to the training set to obtain a first preset number of candidate feature extraction models, and each candidate feature extraction model is tested according to the test set to screen and obtain a second preset number of feature extraction models; the second preset number is less than the first preset number;

[0011] According to each feature extraction model, feature extraction is performed on the oral pathology image to be identified, and corresponding image features to be optimized are obtained respectively;

[0012] Adopting the nearest component analysis method to perform dimension reduction optimization on the image features to be optimized, so as to obtain the image features to be identified;

[0013] The image features to be identified are input into a preset integrated classifier for predictive identification to obtain a corresponding identification result; the integrated classifier is obtained by integrating a third preset number of machine learning methods.

[0014] Furthermore, the first preset number of deep neural network models includes a VGG network model, a Resnet network model, a Densenet network model, an Inception network model, a Mobilenet network model and an Inception-Resnet network model.

[0015] Furthermore, the step of testing each candidate feature extraction model according to the test set to screen out a second preset number of feature extraction models includes:

[0016] Testing each candidate feature extraction model according to the test set to obtain the highest accuracy and operation efficiency corresponding to each candidate feature extraction model;

[0017] The highest accuracy and computational efficiency corresponding to each candidate feature extraction model are comprehensively ranked, and a second preset number of feature extraction models are screened out based on the ranking results.

[0018] Furthermore, the highest precision is the highest accuracy of the corresponding prediction; the computational efficiency is expressed as:

[0019] q=t / n

[0020] Among them, q represents the computational efficiency of the candidate feature extraction model; t represents the time taken to train the candidate feature extraction model under the same parameters; and n represents the number of parameters of the candidate feature extraction model.

[0021] Furthermore, the step of comprehensively ranking the highest accuracy and computational efficiency corresponding to each candidate feature extraction model, and screening a second preset number of feature extraction models according to the ranking result includes:

[0022] The highest accuracy and operation efficiency corresponding to each candidate feature extraction model are weighted averaged according to the preset weight coefficient to obtain the model screening index; the model screening index is expressed as:

[0023] C=ω1·α+ω2·q

[0024] Where C represents the model screening index of the candidate feature extraction model; α and q represent the highest accuracy and operation efficiency of the candidate feature extraction model respectively; ω1 and ω2 represent the weight coefficients corresponding to α and q respectively;

[0025] The model screening indicators of each candidate feature extraction model are arranged in descending order, and a second preset number of candidate feature extraction models are screened out from top to bottom as the feature extraction models.

[0026] Furthermore, the preset integrated classifier includes a discriminant analysis method, a decision tree method, a K nearest neighbor method, a naive Bayes method, a subspace integration method and a support vector machine method.

[0027] Furthermore, the step of inputting the image features to be identified into a preset integrated classifier for predictive identification to obtain a corresponding identification result includes:

[0028] Presetting the classification categories of the preset integrated classifier, training each machine learning method, and determining the category matching relationship between each classification category and each machine learning method according to the corresponding training results and preset matching rules;

[0029] Using each machine learning method in the integrated classifier to predict and identify the image features to be identified, and obtain corresponding candidate identification results; the candidate identification results include the prediction probabilities corresponding to each classification category;

[0030] The recognition result of the image feature to be recognized is obtained by combining the candidate recognition results corresponding to each machine learning method and the category matching relationship.

[0031] In a second aspect, an embodiment of the present invention provides an oral pathology image automatic recognition system, the system comprising:

[0032] An image acquisition module, used to obtain microscopic images of oral pathological tissue slides;

[0033] A data set construction module, used to establish an oral pathology image data set based on the oral pathology tissue slide microscopic image, and divide the oral pathology image data set into a training set and a test set according to a preset ratio;

[0034] A model screening module, used to train a first preset number of deep neural network models according to the training set to obtain a first preset number of candidate feature extraction models, and to test each candidate feature extraction model according to the test set to screen and obtain a second preset number of feature extraction models; the second preset number is less than the first preset number;

[0035] A feature extraction module, used to extract features of the oral pathology image to be identified according to each feature extraction model, and obtain corresponding image features to be optimized;

[0036] A feature optimization module, used for performing dimension reduction optimization on the image features to be optimized by using a nearest component analysis method to obtain image features to be identified;

[0037] The image recognition module is used to input the image features to be recognized into a preset integrated classifier for predictive recognition to obtain corresponding recognition results; the integrated classifier is obtained by integrating a third preset number of machine learning methods.

[0038] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0040] The above application provides a method, system, computer device and storage medium for automatic recognition of oral pathology images. Through the method, multiple pre-trained deep convolutional neural network models selected based on prediction accuracy and computational efficiency are used as the basic architecture to extract multiple groups of different image features of the oral pathology images to be recognized, and then the nearest component analysis optimization technology is used to select important features from multiple groups of different image features and input them into the integrated classifier to obtain the corresponding automatic recognition results. Compared with the prior art, the method for automatic recognition of oral pathology images not only effectively solves the problems of high recognition training cost and low learning efficiency of the prior art, but also further improves the accuracy of the recognition results, and also provides a reliable basis for the evaluation of the biological behavior of oral tumors, the formulation of individualized treatment plans and the monitoring of efficacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of an application scenario of the method for automatic recognition of oral pathology images in an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of the model architecture of the method for automatic recognition of oral pathology images in an embodiment of the present invention;

[0043] Figure 3 is a schematic flow chart of a method for automatic recognition of oral pathology images in an embodiment of the present invention;

[0044] Figure 4 yes Figure 3 Schematic diagram of the process of screening feature extraction models in step S13;

[0045] Figure 5 yes Figure 3 The schematic diagram of the process of using a preset integrated classifier to perform recognition prediction on the image features to be recognized in step S16;

[0046] Figure 6 is a schematic diagram of the structure of an automatic recognition system for oral pathology images in an embodiment of the present invention;

[0047] Figure 7 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The oral pathology image automatic recognition method provided by the present invention can be applied to Figure 1 The terminal or server shown in the figure. The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server may be implemented as an independent server or a server cluster composed of multiple servers. The server adopts the oral pathology image automatic recognition method provided by the present invention, according to the following example. Figure 2 The process architecture of the hybrid integrated model of the deep convolutional neural network model + the nearest component analysis NCA method + the machine learning method is shown to perform efficient prediction analysis on any oral pathology image to be identified, and obtain accurate and effective image prediction and recognition results, which are used for subsequent learning tasks of the server, or transmitted to the terminal for viewing and use by the terminal user, providing a reliable basis for the formulation of individualized treatment plans and efficacy monitoring. The following is a detailed description of the oral pathology image automatic recognition method of the present invention through examples.

[0050] In one embodiment, Figure 3 As shown, a method for automatic recognition of oral pathology images is provided, the method comprising the following steps:

[0051] S11, obtaining oral pathological tissue slide microscopic images; wherein the oral pathological tissue slide microscopic images can be collected according to actual application requirements, and may include different types of pathological tissue slide microscopic images such as gingival cancer, tongue cancer, soft and hard palate cancer, jaw cancer, floor of mouth cancer, oropharyngeal cancer, salivary gland cancer, lip cancer, and maxillary sinus cancer;

[0052] S12. According to the oral pathological tissue slide microscopic image, an oral pathological image dataset is established, and the oral pathological image dataset is divided into a training set and a test set according to a preset ratio: wherein the oral pathological image dataset is constructed based on the microscopic image obtained after preprocessing such as cropping, denoising, and region of interest screening on the oral pathological tissue slide microscopic image obtained above. Correspondingly, the cropping size, denoising method, and region of interest screening method can be determined according to actual application requirements. For example, according to the recognition application requirements, the original oral pathological tissue slide microscopic image can be cropped to a size of 512 pixels × 512 pixels, and then NL-Means or BM3D is used to perform image denoising, and then grayscale binarization is performed to determine the microscopic image of the region of interest, and the corresponding dataset is constructed using it; at the same time, the preset ratio for dividing the oral pathological image dataset can also be selected according to actual needs, such as the oral pathological image dataset can be divided into a training set and a test set according to 8:2; it should be noted that the above-mentioned method for preprocessing the oral pathological tissue slide microscopic image and the division ratio of the dataset are only exemplary descriptions and do not limit the specific protection scope;

[0053] S13. Train a first preset number of deep neural network models according to the training set to obtain a first preset number of candidate feature extraction models, and test each candidate feature extraction model according to the test set to screen and obtain a second preset number of feature extraction models; the second preset number is less than the first preset number; wherein the first preset number of deep neural network models includes a VGG network model, a Resnet network model, a Densenet network model, an Inception network model, a Mobilenet network model and an Inception-Resnet network model, but more deep neural network models can also be introduced according to actual application requirements; after selecting the first preset number of deep neural network models, it is necessary to use the obtained training set to train each deep neural network model Training is performed to obtain a corresponding stable candidate feature extraction model, and in order to ensure that reliable and effective image features can be extracted and the accuracy of subsequent image recognition results is ensured, this embodiment is further based on the highest accuracy and computational efficiency corresponding to each candidate feature extraction model collected through training, and a second preset number of feature extraction models that can be used for feature extraction are screened out; it should be noted that the above-mentioned first preset number and second preset number can be determined according to actual application requirements, and it is only necessary to meet the basic principle that the second preset number is less than the first preset number. If it is determined that the first preset number corresponding to the deep neural network model is 6 and the second preset number is 3, then 3 feature extraction models are screened out from the 6 candidate feature extraction models obtained through corresponding training for subsequent feature extraction. In addition, this is only an exemplary description and does not specifically limit the scope of protection;

[0054] Specifically, Figure 4 As shown, the step S13 of testing each candidate feature extraction model according to the test set to screen out a second preset number of feature extraction models includes:

[0055] S131, testing each candidate feature extraction model according to the test set to obtain the highest accuracy and operation efficiency corresponding to each candidate feature extraction model; wherein the highest accuracy is the highest accuracy rate of the corresponding prediction, which can be understood as the highest accuracy rate collected during the model training process; the operation efficiency is expressed as:

[0056] q=t / n

[0057] Where q represents the operating efficiency of the candidate feature extraction model; t represents the time taken to train the candidate feature extraction model under the same parameters; n represents the number of parameters of the candidate feature extraction model;

[0058] S132, comprehensively sorting the highest accuracy and operation efficiency corresponding to each candidate feature extraction model, and screening to obtain a second preset number of feature extraction models according to the sorting results;

[0059] Specifically, the step of comprehensively ranking the highest accuracy and computational efficiency corresponding to each candidate feature extraction model, and screening to obtain a second preset number of feature extraction models according to the ranking results includes:

[0060] The highest accuracy and operation efficiency corresponding to each candidate feature extraction model are weighted averaged according to the preset weight coefficient to obtain the model screening index; the model screening index is expressed as:

[0061] C=ω1·α+ω2·q

[0062] Among them, C represents the model screening index of the candidate feature extraction model; α and q represent the highest accuracy and operating efficiency of the candidate feature extraction model respectively; ω1 and ω2 represent the weight coefficients corresponding to α and q respectively; correspondingly, ω1 and ω2 can also be determined according to the actual application requirements, and it is only necessary to satisfy ω1+ω2=1. When the model is screened, the one that attaches more importance to the highest accuracy and operating efficiency will be given a higher weight. For example, the model screening index can be determined as C=0.7α+0.3q according to the requirements. If you want to reflect the importance of prediction accuracy in model screening, you can further increase the value of ω1 and adjust it to C=0.8α+0.2q. On the contrary, if you pay attention to the operation efficiency, you can also increase the value of ω1. There is no specific restriction here.

[0063] The model screening indexes of each candidate feature extraction model are arranged in descending order, and a second preset number of candidate feature extraction models are screened out from top to bottom as the feature extraction models; wherein, the number of feature extraction models can be determined according to actual needs as described above, that is, after the model screening index values ​​corresponding to each candidate feature extraction model are calculated according to the above method, all the model screening indexes are sorted, and feature extraction models with higher model screening indexes are screened out according to the second preset number; it should be noted that the model screening indexes are arranged in ascending order, and the corresponding bottom-up screening can also achieve the corresponding effect;

[0064] S14, according to each feature extraction model, feature extraction is performed on the oral pathology image to be identified, and corresponding image features to be optimized are obtained respectively; wherein, the oral pathology image to be identified can be any image in the above-mentioned oral pathology image data set, or can be any image to be identified obtained by preprocessing any acquired oral pathology tissue slide microscopic image outside the oral pathology image data set; the method of extracting features from the oral pathology image to be identified using each feature extraction model can be implemented using the corresponding existing technology, and no specific limitation is made here; it should be noted that each feature extraction model will extract a set of image features for the oral pathology image to be identified, that is, the image features to be optimized extracted through this step also include the second preset number of image features, which effectively ensures the comprehensiveness and effectiveness of feature extraction and provides guarantee for subsequent accurate prediction; in addition, in principle, all the image features to be optimized obtained above can be directly used for subsequent image prediction and recognition, but in order to ensure the accuracy of the prediction result while also improving the prediction efficiency, this embodiment preferably adopts the following step S15 of neighbor component analysis (Neighbourhood Component The NCA optimization method is used to reduce the dimension of the image features to be optimized. By reducing the size of the feature map, the training speed and test speed of the model are improved.

[0065] S15, using the nearest component analysis method to perform dimensionality reduction optimization on the image features to be optimized, to obtain the image features to be identified;

[0066] S16, inputting the image features to be identified into a preset integrated classifier for prediction and identification, and obtaining a corresponding identification result; the integrated classifier is obtained by integrating a third preset number of machine learning methods; wherein the third preset number can be determined according to actual application requirements. In order to further improve the accuracy of image recognition prediction, this embodiment does not use a single machine learning method to analyze and predict the above-extracted image features to be identified, but preferably uses an integrated classifier that includes multiple different machine learning methods at the same time. The corresponding preset integrated classifier includes but is not limited to a discriminant analysis method, a decision tree method, a K nearest neighbor method, a naive Bayes method, a subspace integration method, and a support vector machine method;

[0067] Specifically, Figure 5 As shown, the step S16 of inputting the image features to be identified into a preset integrated classifier for prediction and identification to obtain the corresponding identification result includes:

[0068] S161, presetting the classification category of the preset integrated classifier, and training each machine learning method, and determining the category matching relationship between each classification category and each machine learning method according to the corresponding training results and the preset matching rules; wherein the classification category of the preset integrated classifier is specifically determined according to the type of pathological tissue slide microscopic image that may be input, such as when the oral pathological image to be identified may be any type of pathological tissue slide microscopic image such as gingival cancer, tongue cancer, soft and hard palate cancer, jaw cancer, floor of mouth cancer, oropharyngeal cancer, salivary gland cancer, lip cancer and maxillary sinus cancer, the corresponding classification category can be set to the corresponding type; at the same time, each machine learning method in the preset integrated classifier can be understood as a stable model obtained by pre-training with all types of pathological tissue slide microscopic image data sets corresponding to the classification category, and according to the preset matching rules, the required matching rule judgment information is collected in the training process of various types of pathological tissue slide microscopic image data sets using each machine learning method, and the category matching relationship between each classification category and each machine learning method is determined according to the collected matching rule judgment information;

[0069] Specifically, if the preset matching rule is to select the classification category with the highest prediction accuracy of each machine learning method, and establish a corresponding category matching relationship between it and the classification category with the highest prediction accuracy, then during the training process, the prediction accuracy of each machine learning method for various types of pathological tissue slide microscopic image data sets is recorded, and a corresponding category matching relationship is established between each machine learning method and the classification category with the highest prediction accuracy. For example, after training with the discriminant analysis method based on the above-mentioned 9 types of pathological tissue slide microscopic image data sets, the prediction accuracy corresponding to the 9 types of diseases is recorded, and the oropharyngeal cancer classification category corresponding to the disease with the highest prediction accuracy is selected from them to establish a matching relationship with the discriminant analysis method. When predicting and identifying any oral pathology image to be identified subsequently, the discriminant analysis method is used to predict the probability of the oropharyngeal cancer classification category, which is used as the prediction probability of the oral pathology image to be identified as the oropharyngeal cancer classification category by the integrated classifier. Similarly, the corresponding category matching relationship of other machine learning methods can be obtained similarly; it should be noted that the above-mentioned discriminant analysis method and the oropharyngeal cancer classification category are used to establish a category matching relationship for exemplary purposes only and do not limit the specific protection scope. The establishment of the actual category matching relationship needs to be based on the corresponding training results and preset matching rules. A method similar to the confusion matrix can be used to screen out a machine learning method that meets the preset matching rules for each classification category of each integrated classifier.

[0070] S162, respectively using each machine learning method in the integrated classifier to predict and identify the image features to be identified, and obtain corresponding candidate identification results; the candidate identification results include the prediction probabilities corresponding to each classification category, that is, each machine learning method predicts and identifies the same group of image features to be identified, and will obtain a corresponding group of prediction probabilities corresponding to each classification category. For example, when the classification categories of the integrated classifier are 9 types, namely, gingival cancer, tongue cancer, soft and hard palate cancer, jaw cancer, floor of mouth cancer, oropharyngeal cancer, salivary gland cancer, lip cancer and maxillary sinus cancer, the corresponding candidate identification results obtained by using any machine learning method are in the form of: the probability of gingival cancer is a%, tongue cancer is b%, soft and hard palate cancer is c%, jaw cancer is d%, floor of mouth cancer is e%, oropharyngeal cancer is f%, salivary gland cancer is h%, lip cancer is i% and maxillary sinus cancer is j%, and a%+b%+c%+d%+e%+f%+h%+i%+j%=100%;

[0071] S163. Combining the candidate recognition results corresponding to each machine learning method and the category matching relationship to obtain the recognition result of the image feature to be recognized; wherein the recognition result of the image feature to be recognized can be understood as being obtained by combining the candidate recognition results corresponding to each machine learning method. If the discriminant analysis method has a category matching relationship with both the gingival cancer classification category and the oropharyngeal cancer classification category, the decision tree method has a category matching relationship with the tongue cancer classification category, the K nearest neighbor method has a category matching relationship with the soft and hard palate cancer classification category, the naive Bayes method has a category matching relationship with the jaw cancer classification category, and the subspace integration method has a category matching relationship with the floor of mouth cancer classification category. There is a category matching relationship. The support vector machine method also has a category matching relationship with the salivary gland cancer classification category, lip cancer classification category and maxillary sinus cancer classification category. The candidate identification results corresponding to the discriminant analysis method, decision tree method, K nearest neighbor method, naive Bayes method, subspace integration method and support vector machine method are: the probability of gingival cancer is a1%, tongue cancer is b1%, soft and hard palate cancer is c1%, jaw cancer is d1%, floor of mouth cancer is e1%, oropharyngeal cancer is f1%, salivary gland cancer is h1%, lip cancer is i1% and maxillary sinus cancer is j1%; the probability of gingival cancer is a2%, tongue cancer is b2%, soft and hard palate cancer is c2%, jaw cancer is d2%, floor of mouth cancer is e1%, oropharyngeal cancer is f1%, salivary gland cancer is h1%, lip cancer is i1% and maxillary sinus cancer is j1%; the probability of gingival cancer is a2%, tongue cancer is b2%, soft and hard palate cancer is c2%, jaw cancer is d2%, oral The probability of floor cancer is e2%, oropharyngeal cancer is f2%, salivary gland cancer is h2%, lip cancer is i2%, and maxillary sinus cancer is j2%; the probability of gingival cancer is a3%, tongue cancer is b3%, soft and hard palate cancer is c3%, jaw cancer is d3%, floor of mouth cancer is e3%, oropharyngeal cancer f3%, salivary gland cancer h3%, lip cancer i3%, and maxillary sinus cancer j3%; the probability of gingival cancer is a4%, tongue cancer is b4%, soft and hard palate cancer is c4%, jaw cancer d4%, floor of mouth cancer is e4%, oropharyngeal cancer f4%, salivary gland cancer h4%, lip cancer i4%, and maxillary sinus cancer j4%; the probability of gingival cancer is a5%, tongue cancer is b5%, soft and hard palate cancer is c5%, jaw cancer d5%, floor of mouth cancer is e5% , oropharyngeal cancer f5%, salivary gland cancer h5%, lip cancer i5% and maxillary sinus cancer j5%; the probability of gingival cancer is a6%, tongue cancer is b6%, soft and hard palate cancer is c6%, jaw cancer is d6%, floor of mouth cancer is e6%, oropharyngeal cancer f6%, salivary gland cancer h6%, lip cancer i6% and maxillary sinus cancer j6%; then, the combined recognition results are: the probability of gingival cancer is a1%, tongue cancer is b2%, soft and hard palate cancer is c3%, jaw cancer is d4%, floor of mouth cancer is e5%, oropharyngeal cancer f1%, salivary gland cancer h6%, lip cancer i6% and maxillary sinus cancer j6%, that is, the corresponding classification with the highest percentage is the predicted classification category of the oral pathology image to be identified.

[0072] In addition, after obtaining the classification category of the oral pathology image to be identified through the above step S16, the machine learning method corresponding to the classification category can be further used for learning and training to further predict the grade and stage of the oral pathology image to be identified corresponding to the classification category, thereby obtaining a more refined recognition result, providing convenient and reliable technical support for oral tumor-related research.

[0073] The embodiment of the present application adopts a method of comprehensively screening a preset number of feature extraction models from multiple preset deep convolutional neural network models according to prediction accuracy and computational efficiency to extract features of oral pathology images to be identified, adopts the nearest component analysis method to perform dimensionality reduction optimization on multiple groups of image features extracted by different network models, and adopts an integrated classifier obtained by integrating multiple machine learning methods to perform combined analysis on image features to obtain identification results such as typing, grading and staging of oral tumors. This method not only effectively solves the problems of high recognition training cost and low learning efficiency in the prior art, but also effectively improves the accuracy of oral pathology image recognition results, and also provides a reliable basis for the evaluation of biological behavior of oral tumors, formulation of individualized treatment plans and efficacy monitoring.

[0074] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0075] In one embodiment, Figure 6 As shown, a system for automatic recognition of oral pathology images is provided, the system comprising:

[0076] Image acquisition module 1, used to obtain microscopic images of oral pathological tissue slides;

[0077] A data set construction module 2 is used to establish an oral pathology image data set based on the oral pathology tissue slide microscopic image, and divide the oral pathology image data set into a training set and a test set according to a preset ratio;

[0078] A model screening module 3 is used to train a first preset number of deep neural network models according to the training set to obtain a first preset number of candidate feature extraction models, and to test each candidate feature extraction model according to the test set to screen and obtain a second preset number of feature extraction models; the second preset number is less than the first preset number;

[0079] A feature extraction module 4 is used to extract features of the oral pathology image to be identified according to each feature extraction model, and obtain corresponding image features to be optimized;

[0080] The feature optimization module 5 is used to optimize the image features to be optimized by using the nearest component analysis method to reduce the dimension and obtain the image features to be identified;

[0081] The image recognition module 6 is used to input the image features to be recognized into a preset integrated classifier for predictive recognition to obtain a corresponding recognition result; the integrated classifier is obtained by integrating a third preset number of machine learning methods.

[0082] It should be noted that the specific definition of the oral pathology image automatic recognition system can be found in the definition of the oral pathology image automatic recognition method above, which will not be repeated here. Each module in the above-mentioned oral pathology image automatic recognition system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0083] Figure 7 FIG. 1 shows an internal structure diagram of a computer device in an embodiment, and the computer device may specifically be a terminal or a server. Figure 7 As shown, the computer device includes a processor, a memory, a network interface, a display and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned method for automatic recognition of oral pathology images is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0084] It can be understood by those skilled in the art that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computing power device may include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0085] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0086] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0087] In summary, the embodiments of the present invention provide an oral pathology image automatic recognition method, system, computer device and storage medium. The oral pathology image automatic recognition method realizes the acquisition of oral pathology tissue slide microscopic images, the establishment of an oral pathology image data set, and the division of the data set into a training set and a test set according to a preset ratio. A first preset number of deep neural network models are trained according to the training set to obtain candidate feature extraction models, and then each candidate feature extraction model is tested according to the test set, and a second preset number of feature extraction models are screened to extract features of the oral pathology images to be identified. After the image features to be optimized are obtained, the nearest component analysis method is used to perform dimensionality reduction optimization and input into a preset integrated classifier for prediction and recognition, so as to obtain a technical solution of the corresponding recognition result. This method for automatic recognition of oral pathology images uses multiple pre-trained deep convolutional neural network models selected based on prediction accuracy and computational efficiency as the basic architecture for feature extraction, then uses the nearest component analysis optimization technology for feature dimensionality reduction optimization, and uses an integrated classifier obtained by integrating multiple machine learning methods to perform combined analysis on image features to obtain recognition results such as oral tumor typing, grading and staging. It not only effectively solves the problems of high recognition training cost and low learning efficiency in existing technologies, but also effectively improves the accuracy of oral pathology image recognition results, and also provides a reliable basis for the evaluation of oral tumor biological behavior, formulation of individualized treatment plans and efficacy monitoring.

[0088] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A method for automatic recognition of oral pathology images, characterized in that: The method comprises the following steps: Obtain microscopic images of oral pathological tissue slides; Establishing an oral pathology image dataset based on the oral pathology tissue slide microscopic image, and dividing the oral pathology image dataset into a training set and a test set according to a preset ratio; A first preset number of deep neural network models are trained according to the training set to obtain a first preset number of candidate feature extraction models, and each candidate feature extraction model is tested according to the test set to screen and obtain a second preset number of feature extraction models; the second preset number is less than the first preset number; According to each feature extraction model, feature extraction is performed on the oral pathology image to be identified, and corresponding image features to be optimized are obtained respectively; Adopting the nearest component analysis method to perform dimension reduction optimization on the image features to be optimized, so as to obtain the image features to be identified; Inputting the image features to be identified into a preset integrated classifier for prediction and identification to obtain a corresponding identification result; the integrated classifier is obtained by integrating a third preset number of machine learning methods; The step of testing each candidate feature extraction model according to the test set to screen out a second preset number of feature extraction models comprises: Testing each candidate feature extraction model according to the test set to obtain the highest accuracy and computational efficiency corresponding to each candidate feature extraction model; Comprehensively sorting the highest accuracy and computational efficiency corresponding to each candidate feature extraction model, and screening out a second preset number of feature extraction models according to the sorting results; The highest precision is the highest accuracy of the corresponding prediction; the computational efficiency is expressed as: Where q represents the computational efficiency of the candidate feature extraction model; t represents the time taken to train the candidate feature extraction model under the same parameters; n represents the number of parameters of the candidate feature extraction model; The step of comprehensively ranking the highest accuracy and operation efficiency corresponding to each candidate feature extraction model, and selecting a second preset number of feature extraction models according to the ranking results includes: The highest accuracy and operation efficiency corresponding to each candidate feature extraction model are weighted averaged according to the preset weight coefficient to obtain the model screening index; the model screening index is expressed as: Wherein, C represents the model screening index of the candidate feature extraction model; and q represent the highest accuracy and computational efficiency of the candidate feature extraction model, respectively; and Respectively represent and the weight coefficient corresponding to q; The model screening indicators of each candidate feature extraction model are arranged in descending order, and a second preset number of candidate feature extraction models are screened out from top to bottom as the feature extraction models.

2. The method for automatic recognition of oral pathology images according to claim 1, characterized in that: The first preset number of deep neural network models includes VGG network model, Resnet network model, Densenet network model, Inception network model, Mobilenet network model and Inception-Resnet network model.

3. The method for automatic recognition of oral pathology images according to claim 1, characterized in that: The preset integrated classifier includes a discriminant analysis method, a decision tree method, a K nearest neighbor method, a naive Bayes method, a subspace integration method and a support vector machine method.

4. The method for automatic recognition of oral pathology images according to claim 1, characterized in that: The step of inputting the image features to be identified into a preset integrated classifier for prediction and identification to obtain the corresponding identification result comprises: Presetting the classification categories of the preset integrated classifier, training each machine learning method, and determining the category matching relationship between each classification category and each machine learning method according to the corresponding training results and preset matching rules; Using each machine learning method in the integrated classifier to predict and identify the image features to be identified, and obtain corresponding candidate identification results; the candidate identification results include the prediction probabilities corresponding to each classification category; The recognition result of the image feature to be recognized is obtained by combining the candidate recognition results corresponding to each machine learning method and the category matching relationship.

5. An oral pathology image automatic recognition system, characterized in that: The system comprises: An image acquisition module, used to obtain microscopic images of oral pathological tissue slides; A data set construction module, used to establish an oral pathology image data set based on the oral pathology tissue slide microscopic image, and divide the oral pathology image data set into a training set and a test set according to a preset ratio; A model screening module, used to train a first preset number of deep neural network models according to the training set to obtain a first preset number of candidate feature extraction models, and to test each candidate feature extraction model according to the test set to screen and obtain a second preset number of feature extraction models; the second preset number is less than the first preset number; A feature extraction module, used to extract features of the oral pathology image to be identified according to each feature extraction model, and obtain corresponding image features to be optimized; A feature optimization module, used for performing dimension reduction optimization on the image features to be optimized by using a nearest component analysis method to obtain image features to be identified; An image recognition module, used for inputting the image features to be recognized into a preset integrated classifier for prediction and recognition, and obtaining a corresponding recognition result; the integrated classifier is obtained by integrating a third preset number of machine learning methods; The step of testing each candidate feature extraction model according to the test set to screen out a second preset number of feature extraction models includes: Testing each candidate feature extraction model according to the test set to obtain the highest accuracy and computational efficiency corresponding to each candidate feature extraction model; Comprehensively sorting the highest accuracy and computational efficiency corresponding to each candidate feature extraction model, and screening out a second preset number of feature extraction models according to the sorting results; The highest precision is the highest accuracy of the corresponding prediction; the computational efficiency is expressed as: Where q represents the computational efficiency of the candidate feature extraction model; t represents the time taken to train the candidate feature extraction model under the same parameters; n represents the number of parameters of the candidate feature extraction model; The step of comprehensively ranking the highest accuracy and operation efficiency corresponding to each candidate feature extraction model, and selecting a second preset number of feature extraction models according to the ranking result, comprises: The highest accuracy and operation efficiency corresponding to each candidate feature extraction model are weighted averaged according to the preset weight coefficient to obtain the model screening index; the model screening index is expressed as: Wherein, C represents the model screening index of the candidate feature extraction model; and q represent the highest accuracy and computational efficiency of the candidate feature extraction model, respectively; and Respectively represent and the weight coefficient corresponding to q; The model screening indicators of each candidate feature extraction model are arranged in descending order, and a second preset number of candidate feature extraction models are screened out from top to bottom as the feature extraction models.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images