Method and device for determining nasopharyngeal carcinoma pathological recognition model based on intelligent labeling

Abstract

The invention relates to the technical field of image processing, and discloses a nasopharyngeal carcinoma pathological recognition model determination method and device based on intelligent labeling, and the method comprises the steps: determining a nasopharyngeal carcinoma pathological image set and a non-nasopharyngeal carcinoma pathological image set; marking a nasopharyngeal carcinoma tumor area of the nasopharyngeal carcinoma pathological image based on the nasopharyngeal carcinoma pathological image intelligent marking model; dividing the marked nasopharyngeal carcinoma pathological image and non-nasopharyngeal carcinoma pathological image into a training set and a verification set; training the neural network model according to the training set and the verification set to obtain a nasopharynx cancer pathology recognition model; testing the nasopharyngeal carcinoma pathological recognition model to obtain a model recognition effect of the nasopharyngeal carcinoma pathological recognition model; and when the model recognition effect is a preset recognition effect, determining a nasopharyngeal carcinoma pathological recognition model as a final nasopharyngeal carcinoma pathological recognition model. Therefore, the nasopharyngeal carcinoma pathological image can be recognized more efficiently by training the nasopharyngeal carcinoma pathological recognition model, and the intelligence of nasopharyngeal carcinoma pathological recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation. Background Art

[0002] Currently, the recognition of nasopharyngeal carcinoma (NPC) pathological images mainly relies on pathologists manually observing the changed image features in tissue cells through a microscope for identification.

[0003] However, this manual recognition method of pathological images has many limitations (such as subjective limitations and experience limitations of pathologists, etc.), which can easily lead to reduced recognition accuracy, especially in grassroots hospitals. It is not intelligent enough, resulting in non-standard selection of treatment strategies based on pathological image recognition, which greatly affects the survival of nasopharyngeal carcinoma patients. It can be seen that it is particularly important to propose a technical solution to improve the recognition accuracy of nasopharyngeal carcinoma pathological images. Summary of the invention

[0004] The present invention provides a method and device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, which can more accurately recognize nasopharyngeal carcinoma pathology images and is conducive to improving the intelligence of nasopharyngeal carcinoma recognition.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, the method comprising:

[0006] Determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set, wherein the nasopharyngeal carcinoma pathology image set includes a plurality of nasopharyngeal carcinoma pathology images, and the non-nasopharyngeal carcinoma pathology image set includes a plurality of non-nasopharyngeal carcinoma pathology images;

[0007] Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotating the nasopharyngeal carcinoma tumor area in each of the nasopharyngeal carcinoma pathology images;

[0008] Dividing all the annotated nasopharyngeal carcinoma pathology images and all the non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio; and training a pre-constructed neural network model according to the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model;

[0009] Based on a preset model testing method, the nasopharyngeal carcinoma pathology recognition model is tested to obtain a test result of the nasopharyngeal carcinoma pathology recognition model;

[0010] When the test result is a preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model, and the final nasopharyngeal carcinoma pathology recognition model is used to identify the pathological image to be identified to obtain a recognition result of the pathological image to be identified, and the recognition result includes that the pathological image to be identified is the nasopharyngeal carcinoma pathology image or the pathological image to be identified is the non-nasopharyngeal carcinoma pathology image.

[0011] As an optional implementation, in the first aspect of the present invention, determining the nasopharyngeal carcinoma pathology image set and the non-nasopharyngeal carcinoma pathology image set includes:

[0012] Collecting pathological image information of each of the multiple cases with identified pathology, wherein the pathological image information includes image information of the pathological image of the corresponding case and diagnostic information of the pathological image, wherein the diagnostic information includes a diagnostic type and a diagnostic result, and the diagnostic type includes a biopsy type or a non-biopsy type;

[0013] According to the image information of each of the pathological images, the image information of each of the pathological images is evaluated to obtain the pathological slide quality corresponding to each of the pathological images;

[0014] According to the diagnosis types of all the pathological images and the pathological film preparation qualities corresponding to all the pathological images, a target pathological image whose diagnosis type is the biopsy type and whose pathological film preparation quality is greater than or equal to a preset quality is screened out from all the pathological images;

[0015] According to the diagnosis results of all the target pathological images, all the target pathological images are classified to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images.

[0016] As an optional implementation, in the first aspect of the present invention, all the target pathological images are classified according to the diagnosis results of all the target pathological images to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images, including:

[0017] Determining the pathological type of each target pathological image according to the diagnosis result of each target pathological image, wherein the pathological type includes a nasopharyngeal carcinoma pathological type or a non-nasopharyngeal carcinoma pathological type;

[0018] For all first pathology images whose pathology type is the nasopharyngeal carcinoma pathology type in all the target pathology images, classify all the first pathology images into a nasopharyngeal carcinoma pathology image set;

[0019] For the second pathology images whose pathology type is the non-nasopharyngeal carcinoma pathology type in all the target pathology images, all the second pathology images are classified into a non-nasopharyngeal carcinoma pathology image set.

[0020] As an optional implementation, in the first aspect of the present invention, the step of labeling the nasopharyngeal carcinoma tumor region in each nasopharyngeal carcinoma pathology image based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model includes:

[0021] Performing registration processing on all the nasopharyngeal carcinoma pathology images to obtain multiple registered image groups, wherein the registration processing includes rigid registration processing and non-rigid registration processing, and all the nasopharyngeal carcinoma pathology images in the same registered image group are obtained by processing the same nasopharyngeal carcinoma initial pathology image according to different staining processing methods;

[0022] Verifying each of the registered image groups according to one or more preset registration verification methods, wherein the registration verification method includes a verification method based on a preset registration evaluation index or a verification method based on a registration quality control CNN model for effect evaluation;

[0023] When the verification passes, the registered image group is annotated based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group, and the marking information is used to annotate the nasopharyngeal carcinoma tumor area of ​​the nasopharyngeal carcinoma pathology image in the corresponding nasopharyngeal carcinoma pathology image.

[0024] As an optional implementation, in the first aspect of the present invention, for each of the registered image groups, the registered image group is labeled based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model to obtain labeling information of all nasopharyngeal carcinoma pathology images in the registered image group, including:

[0025] Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotate each of the nasopharyngeal carcinoma pathology images in the registration image group to obtain initial labeling information for each of the nasopharyngeal carcinoma pathology images in the registration image group, wherein all nasopharyngeal carcinoma pathology images contained in the registration image group include CK staining images, EBERs staining images, and H&E images;

[0026] Transferring the initial marking information of the CK staining image and the initial marking information of the EBERs staining image in the registration image group to the initial marking information of the H&E image to obtain the target marking information of the H&E image;

[0027] The target label information of the H&E image is determined to replace the original initial label information as new label information for all nasopharyngeal carcinoma pathology images in the registered image group.

[0028] As an optional implementation, in the first aspect of the present invention, the registration processing is performed on all the nasopharyngeal carcinoma pathology images to obtain a plurality of registered image groups, including:

[0029] Based on the VGG model, extracting feature information of each nasopharyngeal carcinoma pathological image;

[0030] performing registration processing on all the nasopharyngeal carcinoma pathology images according to the feature information of all the nasopharyngeal carcinoma pathology images to obtain a plurality of registered image groups;

[0031] And, extracting feature information of each nasopharyngeal carcinoma pathological image based on the VGG model includes:

[0032] Performing grayscale conversion on each of the nasopharyngeal carcinoma pathology images to obtain a grayscale image corresponding to each of the nasopharyngeal carcinoma pathology images;

[0033] Normalizing the brightness values ​​of the grayscale images corresponding to all the nasopharyngeal carcinoma pathology images to obtain a normalized grayscale image corresponding to each of the nasopharyngeal carcinoma pathology images;

[0034] Based on the VGG model, characteristic information of the nasopharyngeal carcinoma pathology image is extracted from each nasopharyngeal carcinoma pathology image according to the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image.

[0035] As an optional implementation, in the first aspect of the present invention, it is characterized in that the model testing method includes a model index evaluation method and also includes a pathology recognition result comparison method;

[0036] And, the nasopharyngeal carcinoma pathology recognition model is tested based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model, including:

[0037] Based on one or more preset model evaluation indicators, the performance of the nasopharyngeal carcinoma pathology recognition model is evaluated to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model;

[0038] When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, a preset nasopharyngeal carcinoma biopsy case set is recognized by the nasopharyngeal carcinoma pathology recognition model to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model;

[0039] Obtaining second pathology recognition results of one or more predetermined pathologist groups on the nasopharyngeal carcinoma biopsy case set, wherein all pathologists in each of the pathologist groups correspond to a pathology recognition level;

[0040] The first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model is compared with the second pathology recognition results of all the pathologist groups to obtain a pathology recognition ability comparison result, and the test result of the nasopharyngeal carcinoma pathology recognition model is determined based on the pathology recognition ability comparison result.

[0041] The second aspect of the present invention discloses a device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, the device comprising:

[0042] A determination module, configured to determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set, wherein the nasopharyngeal carcinoma pathology image set includes a plurality of nasopharyngeal carcinoma pathology images, and the non-nasopharyngeal carcinoma pathology image set includes a plurality of non-nasopharyngeal carcinoma pathology images;

[0043] A labeling module, used for labeling the nasopharyngeal carcinoma tumor area in each of the nasopharyngeal carcinoma pathology images based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model;

[0044] A division module, used for dividing all the annotated nasopharyngeal carcinoma pathology images and all the non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio;

[0045] A training module, used to train the pre-built neural network model according to the training set and the verification set to obtain a nasopharyngeal carcinoma pathology recognition model;

[0046] A testing module, used to test the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model;

[0047] The determination module is also used to determine the nasopharyngeal carcinoma pathology recognition model as the final nasopharyngeal carcinoma pathology recognition model when the test result is a preset result. The final nasopharyngeal carcinoma pathology recognition model is used to identify the pathological image to be identified to obtain a recognition result of the pathological image to be identified, and the recognition result includes that the pathological image to be identified is the nasopharyngeal carcinoma pathology image or the pathological image to be identified is the non-nasopharyngeal carcinoma pathology image.

[0048] As an optional implementation, in the second aspect of the present invention, the manner in which the determination module determines the nasopharyngeal carcinoma pathology image set and the non-nasopharyngeal carcinoma pathology image set specifically includes:

[0049] Collecting pathological image information of each of the multiple cases with identified pathology, wherein the pathological image information includes image information of the pathological image of the corresponding case and diagnostic information of the pathological image, wherein the diagnostic information includes a diagnostic type and a diagnostic result, and the diagnostic type includes a biopsy type or a non-biopsy type;

[0050] According to the image information of each of the pathological images, the image information of each of the pathological images is evaluated to obtain the pathological slide quality corresponding to each of the pathological images;

[0051] According to the diagnosis types of all the pathological images and the pathological film preparation qualities corresponding to all the pathological images, a target pathological image whose diagnosis type is the biopsy type and whose pathological film preparation quality is greater than or equal to a preset quality is screened out from all the pathological images;

[0052] According to the diagnosis results of all the target pathological images, all the target pathological images are classified to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images.

[0053] As an optional implementation, in the second aspect of the present invention, the determination module classifies all the target pathology images according to the diagnosis results of all the target pathology images to obtain a set of nasopharyngeal carcinoma pathology images and a set of non-nasopharyngeal carcinoma pathology images, specifically including:

[0054] Determining the pathological type of each target pathological image according to the diagnosis result of each target pathological image, wherein the pathological type includes a nasopharyngeal carcinoma pathological type or a non-nasopharyngeal carcinoma pathological type;

[0055] For all first pathology images whose pathology type is the nasopharyngeal carcinoma pathology type in all the target pathology images, classify all the first pathology images into a nasopharyngeal carcinoma pathology image set;

[0056] For the second pathology images whose pathology type is the non-nasopharyngeal carcinoma pathology type in all the target pathology images, all the second pathology images are classified into a non-nasopharyngeal carcinoma pathology image set.

[0057] As an optional implementation, in the second aspect of the present invention, the annotation module, based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotates the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image in a manner specifically including:

[0058] Performing registration processing on all the nasopharyngeal carcinoma pathology images to obtain multiple registered image groups, wherein the registration processing includes rigid registration processing and non-rigid registration processing, and all the nasopharyngeal carcinoma pathology images in the same registered image group are obtained by processing the same nasopharyngeal carcinoma initial pathology image according to different staining processing methods;

[0059] Verifying each of the registered image groups according to one or more preset registration verification methods, wherein the registration verification method includes a verification method based on a preset registration evaluation index or a verification method based on a registration quality control CNN model for effect evaluation;

[0060] When the verification passes, the registered image group is annotated based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group, and the marking information is used to annotate the nasopharyngeal carcinoma tumor area of ​​the nasopharyngeal carcinoma pathology image in the corresponding nasopharyngeal carcinoma pathology image.

[0061] As an optional implementation, in the second aspect of the present invention, for each of the registered image groups, the annotation module annotates the registered image group based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, and the method of obtaining the labeling information of all nasopharyngeal carcinoma pathology images in the registered image group specifically includes:

[0062] Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotate each of the nasopharyngeal carcinoma pathology images in the registration image group to obtain initial labeling information for each of the nasopharyngeal carcinoma pathology images in the registration image group, wherein all nasopharyngeal carcinoma pathology images contained in the registration image group include CK staining images, EBERs staining images, and H&E images;

[0063] Transferring the initial marking information of the CK staining image and the initial marking information of the EBERs staining image in the registration image group to the initial marking information of the H&E image to obtain the target marking information of the H&E image;

[0064] The target label information of the H&E image is determined to replace the original initial label information as new label information for all nasopharyngeal carcinoma pathology images in the registered image group.

[0065] As an optional implementation, in the second aspect of the present invention, the annotation module performs registration processing on all the nasopharyngeal carcinoma pathology images to obtain a plurality of registered image groups, specifically including:

[0066] Based on the VGG model, extracting feature information of each nasopharyngeal carcinoma pathological image;

[0067] performing registration processing on all the nasopharyngeal carcinoma pathology images according to the feature information of all the nasopharyngeal carcinoma pathology images to obtain a plurality of registered image groups;

[0068] Furthermore, the annotation module extracts the characteristic information of each nasopharyngeal carcinoma pathological image based on the VGG model, specifically including:

[0069] Performing grayscale conversion on each of the nasopharyngeal carcinoma pathology images to obtain a grayscale image corresponding to each of the nasopharyngeal carcinoma pathology images;

[0070] Normalizing the brightness values ​​of the grayscale images corresponding to all the nasopharyngeal carcinoma pathology images to obtain a normalized grayscale image corresponding to each of the nasopharyngeal carcinoma pathology images;

[0071] Based on the VGG model, characteristic information of the nasopharyngeal carcinoma pathology image is extracted from each nasopharyngeal carcinoma pathology image according to the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image.

[0072] As an optional implementation, in the second aspect of the present invention, the model testing method includes a model index evaluation method and also includes a pathology identification result comparison method;

[0073] Furthermore, the testing module tests the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method, and the method of obtaining the test result of the nasopharyngeal carcinoma pathology recognition model specifically includes:

[0074] Based on one or more preset model evaluation indicators, the performance of the nasopharyngeal carcinoma pathology recognition model is evaluated to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model;

[0075] When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, a preset nasopharyngeal carcinoma biopsy case set is recognized by the nasopharyngeal carcinoma pathology recognition model to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model;

[0076] Obtaining second pathology recognition results of one or more predetermined pathologist groups on the nasopharyngeal carcinoma biopsy case set, wherein all pathologists in each of the pathologist groups correspond to a pathology recognition level;

[0077] The first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model is compared with the second pathology recognition results of all the pathologist groups to obtain a pathology recognition ability comparison result, and the test result of the nasopharyngeal carcinoma pathology recognition model is determined based on the pathology recognition ability comparison result.

[0078] The third aspect of the present invention discloses another device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, the device comprising:

[0079] A memory storing executable program code;

[0080] a processor coupled to the memory;

[0081] The processor calls the executable program code stored in the memory to execute the method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in the first aspect of the present invention.

[0082] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the method for determining the nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in the first aspect of the present invention.

[0083] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0084] In an embodiment of the present invention, a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set are determined, the nasopharyngeal carcinoma pathology image set includes multiple nasopharyngeal carcinoma pathology images, and the non-nasopharyngeal carcinoma pathology image set includes multiple non-nasopharyngeal carcinoma pathology images; based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image is annotated; all annotated nasopharyngeal carcinoma pathology images are used as a training set, and all non-nasopharyngeal carcinoma pathology images are used as a verification set to train a pre-constructed neural network model to obtain a nasopharyngeal carcinoma pathology recognition model; based on a preset model testing method, the nasopharyngeal carcinoma pathology recognition model is tested to obtain a test result of the nasopharyngeal carcinoma pathology recognition model, and the test result is used to represent the model recognition effect of the nasopharyngeal carcinoma pathology recognition model; when the model recognition effect is the preset recognition effect, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model, and the final nasopharyngeal carcinoma pathology recognition model is used to identify the pathology image to be identified to obtain a recognition result of the pathology image to be identified, and the recognition result includes that the pathology image to be identified is a nasopharyngeal carcinoma pathology image or the pathology image to be identified is a non-nasopharyngeal carcinoma pathology image. It can be seen that the implementation of the present invention can accurately annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image in the determined nasopharyngeal carcinoma pathology image set through the nasopharyngeal carcinoma pathology image intelligent annotation model to obtain the annotated nasopharyngeal carcinoma pathology image, which is beneficial to improving the accuracy and reliability of the annotated nasopharyngeal carcinoma pathology image, and is beneficial to improving the annotation efficiency and intelligence of the nasopharyngeal carcinoma pathology image through the nasopharyngeal carcinoma pathology image intelligent annotation model, and can more accurately train the pre-constructed neural network model to obtain a nasopharyngeal carcinoma pathology recognition model by using all annotated nasopharyngeal carcinoma pathology images as a training set and all non-nasopharyngeal carcinoma pathology images as a verification set, which is beneficial to improving the accuracy and reliability of the trained nasopharyngeal carcinoma pathology recognition model, and can improve the accuracy of the nasopharyngeal carcinoma pathology recognition model through the preset model testing method. The testing accuracy and efficiency of the pathology recognition model can improve the research and development accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model by using the intelligently labeled nasopharyngeal carcinoma pathology images as the training set of the nasopharyngeal carcinoma pathology recognition model and verifying and testing the nasopharyngeal carcinoma pathology recognition model. This is conducive to improving the recognition accuracy, recognition efficiency and intelligence of the subsequent pathology images to be identified by the model based on the accurately determined nasopharyngeal carcinoma pathology recognition model, and then it is conducive to accurately selecting the treatment strategy targeting the nasopharyngeal carcinoma pathology image when the nasopharyngeal carcinoma pathology image is identified, and it is conducive to improving the accuracy, reliability and rationality of the treatment strategy targeting the nasopharyngeal carcinoma pathology image, and it is also conducive to improving the safety of patients corresponding to the nasopharyngeal carcinoma pathology image by adopting accurate treatment strategies for treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0086] Figure 1 It is a flow chart of a method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention;

[0087] Figure 2 It is a flow chart of another method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention;

[0088] Figure 3 It is a structural schematic diagram of a device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention;

[0089] Figure 4 It is a structural schematic diagram of another device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent labeling disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0091] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0092] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0093] The present invention discloses a method and device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation. The method and device can determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set, and annotate a nasopharyngeal carcinoma tumor region in each nasopharyngeal carcinoma pathology image based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model. Subsequently, all annotated nasopharyngeal carcinoma pathology images are used as a training set, and all non-nasopharyngeal carcinoma pathology images are used as a verification set to train a pre-constructed neural network model to obtain a nasopharyngeal carcinoma pathology recognition model. Further, based on a preset model testing method, the nasopharyngeal carcinoma pathology recognition model is tested to obtain a test result of the nasopharyngeal carcinoma pathology recognition model. When the test result is a preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model. The nasopharyngeal carcinoma pathology image intelligent annotation model can be used to accurately annotate the nasopharyngeal carcinoma tumor region in each nasopharyngeal carcinoma pathology image in the determined nasopharyngeal carcinoma pathology image set to obtain an annotated nasopharyngeal carcinoma pathology image, which is beneficial to improving the accuracy and reliability of the annotated nasopharyngeal carcinoma pathology image, and is beneficial to improving the annotation efficiency and reliability of the nasopharyngeal carcinoma pathology image through the nasopharyngeal carcinoma pathology image intelligent annotation model. The intelligent degree can be improved by taking all the annotated nasopharyngeal carcinoma pathology images as training sets and all the non-nasopharyngeal carcinoma pathology images as verification sets to more accurately train the pre-built neural network model to obtain the nasopharyngeal carcinoma pathology recognition model, which is conducive to improving the accuracy and reliability of the trained nasopharyngeal carcinoma pathology recognition model. The preset model testing method can improve the test accuracy and efficiency of the nasopharyngeal carcinoma pathology recognition model. The intelligently annotated nasopharyngeal carcinoma pathology images can be used as the training set of the nasopharyngeal carcinoma pathology recognition model and a series of processes such as verification and testing of the nasopharyngeal carcinoma pathology recognition model can be used to improve the research and development accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model, which is conducive to improving the recognition accuracy, recognition efficiency and intelligent degree of the subsequent pathological images to be recognized by the model based on the accurately determined nasopharyngeal carcinoma pathology recognition model, and then it is conducive to accurately selecting the treatment strategy targeting the nasopharyngeal carcinoma pathology image when the nasopharyngeal carcinoma pathology image is identified, and it is conducive to improving the accuracy, reliability and rationality of the treatment strategy targeting the nasopharyngeal carcinoma pathology image, and it is also conducive to improving the safety of patients corresponding to the nasopharyngeal carcinoma pathology image by adopting accurate treatment strategies for treatment. The following are detailed descriptions.

[0094] Embodiment 1

[0095] See also Figure 1 , Figure 1 1 is a flow chart of a method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention. Figure 1 The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation described above can be applied to a device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, wherein the device may include a determination device or a determination server, wherein the determination server may include a cloud server or a local server, which is not limited in the embodiments of the present invention. Figure 1 As shown, the method for determining the nasopharyngeal carcinoma pathology recognition model based on intelligent annotation may include the following operations:

[0096] 101. Determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set.

[0097] In an embodiment of the present invention, the nasopharyngeal carcinoma pathology image set may include multiple nasopharyngeal carcinoma pathology images, wherein each nasopharyngeal carcinoma pathology image is obtained by staining the pathology image corresponding to the corresponding nasopharyngeal carcinoma case. The nasopharyngeal carcinoma pathology image set may include multiple non-nasopharyngeal carcinoma pathology images.

[0098] Specifically, the method may further include: staining all the acquired pathological images corresponding to one or more nasopharyngeal carcinoma cases according to different staining processing methods to obtain all the nasopharyngeal carcinoma pathological images corresponding to each nasopharyngeal carcinoma case. Each staining processing method includes one of H&E, CK and EBERs.

[0099] Among them, H&E, or hematoxylin-eosin staining, is the most commonly used staining method for paraffin sections. Its specific steps include: sectioning, dewaxing, washing, hematoxylin and eosin staining, dehydration, transparency and sealing, and finally quality control by specialized personnel. CK staining is the use of immunohistochemistry to detect epithelial cell markers. Its specific steps include: using Dako Autostainer for automatic staining, EliVisionTM plus two-step method, operating according to the reagent instructions, the CK antibody clone number is CK AE1 / AE3, ready-to-use, DAB color development, and hematoxylin counterstaining. EBERs staining is the use of in situ hybridization to detect Epstein-Barr virus-encoded small RNA (EBERs). Its specific steps include: using Dako Autostainer for automatic staining, using the EBERs in situ hybridization detection kit, operating according to the reagent instructions, DAB color development, and hematoxylin counterstaining.

[0100] Specifically, the method may also include: preprocessing the nasopharyngeal carcinoma pathology image set to obtain a preprocessed nasopharyngeal carcinoma pathology image set, and triggering the execution of the following step 102, based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, to annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image.

[0101] Specifically, the method may further include: preprocessing the non-nasopharyngeal carcinoma pathology image set to obtain a preprocessed non-nasopharyngeal carcinoma pathology image set, and triggering the execution of the following step 103 of dividing all annotated nasopharyngeal carcinoma pathology images and all non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio.

[0102] 102. Based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model, label the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image.

[0103] 103. Divide all annotated nasopharyngeal carcinoma pathology images and all non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio.

[0104] In an embodiment of the present invention, optionally, when all annotated nasopharyngeal carcinoma pathology images and all non-nasopharyngeal carcinoma pathology images are divided, in addition to dividing into a training set and a validation set, a test set can also be divided, such as: according to the division ratio of 60% for the training set, 20% for the validation set and 20% for the test set, at this time, nasopharyngeal carcinoma pathology images accounting for 60% of all annotated nasopharyngeal carcinoma pathology images and non-nasopharyngeal carcinoma pathology images accounting for 60% of all non-nasopharyngeal carcinoma pathology images can be classified into the training set; nasopharyngeal carcinoma pathology images accounting for 20% of all annotated nasopharyngeal carcinoma pathology images and non-nasopharyngeal carcinoma pathology images accounting for 20% of all non-nasopharyngeal carcinoma pathology images can be classified into the validation set; nasopharyngeal carcinoma pathology images accounting for 20% of all annotated nasopharyngeal carcinoma pathology images and non-nasopharyngeal carcinoma pathology images accounting for 20% of all non-nasopharyngeal carcinoma pathology images can be classified into the test set.

[0105] 104. The pre-built neural network model is trained according to the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model.

[0106] In an embodiment of the present invention, the pre-constructed neural network model can be a semantic segmentation model similar to DeepLabV3, or a classification model of a ViT structure pre-trained based on DINOv2 or a classification model of a CNN structure, or other pre-set network models, which is not limited in the embodiment of the present invention.

[0107] Specifically, the above step 103 of training the pre-built neural network model according to the training set and the validation set to obtain the nasopharyngeal carcinoma pathology recognition model may include:

[0108] Input the training set into the pre-built DeepLabV3 model for training to obtain the trained DeepLabV3 model;

[0109] Input the validation set into the trained DeepLabV3 model for recognition, and obtain the recognition result of the trained DeepLabV3 model on the validation set;

[0110] According to the recognition results of the trained DeepLabV3 model on the validation set, the trained DeepLabV3 model is evaluated to obtain the model evaluation results;

[0111] According to the model evaluation results, determine whether the amount of training data of the trained DeepLabV3 model is greater than or equal to the preset amount of data, and whether the hyperparameters of the DeepLabV3 model reach the optimal hyperparameters, and obtain the judgment result;

[0112] When the judgment result is no, the trained DeepLabV3 model is iteratively trained, and then the above operation of inputting the verification set into the trained DeepLabV3 model for recognition is re-executed to obtain the recognition result of the verification set by the trained DeepLabV3 model;

[0113] When the judgment result is yes, the trained DeepLabV3 model is determined as the nasopharyngeal carcinoma pathology recognition model.

[0114] 105. Based on the preset model testing method, the nasopharyngeal carcinoma pathology recognition model is tested to obtain the test result of the nasopharyngeal carcinoma pathology recognition model.

[0115] In an embodiment of the present invention, the model testing method may include a model index evaluation method, and the model testing method may also include a pathology recognition result comparison method. The model index evaluation method may include an evaluation method based on one or more preset model evaluation indicators, and the test result of the nasopharyngeal carcinoma pathology recognition model may include a performance evaluation result of the nasopharyngeal carcinoma pathology recognition model, and may also include a pathology recognition ability comparison result corresponding to the nasopharyngeal carcinoma pathology recognition model.

[0116] 106. When the test result is the preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model.

[0117] In the embodiment of the present invention, optionally, the preset result may include the result that the performance of the nasopharyngeal carcinoma pathology recognition model represented by the performance evaluation result reaches the expected performance, and may also include the result that the recognition ability of the nasopharyngeal carcinoma pathology recognition model represented by the pathology recognition ability comparison result reaches the expected recognition ability. The final nasopharyngeal carcinoma pathology recognition model is used to identify the pathological image to be identified to obtain the recognition result of the pathological image to be identified, and the recognition result includes that the pathological image to be identified is a nasopharyngeal carcinoma pathological image or the pathological image to be identified is a non-nasopharyngeal carcinoma pathological image.

[0118] It can be seen that the implementation Figure 1 The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation described herein can determine a set of nasopharyngeal carcinoma pathology images and a set of non-nasopharyngeal carcinoma pathology images, and based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image, and then divide all annotated nasopharyngeal carcinoma pathology images and all non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio, and train a pre-constructed neural network model according to the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model, and then test the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model. When the test result is a preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model, and the nasopharyngeal carcinoma pathology recognition model can be used to accurately annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image in the determined nasopharyngeal carcinoma pathology image set through the nasopharyngeal carcinoma pathology image intelligent annotation model to obtain an annotated nasopharyngeal carcinoma pathology image, which is beneficial to improving the accuracy and reliability of the annotated nasopharyngeal carcinoma pathology image, and is beneficial to The invention is to improve the annotation efficiency and intelligence of nasopharyngeal carcinoma pathology images through the intelligent annotation model of nasopharyngeal carcinoma pathology images, and can more accurately train the pre-constructed neural network model to obtain the nasopharyngeal carcinoma pathology recognition model through the training set and the verification set divided by all the annotated nasopharyngeal carcinoma pathology images and all the non-nasopharyngeal carcinoma pathology images, which is conducive to improving the accuracy and reliability of the trained nasopharyngeal carcinoma pathology recognition model, and can improve the test accuracy and efficiency of the nasopharyngeal carcinoma pathology recognition model through the preset model testing method, which is conducive to improving the research and development accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model, so as to improve the recognition accuracy, recognition efficiency and intelligence of the subsequent pathological images to be identified by the model based on the accurately determined nasopharyngeal carcinoma pathology recognition model, and then it is conducive to accurately selecting the treatment strategy targeting the nasopharyngeal carcinoma pathology image when the nasopharyngeal carcinoma pathology image is identified, and is conducive to improving the accuracy, reliability and rationality of the treatment strategy targeting the nasopharyngeal carcinoma pathology image, and is also conducive to improving the safety of patients corresponding to the nasopharyngeal carcinoma pathology image by adopting accurate treatment strategies for treatment.

[0119] In an optional embodiment, determining the NPC pathology image set and the non-NPC pathology image set in step 101 may include:

[0120] collecting pathology image information for each of a plurality of cases in which pathology has been identified;

[0121] According to the image information of each pathological image, the image information of each pathological image is evaluated to obtain the pathological film preparation quality corresponding to each pathological image;

[0122] According to the diagnosis types of all pathological images and the pathological film preparation qualities corresponding to all pathological images, target pathological images whose diagnosis types are biopsy types and whose pathological film preparation qualities are greater than or equal to a preset quality are screened out from all pathological images;

[0123] According to the diagnosis results of all target pathological images, all target pathological images are classified to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images.

[0124] In an embodiment of the present invention, specifically, collecting the pathological image information of each case in multiple cases of identified pathology may include: for a case information database of identified pathology, filtering out the pathological image information of all cases whose consultation time is within a preset time period from the case information database. The pathological image information includes the image information of the pathological image of the corresponding case and the diagnostic information of the pathological image. The image information of the pathological image may include one or more combinations of the image clarity of the pathological image, the image pixels of the pathological image, the image size of the pathological image, and the image brightness of the pathological image. The diagnostic information of the pathological image may include the diagnostic type of the pathological image and the diagnostic result of the pathological image, wherein the diagnostic type may include a biopsy type or a non-biopsy type.

[0125] It can be seen that this optional embodiment can collect pathological image information of each case in multiple cases with identified pathology, and evaluate the image information of each pathological image based on the image information of each pathological image to obtain the pathological film quality corresponding to each pathological image, and then screen out target pathological images with a diagnosis type of biopsy type and a pathological film quality greater than or equal to a preset quality from all pathological images based on the diagnosis type of all pathological images and the pathological film quality corresponding to all pathological images, and then classify all target pathological images based on the diagnosis results of all target pathological images to obtain a nasopharyngeal carcinoma pathological image set and a non-nasopharyngeal carcinoma pathological image set, which can be classified through the image information of each pathological image. The image information of each pathological image can be accurately evaluated to obtain the pathological film quality corresponding to each pathological image, which is beneficial to improving the accuracy and reliability of the evaluated pathological film quality corresponding to each pathological image, and the screening accuracy and screening efficiency of the target pathological images can be improved through the diagnostic types of all pathological images and the pathological film quality corresponding to all pathological images, so that all target pathological images can be quickly and accurately classified through the diagnostic results of the target pathological images obtained by accurate screening to obtain a nasopharyngeal carcinoma pathological image set and a non-nasopharyngeal carcinoma pathological image set, which is beneficial to improving the accuracy, efficiency and reliability of the classified nasopharyngeal carcinoma pathological image set and the non-nasopharyngeal carcinoma pathological image set.

[0126] In this optional embodiment, as an optional implementation, all target pathological images are classified according to the diagnosis results of all target pathological images to obtain a nasopharyngeal carcinoma pathological image set and a non-nasopharyngeal carcinoma pathological image set, which may include:

[0127] Determine the pathological type of each target pathological image according to the diagnosis result of each target pathological image;

[0128] For all first pathology images whose pathology type is nasopharyngeal carcinoma pathology type among all target pathology images, classify all first pathology images into a nasopharyngeal carcinoma pathology image set;

[0129] For the second pathology images whose pathology types are non-nasopharyngeal carcinoma pathology types among all target pathology images, all the second pathology images are classified into a non-nasopharyngeal carcinoma pathology image set.

[0130] In an embodiment of the present invention, optionally, the diagnostic result of each pathological image may include a combination of one or more results of the pathological image being diagnosed as non-keratinizing nasopharyngeal carcinoma, the pathological image being diagnosed as chronic nonspecific inflammation of the nasopharynx, and the pathological image being diagnosed as a malignant tumor other than nasopharyngeal carcinoma (for example, lymphoma, malignant melanoma, etc.), which is not limited in the embodiment of the present invention.

[0131] In the embodiment of the present invention, the pathological type includes a nasopharyngeal carcinoma pathological type or a non-nasopharyngeal carcinoma pathological type. Specifically, according to the diagnosis result of each target pathological image, determining the pathological type of each target pathological image may include:

[0132] For each target pathological image, judging, according to the diagnosis result of the target pathological image, whether the diagnosis result of the target pathological image includes a result that the pathological image is diagnosed as nasopharyngeal non-keratinizing carcinoma;

[0133] When the diagnosis result of the target pathological image includes a result that the pathological image is diagnosed as nasopharyngeal non-keratinizing carcinoma, determining the pathological type of the target pathological image as a nasopharyngeal carcinoma pathological type;

[0134] When the diagnosis result of the target pathological image does not include a result that the pathological image is diagnosed as nasopharyngeal non-keratinizing carcinoma, the pathological type of the target pathological image is determined to be a non-nasopharyngeal carcinoma pathological type.

[0135] It can be seen that this optional implementation can determine the pathological type of each target pathological image according to the diagnosis result of each target pathological image, and for all first pathological images whose pathological type is nasopharyngeal carcinoma pathological type among all target pathological images, all first pathological images are classified into the nasopharyngeal carcinoma pathological image set, and for all second pathological images whose pathological type is non-nasopharyngeal carcinoma pathological type among all target pathological images, all second pathological images are classified into the non-nasopharyngeal carcinoma pathological image set. The pathological type of each target pathological image can be accurately determined through the diagnosis result of each target pathological image, and the classification accuracy and reliability of the nasopharyngeal carcinoma pathological image set and the non-nasopharyngeal carcinoma pathological image set can be improved through the pathological types of all target pathological images.

[0136] Embodiment 2

[0137] See also Figure 2 , Figure 2 1 is a flow chart of a method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention. Figure 2 The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation described above can be applied to a device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, wherein the device may include a determination device or a determination server, wherein the determination server may include a cloud server or a local server, which is not limited in the embodiments of the present invention. Figure 2 As shown, the method for determining the nasopharyngeal carcinoma pathology recognition model based on intelligent annotation may include the following operations:

[0138] 201. Determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set.

[0139] 202. Based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model, label the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image.

[0140] 203. Divide all annotated nasopharyngeal carcinoma pathology images and all non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio.

[0141] 204. The pre-built neural network model is trained according to the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model.

[0142] 205. Based on one or more preset model evaluation indicators, the performance of the nasopharyngeal carcinoma pathology recognition model is evaluated to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model.

[0143] In the embodiment of the present invention, each model evaluation index may include one of ROC curve, AUC, sensitivity, specificity, false positive rate, etc. Optionally, the performance evaluation result may be used to indicate that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, or that the performance of the nasopharyngeal carcinoma pathology recognition model does not reach the expected performance.

[0144] 206. When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, the nasopharyngeal carcinoma pathology recognition model is used to recognize a preset nasopharyngeal carcinoma biopsy case set to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model.

[0145] In the embodiment of the present invention, the above-mentioned preset nasopharyngeal carcinoma biopsy case set can be used as a test set of the nasopharyngeal carcinoma pathology recognition model to test the recognition ability of the nasopharyngeal carcinoma pathology recognition model for nasopharyngeal carcinoma pathology images. Optionally, the nasopharyngeal carcinoma biopsy case set can include H&E sections of multiple nasopharyngeal carcinoma biopsy cases, for example: it can be 100 nasopharyngeal carcinoma biopsy cases, or it can be other numbers of nasopharyngeal carcinoma biopsy cases, which is not limited in the embodiment of the present invention.

[0146] 207. Obtain a second pathology recognition result of one or more predetermined pathologist groups on a set of nasopharyngeal carcinoma biopsy cases.

[0147] In an embodiment of the present invention, all pathologists in each pathologist group correspond to a pathology identification level. Optionally, the pathology identification levels may include primary, intermediate and advanced levels according to the classification from small to large. For example, three primary pathologists, three intermediate pathologists and three senior pathologists may be obtained to perform pathological diagnosis on the 100 nasopharyngeal biopsy cases, and the pathological diagnosis results of each pathologist may be obtained, and the pathological diagnosis results of all pathologists in each pathologist group may be determined as the second case identification results.

[0148] In the embodiment of the present invention, there is no order of precedence between step 207 and step 206, that is, step 207 may occur before step 206, may occur after step 206, or may occur simultaneously with step 206, which is not limited in the embodiment of the present invention.

[0149] 208. Compare the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model with the second pathology recognition results of all the pathologist groups to obtain a pathology recognition capability comparison result.

[0150] In an embodiment of the present invention, optionally, the pathology recognition ability comparison result may include the similarity between the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model and the second pathology recognition result of each pathologist group among the second pathology recognition results of all pathologist groups, and may also include the difference between the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model and the second pathology recognition result of each pathologist group among the second pathology recognition results of all pathologist groups, which is not limited in the embodiment of the present invention.

[0151] 209. Based on the pathology recognition capability comparison results, determine the test results of the nasopharyngeal carcinoma pathology recognition model.

[0152] In the embodiment of the present invention, specifically, determining the test result of the nasopharyngeal carcinoma pathology recognition model according to the pathology recognition ability comparison result may include:

[0153] According to the similarity between the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model and the second pathology recognition result of each pathologist group among the second pathology recognition results of all pathologist groups, one of the pathologist groups having the greatest similarity between the second pathology recognition result and the first pathology recognition result is selected from all the pathologist groups;

[0154] According to the pathology recognition level corresponding to the pathologist group with the greatest similarity, the pathology recognition ability of the nasopharyngeal carcinoma pathology recognition model is determined as the test result of the nasopharyngeal carcinoma pathology recognition model.

[0155] 210. When the test result is a preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model.

[0156] In an embodiment of the present invention, the preset result may be that the pathology recognition capability of the nasopharyngeal carcinoma pathology recognition model reaches the recognition capability equivalent to that of a senior pathologist, or the pathology recognition capability of the nasopharyngeal carcinoma pathology recognition model reaches the recognition capability equivalent to that of a mid-level pathologist, and the embodiment of the present invention does not limit this.

[0157] In the embodiment of the present invention, for other descriptions of steps 201 to 204 and step 210, please refer to the detailed description of steps 101 to 104 and step 106 in the first embodiment, which will not be repeated in the embodiment of the present invention.

[0158] It can be seen that the implementation Figure 2The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation described herein can determine a set of nasopharyngeal carcinoma pathology images and a set of non-nasopharyngeal carcinoma pathology images, and based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image, and then use all the annotated nasopharyngeal carcinoma pathology images as a training set, and all the non-nasopharyngeal carcinoma pathology images as a verification set, to train a pre-constructed neural network model to obtain a nasopharyngeal carcinoma pathology recognition model, and then test the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model. When the test result is a preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model, and the nasopharyngeal carcinoma pathology image intelligent annotation model can be used to accurately annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image in the determined nasopharyngeal carcinoma pathology image set to obtain annotated nasopharyngeal carcinoma pathology images, which is beneficial to improving the accuracy and reliability of the annotated nasopharyngeal carcinoma pathology images, and is beneficial to improving the annotation efficiency and intelligence of nasopharyngeal carcinoma pathology images through the intelligent annotation model for nasopharyngeal carcinoma pathology images. The invention can improve the accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model obtained by training, and can more accurately train the pre-built neural network model by taking all the annotated nasopharyngeal carcinoma pathology images as the training set and all the non-nasopharyngeal carcinoma pathology images as the verification set, so as to obtain the nasopharyngeal carcinoma pathology recognition model, which is conducive to improving the accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model obtained by training. The preset model testing method can improve the test accuracy and efficiency of the nasopharyngeal carcinoma pathology recognition model, and can improve the research and development accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model by taking the intelligently annotated nasopharyngeal carcinoma pathology images as the training set of the nasopharyngeal carcinoma pathology recognition model and verifying and testing the nasopharyngeal carcinoma pathology recognition model through a series of processes, thereby facilitating the improvement of the recognition accuracy, recognition efficiency and intelligence of the subsequent pathological images to be recognized by the model based on the accurately determined nasopharyngeal carcinoma pathology recognition model, and then facilitating the accurate selection of the treatment strategy targeting the nasopharyngeal carcinoma pathology image when the nasopharyngeal carcinoma pathology image is identified, and is conducive to improving the accuracy, reliability and rationality of the treatment strategy targeting the nasopharyngeal carcinoma pathology image, and is also conducive to improving the safety of patients corresponding to the nasopharyngeal carcinoma pathology image by adopting an accurate treatment strategy for treatment.In addition, the performance of the nasopharyngeal carcinoma pathology recognition model can be evaluated based on one or more preset model evaluation indicators to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model. When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, the preset nasopharyngeal carcinoma biopsy case set is identified by the nasopharyngeal carcinoma pathology recognition model to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model, and a second pathology recognition result of the nasopharyngeal carcinoma biopsy case set by one or more predetermined pathologist groups is obtained. The first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model is compared with the second pathology recognition results of all pathologist groups to obtain a pathology recognition ability comparison result, and then the nasopharyngeal carcinoma is determined based on the pathology recognition ability comparison result. The test results of the cancer pathology recognition model can accurately evaluate the performance of the nasopharyngeal carcinoma pathology recognition model through model evaluation indicators, and when the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, the preset nasopharyngeal carcinoma biopsy case set can be accurately identified through the nasopharyngeal carcinoma pathology recognition model to obtain the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model, which is conducive to improving the accuracy and efficiency of the first pathology recognition result obtained, and can accurately obtain the pathology recognition ability comparison result by comparing the first pathology recognition result with the second pathology recognition result obtained from the pathologist groups of different levels, so that the determination accuracy and reliability of the test results of the nasopharyngeal carcinoma pathology recognition model can be improved based on the accurately determined pathology recognition ability comparison result.

[0159] In an optional embodiment, the above step 202 of labeling the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model may include:

[0160] Perform registration processing on all nasopharyngeal carcinoma pathological images to obtain multiple registered image groups;

[0161] Verify each registered image group according to one or more preset registration verification methods;

[0162] When the verification passes, the registered image group is annotated based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group. The marking information is used to annotate the nasopharyngeal carcinoma tumor area of ​​the nasopharyngeal carcinoma pathology image in the corresponding nasopharyngeal carcinoma pathology image.

[0163] In an embodiment of the present invention, the registration processing may include rigid registration processing and non-rigid registration processing (such as: DeepFlow non-rigid registration). All nasopharyngeal carcinoma pathology images in the same registration image group are obtained by processing the same nasopharyngeal carcinoma initial pathology image according to different staining processing methods, and are used to achieve cross-modal registration between the nasopharyngeal carcinoma initial pathology image based on immunohistochemistry CK and the nasopharyngeal carcinoma initial pathology image based on in situ hybridization EBERs and the nasopharyngeal carcinoma initial pathology image based on H&E, so as to accurately determine the nasopharyngeal carcinoma tumor area.

[0164] In an embodiment of the present invention, the registration verification method may include a verification method based on a preset registration evaluation index or a verification method based on a registration quality control CNN model for effect evaluation. Among them, the verification method based on the preset registration evaluation index may specifically include: using DICE similarity coefficient (Dice, DSC), target matching error (TRE), relative target matching error (rTRE), Hausdorff distance (HD) and other evaluation indicators to evaluate the registration effect.

[0165] It should be noted that before adopting the verification method based on the registration quality control CNN model for effect evaluation, the registration quality control CNN model needs to be trained first. Specifically, the registration quality control CNN model is trained in the following way: pathologists evaluate the registration effect corresponding to the historical registration image group, and collect samples with high registration quality and low registration quality included in the registration effect as training data. Through the above training data, the preset initial CNN model is trained, and the trained initial CNN model is verified and tested to train the registration quality control CNN model.

[0166] It can be seen that this optional embodiment can perform registration processing on all nasopharyngeal carcinoma pathology images to obtain multiple registered image groups, and verify each registered image group according to one or more preset registration verification methods. When the verification passes, based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, the registered image group is annotated to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group. The accuracy and reliability of the registered image group obtained by the registration can be improved by performing rigid registration and non-rigid registration processing on all nasopharyngeal carcinoma pathology images, and the verification accuracy, diversity and flexibility of each registered image group can be achieved through the preset multiple registration verification methods, so that when the verification passes, the marking information of the nasopharyngeal carcinoma pathology image in each registered image group can be accurately annotated through the determined nasopharyngeal carcinoma pathology image intelligent annotation model, which is beneficial to improving the annotation efficiency and intelligence of the marking information of the nasopharyngeal carcinoma pathology image, and the nasopharyngeal carcinoma pathology image intelligent annotation model can be used to achieve accurate and intelligent annotation of the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image.

[0167] In this optional embodiment, as an optional implementation, for each registered image group, based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, the registered image group is annotated to obtain the labeling information of all nasopharyngeal carcinoma pathology images in the registered image group, including:

[0168] Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotate each nasopharyngeal carcinoma pathology image in the registration image group to obtain initial labeling information for each nasopharyngeal carcinoma pathology image in the registration image group, wherein all nasopharyngeal carcinoma pathology images contained in the registration image group include CK staining images, EBERs staining images and H&E images;

[0169] Transferring the initial labeling information of the CK staining image and the initial labeling information of the EBERs staining image in the registration image group to the initial labeling information of the H&E image, so as to obtain the target labeling information of the H&E image;

[0170] The target label information of the H&E image is determined to replace the original initial label information as the new label information of all nasopharyngeal carcinoma pathology images in the registration image group.

[0171] In an embodiment of the present invention, positive cells in CK and EBERs staining are the targets of NPC pathology intelligent annotation, so positive cell extraction and label transfer are required. Cell extraction mainly includes steps such as cell detection based on QuPath software, cell pairing of pathological images with different staining after registration, and assignment of positive and negative labels. The labeling of positive and negative cells can be set using color thresholds (for example: for CK staining images, cells with darker colors in the CK staining image are marked as positive cells, and cells with lighter colors in the CK staining image are marked as negative cells). However, in the extraction of positive and negative cells in pathological images, due to the diversity and uncertainty of cell size, shape, color, whether necrosis and other characteristics, there will be problems of false negatives and false positives, that is, unlabeled or wrongly labeled cells; at the same time, because the accuracy of registration may have errors at the cell level, it will also cause problems of missed labels and wrong labels. Therefore, it is necessary to perform false negative and false positive quality control in the process of training the intelligent annotation model of nasopharyngeal carcinoma pathology images.

[0172] Specifically, the intelligent annotation model for nasopharyngeal carcinoma pathology images is trained in the following way:

[0173] Acquire each pathological sample image in the registration sample image group and labeling information corresponding to each pathological sample, wherein the labeling information includes positive cell labeling information and / or negative cell labeling information;

[0174] Taking all the pathological sample images as the input parameters of the training set of the CNN model, and taking the label information corresponding to all the pathological sample images as the output parameters of the training set of the CNN model, the CNN model is trained to obtain an initial labeling model;

[0175] Inputting the acquired registered pathological image into the initial annotation model for annotation, and obtaining the annotation result of the registered pathological image;

[0176] Obtaining a quality control result of a pathologist's annotation result of the registered pathology image;

[0177] According to the quality control result, it is determined whether the annotation result of the registered pathological image is correct;

[0178] When it is determined that the annotation result of the registered pathological image is incorrect, the annotation result of the registered pathological image is updated according to the quality control result to obtain an updated annotation result, and the registered pathological image and the updated annotation result are updated to the training set of the above-mentioned CNN model to continue training the initial annotation model to obtain an intelligent annotation model for nasopharyngeal carcinoma pathological images.

[0179] In an embodiment of the present invention, optionally, the pathological sample image may include a combination of one or more H&E staining images, CK staining images and EBERs staining images. Optionally, the annotation results include the annotation results of positive cells and / or the annotation results of negative cells. Specifically, through the quality control results of the annotation results of each pathological sample image obtained by the pathologist based on the initial annotation model, it can be determined whether the annotation result has false positive and / or false negative problems. Among them, the annotation CNN model (that is, the intelligent annotation model of nasopharyngeal carcinoma pathology images) that finally completes the training can automatically complete the assignment of cell positive / negative labels, and in the subsequent use process, it is necessary to evaluate and improve the algorithm according to the specific verification effect, and perform corresponding parameter optimization and model training. By continuously optimizing the process and algorithm, the accuracy and efficiency of digital pathology intelligent annotation can be improved, providing better support for digital pathology research.

[0180] It can be seen that this optional implementation can, based on the determined NPC pathology image intelligent annotation model, annotate each NPC pathology image in the registered image group to obtain the initial marking information of each NPC pathology image in the registered image group, and transfer the initial marking information of the CK staining image and the initial marking information of the EBERs staining image in the registered image group to the initial marking information of the H&E image to obtain the target marking information of the H&E image, and then determine the target marking information of the H&E image to replace the original initial marking information as the new marking information of all NPC pathology images in the registered image group. The NPC pathology image intelligent annotation model can be used to quickly and accurately annotate each NPC pathology image to obtain the initial marking information, which is beneficial to improving the accuracy and reliability of the initial marking information obtained by annotation, and can improve the accuracy of the transfer of the initial marking information through accurate initial marking information to accurately obtain the target marking information.

[0181] In this optional embodiment, as another optional implementation, all nasopharyngeal carcinoma pathology images are registered to obtain multiple registered image groups, including:

[0182] Based on the VGG model, the feature information of each NPC pathological image is extracted;

[0183] According to the characteristic information of all nasopharyngeal carcinoma pathological images, all nasopharyngeal carcinoma pathological images are registered to obtain multiple registered image groups.

[0184] In this optional implementation, optionally, based on the VGG model, feature information of each nasopharyngeal carcinoma pathological image is extracted, including:

[0185] Performing grayscale conversion on each nasopharyngeal carcinoma pathology image to obtain a grayscale image corresponding to each nasopharyngeal carcinoma pathology image;

[0186] Normalizing the brightness values ​​of the grayscale images corresponding to all nasopharyngeal carcinoma pathological images to obtain a normalized grayscale image corresponding to each nasopharyngeal carcinoma pathological image;

[0187] Based on the VGG model, the characteristic information of each nasopharyngeal carcinoma pathology image is extracted from the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image.

[0188] In the embodiment of the present invention, the feature information of each nasopharyngeal carcinoma pathological image can be the CNN features of each nasopharyngeal carcinoma pathological image extracted by the VGG model. Specifically, normalizing the brightness values ​​of the grayscale images corresponding to all nasopharyngeal carcinoma pathological images can make pathological images of different staining types as similar as possible in the grayscale domain, thereby facilitating the subsequent execution of various processing operations on each pathological image.

[0189] It can be seen that this optional implementation can perform grayscale conversion on each nasopharyngeal carcinoma pathology image to obtain a grayscale image corresponding to each nasopharyngeal carcinoma pathology image, and normalize the brightness values ​​of the grayscale images corresponding to all nasopharyngeal carcinoma pathology images to obtain a normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image. Subsequently, based on the VGG model, according to the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image, the feature information of the nasopharyngeal carcinoma pathology image is extracted from each nasopharyngeal carcinoma pathology image. Then, according to the feature information of all nasopharyngeal carcinoma pathology images, all nasopharyngeal carcinoma pathology images are registered to obtain multiple registered image groups. The brightness value of each grayscale image can be accurately normalized by the brightness value of the grayscale images after conversion of all nasopharyngeal carcinoma pathology images, which is beneficial to make each nasopharyngeal carcinoma pathology image as similar as possible in the grayscale domain, and the feature information of the nasopharyngeal carcinoma pathology image can be accurately extracted from the normalized nasopharyngeal carcinoma pathology image through the VGG model, which is beneficial to improve the accuracy and reliability of the feature information of the extracted nasopharyngeal carcinoma pathology image.

[0190] Embodiment 3

[0191] See also Figure 3 , Figure 3 Schematic diagram of a structure of a device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention. Figure 3 The described determination device of the nasopharyngeal carcinoma pathology recognition model based on intelligent annotation may include a determination device or a determination server, wherein the determination server may include a cloud server or a local server, which is not limited in the embodiment of the present invention. Figure 3 As shown, the device for determining the nasopharyngeal carcinoma pathology recognition model based on intelligent annotation may include:

[0192] A determination module 301 is used to determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set, wherein the nasopharyngeal carcinoma pathology image set includes a plurality of nasopharyngeal carcinoma pathology images, and the non-nasopharyngeal carcinoma pathology image set includes a plurality of non-nasopharyngeal carcinoma pathology images;

[0193] A labeling module 302 is used to label the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model;

[0194] A division module 303 is used to divide all the annotated NPC pathology images and all the non-NPC pathology images into a training set and a validation set according to a preset division ratio;

[0195] A training module 304 is used to train the pre-built neural network model according to the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model;

[0196] The testing module 305 is used to test the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model;

[0197] The determination module 301 is also used to determine the nasopharyngeal carcinoma pathology recognition model as the final nasopharyngeal carcinoma pathology recognition model when the test result is a preset result. The final nasopharyngeal carcinoma pathology recognition model is used to identify the pathological image to be identified to obtain a recognition result of the pathological image to be identified, and the recognition result includes that the pathological image to be identified is a nasopharyngeal carcinoma pathology image or the pathological image to be identified is a non-nasopharyngeal carcinoma pathology image.

[0198] It can be seen that the implementation Figure 3The described device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation is capable of determining a set of nasopharyngeal carcinoma pathology images and a set of non-nasopharyngeal carcinoma pathology images, and based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotates the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image, and then divides all the annotated nasopharyngeal carcinoma pathology images and all non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio, and trains a pre-constructed neural network model based on the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model, and then based on a preset model testing method, the nasopharyngeal carcinoma pathology recognition model is obtained. The nasopharyngeal carcinoma pathology recognition model is tested to obtain the test result of the nasopharyngeal carcinoma pathology recognition model. When the test result is the preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model. The nasopharyngeal carcinoma pathology image intelligent annotation model can accurately annotate the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image in the determined nasopharyngeal carcinoma pathology image set to obtain an annotated nasopharyngeal carcinoma pathology image, which is beneficial to improving the accuracy and reliability of the annotated nasopharyngeal carcinoma pathology image, and is beneficial to improving the labeling of nasopharyngeal carcinoma pathology images through the nasopharyngeal carcinoma pathology image intelligent annotation model. The method can improve the efficiency and intelligence of the image recognition system, and can more accurately train the pre-built neural network model to obtain the nasopharyngeal carcinoma pathology recognition model through the training set and verification set obtained by dividing all the annotated nasopharyngeal carcinoma pathology images and all the non-nasopharyngeal carcinoma pathology images, which is conducive to improving the accuracy and reliability of the trained nasopharyngeal carcinoma pathology recognition model. The preset model testing method can improve the test accuracy and efficiency of the nasopharyngeal carcinoma pathology recognition model. The intelligently annotated nasopharyngeal carcinoma pathology images can be used as the training set of the nasopharyngeal carcinoma pathology recognition model and a series of processes such as verification and testing of the nasopharyngeal carcinoma pathology recognition model can be used to improve the research and development accuracy and reliability of the nasopharyngeal carcinoma pathology recognition model, which is conducive to improving the recognition accuracy, recognition efficiency and intelligence of the subsequent pathological images to be recognized by the model based on the accurately determined nasopharyngeal carcinoma pathology recognition model, and then it is conducive to accurately selecting the treatment strategy targeting the nasopharyngeal carcinoma pathology image when the nasopharyngeal carcinoma pathology image is identified, and it is conducive to improving the accuracy, reliability and rationality of the treatment strategy targeting the nasopharyngeal carcinoma pathology image, and it is also conducive to improving the safety of patients corresponding to the nasopharyngeal carcinoma pathology image by adopting accurate treatment strategies for treatment.

[0199] In an optional embodiment, the manner in which the determination module 301 determines the nasopharyngeal carcinoma pathology image set and the non-nasopharyngeal carcinoma pathology image set may specifically include:

[0200] collecting pathological image information of each case among multiple cases of identified pathology, the pathological image information including image information of the pathological image of the corresponding case and diagnostic information of the pathological image, the diagnostic information including diagnostic type and diagnostic result, the diagnostic type including biopsy type or non-biopsy type;

[0201] According to the image information of each pathological image, the image information of each pathological image is evaluated to obtain the pathological film preparation quality corresponding to each pathological image;

[0202] According to the diagnosis types of all pathological images and the pathological film preparation qualities corresponding to all pathological images, target pathological images whose diagnosis types are biopsy types and whose pathological film preparation qualities are greater than or equal to a preset quality are screened out from all pathological images;

[0203] According to the diagnosis results of all target pathological images, all target pathological images are classified to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images.

[0204] It can be seen that this optional embodiment can collect pathological image information of each case in multiple cases with identified pathology, and evaluate the image information of each pathological image based on the image information of each pathological image to obtain the pathological film quality corresponding to each pathological image, and then screen out target pathological images with a diagnosis type of biopsy type and a pathological film quality greater than or equal to a preset quality from all pathological images based on the diagnosis type of all pathological images and the pathological film quality corresponding to all pathological images, and then classify all target pathological images based on the diagnosis results of all target pathological images to obtain a nasopharyngeal carcinoma pathological image set and a non-nasopharyngeal carcinoma pathological image set, which can be classified through the image information of each pathological image. The image information of each pathological image can be accurately evaluated to obtain the pathological film quality corresponding to each pathological image, which is beneficial to improving the accuracy and reliability of the evaluated pathological film quality corresponding to each pathological image, and the screening accuracy and screening efficiency of the target pathological images can be improved through the diagnostic types of all pathological images and the pathological film quality corresponding to all pathological images, so that all target pathological images can be quickly and accurately classified through the diagnostic results of the target pathological images obtained by accurate screening to obtain a nasopharyngeal carcinoma pathological image set and a non-nasopharyngeal carcinoma pathological image set, which is beneficial to improving the accuracy, efficiency and reliability of the classified nasopharyngeal carcinoma pathological image set and the non-nasopharyngeal carcinoma pathological image set.

[0205] In this optional embodiment, as an optional implementation, the determination module 301 classifies all target pathological images according to the diagnosis results of all target pathological images, and the method of obtaining the nasopharyngeal carcinoma pathological image set and the non-nasopharyngeal carcinoma pathological image set may specifically include:

[0206] Determine the pathological type of each target pathological image according to the diagnosis result of each target pathological image, where the pathological type includes a nasopharyngeal carcinoma pathological type or a non-nasopharyngeal carcinoma pathological type;

[0207] For all first pathology images whose pathology type is nasopharyngeal carcinoma pathology type among all target pathology images, classify all first pathology images into a nasopharyngeal carcinoma pathology image set;

[0208] For the second pathology images whose pathology types are non-nasopharyngeal carcinoma pathology types among all target pathology images, all the second pathology images are classified into a non-nasopharyngeal carcinoma pathology image set.

[0209] It can be seen that this optional implementation can determine the pathological type of each target pathological image according to the diagnosis result of each target pathological image, and for all first pathological images whose pathological type is nasopharyngeal carcinoma pathological type among all target pathological images, all first pathological images are classified into the nasopharyngeal carcinoma pathological image set, and for all second pathological images whose pathological type is non-nasopharyngeal carcinoma pathological type among all target pathological images, all second pathological images are classified into the non-nasopharyngeal carcinoma pathological image set. The pathological type of each target pathological image can be accurately determined through the diagnosis result of each target pathological image, and the classification accuracy and reliability of the nasopharyngeal carcinoma pathological image set and the non-nasopharyngeal carcinoma pathological image set can be improved through the pathological types of all target pathological images.

[0210] In another optional embodiment, the labeling module 302 labels the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model, and the specific method may include:

[0211] All nasopharyngeal carcinoma pathological images are registered to obtain multiple registered image groups. The registration processing includes rigid registration processing and non-rigid registration processing. All nasopharyngeal carcinoma pathological images in the same registered image group are obtained by processing the same nasopharyngeal carcinoma initial pathological image according to different staining processing methods.

[0212] Verify each registered image group according to one or more preset registration verification methods, where the registration verification method includes a verification method based on a preset registration evaluation index or a verification method based on a registration quality control CNN model for effect evaluation;

[0213] When the verification passes, the registered image group is annotated based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group. The marking information is used to annotate the nasopharyngeal carcinoma tumor area of ​​the nasopharyngeal carcinoma pathology image in the corresponding nasopharyngeal carcinoma pathology image.

[0214] It can be seen that this optional embodiment can perform registration processing on all nasopharyngeal carcinoma pathology images to obtain multiple registered image groups, and verify each registered image group according to one or more preset registration verification methods. When the verification passes, based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, the registered image group is annotated to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group. The accuracy and reliability of the registered image group obtained by the registration can be improved by performing rigid registration and non-rigid registration processing on all nasopharyngeal carcinoma pathology images, and the verification accuracy, diversity and flexibility of each registered image group can be achieved through the preset multiple registration verification methods, so that when the verification passes, the marking information of the nasopharyngeal carcinoma pathology image in each registered image group can be accurately annotated through the determined nasopharyngeal carcinoma pathology image intelligent annotation model, which is beneficial to improving the annotation efficiency and intelligence of the marking information of the nasopharyngeal carcinoma pathology image, and the nasopharyngeal carcinoma pathology image intelligent annotation model can be used to achieve accurate and intelligent annotation of the nasopharyngeal carcinoma tumor area in each nasopharyngeal carcinoma pathology image.

[0215] In this optional embodiment, as an optional implementation, for each registered image group, the annotation module 302 annotates the registered image group based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model, and the method of obtaining the labeling information of all nasopharyngeal carcinoma pathology images in the registered image group may specifically include:

[0216] Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotate each nasopharyngeal carcinoma pathology image in the registration image group to obtain initial labeling information for each nasopharyngeal carcinoma pathology image in the registration image group, wherein all nasopharyngeal carcinoma pathology images contained in the registration image group include CK staining images, EBERs staining images and H&E images;

[0217] Transferring the initial labeling information of the CK staining image and the initial labeling information of the EBERs staining image in the registration image group to the initial labeling information of the H&E image, so as to obtain the target labeling information of the H&E image;

[0218] The target label information of the H&E image is determined to replace the original initial label information as the new label information of all nasopharyngeal carcinoma pathology images in the registration image group.

[0219] It can be seen that this optional implementation can, based on the determined NPC pathology image intelligent annotation model, annotate each NPC pathology image in the registration image group to obtain the initial marking information of each NPC pathology image in the registration image group, and transfer the initial marking information of the CK staining image and the initial marking information of the EBERs staining image in the registration image group to the initial marking information of the H&E image to obtain the target marking information of the H&E image, and then determine the target marking information of the NPC pathology image where the reference marking information is located, as the target marking information of all NPC pathology images in the registration image group, and can quickly and accurately annotate each NPC pathology image to obtain the initial marking information through the NPC pathology image intelligent annotation model, which is conducive to improving the accuracy and reliability of the initial marking information obtained by annotation, and can improve the accuracy of the transfer of the initial marking information through accurate initial marking information to accurately obtain the target marking information.

[0220] In this optional embodiment, as another optional implementation, the marking module 302 performs registration processing on all nasopharyngeal carcinoma pathology images to obtain multiple registered image groups, which may specifically include:

[0221] Based on the VGG model, the feature information of each NPC pathological image is extracted;

[0222] According to the characteristic information of all nasopharyngeal carcinoma pathological images, all nasopharyngeal carcinoma pathological images are registered to obtain multiple registered image groups.

[0223] In this optional implementation, the labeling module 302 may optionally extract feature information of each nasopharyngeal carcinoma pathology image based on the VGG model, specifically including:

[0224] Performing grayscale conversion on each nasopharyngeal carcinoma pathology image to obtain a grayscale image corresponding to each nasopharyngeal carcinoma pathology image;

[0225] Normalizing the brightness values ​​of the grayscale images corresponding to all nasopharyngeal carcinoma pathological images to obtain a normalized grayscale image corresponding to each nasopharyngeal carcinoma pathological image;

[0226] Based on the VGG model, the characteristic information of each nasopharyngeal carcinoma pathology image is extracted from the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image.

[0227] It can be seen that this optional implementation can also perform grayscale conversion on each nasopharyngeal carcinoma pathology image to obtain a grayscale image corresponding to each nasopharyngeal carcinoma pathology image, and normalize the brightness values ​​of the grayscale images corresponding to all nasopharyngeal carcinoma pathology images to obtain a normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image. Subsequently, based on the VGG model, according to the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image, the feature information of the nasopharyngeal carcinoma pathology image is extracted from each nasopharyngeal carcinoma pathology image. Then, according to the feature information of all nasopharyngeal carcinoma pathology images, all nasopharyngeal carcinoma pathology images are registered to obtain multiple registered image groups. The brightness value of each grayscale image can be accurately normalized by the brightness value of the grayscale images after conversion of all nasopharyngeal carcinoma pathology images, which is conducive to making each nasopharyngeal carcinoma pathology image as similar as possible in the grayscale domain, and the feature information of the nasopharyngeal carcinoma pathology image can be accurately extracted from the normalized nasopharyngeal carcinoma pathology image through the VGG model, which is conducive to improving the accuracy and reliability of the feature information of the extracted nasopharyngeal carcinoma pathology image.

[0228] In another optional embodiment, the model testing method includes a model index evaluation method and a pathology recognition result comparison method. And, the testing module 305 tests the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method, and the method of obtaining the test result of the nasopharyngeal carcinoma pathology recognition model may specifically include:

[0229] Based on one or more preset model evaluation indicators, the performance of the nasopharyngeal carcinoma pathology recognition model is evaluated to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model;

[0230] When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, the nasopharyngeal carcinoma pathology recognition model is used to recognize a preset nasopharyngeal carcinoma biopsy case set to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model;

[0231] Obtaining second pathology recognition results of one or more predetermined pathologist groups on a set of nasopharyngeal carcinoma biopsy cases, wherein all pathologists in each pathologist group correspond to a pathology recognition level;

[0232] The first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model is compared with the second pathology recognition results of all pathologist groups to obtain a pathology recognition ability comparison result, and the test result of the nasopharyngeal carcinoma pathology recognition model is determined based on the pathology recognition ability comparison result.

[0233] It can be seen that this optional embodiment can evaluate the performance of the nasopharyngeal carcinoma pathology recognition model based on one or more preset model evaluation indicators to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model. When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, the preset nasopharyngeal carcinoma biopsy case set is identified by the nasopharyngeal carcinoma pathology recognition model to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model, and obtain a second pathology recognition result of the nasopharyngeal carcinoma biopsy case set by one or more predetermined pathologist groups, compare the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model with the second pathology recognition results of all pathologist groups, obtain a pathology recognition ability comparison result, and then determine the pathology recognition ability comparison result based on the pathology recognition ability comparison result. The test result of the nasopharyngeal carcinoma pathology recognition model can be determined, and the performance of the nasopharyngeal carcinoma pathology recognition model can be accurately evaluated through the model evaluation indicators. When the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, the preset nasopharyngeal carcinoma biopsy case set can be accurately identified through the nasopharyngeal carcinoma pathology recognition model to obtain the first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model, which is conducive to improving the accuracy and efficiency of the first pathology recognition result obtained, and the pathology recognition ability comparison result can be accurately obtained by comparing the first pathology recognition result with the second pathology recognition results obtained from the groups of pathologists of different levels, so that the determination accuracy and reliability of the test result of the nasopharyngeal carcinoma pathology recognition model can be improved based on the accurately determined pathology recognition ability comparison result.

[0234] Embodiment 4

[0235] See also Figure 4 , Figure 4 FIG. 1 is a schematic diagram of another structure of a device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in an embodiment of the present invention. Figure 4 As shown, the device for determining the nasopharyngeal carcinoma pathology recognition model based on intelligent annotation may include:

[0236] A memory 401 storing executable program codes;

[0237] a processor 402 coupled to the memory 401;

[0238] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation described in the first embodiment of the present invention or the second embodiment of the present invention.

[0239] Embodiment 5

[0240] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent labeling described in Embodiment 1 or Embodiment 2 of the present invention.

[0241] Embodiment 6

[0242] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation described in Example 1 or Example 2.

[0243] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0244] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0245] Finally, it should be noted that the method and device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation disclosed in the embodiment of the present invention disclose only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, characterized in that: The method comprises: Determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set, wherein the nasopharyngeal carcinoma pathology image set includes a plurality of nasopharyngeal carcinoma pathology images, and the non-nasopharyngeal carcinoma pathology image set includes a plurality of non-nasopharyngeal carcinoma pathology images; Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotating the nasopharyngeal carcinoma tumor area in each of the nasopharyngeal carcinoma pathology images; Dividing all the annotated nasopharyngeal carcinoma pathology images and all the non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio; and training a pre-constructed neural network model according to the training set and the validation set to obtain a nasopharyngeal carcinoma pathology recognition model; Based on a preset model testing method, the nasopharyngeal carcinoma pathology recognition model is tested to obtain a test result of the nasopharyngeal carcinoma pathology recognition model; When the test result is a preset result, the nasopharyngeal carcinoma pathology recognition model is determined as the final nasopharyngeal carcinoma pathology recognition model, and the final nasopharyngeal carcinoma pathology recognition model is used to identify the pathological image to be identified to obtain a recognition result of the pathological image to be identified, and the recognition result includes that the pathological image to be identified is the nasopharyngeal carcinoma pathology image or the pathological image to be identified is the non-nasopharyngeal carcinoma pathology image.

2. The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation according to claim 1, characterized in that: The step of determining a set of nasopharyngeal carcinoma pathology images and a set of non-nasopharyngeal carcinoma pathology images includes: Collecting pathological image information of each of the multiple cases with identified pathology, wherein the pathological image information includes image information of the pathological image of the corresponding case and diagnostic information of the pathological image, wherein the diagnostic information includes a diagnostic type and a diagnostic result, and the diagnostic type includes a biopsy type or a non-biopsy type; According to the image information of each of the pathological images, the image information of each of the pathological images is evaluated to obtain the pathological slide quality corresponding to each of the pathological images; According to the diagnosis types of all the pathological images and the pathological film preparation qualities corresponding to all the pathological images, a target pathological image whose diagnosis type is the biopsy type and whose pathological film preparation quality is greater than or equal to a preset quality is screened out from all the pathological images; According to the diagnosis results of all the target pathological images, all the target pathological images are classified to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images.

3. The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation according to claim 2, characterized in that: According to the diagnosis results of all the target pathological images, all the target pathological images are classified to obtain a set of nasopharyngeal carcinoma pathological images and a set of non-nasopharyngeal carcinoma pathological images, including: Determining the pathological type of each target pathological image according to the diagnosis result of each target pathological image, wherein the pathological type includes a nasopharyngeal carcinoma pathological type or a non-nasopharyngeal carcinoma pathological type; For all first pathology images whose pathology type is the nasopharyngeal carcinoma pathology type in all the target pathology images, classify all the first pathology images into a nasopharyngeal carcinoma pathology image set; For the second pathology images whose pathology type is the non-nasopharyngeal carcinoma pathology type in all the target pathology images, all the second pathology images are classified into a non-nasopharyngeal carcinoma pathology image set.

4. The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation according to any one of claims 1 to 3, characterized in that: The step of labeling the nasopharyngeal carcinoma tumor region in each nasopharyngeal carcinoma pathology image based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model includes: Performing registration processing on all the nasopharyngeal carcinoma pathology images to obtain multiple registered image groups, wherein the registration processing includes rigid registration processing and non-rigid registration processing, and all the nasopharyngeal carcinoma pathology images in the same registered image group are obtained by processing the same nasopharyngeal carcinoma initial pathology image according to different staining processing methods; Verifying each of the registered image groups according to one or more preset registration verification methods, wherein the registration verification method includes a verification method based on a preset registration evaluation index or a verification method based on a registration quality control CNN model for effect evaluation; When the verification passes, the registered image group is annotated based on the determined nasopharyngeal carcinoma pathology image intelligent annotation model to obtain the marking information of all nasopharyngeal carcinoma pathology images in the registered image group, and the marking information is used to annotate the nasopharyngeal carcinoma tumor area of ​​the nasopharyngeal carcinoma pathology image in the corresponding nasopharyngeal carcinoma pathology image.

5. The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation according to claim 4, characterized in that: For each of the registered image groups, the registered image group is labeled based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model to obtain labeling information of all nasopharyngeal carcinoma pathology images in the registered image group, including: Based on the determined intelligent annotation model for nasopharyngeal carcinoma pathology images, annotate each of the nasopharyngeal carcinoma pathology images in the registration image group to obtain initial labeling information for each of the nasopharyngeal carcinoma pathology images in the registration image group, wherein all nasopharyngeal carcinoma pathology images contained in the registration image group include CK staining images, EBERs staining images, and H&E images; Transferring the initial marking information of the CK staining image and the initial marking information of the EBERs staining image in the registration image group to the initial marking information of the H&E image to obtain the target marking information of the H&E image; The target label information of the H&E image is determined to replace the original initial label information as new label information for all nasopharyngeal carcinoma pathology images in the registered image group.

6. The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation according to claim 4, characterized in that: The registering process is performed on all the nasopharyngeal carcinoma pathological images to obtain a plurality of registered image groups, including: Based on the VGG model, extracting feature information of each nasopharyngeal carcinoma pathological image; performing registration processing on all the nasopharyngeal carcinoma pathology images according to the feature information of all the nasopharyngeal carcinoma pathology images to obtain a plurality of registered image groups; And, extracting feature information of each nasopharyngeal carcinoma pathological image based on the VGG model includes: Performing grayscale conversion on each of the nasopharyngeal carcinoma pathology images to obtain a grayscale image corresponding to each of the nasopharyngeal carcinoma pathology images; Normalizing the brightness values ​​of the grayscale images corresponding to all the nasopharyngeal carcinoma pathology images to obtain a normalized grayscale image corresponding to each of the nasopharyngeal carcinoma pathology images; Based on the VGG model, characteristic information of the nasopharyngeal carcinoma pathology image is extracted from each nasopharyngeal carcinoma pathology image according to the normalized grayscale image corresponding to each nasopharyngeal carcinoma pathology image.

7. The method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation according to any one of claims 1, 2, 3, 5 and 6, characterized in that: The model testing method includes a model index evaluation method and a pathology recognition result comparison method; And, the nasopharyngeal carcinoma pathology recognition model is tested based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model, including: Based on one or more preset model evaluation indicators, the performance of the nasopharyngeal carcinoma pathology recognition model is evaluated to obtain a performance evaluation result corresponding to the nasopharyngeal carcinoma pathology recognition model; When the performance evaluation result indicates that the performance of the nasopharyngeal carcinoma pathology recognition model reaches the expected performance, a preset nasopharyngeal carcinoma biopsy case set is recognized by the nasopharyngeal carcinoma pathology recognition model to obtain a first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model; Obtaining a second pathology recognition result of one or more predetermined pathologist groups on the nasopharyngeal carcinoma biopsy case set, wherein all pathologists in each of the pathologist groups correspond to a pathology recognition level; The first pathology recognition result of the nasopharyngeal carcinoma pathology recognition model is compared with the second pathology recognition results of all the pathologist groups to obtain a pathology recognition ability comparison result, and the test result of the nasopharyngeal carcinoma pathology recognition model is determined based on the pathology recognition ability comparison result.

8. A device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, characterized in that: The device comprises: A determination module, configured to determine a nasopharyngeal carcinoma pathology image set and a non-nasopharyngeal carcinoma pathology image set, wherein the nasopharyngeal carcinoma pathology image set includes a plurality of nasopharyngeal carcinoma pathology images, and the non-nasopharyngeal carcinoma pathology image set includes a plurality of non-nasopharyngeal carcinoma pathology images; A labeling module, used for labeling the nasopharyngeal carcinoma tumor area in each of the nasopharyngeal carcinoma pathology images based on the determined nasopharyngeal carcinoma pathology image intelligent labeling model; A division module, used for dividing all the annotated nasopharyngeal carcinoma pathology images and all the non-nasopharyngeal carcinoma pathology images into a training set and a validation set according to a preset division ratio; A training module, used to train the pre-built neural network model according to the training set and the verification set to obtain a nasopharyngeal carcinoma pathology recognition model; A testing module, used to test the nasopharyngeal carcinoma pathology recognition model based on a preset model testing method to obtain a test result of the nasopharyngeal carcinoma pathology recognition model; The determination module is also used to determine the nasopharyngeal carcinoma pathology recognition model as the final nasopharyngeal carcinoma pathology recognition model when the test result is a preset result. The final nasopharyngeal carcinoma pathology recognition model is used to identify the pathological image to be identified to obtain a recognition result of the pathological image to be identified, and the recognition result includes that the pathological image to be identified is the nasopharyngeal carcinoma pathology image or the pathological image to be identified is the non-nasopharyngeal carcinoma pathology image.

9. A device for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the method for determining a nasopharyngeal carcinoma pathology recognition model based on intelligent annotation as described in any one of claims 1 to 7.

Images