Intelligent measuring system for depth of invasion of oral squamous cell carcinoma

Abstract

The oral squamous cell carcinoma invasion depth intelligent measurement system of the present invention combines pathology and artificial intelligence. It obtains segmented images containing tumor areas and mucosal areas from oral squamous cell carcinoma pathological images through a depth segmentation model, and locates two reference points on the mucosal basement membrane that are closest to the target tumor area to predict the invasion depth of the oral squamous cell carcinoma pathological images. It realizes intelligent and automated measurement of invasion depth, and can also replace pathologists in completing repetitive and tedious work, greatly improving efficiency and standardizing the process.

Description

Technical Field

[0001] This invention relates to the field of medical data processing, and in particular to an intelligent measurement system for the invasion depth of oral squamous cell carcinoma. Background Technology

[0002] Squamous cell carcinoma (SCC) of the oral mucosa is the most common malignant tumor of the head and neck, accounting for more than 90% of oral and maxillofacial malignancies. The key and core of its standardized treatment is to adopt a specific comprehensive sequential treatment plan based on the results of prognostic evaluation (individualized evaluation). The most commonly used first-line evaluation indicator in the individualized evaluation system for oral mucosal SCC is TNM staging. Depth of invasion (DOI) has been added as an important indicator for determining the pT stage of oral cancer; for every 5 mm increase in DOI, the T stage rises by one level. In clinical pathology practice, depth of invasion (DOI) is measured manually. The specific method is as follows: Under a microscope, the slide is marked with a marker. The horizontal line is drawn from the basement membrane of the normal mucosa closest to the tumor to the deepest point of tumor invasion. The length of this vertical line is the DOI value, and the specific value is measured using a millimeter ruler. The cumbersome manual measurement method not only wastes manpower and time in measuring tumor invasion depth, but is also prone to errors. Furthermore, inconsistent individual measurement standards lead to low accuracy in measuring invasion depth. Summary of the Invention

[0003] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide an intelligent measurement system for the invasion depth of oral squamous cell carcinoma, which solves the problems of the cumbersome manual measurement method in the prior art, which not only wastes manpower and time, but is also prone to errors, and the inconsistent measurement standards of individuals lead to low accuracy in measuring invasion depth.

[0004] To achieve the above and other related objectives, the present invention provides an intelligent measurement system for the invasion depth of oral squamous cell carcinoma. The system includes: a segmentation module, used to obtain a segmented image containing the tumor region and the mucosal region based on a depth segmentation model and an oral squamous cell carcinoma pathological image; and an invasion depth prediction module, connected to the segmentation module, used to locate two reference points located on the mucosal basement membrane that are closest to the target tumor region based on the segmented image, so as to predict the invasion depth of the oral squamous cell carcinoma pathological image.

[0005] In one embodiment of the present invention, the deep segmentation model includes: a segmentation sub-model, an attention sub-model, and a FR prediction sub-model, respectively used to output the preliminary segmentation result, attention map, and FR value of the corresponding tumor or mucosal area ratio of the corresponding oral squamous cell carcinoma pathological image; wherein, the training method of the deep segmentation model includes: training the deep segmentation model according to the pathological image training set based on the segmentation loss function related to the preliminary segmentation result, attention map, and FR value; and wherein, the pathological image training set includes: multiple pathological training images and labels corresponding to each pathological training image.

[0006] In one embodiment of the present invention, the segmentation loss function includes: a TopK cross-entropy loss function, a mean squared error term based on the attention map, and a FR regularization term.

[0007] In one embodiment of the present invention, the segmentation loss function includes:

[0008]

[0009] Among them, l CE For the TopK cross-entropy loss function, l MSE For the mean squared error term based on the attention map and αFR as the FR regularization term, the p i Let y be the probability value of pixel i corresponding to each category in the prediction result. i FR is the category label of pixel i in the label of the pathological training image, FR is the proportion of tumor or mucosal area, and α is a hyperparameter.

[0010] In one embodiment of the present invention, the segmentation module includes: a global segmentation submodule and / or a local segmentation submodule; wherein, the global segmentation submodule is used to perform global segmentation on a global oral squamous cell carcinoma pathological image obtained by scaling up the oral squamous cell carcinoma pathological image based on the depth segmentation model, so as to obtain a global segmentation image corresponding to the oral squamous cell carcinoma pathological image; the local segmentation submodule is used to perform local segmentation on one or more local oral squamous cell carcinoma pathological images obtained by segmentation of the oral squamous cell carcinoma pathological image based on the depth segmentation model, so as to obtain a local segmentation image corresponding to each local oral squamous cell carcinoma pathological image.

[0011] In one embodiment of the present invention, the segmentation module further includes a global-local fusion submodule, used to fuse the global segmentation image corresponding to the oral squamous cell carcinoma pathological image and each local segmentation image to obtain a segmentation image containing the tumor region and the mucosal region.

[0012] In one embodiment of the present invention, the method of locating two reference points located on the mucosal basement membrane closest to the target tumor region based on the segmented image to predict the invasion depth of the oral squamous cell carcinoma pathological image includes: determining the location information of the target tumor region based on the segmented image; determining the location information of the two left and right mucosal regions located on the tissue surface closest to the target tumor region based on the location information of the target tumor region; obtaining the location information of the two reference points located on the mucosal basement membrane closest to the target tumor region based on the location information of the target tumor region and the location information of the two corresponding mucosal regions; and calculating the invasion depth based on the location information of the two reference points.

[0013] In one embodiment of the present invention, determining the location information of the target tumor region based on the segmented image includes: preprocessing the segmented image; extracting the tissue outer contour in the preprocessed segmented image; determining the outer contour of the target tumor region in the tissue outer contour, and obtaining the location information of the outer contour of the target tumor region.

[0014] In one embodiment of the present invention, the calculation of invasion depth based on the position information of two reference points includes: calculating the tumor invasion direction and the deepest point of tumor invasion based on the position information of the two reference points and all tumor regions of the segmented image, so as to obtain the invasion depth.

[0015] In one embodiment of the present invention, the pathological training image includes: a coarsely labeled image, wherein the tumor or mucosal area ratio is greater than 0.7.

[0016] As described above, the present invention is an intelligent measurement system for the invasion depth of oral squamous cell carcinoma, which has the following beneficial effects: The present invention obtains segmented images containing tumor regions and mucosal regions from oral squamous cell carcinoma pathological images through a depth segmentation model, and locates two reference points located on the mucosal basement membrane that are closest to the target tumor region, so as to predict the invasion depth of the oral squamous cell carcinoma pathological images. This realizes intelligent and automated measurement of invasion depth, and can also replace pathologists in completing repetitive and tedious work, greatly improving efficiency and standardizing the process, thus solving the problems of the prior art. Attached Figure Description

[0017] Figure 1 The diagram shown is a structural schematic of an intelligent measurement system for the invasion depth of oral squamous cell carcinoma according to an embodiment of the present invention.

[0018] Figure 2 The diagram shown is a schematic diagram of pathological segmentation model training in one embodiment of the present invention.

[0019] Figure 3 The diagram shown is a schematic representation of a segmentation module in one embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram illustrating the measurement of invasion depth in one embodiment of the present invention.

[0021] Figure 5 The diagram shown is a structural schematic of a segmentation module according to an embodiment of the present invention;

[0022] Figure 6 The diagram shown is a schematic diagram of pathological segmentation model training in one embodiment of the present invention. Detailed Implementation

[0023] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0024] It should be noted that in the following description, reference is made to the accompanying drawings, which illustrate several embodiments of the invention. It should be understood that other embodiments may also be used, and changes in mechanical composition, structure, electrical system, and operation may be made without departing from the spirit and scope of the invention. The following detailed description should not be considered limiting, and the scope of the embodiments of the invention is defined only by the claims of the published patents. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. Spatially related terms, such as “upper,” “lower,” “left,” “right,” “below,” “below,” “lower part,” “above,” “upper part,” etc., may be used herein to illustrate the relationship between one element or feature shown in the figures and another element or feature.

[0025] Throughout this specification, when it is said that a part is "connected" to another part, this includes not only "direct connection" but also "indirect connection" by placing other elements in between. Furthermore, when it is said that a part "includes" a certain constituent element, unless otherwise stated otherwise, this does not exclude other constituent elements, but rather means that other constituent elements may also be included.

[0026] The terms "first," "second," and "third," etc., used herein are for the purpose of describing various parts, components, regions, layers, and / or segments, but are not limiting. These terms are used only to distinguish one part, component, region, layer, or segment from others. Therefore, the "first part," "component," "region," "layer," or "segment" described below may refer to a "second part," "component," "region," "layer," or "segment" without departing from the scope of this invention.

[0027] Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, operation, element, component, item, kind, and / or group, but do not preclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition arise only when combinations of elements, functions, or operations are inherently mutually exclusive in some manner.

[0028] In current clinical practice in pathology departments, the depth of invasion (DOI) is measured manually. This cumbersome manual measurement method not only wastes manpower and time, but is also prone to errors. Furthermore, inconsistent individual measurement standards lead to low accuracy in the measured depth of invasion.

[0029] With the rise of big data computing, artificial intelligence (AI) has rapidly developed in the medical field. Deep learning, as a new generation technology in AI, has achieved great success in automatic image classification and recognition. Deep neural networks possess powerful self-learning capabilities for features, but also require a large amount of labeled data for training. They also incorporate, aggregate, and optimize the "experience" of numerous doctors by combining actual diagnostic structures. Depending on the learning scheme, deep neural network learning can be divided into supervised learning, weakly supervised learning, unsupervised learning, and transfer learning, applied to various histopathological image analysis tasks. In supervised learning techniques, deep learning models can be categorized into three main types based on the nature of the task in histopathology: classification-based models (including object detection and image classification), regression, and segmentation models. The pioneering work in object detection was R-CNN based on the region proposal method, followed by deep learning frameworks such as SPP-Net, Fast R-CNN, Faster R-CNN, and Mask R-CNN. Deep learning neural networks for classification problems, starting with AlexNet, have evolved into a series of models such as VGGNet, GoogleNet, Inception, ResNet, Xception, and DenseNet, continuously improving performance metrics such as classification accuracy. These models differ in structure and network depth, but their basic idea is to use deep neural networks for feature extraction, and then feed the extracted features into a designed classifier to complete the classification. For semantic segmentation problems, deep learning neural networks have made breakthroughs by proposing Fully Convolutional Networks (FCNs), including DeepLab, PSPNet, U-Net, and the aforementioned Mask R-CNN, all achieving excellent segmentation results. These segmentation models share the common characteristic of first extracting deep features from the image and then combining these deep features with shallow features to achieve region localization and accurate segmentation.

[0030] In recent years, researchers have achieved remarkable results and made significant progress in pathological image analysis using supervised deep learning neural networks. These advancements include detecting mitotic cells, detecting tumor cells, segmenting tumors from normal tissues, classifying different cells based on nuclei with varying morphologies in multiple tumors, classifying different histological subtypes of the same tumor, detecting specific tumors, predicting tumor grading, and predicting disease-free survival using tumor-infiltrating lymphocyte abundance scores. These successful experiences demonstrate the enormous potential and bright future of artificial intelligence technology in the field of pathological image analysis.

[0031] Therefore, this invention provides an intelligent measurement system for the invasion depth of oral squamous cell carcinoma that combines pathology and artificial intelligence. This invention obtains segmented images containing the tumor region and the mucosal region from oral squamous cell carcinoma pathological images through a depth segmentation model, and locates two reference points located on the mucosal basement membrane that are closest to the target tumor region to predict the invasion depth of the oral squamous cell carcinoma pathological images. This achieves intelligent and automated measurement of invasion depth, and can also replace pathologists in completing repetitive and tedious work, greatly improving efficiency and standardizing the process, thus solving the problems of existing technologies.

[0032] The present invention will now be described in detail with reference to the accompanying drawings, so that those skilled in the art can readily implement it. The present invention can be embodied in many different forms and is not limited to the embodiments described herein.

[0033] like Figure 1 This invention presents a schematic diagram of the structure of an intelligent measurement system for the invasion depth of oral squamous cell carcinoma according to an embodiment of the present invention.

[0034] The system includes:

[0035] Segmentation module 11 is used to obtain segmented images containing tumor regions and mucosal regions based on oral squamous cell carcinoma pathological images using a depth segmentation model.

[0036] The invasion depth prediction module 12 is connected to the segmentation module 11 and is used to locate two reference points located on the mucosal basement membrane that are closest to the target tumor region based on the segmented image, so as to predict the invasion depth of the oral squamous cell carcinoma pathological image.

[0037] Optionally, current semantic segmentation based on noise labels in natural images often employs CAM combined with self-supervised learning, noise-robust loss functions, and label correction methods. However, since pathological images are composed of basic structures such as cells, with more texture than structural and edge information, CAM-based methods are not applicable, and their effectiveness is very limited from the perspective of loss functions and label correction.

[0038] Therefore, as Figure 2 As shown, the deep segmentation model includes a segmentation sub-model, an attention sub-model, and an FR prediction sub-model, which are used to output the preliminary segmentation results, attention map, and FR value of the corresponding tumor or mucosal area ratio of the corresponding oral squamous cell carcinoma pathological image, respectively.

[0039] The training method of the deep segmentation model includes: training the deep segmentation model based on the segmentation loss function related to the preliminary segmentation result, attention map and FR value, according to the pathological image training set; and wherein the pathological image training set includes: multiple pathological training images and corresponding labels for each pathological training image.

[0040] We propose using the metric FR (Filling Rate), which represents the percentage of tumor or mucosal area in the input image, to estimate this percentage through a model. This helps avoid overfitting the segmentation network to incorrect labels and improves the network's segmentation performance.

[0041] It should be noted that the multiple pathological training images in the pathological image training set can be preprocessed images, and the preprocessing methods include, but are not limited to, scaling, filtering, and cropping, or one or more of these.

[0042] Optionally, the deep segmentation model employs a U-Net encoder-decoder structure as the segmentation sub-model; then, the attention sub-model is connected in parallel to the decoder to obtain an attention map representing the saliency of tumor and mucosal regions; finally, the FR prediction sub-model is connected in parallel to the decoder to predict the tumor and mucosal area proportion (FR). Based on the segmentation loss function associated with the preliminary segmentation results, attention map, and FR value, a noise-label-based segmentation model is trained using a pathological image training set.

[0043] Optionally, the pathological training images include: coarsely annotated images, each with a tumor or mucosal area ratio greater than 0.7. This coarse annotation scheme can ensure a balance between annotation difficulty and workload, providing the most accurate annotation results possible at the lowest cost. The coarsely annotated data will provide supervision information for the training of the segmentation network, while a portion of the data will serve as a coarsely annotated test set to verify the network's segmentation performance to some extent.

[0044] Optionally, the segmentation loss function includes: a TopK cross-entropy loss function, a mean squared error term based on the attention map, and a FR regularization term. Preferably, the magnitude of the segmentation loss function is positively correlated with the proportion of tumor or mucosal area, forming an adversarial relationship to ensure that the network learns high-confidence supervised regions as much as possible, avoiding interference from noise labels and improving the network's segmentation performance.

[0045] Optionally, the segmentation loss function includes:

[0046]

[0047] Among them, l CE For the TopK cross-entropy loss function, l MSE For the mean squared error term based on the attention map and αFR as the FR regularization term, the p i Let y be the probability value of pixel i corresponding to each category in the prediction result. i FR is the category label of pixel i in the label of the pathological training image, FR is the proportion of tumor or mucosal area, and α is a hyperparameter.

[0048] Optionally, the segmentation module includes: a global segmentation submodule and / or a local segmentation submodule;

[0049] The global segmentation submodule is used to perform global segmentation on the global oral squamous cell carcinoma pathological image obtained by scaling the oral squamous cell carcinoma pathological image based on the depth segmentation model, so as to obtain a global segmentation image corresponding to the oral squamous cell carcinoma pathological image.

[0050] The local segmentation submodule is used to perform local segmentation on one or more local oral squamous cell carcinoma pathological images obtained by segmentation of oral squamous cell carcinoma pathological images based on the deep segmentation model, so as to obtain the local segmentation images corresponding to each local oral squamous cell carcinoma pathological image.

[0051] It should be noted that if only the global segmentation submodule is used for segmentation, the global segmentation image containing both the tumor region and the mucosal region can be directly used as the final output segmentation image for subsequent invasion depth prediction; if only the local segmentation submodule is used for segmentation, the final output segmentation image can be obtained directly from each local segmentation image containing both the tumor region and the mucosal region for subsequent invasion depth prediction; if both the global segmentation submodule and the local segmentation submodule are used for segmentation, the final output segmentation image can be obtained from both the global segmentation image containing both the tumor region and the mucosal region and each local segmentation image for subsequent invasion depth prediction.

[0052] Optionally, since the boundary between the tumor and the mucosa in pathological images is difficult to distinguish based on tissue cell morphology in local images, to ensure the network can correctly segment the tumor and mucosa regions, we need to use a global segmentation submodule and a local segmentation submodule. That is, we use two branches for segmentation, including a global branch and a local branch. The inputs are a scaled-down oral squamous cell carcinoma pathological image and a segmented oral squamous cell carcinoma pathological image, respectively. The global branch learns global structural information, while the local branch acquires local detail features of the image. Then, the features of the two branches are fused together to obtain greater global information while maintaining the image detail features, thereby enhancing the network's segmentation effect on oral pathological WSI images.

[0053] Therefore, as Figure 3 As shown, the segmentation module includes: a global segmentation submodule corresponding to a global branch, a local segmentation submodule corresponding to a local branch, and a global-local fusion submodule;

[0054] The global segmentation submodule is used to perform global segmentation on the global oral squamous cell carcinoma pathological image obtained by scaling the oral squamous cell carcinoma pathological image based on the depth segmentation model, so as to obtain a global segmentation image corresponding to the oral squamous cell carcinoma pathological image.

[0055] The local segmentation submodule is used to perform local segmentation on one or more local oral squamous cell carcinoma pathological images obtained by segmentation of oral squamous cell carcinoma pathological images based on the deep segmentation model, so as to obtain the local segmentation images corresponding to each local oral squamous cell carcinoma pathological image.

[0056] The global-local fusion submodule is used to fuse the global segmentation image and each local segmentation image corresponding to the oral squamous cell carcinoma pathological image to obtain a segmentation image containing the tumor region and the mucosal region.

[0057] Optionally, the method of fusing the global segmentation image and each local segmentation image corresponding to the oral squamous cell carcinoma pathological image includes: applying a regularization term constraint to the global segmentation image and each local segmentation image corresponding to the oral squamous cell carcinoma pathological image; preferably, the mean square error between the global segmentation image and each local segmentation image corresponding to the oral squamous cell carcinoma pathological image is used as a regularization term constraint.

[0058] Optionally, since the average resolution of a 200x pathological image is over 10000x10000, directly using a 200x pathological image for data preprocessing would lead to insufficient computer memory or excessive computational load. Therefore, the oral squamous cell carcinoma pathological image is scaled down from 200x to 50x to obtain a global detection image.

[0059] Optionally, considering the large size of whole-slice pathological images and the limited GPU memory, these images cannot be directly input into deep learning models. They need to be segmented into one or more localized oral squamous cell carcinoma pathological images. We use a 1024x1024 image size, with a 200-pixel overlap between adjacent localized oral squamous cell carcinoma pathological images to ensure effective segmentation of the edge regions. Simultaneously, based on the tissue area ratio, localized oral squamous cell carcinoma pathological images with excessively small tissue areas are filtered out to reduce computational load.

[0060] Optionally, since the algorithm is based on reference points, the most direct and simple method for fitting the horizontal line is to locate two reference points on the left and right, connect them to form a horizontal line, and then measure the depth of tumor invasion. This reference point refers to the point on the mucosal basement membrane that is closest to the tumor. Considering that the basement membrane is undulating in pathological images, we need to further clarify and simplify the location of the reference point, limiting it to a position close to the average thickness of the mucosa at a distance from the lesion surface.

[0061] Therefore, the invasion depth prediction module 12 is further configured to: determine the location information of the target tumor region based on the segmented image; determine the location information of the two mucosal regions on the tissue surface that are closest to the target tumor region based on the location information of the target tumor region; obtain the location information of the two reference points on the mucosal basement membrane that are closest to the target tumor region based on the location information of the target tumor region and the location information of the two corresponding mucosal regions; and calculate the invasion depth based on the location information of the two reference points.

[0062] Optionally, determining the location information of the target tumor region based on the segmented image includes: preprocessing the segmented image; extracting the tissue outer contour in the preprocessed segmented image; determining the outer contour of the target tumor region in the tissue outer contour, and obtaining the location information of the outer contour of the target tumor region.

[0063] Optionally, preprocessing the segmented image includes filtering the segmented image, specifically filtering non-tissue regions, including background, shadows, stains, residual tissue, and handwriting. Statistical analysis of the grayscale values ​​of different regions of the image reveals that the RGB channel values ​​of the background region are generally similar and can be processed using grayscale filtering; the RGB channel grayscale values ​​of the tissue region mostly follow a certain distribution, and stains and handwriting can be filtered out using RGB channels. Furthermore, connected component analysis is needed for the pathological image to filter out interference from fine tissue by filtering connected components with smaller areas.

[0064] Optional, such as Figure 4 As shown, calculating the invasion depth based on the positional information of two mucosal reference points A and B includes: calculating the tumor invasion direction based on the positional information of the two mucosal reference points A and B and all tumor regions in the segmented image. To obtain the depth of invasion, we need to find the deepest point of tumor invasion, C.

[0065] To better illustrate the above-mentioned intelligent measurement system for the invasion depth of oral squamous cell carcinoma, the present invention provides the following specific embodiments.

[0066] Example: Construction method and application of an intelligent measurement system for the invasion depth of oral squamous cell carcinoma.

[0067] I. Construct a digital image database of oral mucosal squamous cell carcinoma of a certain scale:

[0068] 1. Data Acquisition: Images of 362 pathological sections from 244 cases of oral mucosal SCC were acquired using a slide scanner, resulting in 200x pathological images.

[0069] 2. Data labeling

[0070] 2.1 Coarse-labeled dataset: 183 cases were used for coarse labeling, of which 138 were used as the training set and 45 as the test set. The coarse labeling scheme used ImageScope as the labeling software. The tumor and mucosal regions were completely enclosed in polygons by drawing a pen, ensuring that no tumor or mucosal tissue was missed. However, normal fibrous connective tissue between tumor nests could be enclosed to ensure that the tumor or mucosal area accounted for more than 70%.

[0071] 2.2 Fine-Annotated Dataset: 30 examples were used for fine-annotation, serving as a test set. The fine-annotation scheme first divided the 200x WSI image into 5000x5000 sub-images. Then, Labelme was used as the annotation software to accurately annotate tumors and mucosal tissues by marking dots, strictly distinguishing between tumors, mucosa, and normal tissues. Using this fine-annotated data as a test set avoids interference from noisy labels and truly reflects the actual segmentation performance of the segmentation network.

[0072] II. Generation of the segmentation module;

[0073] like Figure 5 As shown, this includes: global branch training and local branch training;

[0074] For training the global segmentation submodule; such as Figure 6 As shown, segmented images, attention maps, and FR values ​​are obtained by training sequentially based on a scaled coarsely labeled dataset: based on the segmentation loss function, the segmentation loss is calculated based on the segmented images, attention maps, and FR values, and then gradient backpropagation is calculated to update the deep segmentation model; wherein, the scaled coarsely labeled dataset includes: multiple scaled coarsely labeled images and their corresponding scaled coarse labels.

[0075] For the training of the local segmentation submodule: the segmented image, attention map, and FR value are obtained by training sequentially according to the coarsely labeled dataset of the cuts: based on the segmentation loss function, the segmentation loss is calculated according to the segmented image, attention map, and FR value, and then the gradient backpropagation is calculated to update the deep segmentation model; wherein, the coarsely labeled dataset of the cuts includes: multiple coarsely labeled images after cutting and their corresponding coarse labels for the cuts.

[0076] The global-local fusion submodule fuses the outputs of the global segmentation submodule and the local submodule to generate the final overall segmentation image.

[0077] We used a finely labeled dataset as the test set to ensure an objective and accurate evaluation of the segmentation module's performance. We used mean Intersection of Union (mIOU) and Mean Pixel Accuracy (MPA) as evaluation metrics to measure segmentation performance. Our software currently has an mIOU of 0.836 and an MPA value of 0.925.

[0078] II. Construction of an Intelligent Measurement System for Oral Squamous Cell Carcinoma Invasion Depth

[0079] The system includes: a segmentation module for performing global and local segmentation on oral squamous cell carcinoma pathological images based on a depth segmentation model to obtain segmented images containing tumor regions and mucosal regions; and an invasion depth prediction module connected to the segmentation module for determining the location information of the target tumor region based on the segmented image; determining the location information of the two mucosal regions located on the tissue surface closest to the target tumor region based on the location information of the target tumor region; obtaining the location information of the two reference points located on the mucosal basement membrane closest to the target tumor region based on the location information of the target tumor region and the expected location information of the two corresponding mucosal regions; and calculating the invasion depth based on the location information of the two reference points.

[0080] III. Evaluation and validation of the intelligent measurement system for oral squamous cell carcinoma invasion depth;

[0081] 1. Model Performance Evaluation

[0082] We used 78 cases of oral mucosal SCC as the test set to compare the tumor invasion depth prediction results based on segmentation results with the results of manual measurement by the physicians. Two evaluation metrics were proposed: mean invasion depth offset and tumor T-staging accuracy. Furthermore, we need to analyze the stability and robustness of the prediction model from a statistical perspective, specifically the variance, worst-case scenario, and distribution map of the statistical horizontal deviation and invasion depth offset.

[0083] 2. Verification through clinical pathology practice

[0084] Clinical pathology practice validation will examine the model's accuracy, safety, speed, and usability. This project will compare the model with the diagnostic results of multiple doctors in actual work to verify its practical accuracy. Secondly, the project will statistically analyze the model's reliability and serious error rates to determine its usability range and issue warnings for unreliable cases to ensure safety. Next, the project will compare the model's inference speed with the diagnostic speed of doctors to verify its theoretical speed. Finally, the project will validate the software system's usability from the perspectives of ease of operation, data visualization, data management, and user feedback.

[0085] We can also develop application software systems using the intelligent measurement system for the invasion depth of oral squamous cell carcinoma. This system, based on Python and the Django framework, establishes a visual interface and database management backend, enabling intelligent measurement of DOI and pathological staging for oral squamous cell carcinoma. In the software system's import interface, users select the digitized pathological image to be analyzed. After import, automatic processing yields the visualization results. The left side of the visualization interface displays the original image, the middle shows a schematic diagram obtained after model segmentation, and the right side displays specific parameters (including maximum tumor diameter, invasion depth, tumor T stage, image description, etc.) and functional modules (including manually annotated DOI, manually annotated maximum diameter, overlay mode, dual-image mode, etc.). Clicking the "Manually Annotated DOI" module leads to a manual annotation interface, allowing for manual correction of cases with automatic measurement deviations. Correction is extremely convenient; only three reference points need to be determined for automatic DOI value calculation.

[0086] The application software system developed based on the intelligent measurement system for oral squamous cell carcinoma invasion depth is more objective and accurate for invasion depth measurement: the DOI value is accurate to 0.1mm; the measurement is more efficient, faster and less tiring: a large number of pathological images can be imported at the same time and automatically analyzed at one time. The measurement of one case takes only 64 seconds and there is no limit to the working time. It is also simple to operate, easy to promote and can be operated by people without professional pathology knowledge.

[0087] In summary, the intelligent measurement system for the invasion depth of oral squamous cell carcinoma of the present invention obtains segmented images of the tumor region and mucosal region from oral squamous cell carcinoma pathological images through a depth segmentation model, and locates two reference points on the mucosal basement membrane closest to the target tumor region to predict the invasion depth of the oral squamous cell carcinoma pathological image. This achieves intelligent and automated measurement of invasion depth, and can also replace pathologists in performing repetitive and tedious tasks, greatly improving efficiency and standardizing procedures, thus solving the problems of existing technologies. Therefore, the present invention effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0088] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. An intelligent measurement system for the invasion depth of oral squamous cell carcinoma, characterized in that, The system includes: The segmentation module is used to obtain segmented images containing both the tumor region and the mucosal region based on oral squamous cell carcinoma pathological images using a depth segmentation model. An invasion depth prediction module, connected to the segmentation module, is used to locate two reference points located on the mucosal basement membrane that are closest to the target tumor region based on the segmented image, so as to predict the invasion depth of the oral squamous cell carcinoma pathological image. The deep segmentation model includes a segmentation sub-model, an attention sub-model, and an FR prediction sub-model, which are used to output the preliminary segmentation results, attention map, and FR value of the corresponding tumor or mucosal area ratio of the corresponding oral squamous cell carcinoma pathological image, respectively. The training methods for the deep segmentation model include: Based on the segmentation loss function associated with the preliminary segmentation results, attention map, and FR value, the deep segmentation model is trained using a pathological image training set. Furthermore, the pathological image training set includes: multiple pathological training images and corresponding labels for each pathological training image; The segmentation loss function includes: ; in, The TopK cross-entropy loss function, For the mean squared error term based on the attention map and αFR as the FR regularization term, the... This represents the probability value of pixel i corresponding to each category in the prediction result. FR is the category label of pixel i in the label of the pathological training image, FR is the proportion of tumor or mucosal area, and α is a hyperparameter.

2. The oral squamous cell carcinoma invasion depth intelligent measuring system according to claim 1, characterized in that, The segmentation loss function includes: TopK cross-entropy loss function, mean squared error term based on attention map, and FR regularization term.

3. The oral squamous cell carcinoma invasion depth intelligent measuring system according to claim 1, characterized in that, The segmentation module includes: a global segmentation submodule and / or a local segmentation submodule; The global segmentation submodule is used to perform global segmentation on the global oral squamous cell carcinoma pathological image obtained by scaling the oral squamous cell carcinoma pathological image based on the depth segmentation model, so as to obtain a global segmentation image corresponding to the oral squamous cell carcinoma pathological image. The local segmentation submodule is used to perform local segmentation on one or more local oral squamous cell carcinoma pathological images obtained by segmentation of oral squamous cell carcinoma pathological images based on the deep segmentation model, so as to obtain the local segmentation images corresponding to each local oral squamous cell carcinoma pathological image.

4. The oral squamous cell carcinoma invasion depth intelligent measuring system according to claim 3, characterized in that, The segmentation module further includes: The global-local fusion submodule is used to fuse the global segmentation image and each local segmentation image corresponding to the oral squamous cell carcinoma pathological image to obtain a segmentation image containing the tumor region and the mucosal region.

5. The oral squamous cell carcinoma invasion depth intelligent measuring system according to claim 1, wherein The method of locating two reference points located on the mucosal basement membrane closest to the target tumor region based on the segmented image to predict the invasion depth of the oral squamous cell carcinoma pathological image includes: Based on the segmented image, the location information of the target tumor region is determined; Based on the segmented image, the location information of the two mucosal regions on the tissue surface that are closest to the target tumor region is determined according to the location information of the target tumor region. Based on the location information of the target tumor region and the location information of the two corresponding mucosal regions, the location information of the two reference points located on the mucosal basement membrane that are closest to the target tumor region is obtained. The invasion depth is calculated based on the location information of two reference points.

6. The intelligent measurement system for oral squamous cell carcinoma invasion depth according to claim 5, characterized in that, The step of determining the location information of the target tumor region based on the segmented image includes: The segmented image is preprocessed; Extract the outer contour of tissue from the preprocessed segmented image; The outer contour of the target tumor region is determined within the outer contour of the tissue, and the positional information of the outer contour of the target tumor region is obtained.

7. The oral squamous cell carcinoma invasion depth intelligent measuring system according to claim 5, characterized in that, The calculation of the invasion depth based on the location information of two reference points includes: Based on the location information of the two reference points and all tumor regions in the segmented image, the tumor invasion direction and the deepest point of tumor invasion are calculated to obtain the invasion depth.

8. The oral squamous cell carcinoma invasion depth intelligent measuring system according to claim 1, wherein The pathological training images include: roughly annotated images, where the tumor or mucosal area ratio is greater than 0.7.

Images