Oral mucosa malignant transformation risk model training method, system, medium and device

Abstract

The application relates to the technical field of oral lesion risk screening, and particularly relates to a method, system, medium and equipment for training an oral mucosa malignant transformation risk model, which comprises the following steps: acquiring a first oral sample set, wherein the first oral sample set comprises a plurality of groups of first spectrum selection images and second spectrum selection images obtained by photographing a plurality of first objects and second objects; the oral condition of the first objects is healthy, and the oral condition of the second objects is pathological or has a pathological tendency; performing sample expansion on the first oral sample set by using an image processing method to generate a second oral sample set; and performing deep learning on the second oral sample set by using a multi-order dropout mode to train an output oral mucosa malignant transformation risk model. The oral mucosa malignant transformation risk model provided by the application can effectively avoid problems such as gradient disappearance and overfitting under a small oral sample set, and improves the accuracy of identification.

Description

Technical Field

[0001] This invention relates to the field of oral lesion risk screening technology, specifically to a method, system, medium, and equipment for training a risk model of malignant transformation of oral mucosa. Background Technology

[0002] Currently, AI-based image recognition technology has been gradually applied in the field of medical testing.

[0003] For example, patent application CN119540261A discloses an automatic medical image segmentation method based on a dual-convergent hybrid neural network. This method includes: inputting the medical image to be segmented into a pre-trained medical image segmentation network and outputting the medical image segmentation result; the medical image segmentation network employs a dual-convergent hybrid U-shaped network, which includes two parallel encoders and a single decoder, the two parallel encoders being a convolutional neural network and a Transformer, respectively; simultaneously extracting local and global features of the medical image through the convolutional neural network and the Transformer, respectively; performing multi-scale feature fusion on the extracted local and global features through a collaborative feature aggregation module (CFA) and a non-collaborative feature aggregation module (NCFA); and inputting the multi-scale fused features into the decoder to output the medical image segmentation result.

[0004] For example, patent application CN118115740A discloses a medical image referencing segmentation method and apparatus based on multi-scale deep learning, including: acquiring a patient's tumor CT image and corresponding text description; embedding the text description into several tokens, and then inputting them into a Transformer-based language encoder to form high-dimensional word vectors to obtain language features; based on the tumor CT image and language features, performing visual feature encoding and multi-modal feature fusion using MSFA through four stages with decreasing spatial resolution; upsampling the fused visual features output from each stage using bilinear interpolation, scaling the number of channels of the multi-modal feature map output from each stage to a uniform dimension, and then stitching them together to obtain the final feature map, which is then input into a full-scale decoder for decoding to obtain a pixel-level segmentation mask marking the target object.

[0005] For example, patent application CN 119446490 A discloses an early tongue cancer occult lymph node metastasis prediction system, electronic device, and storage medium. The early tongue cancer occult lymph node metastasis prediction system includes: a data acquisition module for acquiring head and neck MRI images, wherein the head and neck MRI images are MRI images of the acquired target object; an automatic tumor detection model for performing target detection on the head and neck MRI images to obtain tumor region images; and a lymph node metastasis prediction model for performing lymph node metastasis prediction classification on the tumor region images to obtain lymph node metastasis classification results.

[0006] However, the applicant noted that existing AI model networks will have many problems when dealing with small sample images with highly repetitive surface information, such as vanishing gradients and overfitting. Summary of the Invention

[0007] The purpose of this invention is to provide a training method, system, medium, and device for a risk model of malignant transformation of oral mucosa, which partially solves or alleviates the above-mentioned deficiencies in the prior art and can solve the overfitting problem in scenarios where oral samples are scarce.

[0008] To solve the aforementioned technical problems, the present invention specifically adopts the following technical solution: A first aspect of the present invention is to provide a training method for a risk model of malignant transformation of oral mucosa, the risk model of malignant transformation of oral mucosa comprising: a first model network and a second model network connected to each other, comprising the steps of: S101, Obtain a first oral cavity sample set, the first oral cavity sample set including: multiple sets of first spectral selection images taken for multiple first objects, multiple sets of second spectral selection images taken for second objects, and at least two of the first spectral selection images and / or the at least two of the second spectral selection images simultaneously cover a first region and a second region, wherein the first region is the inner wall of the oral cavity, and the second region includes one or more of the following: teeth, gums, and tongue; the oral cavity condition of the first objects is healthy, and the oral cavity condition of the second objects is diseased or has a tendency to become diseased; S102, at least one type of image processing method is used to expand the first oral cavity sample set to generate a second oral cavity sample set; S103, employing a multi-stage discarding pattern to perform deep learning on the second oral sample set to train and output the oral mucosal malignant transformation risk model; wherein, S103 includes the following steps: S1031, a first training sample set is selected from the second oral cavity sample set, and the first training sample set is input into the first model network to extract local features from the first training sample set; wherein, the first model network includes: (1) an image input layer for receiving an image; (2) an image segmentation layer for segmenting the received image to obtain multiple image blocks; (3) a first mapping layer for projecting the image blocks into low-dimensional embedding vectors through linear mapping; (4) a second mapping layer for mapping the low-dimensional embedding vectors to a high-dimensional embedding space and outputting the corresponding vector results through position embedding; S1032, The vector result is input into the second model network, and the second model network sets a first drop-off function to drop some paths during the first learning stage; S1033, the second network model sets a second dropout function to drop some neurons during the second learning phase; after the second model network converges, the oral mucosa malignant transformation risk model is trained.

[0009] In some embodiments, the steps further include: S104, verify the accuracy of the oral mucosal malignant transformation risk model.

[0010] In some embodiments, S104 includes the step of: S1041, Obtain the first accuracy of the oral mucosal malignant transformation risk model on the first training sample set; S1042, determine whether the first accuracy is greater than the first preset threshold; If so, proceed with the following steps: S1043, Select a first validation sample set from the second oral sample set, and calculate the second accuracy of the oral mucosal malignant transformation risk model on the first validation sample set; S1044, Calculate the first difference between the first accuracy and the second accuracy; S1045, determine whether the first difference is greater than a preset first threshold; If so, proceed with the following steps: S1046, At least one type of image processing method is used to perform sample expansion on at least two sets of the second spectral selection images to form a supplementary oral sample set; S1047, update the supplementary oral sample set to the second oral sample set, and return to step S103.

[0011] In some embodiments, if the determination result of S1042 is negative, then the following steps are executed: S1048, at least one type of image processing method is used to perform sample expansion on at least one set of the first spectral selection images and at least one set of the second spectral selection images to form supplementary oral samples, and then the process returns to step S1047.

[0012] In some embodiments, in S102, at least two types of image processing techniques are used to expand the first oral cavity sample set to generate two second oral cavity sample sets. Correspondingly, the method further includes the step of: If the judgment result of S1045 is negative, the oral mucosa malignant transformation risk model is defined as a qualified model. At least two qualified models are obtained, and the at least two qualified models are fused to obtain a new oral mucosal malignant transformation risk model.

[0013] In some embodiments, the spectral selection images corresponding to the two second oral sample sets are not repeated.

[0014] In some embodiments, the two second oral cavity sample sets are generated using different image processing techniques.

[0015] In some embodiments, the image processing means is a combination of two or more image processing methods, wherein the types of the image processing methods include one or more of the following: adaptive histogram equalization, noise reduction, color correction, local occlusion, background segmentation, contrast stretching, image pyramid, rotation, scaling, translation, brightness adjustment, contrast adjustment, and saturation adjustment.

[0016] This invention also provides a training system for a risk model of malignant transformation of oral mucosa, the risk model of malignant transformation of oral mucosa comprising: a first model network and a second model network connected together, the training system comprising: The sample acquisition module is used to acquire a first oral cavity sample set, which includes: multiple sets of first spectral selection images taken for multiple first subjects, and multiple sets of second spectral selection images taken for second subjects, wherein at least two of the first spectral selection images and / or the at least two of the second spectral selection images simultaneously cover a first region and a second region, wherein the first region is the inner wall of the oral cavity, and the second region includes one or more of the following: teeth, gums, and tongue; the oral cavity condition of the first subjects is healthy, and the oral cavity condition of the second subjects is diseased or has a tendency to become diseased; The sample expansion module is used to expand the first oral cavity sample set using at least one type of image processing method to generate a second oral cavity sample set; A multi-stage training module is used to perform deep learning on the second oral sample set using a multi-stage dropout pattern to train and output the oral mucosal malignant transformation risk model; wherein, the multi-stage training module includes: A first-order training unit is used to select a first training sample set from the second oral cavity sample set and input the first training sample set into a first model network to extract local features from the first training sample set; wherein, the first model network includes: (1) an image input layer for receiving an image; (2) an image segmentation layer for segmenting the received image to obtain multiple image blocks; (3) a first mapping layer for projecting the image blocks into low-dimensional embedding vectors through linear mapping; and (4) a second mapping layer for mapping the low-dimensional embedding vectors to a high-dimensional embedding space and outputting the corresponding vector results through position embedding. The second-order training unit is used to input the vector results into the second model network, and the second model network sets a first drop-off function to drop some paths during the first learning stage; The third-order training unit is used to continue the second network model in the second learning phase by setting a second dropout function to drop some neurons; after the second model network converges, the oral mucosa malignant transformation risk model is trained.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a risk model of malignant transformation of the oral mucosa as described in any embodiment.

[0018] The present invention also provides an electronic device, the electronic device comprising: a memory for storing computer-executable instructions; and a processor for executing the computer-executable instructions stored in the memory to implement the oral mucosal malignant transformation risk model training method described in any embodiment.

[0019] Beneficial technical effects: Traditional detection network designs used in medical images have three problems when applied to spectral selection images: 1) The number of spectral selection images is relatively limited; 2) The spectral selection images have extremely high repeatability; 3) There are many interference factors in spectral selection images due to saliva, teeth and other reasons.

[0020] To address the learning scenario involving small sample sets with numerous and highly repetitive interference factors, this invention provides a training method for a multi-stage dropout model based on randomly expanded sample sets. Specifically, this invention employs multiple processing combinations to randomly expand a small number of spectral selection images, thereby expanding the learnable information of the model. Furthermore, during training, a multi-stage dropout pattern is used to progressively discard the expanded learning information, thus mitigating or avoiding overfitting issues that may arise in scenarios with random expansion and highly repetitive image information.

[0021] Furthermore, tests show that this new training method can perform extensive deep learning with limited small samples, thus significantly mitigating problems such as sample repetition and excessive interference. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0023] Figure 1 This is a schematic diagram of the method flow in an exemplary embodiment of the present invention; Figure 2 This is a schematic diagram of a set of spectral selection images in a specific embodiment of the present invention; Figure 3 This is a schematic diagram of another set of spectral selection images in a specific embodiment of the present invention; Figure 4 This is a network model architecture diagram in an exemplary embodiment of the present invention; Figure 5 This is a schematic diagram of the display interface of the training results in a verification embodiment of the present invention; Figure 6 This is a graph showing the accuracy of a model trained on the same oral cavity sample using existing technology. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0025] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0026] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0027] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0028] In this document, "and / or" includes any and all combinations of one or more of the listed related items.

[0029] In this article, "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0030] As used in this specification, the term "about" typically means + / -5% of the value, more typically + / -4% of the value, more typically + / -3% of the value, more typically + / -2% of the value, even more typically + / -1% of the value, and even more typically + / -0.5% of the value.

[0031] In this specification, certain embodiments may be disclosed in a range-bound format. It should be understood that this "range-bound" description is merely for convenience and brevity and should not be construed as a rigid limitation on the disclosed range. Therefore, the description of a range should be considered as having specifically disclosed all possible subranges and the individual numerical values ​​within those ranges. For example, a description of the range 1-6 should be considered as having specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and the individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This rule applies regardless of the breadth of the range.

[0032] In this article, the oral condition of the second subject (or patient) is defined as a lesion, referring to medical lesions such as oral mucosal ulceration, erosion, plaques, and herpes. In other words, in the field of medical testing, there are usually conventional testing indicators or methods for various oral diseases. When the oral cavity exhibits characteristic manifestations that would be identified as lesions under conventional testing indicators or methods, the subject's oral cavity is considered to have a lesion.

[0033] Typically, before significant pathological manifestations appear in a subject's oral cavity, some associated features (such as abnormal color, inconspicuous or small ulcers or plaques) may emerge. These associated features often indicate a tendency for later pathological development. Therefore, when these associated features are observed in a subject's oral cavity, it can be considered that the subject has a tendency for pathological development.

[0034] In this article, "spectrally selected images" can refer to images (such as photographs) captured by an endoscope under blue light 415nm and green light 540nm spectral illumination.

[0035] Oral cancer is one of the major diseases threatening human health worldwide. According to the World Health Organization (WHO), in 2020, there were over 377,000 new cases of oral cancer globally, with 177,000 deaths. The five-year survival rate is less than 50%, mainly attributed to the high rate of late-stage diagnosis (>60%). Oral squamous cell carcinoma (OSCC) accounts for more than 90% of oral cancer cases, and most develop from underlying oral malignancies (OPMD). OPMD is a group of oral mucosal diseases or conditions with a risk of malignant transformation, including oral leukoplakia, oral lichen planus, and oral erythroplakia. Studies show that the malignant transformation rate of OPMD varies significantly, ranging from 1.4% to 49.5%, with an overall malignant transformation rate of 7.9%. The risk of malignant transformation in OPMD is closely related to the degree of epithelial dysplasia; the risk of malignant transformation is significantly higher in individuals with moderate / severe epithelial dysplasia than in those with no or mild epithelial dysplasia. In current clinical practice, the risk assessment of malignancy in OPMD is mainly guided by routine oral examinations, primarily visual and tactile examinations, with histopathological results serving as the gold standard.

[0036] However, the applicant noted that testing in clinical practice often has significant limitations: 1) Different risk levels of OPMD (such as low-risk mild dysplasia and high-risk moderate / severe dysplasia) have overlapping clinical manifestations, which can easily lead to misjudgment of malignancy risk during routine artificial oral examinations. 2) Pathological diagnosis is subject to sampling errors and inter-observer variability, which may delay the intervention time for high-risk cases; biopsy is invasive and cannot dynamically monitor the evolution of lesions.

[0037] Furthermore, since the risk assessment of malignancy in OPMD is highly dependent on the experience of specialist physicians, even professional medical personnel need to undergo complex training to identify the vascular morphology grading characteristics. This also makes it easier for low-risk cases to be overtreated or high-risk cases to be missed in areas with scarce medical resources.

[0038] In response, the applicant proposed a risk model for malignant transformation of oral mucosa (or simply: risk model) to achieve intelligent recognition of spectral selection images through AI technology. This intelligent recognition method can provide doctors with auxiliary suggestions, reduce the high dependence on the professional experience of specialist doctors to a certain extent, and reduce the difficulty of assessing the risk of malignant transformation of OPMD.

[0039] It should be noted that the applicant observed numerous problems with traditional detection network designs used in medical imaging when applied to spectral selection images. In particular, spectral selection images differ significantly in information from traditional pathological images, making image learning more challenging. 1) The complex internal environment of the oral cavity leads to numerous interfering elements in the images. For example, the mucous membrane and saliva can cause reflections during the photographing process. Furthermore, the complex environment may result in the capture of images of teeth, gums, and other interfering areas. Additionally, the oral mucosa is not perfectly smooth, especially near the teeth, where small bumps and depressions often appear, potentially creating irregular shadows in the image.

[0040] 2) The spectral selection images have extremely high repeatability. Especially to achieve observation of key areas, some images will focus on the same (or similar) areas of the oral cavity wall. For example, see... Figure 3 See Figures (a) and (b) in the text, or refer to Figure (a) and (b) in the text. Figure 3 As can be seen from Figures (c) and (d), the information repetition (or content similarity) of the spectral selection images is very high, which makes it very easy for traditional model learning methods to overfit.

[0041] 3) The quantity of existing sample data (such as spectral selection images) is relatively limited, and training models with small sample sets increases the difficulty of improving model accuracy. On the one hand, unlike conventional data collection, medical data collection must comply with strict ethical standards, and patient privacy protection requirements increase the complexity of data acquisition. On the other hand, taking spectral selection image acquisition as an example, specialists must undergo long-term professional training to operate spectral selection technology proficiently, and the recognition of spectral selection images also relies heavily on the judgment of clinical specialists, resulting in long sample acquisition timelines and high economic costs.

[0042] In response to this learning scenario involving small sample sets with numerous and highly repetitive interference factors, this invention provides a method with multi-level discarding modes to progressively filter information from spectral selection images, thereby eliminating the impact of repetitive and interference information from small sample sets on model learning.

[0043] Example 1 See Figure 1 As shown, this invention provides a training method for an oral mucosal malignant transformation risk model, the oral mucosal malignant transformation risk model comprising: a first model network and a second model network connected together, including the following steps: S101, Obtain a first oral cavity sample set, the first oral cavity sample set including: multiple sets of first spectral selection images taken for multiple first objects, multiple sets of second spectral selection images taken for second objects, and at least two of the first spectral selection images and / or at least two of the second spectral selection images simultaneously cover a first region and a second region, wherein the first region is the inner wall of the oral cavity, and the second region includes one or more of the following: teeth, gums, and tongue; the oral cavity condition of the first objects is healthy, and the oral cavity condition of the second objects is diseased or has a tendency to become diseased; For example, in some embodiments, the spectral selection image can be an image obtained by spectral selection imaging based on an endoscope.

[0044] In this embodiment, the oral cavity inner wall surface generally refers to the oral mucosa layer, which is a type of connective tissue covering the inside of the oral cavity. It is soft in texture and pink in color, and serves as a protective barrier inside the oral cavity. For example, the oral cavity inner wall surface generally refers to one or more of the following areas: buccal mucosa, hard palate, soft palate, floor of mouth, retromolar triangle, oral vestibule, oral cavity proper, palatopharyngeal arch, palatoglossal arch, and inner labial mucosa.

[0045] S102, at least one type of image processing method is used to expand the first oral cavity sample set to generate a second oral cavity sample set; In this invention, image processing techniques are used to expand the number of spectral selection images (or sample photos), thereby significantly increasing the number of samples.

[0046] For example, in some embodiments, an image processing means may include at least one image processing method. Preferably, an image processing means may include a combination of two or more image processing methods, and the two or more image processing methods may have a specific execution order. Furthermore, an image processing means may include one or more combinations of processing methods.

[0047] For example, in some embodiments, an image processing method may include: processing combination 1, which uses at least two image processing methods to process a spectral selection image in a specific order to obtain a new spectral selection image (or a supplementary image); and processing combination 2, which uses at least two different image processing methods to process a spectral selection image to obtain at least two new spectral selection images.

[0048] For example, in some embodiments, the second oral sample set in S102 includes multiple sets of supplementary images, and each set of supplementary images includes multiple supplementary images obtained by using multiple different image processing methods based on the same spectral selection image.

[0049] For example, in some embodiments, the process of sample expansion of a spectral selection image using image processing means is as follows: the spectral selection image is processed using one image processing method (such as partial occlusion) to obtain a new spectral selection image (i.e., supplementary image), and the same spectral selection image is processed using another image processing method (such as contrast enhancement) to obtain a new spectral selection image.

[0050] For example, in some embodiments, a set of supplementary images may include: a new spectral selection image formed by sequentially processing a spectral selection image using multiple image processing methods.

[0051] S103, employing a multi-stage discarding pattern to perform deep learning on the second oral sample set to train and output the oral mucosal malignant transformation risk model; wherein, S103 includes the following steps: S1031, a first training sample set is selected from the second oral cavity sample set, and the first training sample set is input into the first model network to extract local features from the first training sample set; wherein, the first model network includes: (1) an image input layer for receiving an image; (2) an image segmentation layer for segmenting the received image to obtain multiple image blocks; (3) a first mapping layer for projecting the image blocks into low-dimensional embedding vectors through linear mapping; (4) a second mapping layer for mapping the low-dimensional embedding vectors to a high-dimensional embedding space and outputting the corresponding vector results through position embedding; Preferably, the image block can be a local graphic of the inner wall of the oral cavity, and can also be a local graphic of areas such as teeth, gums, and tongue.

[0052] S1032, The vector result is input into the second model network, and the second model network sets a first drop-off function to drop some paths during the first learning stage; S1033, the second network model sets a second dropout function to drop some neurons during the second learning phase; after the second model network converges, the oral mucosa malignant transformation risk model is trained.

[0053] Preferably, in this embodiment, in order to expand the number of samples, multiple image processing methods can be used to randomly expand the first sample image set. That is, the image processing method (or specific processing combination) can be randomly selected under the preferred image processing method according to the expansion quantity requirements. This random sample expansion can also reduce the user's image processing pressure to a certain extent.

[0054] Preferably, see Figure 4As shown, the network architecture of this invention employs a combination of a first model network and a second model network. A multilayer perceptron and a classification output layer are sequentially connected to the end of the second model network. The second model network incorporates a dual-dropout mode (e.g., using first and second dropout functions for information filtering).

[0055] Specifically, in the first learning phase, the second model network sequentially includes the following network layers: LayerNorm, multi-head self-attention, layer normalization, and regularization (equivalent to the first dropout function). In the second learning phase, the second model network sequentially includes the following network layers: layer normalization, linear layer, activation function, random dropout function (equivalent to one of the second dropout functions), linear layer, and random dropout function (equivalent to one of the second dropout functions). In this embodiment, the setting of multi-level dropout functions can effectively filter the augmented and duplicate information in the sample set, thereby effectively avoiding problems such as gradient vanishing and overfitting caused by sample set defects.

[0056] In summary, this invention provides a training method for a multi-stage dropout model based on a randomly expanded sample set. Preferably, this invention employs multiple processing combinations to randomly expand a small number of spectral selection images to expand the learnable information of the model. Furthermore, during training, a multi-stage dropout mode is used to progressively discard the expanded learning information, thereby mitigating or avoiding overfitting problems that may be encountered in scenarios with random expansion and highly repetitive image information.

[0057] In other words, to address the triple shortcomings of small sample size, high sample redundancy, and abundant interference information, this invention provides a model training method based on a multi-stage discarding pattern of randomly expanded sample sets. Furthermore, the applicant notes that this novel training method can leverage a limited number of small samples for extensive deep learning, thus significantly mitigating problems such as sample redundancy and abundant interference information.

[0058] In some embodiments, the steps further include: S104, verify the accuracy of the oral mucosal malignant transformation risk model.

[0059] In some embodiments, S104 includes the step of: S1041, Obtain the first accuracy of the oral mucosal malignant transformation risk model on the first training sample set; S1042, determine whether the first accuracy is greater than the first preset threshold; For example, if the judgment result of S1042 is yes, it can usually be preliminarily considered that the accuracy of the model on the training sample set is relatively reliable.

[0060] If the judgment result of S1042 is yes, then execute the following steps: S1043, Select a first validation sample set from the second oral sample set, and calculate the second accuracy of the oral mucosal malignant transformation risk model on the first validation sample set; S1044, Calculate the first difference between the first accuracy and the second accuracy; S1045, determine whether the first difference is greater than a preset first threshold; If so, proceed with the following steps: S1046, at least one type of image processing method is used to perform sample expansion on at least two sets of the second spectral selection images to form a supplementary oral sample set; preferably, the image processing method is different from the image processing method in S102.

[0061] S1047, update the supplementary oral sample set to the second oral sample set, and return to step S103.

[0062] In other words, in this embodiment, when the first accuracy is greater than the second accuracy and the first difference between the two is too large, it is considered that the risk model may have an overfitting problem. Therefore, the second spectral selection image is expanded in an attempt to alleviate the overfitting problem through random expansion of a small number of samples.

[0063] In some embodiments, if the determination result of S1042 is negative, then the following steps are executed: S1048, at least one type of image processing method is used to perform sample expansion on at least one set of the first spectral selection images and at least one set of the second spectral selection images to form supplementary oral samples, and then the process returns to step S1047.

[0064] In this embodiment, when the risk model performs poorly on the training sample set, it is preferable to simultaneously and randomly expand the first spectral selection image and the second spectral selection image.

[0065] In other words, in order to improve the performance of the risk model on different sample sets, this invention also provides a method of random expansion based on different sample types (such as spectral selection images of different ranges) to try to optimize the performance of the model through random expansion.

[0066] Specifically, when the difference in performance between the risk model and the validation sample set is slightly large, it is preferable to try processing the second spectral selection image with slightly lower repeatability. When the risk model performs poorly on the training sample set, it is preferable to try synchronous random expansion of the second spectral selection image with different repeatability and the spectral selection image.

[0067] In some embodiments, in S102, at least two different image processing techniques are used to expand the first oral cavity sample set to generate two second oral cavity sample sets. Correspondingly, the method further includes the step of: If the judgment result of S1045 is negative, the oral mucosa malignant transformation risk model is defined as a qualified model. At least two qualified models are obtained, and the at least two qualified models are fused to obtain a new oral mucosal malignant transformation risk model.

[0068] For example, in some embodiments, the process of fusing two or more qualified models can be: weighted summation of the feature vectors output by the two (or more) qualified models, where the weights can be dynamically adjusted through training.

[0069] For example, in some embodiments, the process of fusing two or more qualified models can be as follows: the output features of two (or more) qualified models are concatenated and then input into a fully connected layer or a nonlinear activation function (such as ReLU) to enhance the ability to express complex relationships.

[0070] For example, in some embodiments, multiple different image processing techniques can be used to augment the spectral selection image to obtain multiple second oral cavity sample sets, and multiple small risk models can be created based on these sample sets. In this case, by simultaneously training multiple small risk models, the performance of each second oral cavity sample set can be preliminarily evaluated more quickly. If the performance of two second oral cavity sample sets is deemed satisfactory (or the risk models are considered preliminarily satisfactory), the two corresponding small risk models can then be fused. This method of simultaneously training multiple small risk models allows for rapid filtering of the randomly augmented results, improving model training efficiency.

[0071] It is worth noting that, for application scenarios with small sample sizes, this embodiment further provides a multi-line synchronous training mode. In other words, this invention provides a group training method based on random expansion, and quickly evaluates the risk model obtained by random expansion through group training. Of course, in other embodiments, the process of fusing at least two qualified models can also be to merge at least two sets of training samples to train a new risk model.

[0072] In some embodiments, the spectral selection images corresponding to the two second oral sample sets are not repeated.

[0073] In some embodiments, the two second oral cavity sample sets are generated using different image processing techniques.

[0074] In some embodiments, the image processing means is a combination of two or more image processing methods, wherein the types of the image processing methods include one or more of the following: adaptive histogram equalization, noise reduction, color correction, local occlusion, background segmentation, contrast stretching, image pyramid, rotation, scaling, translation, brightness adjustment, contrast adjustment, and saturation adjustment.

[0075] It is understandable that in actual clinical applications, the collection of oral samples is quite difficult. Furthermore, different types of oral diseases may require customized risk model creation for different types of diseases in some scenarios that require refined diagnosis. In this regard, the present invention proposes a multi-stage dropout training method based on random expansion, which can effectively improve the model training efficiency.

[0076] Preferably, this invention employs deep learning and computer vision technology, based on spectral selective imaging technology, to perform early screening for potential oral malignant diseases and oral squamous cell carcinoma with high risk of malignant transformation. Specifically, it involves a training method and system for a Vision Transformer (ViT) model based on hierarchical embedding and multi-level dropout. By optimizing image patch embedding, attention mechanism, and regularization strategy, it solves the problems of gradient vanishing, overfitting, and computational efficiency in deep network training, thereby improving diagnostic efficiency and recognition accuracy.

[0077] The technical solution adopted in this application will be explained and illustrated below through an exemplary training scheme, and the beneficial technical effects of the present invention will be demonstrated through verification data. In the first model network, the input spectral selection image is first acquired, segmented into fixed-size image patches, and mapped to low-dimensional embedding vectors through linear projection. The input spectral selection image is a 2D image with shape [H, W, C] (height, width, number of channels). After dividing the image into fixed-size image patches (e.g., 16x16), each image patch is flattened into a 1D vector with shape [N, P*P*C], where N = number of image patches, P = size of the image patch (e.g., 16), and C = number of channels. Further, in the linear projection stage, a linear projection (through a fully connected layer) is applied to each image patch, mapping it to a high-dimensional embedding space with shape [N, D], where D = embedding dimension. Simultaneously, the first model network also includes position embedding, which adds learnable position embeddings with shape [N, D].

[0078] Typically, spectral selection images can be pre-classified and labeled by staff (such as doctors). For example, staff can label the type or degree of lesion reflected in the spectral selection image. Furthermore, they can also label lesion areas or non-lesion areas.

[0079] Furthermore, in the second model network, for example, each transformer encoder layer includes the following steps: LayerNorm: normalizing the input; Multi-Head Attention: computing self-attention; Residual Connections and DropPath: preserving some paths (to prevent overfitting), for example, introducing dynamic path dropping (DropPath) in the residual connections to randomly mask the output paths of sublayers; MLP Block (Multilayer Perceptron): containing two linear layers and an activation function (GELU), with Dropout added in between.

[0080] In this approach, residual connections are introduced into sub-layers of each layer (such as multi-head attention mechanisms or MLP blocks), directly adding the input to the sub-layer output. The residual connection formula is expressed as: Output = Input + SubLayer Output.

[0081] The formula for dynamic path dropping that combines residual joins and DropPath is as follows: Output=Input+DropPath(SubLayer(LayerNorm(Input))) In this embodiment, in the deep neural network, the output of each layer is calculated by combining residual connections and DropPath.

[0082] In this embodiment, an interleaved Dropout design is used in the MLP block to achieve double regularization. The structure of the MLP block is: Linear→GELU→Dropout→Linear→Dropout.

[0083] Specifically, for the residual connection y = x + F(x), the output of DropPath (equivalent to the first drop function) is: Y1= ; Where p1 is the discard probability, x1 is the current input data, Y1 is the output data, and F(x) is the residual function.

[0084] Specifically, for input x2, the output of Dropout (equivalent to the second dropout function) is: Y2= ; Where p2 is the discard probability, x2 is the current input data, and Y2 is the output data.

[0085] Furthermore, the specific model training conditions used in this invention are illustrated below: (a) Hierarchical embedding optimization Image Patches: Divide the input image (i.e., the spectral selection image) into fixed-size (e.g., 16×16 pixels) image patches, generating a tensor of shape [Batch_Size, Num_Patches, Patch_Size^2×Channels].

[0086] Linear Projection: Maps image patches to low-dimensional embedding vectors (e.g., Patch_Size^2×3 → 512) using a learnable weight matrix, reducing computational redundancy.

[0087] Position Embedding: Adds learnable positional codes to each image patch to enhance spatial awareness.

[0088] (ii) Dynamic path dropping (DropPath) The DropPath technique is introduced into the residual connection to randomly discard some sub-layer output paths (with a discard probability of 0.1-0.3), as shown in the following formula: Output=Input+DropPath(SubLayer(LayerNorm(Input)); It is applied to multi-head attention layers and MLP blocks to prevent overfitting and improve model robustness.

[0089] (III) Multi-level Regularization Design Layer Normalization (LayerNorm): Applied before each sub-layer (attention, MLP) to stabilize the input distribution.

[0090] Interleaved Dropout: A Dropout layer (dropout rate 0.1) is inserted after the linear layer in an MLP block, as shown in the following formula: MLP=Linear→GELU→Dropout→Linear→Dropout Double regularization suppresses activation value redundancy and accelerates convergence.

[0091] (iv) Classification head optimization Global average pooling is used to aggregate features instead of the traditional [CLS] label, reducing parameter dependence; The class probabilities are output through a lightweight MLP layer, as shown in the following formula: Class=MLP(GlobalAvgPool(Embeddings)) (v) Optimization Results Improved training efficiency: Reduces computational resource consumption by 30% through low-dimensional projection and hierarchical regularization; Enhanced generalization ability: Dynamic path dropping reduces the overfitting rate of the model by 15% on the ImageNet-1K dataset; Convergence speed optimization: The staggered Dropout design shortens the training cycle by 20%.

[0092] The following process is used to test the model training results. The training mainly includes the visual Transformer encoder layer structure module and the torch pseudocode implementation process as follows: Visual Transformer encoder layer structure modules: (1) Input program: The input image (224×224×3) is divided into 14×14 image blocks (16×16 pixels), generating a vector of shape [1,196, 768].

[0093] Each image patch is mapped to a 512-dimensional embedding vector through linear projection, outputting a vector of shape [1, 196, 512].

[0094] Add learnable positional encoding to generate the final embedded input.

[0095] (2) Encoder layer process: graph TD A[Input Embedding] --> B[LayerNorm] B --> C [Multi-Head Attention] C --> D[DropPath] A -->D D --> E[LayerNorm] E -->F[Linear → GELU → Dropout → Linear → Dropout] F --> G[DropPath] D -->G G --> H [Output Embedding] (3) Parameter configuration: Bullish attention parameter: 8 MLP expansion ratio: 4 DropPath probability: 0.1 Dropout probability: 0.1 (4) The formula for Multi-Head Self-Attention (MHSA) is as follows: Attention(Q,K,V)=Softmax((QK^T) / √d)V Where Q, K, and V are the query, key, and value matrices, and d is the embedding dimension.

[0096] Pseudocode implementation process of torch: import torch import torch.nn as nn class DropPath(nn.Module): def __init__(self, drop_prob=0.1): super().__init__() self.drop_prob = drop_prob def forward(self, x): if not self.training or self.drop_prob == 0: return x keep_prob = 1 - self.drop_prob mask = (torch.rand(x.shape[0], 1, 1, device=x.device) <keep_prob).float() return x * mask / keep_prob class TransformerEncoderLayer(nn.Module): def __init__(self, dim, num_heads, mlp_ratio=4, drop_path=0.1, dropout=0.1): super().__init__() self.norm1 = nn.LayerNorm(dim) self.attn = nn.MultiheadAttention(dim, num_heads) self.drop_path1 = DropPath(drop_path) self.norm2 = nn.LayerNorm(dim) self.mlp = nn.Sequential( nn.Linear(dim, dim * mlp_ratio), nn.GELU(), nn.Dropout(dropout), nn.Linear(dim * mlp_ratio, dim), nn.Dropout(dropout) ) self.drop_path2 = DropPath(drop_path) def forward(self, x): x = x + self.drop_path1(self.attn(self.norm1(x))[0]) x = x + self.drop_path2(self.mlp(self.norm2(x))) return x Finally, see Figure 5 As shown, the test results of the risk model obtained by using the multi-stage training method for multi-type expanded samples provided by this invention are as follows: the accuracy of the risk model on the training sample set is 0.9947, and the accuracy on the validation sample set is 0.9969. That is to say, the risk model does not exhibit overfitting and has high accuracy. The test set also includes spectral selection images of high-risk and low-risk lesions.

[0097] The overall training time for the risk model was 5350 minutes and 16 seconds (approximately 3.7 days).

[0098] To compare the technical advantages of this invention over traditional model training, three open-source neural network (CNN) algorithms were used in an accuracy comparison experiment. The test set also included spectral selection images of high-risk and low-risk lesions. The test results for the CNN model are shown below. Figure 6 The accuracy rates of the Vgg-16 model were 86.12%, the ResNet-50 model was 88.52%, and the ConvNeXt-T model was 90.91%.

[0099] It can be seen that even the best-performing ConvNeXt-T model has an accuracy of only 90.91%, which means that it has an accuracy deviation of nearly 10% from the risk model of this invention. In other words, the risk model of this invention shows a significant advantage in prediction results.

[0100] Example 2 See Figure 2 As shown, the present invention also provides a training system for a risk model of malignant transformation of oral mucosa, the risk model of malignant transformation of oral mucosa comprising: a first model network and a second model network connected to each other, and the training system comprising: The sample acquisition module is used to acquire a first oral cavity sample set, which includes: multiple sets of first spectral selection images taken for multiple first subjects, and multiple sets of second spectral selection images taken for second subjects, wherein at least two of the first spectral selection images and / or the at least two of the second spectral selection images simultaneously cover a first region and a second region, wherein the first region is the inner wall of the oral cavity, and the second region includes one or more of the following: teeth, gums, and tongue; the oral cavity condition of the first subjects is healthy, and the oral cavity condition of the second subjects is diseased or has a tendency to become diseased; The sample expansion module is used to expand the first oral cavity sample set using at least one type of image processing method to generate a second oral cavity sample set; A multi-stage training module is used to perform deep learning on the second oral sample set using a multi-stage dropout pattern to train and output the oral mucosal malignant transformation risk model; wherein, the multi-stage training module includes: A first-order training unit is used to select a first training sample set from the second oral cavity sample set and input the first training sample set into a first model network to extract local features from the first training sample set; wherein, the first model network includes: (1) an image input layer for receiving an image; (2) an image segmentation layer for segmenting the received image to obtain multiple image blocks; (3) a first mapping layer for projecting the image blocks into low-dimensional embedding vectors through linear mapping; and (4) a second mapping layer for mapping the low-dimensional embedding vectors to a high-dimensional embedding space and outputting the corresponding vector results through position embedding. The second-order training unit is used to input the vector results into the second model network, and the second model network sets a first drop-off function to drop some paths during the first learning stage; The third-order training unit is used to set a second dropout function to drop some neurons in the second learning stage of the second network model; after the second model network converges, the oral mucosa malignant transformation risk model is trained.

[0101] The training system in this invention can implement the methods or steps in any of the above embodiments, which will not be repeated here.

[0102] This invention belongs to the field of deep learning and computer vision technology, specifically involving a training method and system for a visual Transformer (ViT) model based on hierarchical embedding and path dropping. By optimizing image patch embedding, attention mechanism and regularization strategy, it solves the problems of gradient vanishing, overfitting and computational efficiency in deep network training, and applies it to the screening and identification of oral mucosal diseases.

[0103] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a risk model of malignant transformation of oral mucosa as described in any of the preceding claims.

[0104] The present invention also provides an electronic device, the electronic device comprising: a memory for storing computer-executable instructions; and a processor for executing the computer-executable instructions stored in the memory to implement a method for training a risk model of malignant transformation of oral mucosa as described in any embodiment.

[0105] The present invention also provides a risk model, which can be trained using any of the above-mentioned model training methods.

[0106] This invention also provides a risk prediction method, which includes: acquiring a spectral selection image to be identified; inputting the spectral selection image to be identified into a risk model; and outputting a classification result from the risk model. For example, the result may be the predicted risk level of a lesion, the type of lesion, etc.

[0107] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0109] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for training a risk model of malignant transformation of oral mucosa, characterized in that, The oral mucosal malignant transformation risk model includes: a first model network and a second model network connected together, and includes the following steps: S101, Obtain a first oral cavity sample set, the first oral cavity sample set including: multiple sets of first spectral selection images taken for multiple first objects, multiple sets of second spectral selection images taken for second objects, and at least two of the first spectral selection images and / or at least two of the second spectral selection images simultaneously cover a first region and a second region, wherein the first region is the inner wall of the oral cavity, and the second region includes one or more of the following: teeth, gums, and tongue; the oral cavity condition of the first objects is healthy, and the oral cavity condition of the second objects is diseased or has a tendency to become diseased; S102, at least one type of image processing method is used to expand the first oral cavity sample set to generate a second oral cavity sample set; S103, employing a multi-stage discarding pattern to perform deep learning on the second oral sample set to train and output the oral mucosal malignant transformation risk model; wherein, S103 includes the following steps: S1031, a first training sample set is selected from the second oral cavity sample set, and the first training sample set is input into the first model network to extract local features from the first training sample set; wherein, the first model network includes: (1) an image input layer for receiving an image; (2) an image segmentation layer for segmenting the received image to obtain multiple image blocks; (3) a first mapping layer for projecting the image blocks into low-dimensional embedding vectors through linear mapping; (4) a second mapping layer for mapping the low-dimensional embedding vectors to a high-dimensional embedding space and outputting the corresponding vector results through position embedding; S1032, The vector result is input into the second model network, and the second model network sets a first drop-off function to drop some paths during the first learning stage; The transformer encoder layer in the second model network contains MLP blocks, the structure of which is as follows: Linear, GELU, Dropout, Linear, Dropout; the output of the first dropout function is: ； Where p1 is the discard probability, x1 is the current input data, Y1 is the output data, and F(x) is the residual function; S1033, the second network model sets a second dropout function to drop some neurons during the second learning phase; the output of the second dropout function is: ； Where p2 is the discard probability, x2 is the current input data, and Y2 is the output data; Once the second model network converges, the oral mucosa malignant transformation risk model is trained.

2. The method for training a risk model of malignant transformation of oral mucosa according to claim 1, characterized in that, It also includes the following steps: S104, verify the accuracy of the oral mucosal malignant transformation risk model.

3. The method for training a risk model of malignant transformation of oral mucosa according to claim 2, characterized in that, S104 includes the following steps: S1041, Obtain the first accuracy of the oral mucosal malignant transformation risk model on the first training sample set; S1042, determine whether the first accuracy is greater than the first preset threshold; If so, proceed with the following steps: S1043, Select a first validation sample set from the second oral sample set, and calculate the second accuracy of the oral mucosal malignant transformation risk model on the first validation sample set; S1044, Calculate the first difference between the first accuracy and the second accuracy; S1045, determine whether the first difference is greater than a preset first threshold; If so, proceed with the following steps: S1046, At least one type of image processing method is used to perform sample expansion on at least two sets of the second spectral selection images to form a supplementary oral sample set; S1047, update the supplementary oral sample set to the second oral sample set, and return to step S103.

4. The method for training a risk model of malignant transformation of oral mucosa according to claim 3, characterized in that, If the result of S1042 is negative, then the following steps are executed: S1048, at least one type of image processing method is used to perform sample expansion on at least one set of the first spectral selection images and at least one set of the second spectral selection images to form supplementary oral samples, and then the process returns to step S1047.

5. The method for training a risk model of malignant transformation of oral mucosa according to claim 3, characterized in that, In S102, at least two types of image processing techniques are used to expand the first oral cavity sample set to generate two second oral cavity sample sets. Correspondingly, the method further includes the following steps: If the judgment result of S1045 is negative, the oral mucosa malignant transformation risk model is defined as a qualified model. At least two qualified models are obtained, and the at least two qualified models are fused to obtain a new oral mucosal malignant transformation risk model.

6. The method for training a risk model of malignant transformation of oral mucosa according to claim 5, characterized in that, The spectral selection images corresponding to the two second oral cavity sample sets are not repeated; and / or, the two second oral cavity sample sets are generated using different image processing methods.

7. A method for training a risk model of malignant transformation of oral mucosa according to any one of claims 1-6, characterized in that, The image processing method is a combination of two or more image processing methods, wherein the types of the image processing methods include one or more of the following: adaptive histogram equalization, noise reduction, color correction, local occlusion, background segmentation, contrast stretching, image pyramid, rotation, scaling, translation, brightness adjustment, contrast adjustment, and saturation adjustment.

8. A training system for a risk model of malignant transformation of oral mucosa, characterized in that, The oral mucosal malignant transformation risk model includes: a first model network and a second model network connected together, and the training system includes: The sample acquisition module is used to acquire a first oral cavity sample set, which includes: multiple sets of first spectral selection images taken for multiple first subjects, and multiple sets of second spectral selection images taken for second subjects, wherein at least two of the first spectral selection images and / or the at least two of the second spectral selection images simultaneously cover a first region and a second region, wherein the first region is the inner wall of the oral cavity, and the second region includes one or more of the following: teeth, gums, and tongue; the oral cavity condition of the first subjects is healthy, and the oral cavity condition of the second subjects is diseased or has a tendency to become diseased; The sample expansion module is used to expand the first oral cavity sample set using at least one type of image processing method to generate a second oral cavity sample set; A multi-stage training module is used to perform deep learning on the second oral sample set using a multi-stage dropout pattern to train and output the oral mucosal malignant transformation risk model; wherein, the multi-stage training module includes: A first-order training unit is used to select a first training sample set from the second oral cavity sample set and input the first training sample set into a first model network to extract local features from the first training sample set; wherein, the first model network includes: (1) an image input layer for receiving an image; (2) an image segmentation layer for segmenting the received image to obtain multiple image blocks; (3) a first mapping layer for projecting the image blocks into low-dimensional embedding vectors through linear mapping; and (4) a second mapping layer for mapping the low-dimensional embedding vectors to a high-dimensional embedding space and outputting the corresponding vector results through position embedding. The second-order training unit is used to input the vector results into the second model network. The second model network sets a first dropout function to drop some paths during the first learning phase. The transformer encoder layer in the second model network contains MLP blocks, where the structure of the MLP blocks is as follows: Linear, GELU, Dropout, Linear, Dropout. The output of the first dropout function is: ； Where p1 is the discard probability, x1 is the current input data, Y1 is the output data, and F(x) is the residual function; The third-order training unit is used to set a second dropout function to drop off some neurons during the second learning phase of the second network model; the output of the second dropout function is: ； Where p2 is the discard probability, x2 is the current input data, and Y2 is the output data; Once the second model network converges, the oral mucosa malignant transformation risk model is trained.

9. A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement a training method for a risk model of malignant transformation of oral mucosa as described in any one of claims 1-7.

10. An electronic device, the electronic device comprising: Memory is used to store executable instructions for a computer; The processor, when executing computer-executable instructions stored in the memory, implements the oral mucosal malignant transformation risk model training method according to any one of claims 1 to 7.

Images