Dynamic tracking system and method for oral mucosa lesion

Abstract

The invention relates to the technical field of oral recognition, in particular to an oral mucosa lesion dynamic tracking system and method, and the system comprises a data collection module which is used for obtaining a two-dimensional color image of an oral mucosa; the data processing module is used for carrying out preprocessing operation on the obtained two-dimensional color image; the intelligent analysis module is used for performing feature extraction and analysis on the preprocessed two-dimensional color image by adopting a deep convolutional neural network architecture; the dynamic tracking module is used for dynamically tracking oral mucosa lesions and recording the development and change conditions of the lesions; and the user interaction module is used for displaying the dynamic tracking result of the oral mucosa lesion to a user. The deep convolutional neural network is combined with the attention mechanism and the multi-scale feature extraction technology, fine differences of oral mucosa lesions are automatically recognized, traditional manual observation errors are greatly reduced, the method is especially good at distinguishing complex textures and early-stage tiny lesions, and a reliable basis is provided for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of oral cavity recognition technology, and in particular to an oral mucosal lesion dynamic tracking system and method. Background Art

[0002] Early diagnosis and dynamic monitoring of oral mucosal lesions are crucial for preventing malignant transformation and optimizing treatment options. However, existing technologies have significant limitations: traditional methods rely on manual observation and measurement, which are inefficient and highly subjective; two-dimensional image analysis is mostly limited to static evaluation and lacks quantitative tracking of the dynamic evolution of lesions; and existing systems often use traditional algorithms for feature extraction, such as threshold segmentation, which makes it difficult to accurately distinguish the morphological and textural characteristics of complex lesions.

[0003] In addition, existing technologies lack multi-dimensional features, such as comprehensive analysis of size, color, and texture, and intelligent early warning mechanisms, resulting in delayed clinical decision-making.

[0004] To address the above problems, the present invention proposes a dynamic tracking system for oral mucosal lesions based on deep learning. By integrating intelligent analysis, time series feature comparison and multimodal visualization technology, it can realize automatic identification, dynamic monitoring and accurate early warning of lesions, thus making up for the shortcomings of existing technologies. Summary of the Invention

[0005] In view of the above shortcomings of the prior art, the present invention provides a method that can effectively solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] The present invention provides a dynamic tracking system for oral mucosal lesions, comprising:

[0008] a data acquisition module, configured to acquire a two-dimensional color image of the oral mucosa, wherein the data acquisition module is configured as an optical imaging device;

[0009] A data processing module, connected to the data acquisition module, is used to perform preprocessing operations on the acquired two-dimensional color image, including image enhancement, denoising and normalization, to improve image quality and consistency;

[0010] An intelligent analysis module, connected to the data processing module, uses a deep convolutional neural network architecture to extract and analyze features from preprocessed two-dimensional color images, identify and segment oral mucosal lesion areas, and classify and locate lesion types and severity;

[0011] A dynamic tracking module, connected to the intelligent analysis module, is used to dynamically track oral mucosal lesions based on continuously acquired two-dimensional color images and the analysis results of the intelligent analysis module, record the development and changes of the lesions, and specifically analyze changes in the size, color, and texture characteristics of the lesion area;

[0012] A user interaction module, connected to the dynamic tracking module, is used to display the dynamic tracking results of oral mucosal lesions to the user, including a visual interface display unit, a report generation unit, and an early warning prompt unit. The visual interface display unit is used to display the evolution process and trend of the lesions, the report generation unit is used to generate a detailed lesion analysis report, and the early warning prompt unit is used to promptly remind the user when abnormal changes occur in the lesions;

[0013] Among them, the deep convolutional neural network architecture includes input layer, convolution layer 1, ReLU activation function layer 1, attention mechanism unit, residual connection unit, new convolution kernel unit, pooling layer 1, regression layer, convolution layer 2, ReLU activation function layer 2, pooling layer 2, fully connected layer 1 and fully connected layer 2 in sequence;

[0014] The attention mechanism unit includes a convolution layer 3, a ReLU activation function layer 3, a convolution layer 4 and a sigmoid activation function layer in sequence, and the new convolution kernel unit includes a hole convolution layer and a ReLU activation function layer 4 in sequence.

[0015] Furthermore, the specific operation steps of the data processing module are:

[0016] S100, using histogram equalization to process the two-dimensional color image to enhance the contrast of the image, and obtain an enhanced two-dimensional color image I enhanced ;

[0017] S101, denoising the enhanced image using Gaussian filtering to reduce random noise in the image, and obtaining a denoised two-dimensional color image I denoised , the formula is:

[0018] I denoised =I enhanced *G;

[0019] Among them, G is the Gaussian kernel, the parameter is 3×3, and * is the convolution operation;

[0020] S102, the denoised two-dimensional color image I denoised Perform normalization operation to adjust the pixel value range to the [0,1] interval to obtain the normalized two-dimensional color image I normalized , the formula is:

[0021]

[0022] Among them, I min and I max are the minimum and maximum pixel values ​​in the denoised two-dimensional color image, respectively.

[0023] Furthermore, the specific process of the deep convolutional neural network processing image data is as follows:

[0024] S200, the normalized two-dimensional color image I normalized Input convolution layer, convolution kernel size is 3×3, number is 32, padding is 1, stride is 1, extract edge, texture and color change features of two-dimensional color image, and obtain feature map F conv1 , the formula is:

[0025]

[0026] in, is the pixel value of the ith channel of the normalized two-dimensional color image, K (i) is the i-th convolution kernel, * is the convolution operation, b is the bias term, and σ is the activation function;

[0027] S201, feature map F conv1 Apply the ReLU activation function to enhance the nonlinear expression ability of the feature. The calculation formula is:

[0028] F relu1 =max(0,F conv1 );

[0029] Among them, F relu1 It is the feature map after being processed by the ReLU activation function;

[0030] S202, the feature map F processed by the ReLU activation function relu1 Input the attention mechanism unit, first reduce the dimension through the convolution layer 3 with a convolution kernel size of 1×1 and a number of 16 to obtain the feature map F attention_conv1 , and then apply the ReLU activation function 3 to get F attention_relu1 Then, the feature map F is obtained by increasing the dimension through the convolution layer 4 with a convolution kernel size of 1×1 and a number of 32. attention_conv2 ; Use sigmoid activation function to calculate the attention weight A, and add attention weight A to F relu1 Perform element-by-element multiplication to obtain the feature map F attention , the formula is:

[0031] F attention =F relu1 ⊙A;

[0032] Among them, ⊙ is the factorial element-by-element multiplication operation, which enhances the key feature area through the attention weight;

[0033] S203, the feature map F relu1 and feature map F attention Input to the residual connection unit to form a residual connection and obtain the feature map F after residual connection residual , the formula is:

[0034] F residual =F relu1 +F attention ;

[0035] S204, the feature map F after residual connection residual The input is sent to the new convolution kernel unit, and the dilated convolution is used for further feature extraction. The dilated convolution kernel size is 3×3, the number is 64, the padding is 2, the stride is 1, and the dilation rate is set to 1, and the feature map F is obtained. dilated , the formula is:

[0036]

[0037] in, is the i-th channel of the feature map after residual connection, is the i-th hole convolution kernel;

[0038] S205, the feature map F dilated Input to pooling layer 1 for pooling operation, the pooling window is 2×2, the stride is 2, and the feature map F is reduced dilated The spatial size of the pooled feature map F is obtained pool1 , the formula is:

[0039] F pool1 =Pool(F dilated , 2, 2);

[0040] Among them, Pool is the pooling operation; the feature map F pool1 Input to pooling layer 2, repeat the pooling operation, and finally obtain the flattened feature vector X flat ;

[0041] S206, the feature map F pool1 Input regression layer, the convolution kernel size of the regression layer is 1×1, the number is 4, used to predict the bounding box coordinates, the formula of the convolution operation is:

[0042]

[0043] in, is the i-th coordinate component of the predicted bounding box, is the i-th 1×1 convolution kernel, is the bias term corresponding to the i-th 1×1 convolution kernel;

[0044] After the convolution operation, the regression layer outputs the bounding box coordinates B corresponding to each position pred =(x, y, w, h);

[0045] S207, flatten the eigenvector X flat Input to the fully connected layer 1, further integrate features and perform classification. The formula is:

[0046] X fc1 =σ(W1·X flat +b1);

[0047] Among them, X fc1 is the feature vector output by the fully connected layer 1, W1 is the weight matrix of the fully connected layer 1, b1 is the bias term of the fully connected layer 1, and σ is the activation function;

[0048] Finally, the feature vector X output by the fully connected layer 1 fc1 Input to the fully connected layer 2 for classification task, the fully connected layer 2 uses the weight matrix W2 and the bias term b2 to classify the feature vector X fc1 Perform weighted summation and bias adjustment, and then convert it into a probability distribution through the softmax function to obtain the classification result Y. The formula is:

[0049] Y=softmax(W2·X fc1 +b2);

[0050] Wherein, Y is the classification result, W2 is the weight matrix of the fully connected layer 2, and b2 is the bias term of the fully connected layer 2. The classification result includes the presence of lesions and the absence of lesions.

[0051] Furthermore, the implementation of the intelligent analysis module specifically includes the following steps:

[0052] S300, receiving a continuous time series of two-dimensional color images and corresponding lesion area recognition results, wherein the recognition results include the location, type, severity and bounding box coordinates of the lesion area. pred =(x, y, w, h), and get the area S through the bounding box coordinates t =w×h;

[0053] S301, extracting the features of the lesion area at each time point based on the continuous time-series two-dimensional color image and the recognition result, including:

[0054] a. Size feature: Calculate the area S of the lesion region by the bounding box coordinates t and through the lesion area S t To calculate the percentage change in area, the formula is:

[0055]

[0056] Where, ΔS t is the percentage change in area, S t is the area of ​​the lesion at the current time point t, S t-1 is the area of ​​the lesion at the previous time point t-1;

[0057] b. Color features: based on the preprocessed normalized image I normalized , calculate the mean value μ of the RGB three-channel pixels in the lesion area t (k) , where k = 1, 2, 3, and calculate the Euclidean distance of the color mean, the formula is:

[0058]

[0059] Among them, μ t (k) is the pixel mean of the lesion area in the RGB three channels at the current time point t, ΔC t is the Euclidean distance of the color mean, quantifying the overall degree of color shift;

[0060] c. Texture features: Using the dilated convolution feature map F output by the intelligent analysis module dilated , calculate the change of high-frequency texture information, the formula is:

[0061] ΔT t =∑ i，j |P t (i, j)-P t-1 (i, j)|;

[0062] Where, ΔT t is the Manhattan distance of texture difference, P t (i, j) is the probability of occurrence of pixel pair (i, j) in the gray level co-occurrence matrix of the lesion area at the current time point t, P t-1 (i, j) is the occurrence probability of the pixel pair (i, j) in the gray-level co-occurrence matrix of the lesion area at the previous time point t-1;

[0063] S302, aligning the lesion areas at different time points using image registration technology, comparing the changes in the size features, color features, and texture features, and generating a lesion development trend map;

[0064] S303, compare the change with the preset abnormal thresholds α, β and γ, and if ΔS t >α or ΔC t >β or ΔT t>γ, an early warning signal is triggered, and an alarm message is output through the early warning prompt unit of the user interaction module, where α, β and γ are the preset abnormal thresholds of size feature, color feature and texture feature respectively.

[0065] Furthermore, the lesion development trend map in step S302 includes a ΔS t The historical change curve based on ΔC t The spatial distribution heat map and ΔT t Multi-feature overlay comparison chart.

[0066] Furthermore, the dynamic tracking module has a built-in time series prediction model, which uses a Transformer-based architecture and takes as input the lesion feature sequence at historical time points. The output is the predicted value of lesion change at k time points in the future {ΔS n+1 ,...,ΔS n+k}, and present the output results in the form of trend curves through the visual interface display unit of the user interaction module, and associate them with the clinical recommendation database to generate personalized intervention plans.

[0067] A method for dynamically tracking oral mucosal lesions, comprising the steps of:

[0068] S1. Obtain a two-dimensional color image of the oral mucosa using an optical imaging device;

[0069] S2. performing preprocessing operations on the acquired two-dimensional color image, including image enhancement, denoising and normalization, to improve image quality and consistency;

[0070] S3 uses a deep convolutional neural network architecture to extract and analyze features from preprocessed two-dimensional color images, identify and segment oral mucosal lesions, and classify and locate lesion types and severity.

[0071] S4. Based on the continuously acquired two-dimensional color images and the analysis results of the intelligent analysis module, the oral mucosal lesions are dynamically tracked, the development and changes of the lesions are recorded, and the changes in the size, color, and texture characteristics of the lesion area are specifically analyzed;

[0072] S5. Display the dynamic tracking results of oral mucosal lesions to users, show the evolution process and trend of the lesions, generate detailed lesion analysis reports, and promptly remind users when abnormal changes occur in the lesions.

[0073] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0074] This invention adopts a deep convolutional neural network combined with attention mechanism and multi-scale feature extraction technology. The system can automatically identify subtle differences in oral mucosal lesions, accurately segment the lesion area and classify the lesion type, greatly reducing the subjective errors of traditional manual observation. It is particularly good at distinguishing complex textures from early minor lesions, providing a reliable basis for clinical diagnosis.

[0075] The present invention is based on time-series image analysis. The system continuously monitors the size changes, color shifts and texture feature evolution of the lesion area, quantifies the development trend through intelligent algorithms, and automatically triggers graded warnings when abnormal expansion, color mutation or texture deterioration occurs, helping doctors to capture malignant transformation signals in a timely manner and achieve the clinical goal of early detection and early intervention.

[0076] From image preprocessing to visual report generation, the system of this invention realizes automated pipeline operation, greatly shortening the analysis time; the interactive interface intuitively displays the historical trend of lesions, spatial distribution heat map and multi-feature comparison results, simplifying the doctor's decision-making process, while reducing human operational errors, and significantly improving the diagnosis and treatment efficiency and patient management quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0078] Figure 1 Schematic diagram of the method flow of the present invention;

[0079] Figure 2 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0080] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0081] The present invention will be further described below with reference to the embodiments.

[0082] Example 1: Reference Figure 1 , oral mucosal lesion dynamic tracking system, including:

[0083] A data acquisition module, configured to acquire a two-dimensional color image of the oral mucosa, wherein the data acquisition module is configured as an optical imaging device;

[0084] The data processing module is connected to the data acquisition module and is used to perform preprocessing operations on the acquired two-dimensional color image, including image enhancement, denoising and normalization, to improve image quality and consistency;

[0085] The intelligent analysis module is connected to the data processing module and uses a deep convolutional neural network architecture to extract and analyze features from pre-processed two-dimensional color images, identify and segment oral mucosal lesions, and classify and locate lesion types and severity.

[0086] The dynamic tracking module is connected to the intelligent analysis module and is used to dynamically track oral mucosal lesions based on continuously acquired two-dimensional color images and the analysis results of the intelligent analysis module, record the development and changes of the lesions, and specifically analyze the changes in the size, color and texture characteristics of the lesion area;

[0087] The user interaction module is connected to the dynamic tracking module and is used to display the dynamic tracking results of oral mucosal lesions to the user. It includes a visual interface display unit, a report generation unit, and an early warning prompt unit. The visual interface display unit is used to display the evolution process and trend of the lesions. The report generation unit is used to generate a detailed lesion analysis report. The early warning prompt unit is used to promptly remind the user when abnormal changes occur in the lesions.

[0088] Among them, the deep convolutional neural network architecture includes input layer, convolution layer 1, ReLU activation function layer 1, attention mechanism unit, residual connection unit, new convolution kernel unit, pooling layer 1, regression layer, convolution layer 2, ReLU activation function layer 2, pooling layer 2, fully connected layer 1 and fully connected layer 2 in sequence;

[0089] The attention mechanism unit includes convolution layer 3, ReLU activation function layer 3, convolution layer 4 and sigmoid activation function layer in sequence, and the new convolution kernel unit includes void convolution layer and ReLU activation function layer 4 in sequence.

[0090] In a specific embodiment, taking the dynamic tracking of a patient with oral leukoplakia as an example, combined with Figure 1 The specific implementation steps are as follows:

[0091] 1. Data collection and preprocessing

[0092] Optical imaging equipment, such as a high-resolution intraoral camera, is used to obtain two-dimensional color images of the patient's oral mucosa with a resolution of 1024 × 768 pixels.

[0093] The data processing module processes according to steps S100-S102:

[0094] Histogram equalization enhances image contrast and highlights the boundary between the white spot area and the normal mucosa;

[0095] A 3×3 Gaussian kernel was used for denoising to reduce random noise during image acquisition;

[0096] Normalization maps pixel values ​​to the range [0, 1] to eliminate brightness differences between different devices.

[0097] 2. Intelligent analysis and lesion identification

[0098] The deep convolutional neural network is processed according to steps S200-S207:

[0099] Convolution layer 1 (3×3 convolution kernel, 32) extracts image edge and texture features to obtain feature map F conv1 ;

[0100] The attention mechanism unit generates attention weight A through 1×1 convolution to reduce and increase dimensionality, thereby enhancing the feature expression of the white spot area.

[0101] The residual connection unit fuses the original features with the attention features to avoid the gradient disappearance of the deep network;

[0102] The dilated convolution layer (with a dilation rate of 1 and 64 convolution kernels) expands the receptive field and captures the texture details of white spots.

[0103] The regression layer outputs the bounding box coordinates (x, y, w, h) of the lesion area and calculates the area S t =w×h; the fully connected layer uses softmax classification to determine the lesion type as "leukoplakia" and the severity as "moderate".

[0104] 3. Dynamic tracking and feature analysis

[0105] Images were collected for 30 consecutive days, and the intelligent analysis module extracted features at each time point:

[0106] Size characteristics: Area S on the 10th day 10 =8mm 2 , S5=6mm on the 5th day 2 , calculate ΔS 10 =|8-6| / 6×100%≈33.3%;

[0107] Color features: Count the RGB mean and calculate the Euclidean distance ΔC between the 10th day and the 5th day 10 =0.15;

[0108] Texture features: Calculate Manhattan distance ΔT based on gray-level co-occurrence matrix 10 =0.28.

[0109] The preset abnormal thresholds are α=20%, β=0.1, and γ=0.25. Since ΔS_10>α and ΔT_10>γ, the early warning signal is triggered.

[0110] 4. Results presentation and clinical intervention

[0111] User interaction module generation:

[0112] Lesion development trend map (including ΔS historical curve, ΔC spatial heat map, ΔT multi-feature overlay map);

[0113] The analysis report showed that "the area of ​​leukoplakia continued to increase and the texture was abnormal, and an immediate biopsy was recommended";

[0114] The early warning prompt unit notifies patients and doctors through pop-up windows and text messages.

[0115] Furthermore, the specific operation steps of the data processing module are:

[0116] S100, using histogram equalization to process the two-dimensional color image to enhance the contrast of the image, and obtain an enhanced two-dimensional color image I enhanced ;

[0117] S101, denoising the enhanced image using Gaussian filtering to reduce random noise in the image, and obtaining a denoised two-dimensional color image I denoised , the formula is:

[0118] I denoised =I enhanced *G;

[0119] Among them, G is the Gaussian kernel, the parameter is 3×3, and * is the convolution operation;

[0120] S102, the denoised two-dimensional color image I denoised Perform normalization operation to adjust the pixel value range to the [0,1] interval to obtain the normalized two-dimensional color image I normalized , the formula is:

[0121]

[0122] Among them, I min and I max are the minimum and maximum pixel values ​​in the denoised two-dimensional color image, respectively.

[0123] Furthermore, the specific process of deep convolutional neural network processing image data is as follows:

[0124] S200, the normalized two-dimensional color image I normalizedInput convolution layer, convolution kernel size is 3×3, number is 32, padding is 1, stride is 1, extract edge, texture and color change features of two-dimensional color image, and obtain feature map F conv1 , the formula is:

[0125]

[0126] in, is the pixel value of the ith channel of the normalized two-dimensional color image, K (i) is the i-th convolution kernel, * is the convolution operation, b is the bias term, and σ is the activation function;

[0127] S201, feature map F conv1 Apply the ReLU activation function to enhance the nonlinear expression ability of the feature. The calculation formula is:

[0128] F relu1 =max(0,F conv1 );

[0129] Among them, F relu1 It is the feature map after being processed by the ReLU activation function;

[0130] S202, the feature map F processed by the ReLU activation function relu1 Input the attention mechanism unit, first reduce the dimension through the convolution layer 3 with a convolution kernel size of 1×1 and a number of 16 to obtain the feature map F attention_conv1 , and then apply the ReLU activation function 3 to get F attention_relu1 Then, the feature map F is obtained by increasing the dimension through the convolution layer 4 with a convolution kernel size of 1×1 and a number of 32. attention_conv2 ; Use sigmoid activation function to calculate the attention weight A, and add attention weight A to F relu1 Perform element-by-element multiplication to obtain the feature map F attention , the formula is:

[0131] F attention =F relu1 ⊙A;

[0132] Among them, ⊙ is the factorial element-by-element multiplication operation, which enhances the key feature area through the attention weight;

[0133] S203, the feature map F relu1 and feature map F attention Input to the residual connection unit to form a residual connection and obtain the feature map F after residual connection residual , the formula is:

[0134] F residual =F relu1 +F attention ;

[0135] S204, the feature map F after residual connection residual The input is sent to the new convolution kernel unit, and the dilated convolution is used for further feature extraction. The dilated convolution kernel size is 3×3, the number is 64, the padding is 2, the stride is 1, and the dilation rate is set to 1, and the feature map F is obtained. dilated , the formula is:

[0136]

[0137] in, is the i-th channel of the feature map after residual connection, is the i-th hole convolution kernel;

[0138] S205, the feature map F dilated Input to pooling layer 1 for pooling operation, the pooling window is 2×2, the stride is 2, and the feature map F is reduced dilated The spatial size of the pooled feature map F is obtained pool1 , the formula is:

[0139] F pool1 =Pool(F dilated , 2, 2);

[0140] Among them, Pool is the pooling operation; the feature map F pool1 Input to pooling layer 2, repeat the pooling operation, and finally obtain the flattened feature vector X flat ;

[0141] S206, the feature map F pool1 Input regression layer, the convolution kernel size of regression layer is 1×1, the number is 4, which is used to predict the bounding box coordinates. The formula of convolution operation is:

[0142]

[0143] in, is the i-th coordinate component of the predicted bounding box, is the i-th 1×1 convolution kernel, is the bias term corresponding to the i-th 1×1 convolution kernel;

[0144] After the convolution operation, the regression layer outputs the bounding box coordinates B corresponding to each position pred =(x, y, w, h);

[0145] S207, flatten the eigenvector X flat Input to the fully connected layer 1, further integrate features and perform classification. The formula is:

[0146] X fc1=σ(W1·X flat +b1);

[0147] Among them, X fc1 is the feature vector output by the fully connected layer 1, W1 is the weight matrix of the fully connected layer 1, b1 is the bias term of the fully connected layer 1, and σ is the activation function;

[0148] Finally, the feature vector X output by the fully connected layer 1 fc1 Input to the fully connected layer 2 for classification task, the fully connected layer 2 uses the weight matrix W2 and the bias term b2 to classify the feature vector X fc1 Perform weighted summation and bias adjustment, and then convert it into a probability distribution through the softmax function to obtain the classification result Y. The formula is:

[0149] Y=softmax(W2·X fc1 +b2);

[0150] Among them, Y is the classification result, W2 is the weight matrix of the fully connected layer 2, b2 is the bias term of the fully connected layer 2, and the classification results include the presence of lesions and the absence of lesions.

[0151] Furthermore, the implementation of the intelligent analysis module specifically includes the following steps:

[0152] S300, receiving a continuous time series of two-dimensional color images and corresponding lesion area recognition results, the recognition results including the location, type, severity and bounding box coordinates of the lesion area pred =(x, y, w, h), and get the area S through the bounding box coordinates t =w×h;

[0153] S301, based on the continuous time-series two-dimensional color image and recognition results, extract the features of the lesion area at each time point, including:

[0154] a. Size feature: Calculate the area S of the lesion region by the bounding box coordinates t and through the lesion area S t To calculate the percentage change in area, the formula is:

[0155]

[0156] Where, ΔS t is the percentage change in area, S t is the area of ​​the lesion at the current time point t, S t-1 is the area of ​​the lesion at the previous time point t-1;

[0157] b. Color features: based on the preprocessed normalized image I normalized , calculate the mean value μ of the RGB three-channel pixels in the lesion areat (k) , where k = 1, 2, 3, and calculate the Euclidean distance of the color mean, the formula is:

[0158]

[0159] Among them, μ t (k) is the pixel mean of the lesion area in the RGB three channels at the current time point t, ΔC t is the Euclidean distance of the color mean, quantifying the overall degree of color shift;

[0160] c. Texture features: Using the dilated convolution feature map F output by the intelligent analysis module dilated , calculate the change of high-frequency texture information, the formula is:

[0161] ΔT t =∑ i，j |P t (i, j)-P t-1 (i, j)|;

[0162] Where, ΔT t is the Manhattan distance of texture difference, P t (i, j) is the probability of occurrence of pixel pair (i, j) in the gray level co-occurrence matrix of the lesion area at the current time point t, P t-1 (i, j) is the occurrence probability of the pixel pair (i, j) in the gray-level co-occurrence matrix of the lesion area at the previous time point t-1;

[0163] S302, aligning lesion areas at different time points using image registration technology, comparing changes in size, color, and texture features, and generating a lesion development trend map;

[0164] S303, compare the change with the preset abnormal thresholds α, β and γ, if ΔS t >α or ΔC t >β or ΔT t >γ, an early warning signal is triggered, and an alarm message is output through the early warning prompt unit of the user interaction module, where α, β and γ are the preset abnormal thresholds of size feature, color feature and texture feature respectively.

[0165] Furthermore, the lesion development trend map in step S302 includes a ΔS t The historical change curve based on ΔC t Spatial distribution heat map and ΔT-based t Multi-feature overlay comparison chart.

[0166] Furthermore, the dynamic tracking module has a built-in time series prediction model, which uses a Transformer-based architecture and takes as input the lesion feature sequence at historical time points. The output is the predicted value of lesion change at k time points in the future {ΔS n+1 ,...,ΔS n+k}, and present the output results in the form of trend curves through the visual interface display unit of the user interaction module, and associate them with the clinical recommendation database to generate personalized intervention plans.

[0167] Example 2: Reference Figure 2 , a method for dynamic tracking of oral mucosal lesions, comprising the following steps:

[0168] S1. Obtain a two-dimensional color image of the oral mucosa using an optical imaging device;

[0169] S2. performing preprocessing operations on the acquired two-dimensional color image, including image enhancement, denoising and normalization, to improve image quality and consistency;

[0170] S3 uses a deep convolutional neural network architecture to extract and analyze features from preprocessed two-dimensional color images, identify and segment oral mucosal lesions, and classify and locate lesion types and severity.

[0171] S4. Based on the continuously acquired two-dimensional color images and the analysis results of the intelligent analysis module, the oral mucosal lesions are dynamically tracked, the development and changes of the lesions are recorded, and the changes in the size, color, and texture characteristics of the lesion area are specifically analyzed;

[0172] S5. Display the dynamic tracking results of oral mucosal lesions to users, show the evolution process and trend of the lesions, generate detailed lesion analysis reports, and promptly remind users when abnormal changes occur in the lesions.

[0173] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. Oral mucosal lesion dynamic tracking system, characterized by: include: a data acquisition module, configured to acquire a two-dimensional color image of the oral mucosa, wherein the data acquisition module is configured as an optical imaging device; A data processing module, connected to the data acquisition module, is used to perform preprocessing operations on the acquired two-dimensional color image, including image enhancement, denoising and normalization, to improve image quality and consistency; An intelligent analysis module, connected to the data processing module, uses a deep convolutional neural network architecture to extract and analyze features from preprocessed two-dimensional color images, identify and segment oral mucosal lesion areas, and classify and locate lesion types and severity; A dynamic tracking module, connected to the intelligent analysis module, is used to dynamically track oral mucosal lesions based on continuously acquired two-dimensional color images and the analysis results of the intelligent analysis module, record the development and changes of the lesions, and specifically analyze changes in the size, color, and texture characteristics of the lesion area; A user interaction module, connected to the dynamic tracking module, is used to display the dynamic tracking results of oral mucosal lesions to the user, including a visual interface display unit, a report generation unit, and an early warning prompt unit. The visual interface display unit is used to display the evolution process and trend of the lesions, the report generation unit is used to generate a detailed lesion analysis report, and the early warning prompt unit is used to promptly remind the user when abnormal changes occur in the lesions; Among them, the deep convolutional neural network architecture includes input layer, convolution layer 1, ReLU activation function layer 1, attention mechanism unit, residual connection unit, new convolution kernel unit, pooling layer 1, regression layer, convolution layer 2, ReLU activation function layer 2, pooling layer 2, fully connected layer 1 and fully connected layer 2 in sequence; The attention mechanism unit includes a convolution layer 3, a ReLU activation function layer 3, a convolution layer 4 and a sigmoid activation function layer in sequence, and the new convolution kernel unit includes a hole convolution layer and a ReLU activation function layer 4 in sequence.

2. The oral mucosal lesion dynamic tracking system according to claim 1, characterized in that: The specific operation steps of the data processing module are: S100, using histogram equalization to process the two-dimensional color image to enhance the contrast of the image, and obtain an enhanced two-dimensional color image I enhanced ; S101, denoising the enhanced image using Gaussian filtering to reduce random noise in the image, and obtaining a denoised two-dimensional color image I denoised , the formula is: I denoised =I enhanced *G; Among them, G is the Gaussian kernel, the parameter is 3×3, and * is the convolution operation; S102, the denoised two-dimensional color image I denoised Perform normalization operation to adjust the pixel value range to the [0,1] interval to obtain the normalized two-dimensional color image I normalized , the formula is: Among them, I min and I max are the minimum and maximum pixel values ​​in the denoised two-dimensional color image, respectively.

3. The oral mucosal lesion dynamic tracking system according to claim 1, characterized in that: The specific process of the deep convolutional neural network processing image data is as follows: S200, the normalized two-dimensional color image I normalized Input convolution layer, convolution kernel size is 3×3, number is 32, padding is 1, stride is 1, extract edge, texture and color change features of two-dimensional color image, and obtain feature map F conv1 , the formula is: in, is the pixel value of the ith channel of the normalized two-dimensional color image, K (i) is the i-th convolution kernel, * is the convolution operation, b is the bias term, and σ is the activation function; S201, feature map F conv1 Apply the ReLU activation function to enhance the nonlinear expression ability of the feature. The calculation formula is: F relu1 =max(0,F conv1 ); Among them, F relu1 It is the feature map after being processed by the ReLU activation function; S202, the feature map F processed by the ReLU activation function relu1 Input the attention mechanism unit, first reduce the dimension through the convolution layer 3 with a convolution kernel size of 1×1 and a number of 16 to obtain the feature map F attention_conv1 , and then apply the ReLU activation function 3 to get F attention_relu1 Then, the feature map F is obtained by increasing the dimension through the convolution layer 4 with a convolution kernel size of 1×1 and a number of 32. attention_conv2 ; Use sigmoid activation function to calculate the attention weight A, and add attention weight A to F relu1 Perform element-by-element multiplication to obtain the feature map F attention , the formula is: F attention =F relu1 ⊙A; Among them, ⊙ is the factorial element-by-element multiplication operation, which enhances the key feature area through the attention weight; S203, the feature map F relu1 and feature map F attention Input to the residual connection unit to form a residual connection and obtain the feature map F after residual connection residual , the formula is: F residual =F relu1 +F attention ; S204, the feature map F after residual connection residual The input is sent to the new convolution kernel unit, and the dilated convolution is used for further feature extraction. The dilated convolution kernel size is 3×3, the number is 64, the padding is 2, the stride is 1, and the dilation rate is set to 1, and the feature map F is obtained. dilated , the formula is: in, is the i-th channel of the feature map after residual connection, is the i-th hole convolution kernel; S205, the feature map F dilated Input to pooling layer 1 for pooling operation, the pooling window is 2×2, the stride is 2, and the feature map F is reduced dilated The spatial size of the pooled feature map F is obtained pool1 , the formula is: F pool1 =Pool(F dilated ,2,2); Among them, Pool is the pooling operation; the feature map F pool1 Input to pooling layer 2, repeat the pooling operation, and finally obtain the flattened feature vector X flat ; S206, the feature map F pool1 Input regression layer, the convolution kernel size of the regression layer is 1×1, the number is 4, used to predict the bounding box coordinates, the formula of the convolution operation is: in, is the i-th coordinate component of the predicted bounding box, is the i-th 1×1 convolution kernel, is the bias term corresponding to the i-th 1×1 convolution kernel; After the convolution operation, the regression layer outputs the bounding box coordinates B corresponding to each position pred =(x, y, w, h); S207, flatten the eigenvector X flat Input to the fully connected layer 1, further integrate features and perform classification. The formula is: X fc1 =σ(W1·X flat +b1); Among them, X fc1 is the feature vector output by the fully connected layer 1, W1 is the weight matrix of the fully connected layer 1, b1 is the bias term of the fully connected layer 1, and σ is the activation function; Finally, the feature vector X output by the fully connected layer 1 fc1 Input to the fully connected layer 2 for classification task, the fully connected layer 2 uses the weight matrix W2 and the bias term b2 to classify the feature vector X fc1 Perform weighted summation and bias adjustment, and then convert it into a probability distribution through the softmax function to obtain the classification result Y. The formula is: Y=softmax(W2·X fc1 +b2); Wherein, Y is the classification result, W2 is the weight matrix of the fully connected layer 2, and b2 is the bias term of the fully connected layer 2. The classification result includes the presence of lesions and the absence of lesions.

4. The oral mucosal lesion dynamic tracking system according to claim 1, characterized in that: The implementation of the intelligent analysis module specifically includes the following steps: S300, receiving a continuous time series of two-dimensional color images and corresponding lesion area recognition results, wherein the recognition results include the location, type, severity and bounding box coordinates of the lesion area. pred =(x, y, w, h), and get the area S through the bounding box coordinates t =w×h; S301, extracting the features of the lesion area at each time point based on the continuous time-series two-dimensional color image and the recognition result, including: a. Size feature: Calculate the area S of the lesion region by the bounding box coordinates t and through the lesion area S t To calculate the percentage change in area, the formula is: Where, ΔS t is the percentage change in area, S t is the area of ​​the lesion at the current time point t, S t-1 is the area of ​​the lesion at the previous time point t-1; b. Color features: based on the preprocessed normalized image I normalized , calculate the mean value μ of the RGB three-channel pixels in the lesion area t (k) , where k = 1, 2, 3, and calculate the Euclidean distance of the color mean, the formula is: Among them, μ t (k) is the pixel mean of the lesion area in the RGB three channels at the current time point t, ΔC t is the Euclidean distance of the color mean, quantifying the overall degree of color shift; c. Texture features: Using the dilated convolution feature map F output by the intelligent analysis module dilated , calculate the change of high-frequency texture information, the formula is: ΔT t =∑ i，j |P t (i,j)-P t-1 (i,j)|; Where ΔT t is the Manhattan distance of texture difference, P t (i, j) is the probability of occurrence of pixel pair (i, j) in the gray level co-occurrence matrix of the lesion area at the current time point t, P t-1 (i, j) is the occurrence probability of the pixel pair (i, j) in the gray-level co-occurrence matrix of the lesion area at the previous time point t-1; S302, aligning the lesion areas at different time points using image registration technology, comparing the changes in the size features, color features, and texture features, and generating a lesion development trend map; S303, compare the change with the preset abnormal thresholds α, β and γ, and if ΔS t >α or ΔC t >β or ΔT t >γ, an early warning signal is triggered, and an alarm message is output through the early warning prompt unit of the user interaction module, where α, β and γ are the preset abnormal thresholds of size feature, color feature and texture feature respectively.

5. The oral mucosal lesion dynamic tracking system according to claim 4, characterized in that: The lesion development trend map in step S302 includes a ΔS t The historical change curve based on ΔC t Spatial distribution heat map and ΔT-based t Multi-feature overlay comparison chart.

6. The oral mucosal lesion dynamic tracking system according to claim 1, characterized in that: The dynamic tracking module has a built-in time series prediction model, which uses a Transformer-based architecture and takes as input the lesion feature sequence at historical time points. The output is the predicted value of lesion change at k time points in the future {ΔS n+1 ,...,ΔS n+k }, and present the output results in the form of trend curves through the visual interface display unit of the user interaction module, and associate them with the clinical recommendation database to generate personalized intervention plans.

7. A method for dynamically tracking oral mucosal lesions, characterized in that: The steps include: S1. Obtain a two-dimensional color image of the oral mucosa using an optical imaging device; S2. performing preprocessing operations on the acquired two-dimensional color image, including image enhancement, denoising and normalization, to improve image quality and consistency; S3 uses a deep convolutional neural network architecture to extract and analyze features from preprocessed two-dimensional color images, identify and segment oral mucosal lesions, and classify and locate lesion types and severity. S4. Based on the continuously acquired two-dimensional color images and the analysis results of the intelligent analysis module, the oral mucosal lesions are dynamically tracked, the development and changes of the lesions are recorded, and the changes in the size, color, and texture characteristics of the lesion area are specifically analyzed; S5. Display the dynamic tracking results of oral mucosal lesions to users, show the evolution process and trend of the lesions, generate detailed lesion analysis reports, and promptly remind users when abnormal changes occur in the lesions.

Images